What Is the Future of Open Source AI? (fb.com) 22
Tuesday Meta released Llama 3.1, its largest open-source AI model to date. But just one day Mistral released Large 2, notes this report from TechCrunch, "which it claims to be on par with the latest cutting-edge models from OpenAI and Meta in terms of code generation, mathematics, and reasoning...
"Though Mistral is one of the newer entrants in the artificial intelligence space, it's quickly shipping AI models on or near the cutting edge." In a press release, Mistral says one of its key focus areas during training was to minimize the model's hallucination issues. The company says Large 2 was trained to be more discerning in its responses, acknowledging when it does not know something instead of making something up that seems plausible. The Paris-based AI startup recently raised $640 million in a Series B funding round, led by General Catalyst, at a $6 billion valuation...
However, it's important to note that Mistral's models are, like most others, not open source in the traditional sense — any commercial application of the model needs a paid license. And while it's more open than, say, GPT-4o, few in the world have the expertise and infrastructure to implement such a large model. (That goes double for Llama's 405 billion parameters, of course.)
Mistral only has 123 billion parameters, according to the article. But whichever system prevails, "Open Source AI Is the Path Forward," Mark Zuckerberg wrote this week, predicting that open-source AI will soar to the same popularity as Linux: This year, Llama 3 is competitive with the most advanced models and leading in some areas. Starting next year, we expect future Llama models to become the most advanced in the industry. But even before that, Llama is already leading on openness, modifiability, and cost efficiency... Beyond releasing these models, we're working with a range of companies to grow the broader ecosystem. Amazon, Databricks, and NVIDIA are launching full suites of services to support developers fine-tuning and distilling their own models. Innovators like Groq have built low-latency, low-cost inference serving for all the new models. The models will be available on all major clouds including AWS, Azure, Google, Oracle, and more. Companies like Scale.AI, Dell, Deloitte, and others are ready to help enterprises adopt Llama and train custom models with their own data.
"As the community grows and more companies develop new services, we can collectively make Llama the industry standard and bring the benefits of AI to everyone," Zuckerberg writes. He says that he's heard from developers, CEOs, and government officials that they want to "train, fine-tune, and distill" their own models, protecting their data with a cheap and efficient model — and without being locked into a closed vendor. But they also tell him that want to invest in an ecosystem "that's going to be the standard for the long term." Lots of people see that open source is advancing at a faster rate than closed models, and they want to build their systems on the architecture that will give them the greatest advantage long term...
One of my formative experiences has been building our services constrained by what Apple will let us build on their platforms. Between the way they tax developers, the arbitrary rules they apply, and all the product innovations they block from shipping, it's clear that Meta and many other companies would be freed up to build much better services for people if we could build the best versions of our products and competitors were not able to constrain what we could build. On a philosophical level, this is a major reason why I believe so strongly in building open ecosystems in AI and AR/VR for the next generation of computing...
I believe that open source is necessary for a positive AI future. AI has more potential than any other modern technology to increase human productivity, creativity, and quality of life — and to accelerate economic growth while unlocking progress in medical and scientific research. Open source will ensure that more people around the world have access to the benefits and opportunities of AI, that power isn't concentrated in the hands of a small number of companies, and that the technology can be deployed more evenly and safely across society. There is an ongoing debate about the safety of open source AI models, and my view is that open source AI will be safer than the alternatives. I think governments will conclude it's in their interest to support open source because it will make the world more prosperous and safer... [O]pen source should be significantly safer since the systems are more transparent and can be widely scrutinized...
The bottom line is that open source AI represents the world's best shot at harnessing this technology to create the greatest economic opportunity and security for everyone... I believe the Llama 3.1 release will be an inflection point in the industry where most developers begin to primarily use open source, and I expect that approach to only grow from here. I hope you'll join us on this journey to bring the benefits of AI to everyone in the world.
"Though Mistral is one of the newer entrants in the artificial intelligence space, it's quickly shipping AI models on or near the cutting edge." In a press release, Mistral says one of its key focus areas during training was to minimize the model's hallucination issues. The company says Large 2 was trained to be more discerning in its responses, acknowledging when it does not know something instead of making something up that seems plausible. The Paris-based AI startup recently raised $640 million in a Series B funding round, led by General Catalyst, at a $6 billion valuation...
However, it's important to note that Mistral's models are, like most others, not open source in the traditional sense — any commercial application of the model needs a paid license. And while it's more open than, say, GPT-4o, few in the world have the expertise and infrastructure to implement such a large model. (That goes double for Llama's 405 billion parameters, of course.)
Mistral only has 123 billion parameters, according to the article. But whichever system prevails, "Open Source AI Is the Path Forward," Mark Zuckerberg wrote this week, predicting that open-source AI will soar to the same popularity as Linux: This year, Llama 3 is competitive with the most advanced models and leading in some areas. Starting next year, we expect future Llama models to become the most advanced in the industry. But even before that, Llama is already leading on openness, modifiability, and cost efficiency... Beyond releasing these models, we're working with a range of companies to grow the broader ecosystem. Amazon, Databricks, and NVIDIA are launching full suites of services to support developers fine-tuning and distilling their own models. Innovators like Groq have built low-latency, low-cost inference serving for all the new models. The models will be available on all major clouds including AWS, Azure, Google, Oracle, and more. Companies like Scale.AI, Dell, Deloitte, and others are ready to help enterprises adopt Llama and train custom models with their own data.
"As the community grows and more companies develop new services, we can collectively make Llama the industry standard and bring the benefits of AI to everyone," Zuckerberg writes. He says that he's heard from developers, CEOs, and government officials that they want to "train, fine-tune, and distill" their own models, protecting their data with a cheap and efficient model — and without being locked into a closed vendor. But they also tell him that want to invest in an ecosystem "that's going to be the standard for the long term." Lots of people see that open source is advancing at a faster rate than closed models, and they want to build their systems on the architecture that will give them the greatest advantage long term...
One of my formative experiences has been building our services constrained by what Apple will let us build on their platforms. Between the way they tax developers, the arbitrary rules they apply, and all the product innovations they block from shipping, it's clear that Meta and many other companies would be freed up to build much better services for people if we could build the best versions of our products and competitors were not able to constrain what we could build. On a philosophical level, this is a major reason why I believe so strongly in building open ecosystems in AI and AR/VR for the next generation of computing...
I believe that open source is necessary for a positive AI future. AI has more potential than any other modern technology to increase human productivity, creativity, and quality of life — and to accelerate economic growth while unlocking progress in medical and scientific research. Open source will ensure that more people around the world have access to the benefits and opportunities of AI, that power isn't concentrated in the hands of a small number of companies, and that the technology can be deployed more evenly and safely across society. There is an ongoing debate about the safety of open source AI models, and my view is that open source AI will be safer than the alternatives. I think governments will conclude it's in their interest to support open source because it will make the world more prosperous and safer... [O]pen source should be significantly safer since the systems are more transparent and can be widely scrutinized...
The bottom line is that open source AI represents the world's best shot at harnessing this technology to create the greatest economic opportunity and security for everyone... I believe the Llama 3.1 release will be an inflection point in the industry where most developers begin to primarily use open source, and I expect that approach to only grow from here. I hope you'll join us on this journey to bring the benefits of AI to everyone in the world.
Expand, Exploit, Exterimate (Score:3)
Re: (Score:2)
my take away
Zuck>> Open source will ensure that more people around the world have access to the benefits and opportunities of AI, that power isn't concentrated in the hands of a small number of companies, and that the technology can be deployed more evenly and safely across society
Facebook teaches how private peoples around the world would feed subsequent LLMs building power in the hands of one company that has deployed technology in service to providing worldwide access to full benefits and opportun
AI without robotics (Score:2)
AI is useless without an avatar. A human hand has 27 degrees of freedom, there is no robot hand that has anything close to that. Even Tesla admits they can't do it. In other words, if you do something that involves hand dexterity (whacking off doesn't count), then your job is safe. My advice to the young, learn to do artist level shit. Sculpting, jewelry making, plumbing, carpentry, electrician .. things like that. No robot will have the hand dexterity for that for at least a century if not longer. Anyway,
Re: (Score:2, Interesting)
Basically I would say that if we want to improve the ability of robots to function in the human world, teaching them to whack off people is a simple problem with a lot of generalizable subtl
Re: (Score:2)
Only if the robot has hallucination control, you would not want it getting "strange" ideas about the target "unit". If an AI gets kinky, will we know? Kinky is very subjective.
Re:AI without robotics (Score:5, Interesting)
> AI is useless without an avatar.
We have several AIs that are useful without avatars. For example AlphaFold2 and AlphaFold3.
> A human hand has 27 degrees of freedom, there is no robot hand that has anything close to that.
So 27 degrees of freedom is not close enough?
https://www.freethink.com/robo... [freethink.com]
> Even Tesla admits they can't do it.
I really like the Tesla company, but I don't consider it to be the tech leader. I think Boston Dynamics is better at robots and Google Deepmind is better with AI. So if Tesla can't do something, it doesn't mean that others couldn't do it.
> In other words, if you do something that involves hand dexterity (whacking off doesn't count), then your job is safe.
That is incorrect. For example consider a person who makes super detailed sculptures. That person can be easily replaced by a milling machine with a software that can carve 3D-models. Or consider Switchboard Operators, they required hand dexterity to connect calls, they even required to understand human speech. The job was replaced by a simple computer program with pretty much no moving parts at all.
Re: (Score:2)
IIUC, the system the replaced th switchboard operators was full of relay switches. LOTS of moving parts.
Editor fail, "But just one day Mistral". C'mon... (Score:1)
Jfc, it's the second sentence.
At least pretend to read/edit. If editor had made any attempt to actually read the summary before slapping the "post to front page" button, that would have been caught. I couldn't even read past that without my eyes stopping hard right there.
It's pretty insulting to the few people who still visit here.
All Hail the Great Zuck! (Score:1)
Mistral has Open Source Models (Score:2)
That author missed, that Mistral 7B, Mixtral and Mistral 12B are Apache 2.0 licensed.
Changing the definition of Open Source AI (Score:3, Informative)
The OSI which defines what is an "open source" license. They are working on an definition of the equivilant for Open Source AI, but it seems that it has been overtaken by the large AI companies to define it as something that is not what I would consider "open".
https://lunduke.locals.com/pos... [locals.com]
Stop saying Llama is open-source (Score:5, Insightful)
AI is one tiny program running a giant neural network. The only thing that's open-source is the tiny program. The giant neural network is a giant black box.
Re: (Score:2)
The lie makes the hype going a bit longer (before it all comes crashing down). Most of the AI hype is, at this time, nothing more than a scam.
Re: (Score:2)
AI is one tiny program running a giant neural network. The only thing that's open-source is the tiny program. The giant neural network is a giant black box.
Personally consider the giant black box open source because it is freely available and enables people to modify it to do what they want in the most direct way feasible. For all intents and purposes it is the equivalent of source code.
The difference with AI is you are not directly manipulating source code you are applying algorithms and data to modify the model. It's an indirect process.
People could have a separate beef with the licensing terms of some of these contraptions (Large 2407 in particular) yet o
No Future (Score:5, Insightful)
None, there is no open source AI. "Open Source" that is now just a tag that is attached to a product for marketing purposes.
Soon enough I am sure we will see Open Source Windows, Beef and drinks. All will be protected by a wall of patents, but will be called Open Source to make the unwashed masses feel good.
Re: (Score:2)
Indeed. Marketing people have no shame, no honor and no integrity. They will steal anything and everything they think they can get away with. They will gladly do a lot of damage if that benefits them a tiny bit.
Re: (Score:1)
I get it, this isn't open source. To be open source we would need to be able to have access to the corpus that built it plus all the instructions to rebuild it.
What this *is* is "open weights". As opposed to ChatGPT where it's all locked up in the cloud.
Chances are somebody couldn't find "open weights" in their dictionary but found "open source". It was considered close enough for non-technical users, of which management and the unwashed masses tend to qualify as.
Also, with open weights it is able to be mod
Re:Opened source beef (Score:2)
Who cares (Score:2)
The current AI hype will have as many actual useful effects when the hype is over as the last ones had: Very little, and among them the insight that AI still cannot do what its fans think it can.
The One Goal They Seem To Consistently Miss (Score:2)
So the proliferation of largly identical products with only slight differences in actual performance/use suggests that no current variation is likely to "bring the benefits of AI to everyone".
Open source as in KFC (Score:1)
Open source as in KFC, you can eat it where ever you like, but they're not going to give you the secret recipe.