Meta To Release Open-Source Commercial AI Model To Compete With OpenAI, Google

An anonymous reader quotes a report from ZDNet: Meta, formerly known as Facebook, is set to release a commercial version of LLaMA, its open-source large language model (LLM) that uses artificial intelligence (AI) to generate text, images, and code. LLaMA, which stands for Large Language Model Meta AI, was publicly announced in February as a small foundational model, and made available to researchers and academics. Now, the Financial Times is reporting that Meta is prepared to release the commercial version of the model, which would enable developers and businesses to build applications using the foundational model.

Since it's an open-source AI technology, commercial access to LLaMA gives businesses of all sizes the opportunity to adapt and improve the AI, accelerating technological innovation across various sectors and potentially leading to more robust models. Meta's LLaMA is available in 7, 13, 33, and 65 billion parameters, compared to ChatGPT's LLM, GPT-3.5, which has been confirmed to have 175 billion parameters. OpenAI hasn't said how many parameters GPT-4 has, but it's estimated to have over 1 trillion parameters -- the more parameters, the better the model can understand input and generate appropriate output.

Though open-source AI models already exist, launching Meta's LLaMA commercially is still a significant step, due to it being larger than many of the available open-source LLMs on the market, and the fact that it is from one of the biggest tech companies in the world. The launch means Meta is directly competing with Microsoft-backed OpenAI and Google, and that competition could mean significant advancements in the AI field. Closed or proprietary software, like that used in OpenAI's ChatGPT, has drawn criticism over transparency and security.

"understand"

[nsaspook]

OMg, there's the magic word "understand" being used again.

Re:

[Baron_Yam]

What is really odd is that we don't understand how nature made the jump from simple neural nets to things with consciousness. We're just a bunch of inputs and outputs connected to a mass of specialized nerve cells... it's not like there's a soul in there doing supernatural things, we can see similar awareness in other animals to varying degrees.

As best as I can tell the difference between us and the current AI is ongoing feedback, the ability to affect the inputs with the outputs, and a few orders of magni

Re:

[VeryFluffyBunny]

Yeah, that "...more parameters, the better the model can understand input and generate appropriate output." irked me too. Journalists clearly need help with this. They need to be provided with appropriate, comprehensible language which doesn't mislead the public. Something like "...more parameters, the better the model can process patterns* in the input and generate relevant** sounding human-like output."? The more I think about it, the more difficult it seems to be.

*statistical regularities

**correspo

That and "appropriate output".

[DrYak]

there's the magic word "understand" being used again.

Yup, indeed. And then there's the bit after that:

the more parameters, the better the model can understand input and generate appropriate output.

Nope. The correct would be "and generate better sounding output.".
LLMs are good a putting nice sounding words together, given the context.
Whether those words are actually an appropriate or completely made up answer is entirely outside the scope of LLMs - their scope is merely "most likely words, given the training dataset".

When a user asks a question, there is no vi

hurry up

[Anonymous Coward]

because by the time the lawsuits are finished over training data these tech companies are gonna wish they stuck with Eliza.

Llama

[rossdee]

Say, is Jeff Minter still alive?

Late

[systemd-anonymousd]

LLaMA was leaked by researchers three months ago and 4chan users have been going nuts with excellent finetunes ever since. They're gonna have to do better than stock LLaMA.

Re:

[real_nickname]

But now you can use llama in commercial software before it was only for research projects, that's the difference. Maybe some developers are illegally selling software using llama but that's not the point.

Re:

[systemd-anonymousd]

Legal and illegal usage of leaked weights don't matter that much when Meta's model is trained in a manner that could very well be stealing. This is still all a grey area. Before you disagree, remember that you very probably supported the free usage of DeCSS keys and yelled "It's just math! It's just a number!"

Re:

[real_nickname]

Oh I agree, models trained on scrapped data are full of copyrighted contents and should be restricted to research and hobby. It's an on going battle.

Re:

[VeryFluffyBunny]

Oh great. Just what the world needs; yet another racist, misogynistic chat bot. Didn't Microsoft already infamously do that & then pull the plug in hyper-apologetic embarrassment?

Re:

[braden87]

it wasn't the bots fault! When you chat with Nazis and that's your impression of humanity while your task is to be more human.... yeah.

Will there be a 4-bit (or less) versions?

[Stash of Code]

Since the hype around the leak of LLaMA has happened because some great guys made it run without even requiring a GPU, it will be interesting to see if Meta learned something from this, first of all that 4-bit quantization seems the produce good results...