A New White House Report Embraces Open-Source AI 15
An anonymous reader quotes a report from ZDNet: According to a new statement, the White House realizes open source is key to artificial intelligence (AI) development -- much like many businesses using the technology. On Tuesday, the National Telecommunications and Information Administration (NTIA) issued a report supporting open-source and open models to promote innovation in AI while emphasizing the need for vigilant risk monitoring. The report recommends that the US continue to support AI openness while working on new capabilities to monitor potential AI risks but refrain from restricting the availability of open model weights.
According to the NTIA report, these are the key benefits of open-source AI models:
1. Broader accessibility: "Open-weight" models allow developers to build upon and adapt previous work, making AI tools more accessible to small companies, researchers, nonprofits, and individuals.
2. Innovation promotion: The openness of AI systems affects competition and innovation in these revolutionary tools. By embracing openness, the report aims to provide a roadmap for responsible AI innovation and American leadership.
3. Accelerated development: Open models may accelerate the diffusion of AI's benefits and the pace of AI safety research.
4. Democratization of AI: Open models broaden the availability of AI tools, potentially democratizing access to powerful AI capabilities across various sectors and user groups.
5. Transparency and understanding: Open models can contribute to a broader understanding of AI systems, a crucial factor for effective and reliable development.
6. Economic benefits: The wide availability of US-developed open foundation models can serve the national interest by promoting innovation and competitiveness.
7. Research advancement: Open models facilitate academic research on the internals of AI models, enabling deeper study and improvement of the technology.
8. Local deployment: Open weights allow users and organizations to run models locally on their edge devices, which can benefit certain applications and use cases.
9. Customization: Open models enable creative modifications to suit specific user needs and applications.
Intentionally Left Blank (Score:1)
Read TFA, is very light on specifics. Do they even bother to specify what "open source" is? I'm seeing a lot of techbros throwing the term around as if it were just another magical incantation to ward away lawsuits and make line go up.
Re: Intentionally Left Blank (Score:3)
Re: (Score:1)
Can you view the source? If the answer is yes then it's open.
Re: (Score:1, Troll)
Do they even bother to specify what "open source" is?
It only ever meant you could see the sources, that meaning was used starting in the 1980s, only the ignorant and late to the party think otherwise.
What it means for a LLM is even less, though. Does it mean you can see the source to the code used to produce the model? Does it mean you can see the training set? Which you can't make use of unless you have a big fat cluster, of course. But then, Open Source never meant that you had the tools you need to make use of the sources, either. That's why we have not on
Going further towards AI transparency.... (Score:2)
As I suggested in this essay over twenty years ago (originally sent to the Markle Foundation in 2001):
"On funding digital public works"
https://pdfernhout.net/on-fund... [pdfernhout.net]
====
Consider again the self-driving cars mentioned earlier which now cruise some streets in small numbers. The software "intelligence" doing the driving was primarily developed by public money given to universities, which generally own the copyrights and patents as the contractors. Obviously there are related scientific publications, but in p
Likely the White House has no AI plan at all (Score:2)
The administration has no plan for AI and this is a feel good, blue ribbon commission cutting exercise, to say that they have addressed the AI question until the next president is sworn in.
Re: (Score:2)
Terms definition is a problem here.
By the classic definition of Open Source, an Open Source model would include the full set of training data and the algorithms used to train it.
As used here, I understand Open Source to mean "here are the weights used in the model so you can run it on any hardware, retrain it, or extend it".
I prefer to use models opened under the latter definition vs. closed models so I'm not locked in and at the behest of $vendor. To that end, I'm using Mistral in my personal work. If a
Re: (Score:3)
By the classic definition of Open Source, an Open Source model would include the full set of training data and the algorithms used to train it.
The classic definition is "you can see the source".
We were using this definition for literally decades before the OSI was created and members started lying about defining the term.
Where is the LLM stored and processed? (Score:2)
Devil's advocate:
I'm curious who will store the LLM data and process it. The amount of storage would make Chia farmers drool that is needed even for a smaller LLM, and the amount of matrix multiplication hardware needed can require data centers and a county changing its energy profile to handle the gigawatts used for a data center.
An open source AI is nice, but someone has to "bell the cat" and have it run somewhere. One can use smaller LLMs, but they may wind up irrelevant.
Re: Where is the LLM stored and processed? (Score:3)
Re: (Score:3)
Source code?
That is the easiest part:
https://github.com/berkerdemir... [github.com]
The papers are out there as well:
https://arxiv.org/abs/2005.141... [arxiv.org]
Transformers are very powerful, and also extremely simple. They were initially "attention" layers in the more complex LSTM cells in neural networks. Over time we realized "attention is all you need" (https://arxiv.org/abs/1706.03762, the famous Google paper that brings the T to GPT).
The rest is history. Except for Petabytes of data.
Really, you need PBs of training data to m
Re: Where is the LLM stored and processed? (Score:2)
Re: (Score:3)
It is the same one with all other GPTs, except for using "mixture of experts"
You can literally find many implementations using a simple Google search:
https://github.com/lucidrains/... [github.com]
Once again source code is not the problem. Many others including Meta, Anthropic, Mistal, X have done similar networks, and published their research and some of the source code.
Here is llama from meta, which can match GPT4o:
https://github.com/meta-llama/... [github.com]
Good luck running it though. You'd need PBs of data and hundreds of mill
Re: Where is the LLM stored and processed? (Score:2)