Open Source ChatGPT Clone 'LibreChat' Lets You Use Multiple AI Services (thenewstack.io) 39
Slashdot reader DevNull127 writes:
A free and open source ChatGPT clone — named LibreChat — lets its users choose which AI model to use, "to harness the capabilities of cutting-edge language models from multiple providers in a unified interface". This means LibreChat includes OpenAI's models, but also others — both open-source and closed-source — and its website promises "seamless integration" with AI services from OpenAI, Azure, Anthropic, and Google — as well as GPT-4, Gemini Vision, and many others. ("Every AI in one place," explains LibreChat's home page.) Plugins even let you make requests to DALL-E or Stable Diffusion for image generations. (LibreChat also offers a database that tracks "conversation state" — making it possible to switch to a different AI model in mid-conversation...)
Released under the MIT License, LibreChat has become "an open source success story," according to this article, representing "the passionate community that's actively creating an ecosystem of open source AI tools." And its creator, Danny Avila, says in some cases it finally lets users own their own data, "which is a dying human right, a luxury in the internet age and even more so with the age of LLM's." Avila says he was inspired by the day ChatGPT leaked the chat history of some of its users back in March of 2023 — and LibreChat is "inherently completely private". From the article:
With locally-hosted LLMs, Avila sees users finally getting "an opportunity to withhold training data from Big Tech, which many trade at the cost of convenience." In this world, LibreChat "is naturally attractive as it can run exclusively on open-source technologies, database and all, completely 'air-gapped.'" Even with remote AI services insisting they won't use transient data for training, "local models are already quite capable" Avila notes, "and will become more capable in general over time."
And they're also compatible with LibreChat...
Released under the MIT License, LibreChat has become "an open source success story," according to this article, representing "the passionate community that's actively creating an ecosystem of open source AI tools." And its creator, Danny Avila, says in some cases it finally lets users own their own data, "which is a dying human right, a luxury in the internet age and even more so with the age of LLM's." Avila says he was inspired by the day ChatGPT leaked the chat history of some of its users back in March of 2023 — and LibreChat is "inherently completely private". From the article:
With locally-hosted LLMs, Avila sees users finally getting "an opportunity to withhold training data from Big Tech, which many trade at the cost of convenience." In this world, LibreChat "is naturally attractive as it can run exclusively on open-source technologies, database and all, completely 'air-gapped.'" Even with remote AI services insisting they won't use transient data for training, "local models are already quite capable" Avila notes, "and will become more capable in general over time."
And they're also compatible with LibreChat...
You're wrong. (Score:5, Informative)
It's right there, in Slashdot's story...
"With locally-hosted LLMs, Avila sees users finally getting "an opportunity to withhold training data from Big Tech..."
What part of "completely 'air-gapped'" are you not understanding?
Re: You're wrong. (Score:5, Insightful)
Itâ(TM)s a very good open source project that can front open models (or closed ones). What is your problem that you need to ignorantly dump all over a free MIT licensed project?
Re: (Score:2)
I'll add: the documentation does not clearly lay this out. Given that the whole point is 'owning your own data,' these LLMs need big flashing warning labels on them, which I am not seeing.
Re: (Score:3)
Acting as if LibreChat brings one nearer to running a local LLM is like claiming that running a reverse proxy locally, routing requests to google.com gets you a "local Internet search engine".
As I read it, LibreChat lets you connect to almost any LLM, including ones that are hosted locally. So, to address your analogy, the reverse-proxy could connect to a "local internet search engine" if you happen to have one.
Re: (Score:2)
Acting as if LibreChat brings one nearer to running a local LLM is like claiming that running a reverse proxy locally, routing requests to google.com gets you a "local Internet search engine".
As I read it, LibreChat lets you connect to almost any LLM, including ones that are hosted locally. So, to address your analogy, the reverse-proxy could connect to a "local internet search engine" if you happen to have one.
And just like with Internet search engines, connecting to one is trivial, with or without a local proxy in between, while hosting a useful Internet search engine on your own hardware is extremely expensive.
Re:Not really (Score:5, Interesting)
Re: (Score:2)
"Love"? Not at all. I am currently correcting a coding exam where students are allowed to use ChatGPT and the like. This machine is a lot more dumb than the average student. So far, about 20% of the students have managed to fix the crap that ChatGPT delivered them and these are _simple_ coding tasks. The rest is just insightless crap and often outright bullshit.
I don't get it... (Score:3, Insightful)
Sure, using something that isn't the vendor's curious interpretation of what makes a good web UI might well be nice; but it absolutely does not change the structure of what you are doing.
Re: I don't get it... (Score:4, Informative)
It is awesome you have the self awareness to start with âoeI donâ(TM)t get itâ. You can run a decent local LLM with one line and docker from the command line and you can front it with this.
IRC (Score:1)
They could have come up with a name that doesn't conflict with the new-ish F/OSS project focused IRC network that everyone migrated to from that old one that I forget the name to...
Re: (Score:2)
What are you talking about? LibEraChat != LibreChat, I didn't even think of Libera Chat IRC when I read TFS.
Re: (Score:2)
Whoops... I stand corrected. I guess, to me, it was similar enough looking to trick my brain.
Re:If you use APIs you do not have privacy (Score:5, Insightful)
Except you can easily self-host an openAI compatible LLM on your home network.
Re: (Score:2)
In my personal testing, on an AM5 system with a GTX 1070ti, I can get comparable results with llama.cpp and various medium sized models. The conversational history isn't as long, and the generation is substantially slower-- but the results still usually only take a minute or so.
Obviously there's a question of scale-- the models are a bit limited to allow loading in a lowly $1500 PC, and I'm not going to generate an article, let alone a book-- but for general (and sometimes specific) knowledge, it does work
Re: (Score:2)
You can, but the headline says you could use "Multiple AI Services" while owning your data. And that's plain wrong.
You can self-host and own your data OR you can use AI services and don't have that privacy. You can't have both.
Re: (Score:2, Insightful)
From the article:
Down the line, Avila tells the podcasters, he’s even considering the possibility of preconfiguring LibreChat to switch to the best AI for particular tasks — either by creating a kind of “smart rou
Re: (Score:2)
Most API frontends won't even notice when you switch the model in the backend. Just use some frontend with OpenAI API and ooba as backend. The frontends may fetch what model is used when you connect them, but when you switch to another model in the backend after a response, the frontend won't even notice it before it gets the answer to the next request.
LibreChat is Great! (Score:4, Insightful)
It's so easy even I set it up (and bolted in RAG in the Node.js code to hit my own vector database). Completely self hosted LLMs with a ChatGPT UI, all running on a few easy-as-pie to set up Docker containers with the source code easy to fiddle with... These guys are heroes in my book. It literally could not be easier to work with.
Re: (Score:2, Insightful)
Re:LibreChat is Great! (Score:4, Informative)
Re: (Score:2)
Re: LibreChat is Great! (Score:5, Insightful)
Thank you. The amount of stupid in these comments is appalling
Re: (Score:1)
obligatatory (Score:2)
Nonsense (Score:2)
There are already many answers from locally hosting any of the available LLM models and this is not private if connecting to third party servers, though open ai may offer more privacy to commercial api gpt users vs chatgpt that is voluntary on their part, they still get all your data.
I'm seeing no reason I'd choose this over something like Voxta with integration to just about every service, their own privacy focused cloud service, and tts/stt integration for natural interaction as a core feature. Actually I