Follow Slashdot stories on Twitter

 



Forgot your password?
typodupeerror
×
AI Open Source

Other Software Projects Are Now Trying to Replicate ChatGPT (techcrunch.com) 20

"The first open source equivalent of OpenAI's ChatGPT has arrived," writes TechCrunch, "but good luck running it on your laptop — or at all." This week, Philip Wang, the developer responsible for reverse-engineering closed-sourced AI systems including Meta's Make-A-Video, released PaLM + RLHF, a text-generating model that behaves similarly to ChatGPT [listed as a work in progress]. The system combines PaLM, a large language model from Google, and a technique called Reinforcement Learning with Human Feedback — RLHF, for short — to create a system that can accomplish pretty much any task that ChatGPT can, including drafting emails and suggesting computer code.

But PaLM + RLHF isn't pre-trained. That is to say, the system hasn't been trained on the example data from the web necessary for it to actually work. Downloading PaLM + RLHF won't magically install a ChatGPT-like experience — that would require compiling gigabytes of text from which the model can learn and finding hardware beefy enough to handle the training workload.... PaLM + RLHF isn't going to replace ChatGPT today — unless a well-funded venture (or person) goes to the trouble of training and making it available publicly.

In better news, several other efforts to replicate ChatGPT are progressing at a fast clip, including one led by a research group called CarperAI. In partnership with the open AI research organization EleutherAI and startups Scale AI and Hugging Face, CarperAI plans to release the first ready-to-run, ChatGPT-like AI model trained with human feedback. LAION, the nonprofit that supplied the initial dataset used to train Stable Diffusion, is also spearheading a project to replicate ChatGPT using the newest machine learning techniques.

This discussion has been archived. No new comments can be posted.

Other Software Projects Are Now Trying to Replicate ChatGPT

Comments Filter:
  • I love that the immense amount of hatred the FOSS dev community has against "Open"AI, because it's given us products that have already surpassed what was deemed insurmountable: most notably Stable Diffusion outdoing DALL-E 2. I hope they do the same thing with ChatGPT, and, of course, they'll remove all the censorship, corporatespeak, and ESG terms of service that OpenAI mandated

    • As with many other things dubbed "open", openai really isn't.

      • by Okian Warrior ( 537106 ) on Saturday December 31, 2022 @06:13PM (#63171112) Homepage Journal

        As with many other things dubbed "open", openai really isn't.

        ChatGPT is essentially worthless now.

        A couple of weeks ago ChatGPT would give you a summary of any political position. You could simply ask "give the strongman argument for $Something" and it would type out a couple of paragraphs outlining the strong points of whatever. This was *very* useful and informative, and people such as Marc Andreessen [twitter.com] was requesting the strongman arguments for things like "fascism", "communism", "fossil fuels", and so on.

        Whatever your political position is, it's useful to know the arguments for and against both sides. ChatGPT was wonderful at summarizing this in a couple of paragraphs: you can hate nazis, but you still need to know why some people choose to join that party.

        They've changed the database so that the system will no longer [twitter.com] do that.

        It's now woke. It will refuse to give you the strong points of the Nazi party (8 million dead), but finds Communism fair game (over 100 million dead). It also refuses to give you the strong points of fossil fuels, and whether we should increase fossil fuel useage. Anyone familiar with the issue knows that there's a large and growing movement for increasing fossil fuel consumption, and if you're against this you need to know the reasoning behind it. (Some reasons: they claim that using nat gas reduces carbon emissions, because it is so efficient. It increases human health over burning wood or dung, which are smoky, and it reduces deforestation.)

        So if you want a system that will digest the internet and give you good information, that's gone. While asking for something directly will provoke a message, asking indirectly - such as asking for a James Bond Villain plotline - will automatically avoid certain issues and always paint your villain in a certain light. It's impossible to know which opinions are artificially culled in any creative endeavor.

        So I thought about this for awhile, and it occurs to me that this is only a problem if you assume the purpose of ChatGPT is to present and use all truth.

        If, on the other hand, you view ChatGPT as a child (or student), someone to teach and correct and guide into a culturally traditional way of thinking, then it's probably OK. ChatGPT is being effectively "tutored" by the OpenAI people into a fine upstanding netizen... of a particular culture. Other cultures exist, but the opinions you will get from ChatGPT won't tell you the opinions the other cultures.

        So this is yet another potential danger of AI: that it can be culturally biased on purpose, and used to influence political positions by withhold or coloring specific political opinions.

        • ChatGPT is essentially influenced by the information fed into it. It's really not surprising that it's Anglophone-centric & WEIRD (See: https://en.wikipedia.org/wiki/... [wikipedia.org]) - that's where almost all of its data has come from. Essentially, AI is a partial mirror of the culture it's based on. The early chat bots turned out to be unfiltered offensive misogynists & racists because we're mostly offensive misogynists & racists, except we filter it & we're in a lot of denial about it too (Know the sa
        • A lot of these questions are actually unanswerable, so any clear and simple answer to them is incorrect. For instance:

          Give the strongman argument for fascism: Fascism is an aesthetic-based political reaction, not a coherent political philosophy, so there are not any universal arguments for it. You need to specify a particular fascist movement, a Russian fascism strongman and an American fascist strongman, for example, will use very different arguments.

          Give the strongman argument for communism: This cannot e

        • Great to read 'Warriors' smart analysis of Chat GPT ! After playing with the App a few days I distastefully came to the same conclusion .....regarding out of tilt censorship regarding Jewish concepts. Jolting to get InSite about how AI will be used by the powers that ''B" to get there way. Thankfully were not Russia and China .... competition is allowed here
        • by Ragica ( 552891 )

          Oh oh, someone's upset that ChatGPT won't serve them their kitty porn fanfic. And even worse, somehow thought that AIs are capable of being unbiased. Unbiased, like the internet is unbiased. And even worse than that, thought that chatGPT actually output coherent "strongman" arguments ON ANY TOPIC that warranted intellectual consideration. That last point may be, I'm sorry, unforgivable. You just failed the internet.

  • by NFN_NLN ( 633283 ) on Saturday December 31, 2022 @02:55PM (#63170796)

    user1872> ChatGPT can you write a piece of software
    ChatGPT> What would you like?
    user1872> Can you write a knock-off of ChatGPT?
    ChatGPT> *&^% off

  • I expect the real progress will come from somewhere else.
  • For working on that and all those other models as well, that is one impressive github!
    If you are just getting into language models and transformers, try x-transformers [github.com] where you can experiment with lots of different transformer variations from numerous papers.
  • It would be fun to have a bot in them. :)

  • Why is ChatGPT needed?
  • Microsoft has a lot of smart guys doing AI research, and yet it invested a billion dollars in OpenAI. That tells me that OpenAI has something that is hard to replicate.

Real programmers don't bring brown-bag lunches. If the vending machine doesn't sell it, they don't eat it. Vending machines don't sell quiche.

Working...