Wikipedia Editors Reject Founder's AI Review Proposal After ChatGPT Fails Basic Policy Test (404media.co) 35
Wikipedia's volunteer editors have rejected founder Jimmy Wales' proposal to use ChatGPT for article review guidance after the AI tool produced error-filled feedback when Wales tested it on a draft submission. The ChatGPT response misidentified Wikipedia policies, suggested citing non-existent sources and recommended using press releases despite explicit policy prohibitions.
Editors argued automated systems producing incorrect advice would undermine Wikipedia's human-centered model. The conflict follows earlier tensions over the Wikimedia Foundation's AI experiments, including a paused AI summary feature and new policies targeting AI-generated content.
Editors argued automated systems producing incorrect advice would undermine Wikipedia's human-centered model. The conflict follows earlier tensions over the Wikimedia Foundation's AI experiments, including a paused AI summary feature and new policies targeting AI-generated content.
Re: (Score:1)
Re: Al Horse Shit (Score:2)
Advertisers.
As long as we keep arguing about it, they will keep posting it, because social media is about increasing engagement to sell ads.
Re: (Score:2)
How recently did you suggest a new story?
(Slashdot has NEVER been good with news. That's not really what it's about. It's about commentary on news stories.)
Re: (Score:2)
We could always go back to the daily crypto and bitcoin stories.
Good (Score:5, Interesting)
A good thing, overall. Large language models (colloquially called "AI") have too many problems. They don't really "understand" anything in a real sense, they just are able to pattern match.
I can see, however, that Wikipedia (like almost everything else on the internet that isn't locked off) is undergoing a tidal wave of spam (likely much of it generated by these same Large Language Models), and it would indeed be useful to find an automated way to deal with it, and save the human time spent to actually writing articles.
At this point, AI is a hoax! (Score:2, Insightful)
Re:At this point, AI is a hoax! (Score:5, Interesting)
It's a bit worse than that.
It is effectively an "please generate a text that looks as much as possible as the correct answer", with the failure mode being text that LOOKS like the right thing, at a point a human not scrutinizing it enough will think it is the correct response.
It's the perfect liar.
Re: (Score:1)
You know those sci-fi stories where a cunning superintelligence plays humanity like a fiddle?
Clearly humanity has a flattering bias toward itself.
It would take only the best dumbass simulator that VC money could buy before humanity starts slathering pizzas with gorilla glue.
Re: (Score:2)
When talking about 8 billion people the "if" is completely out of question as there's more than zero people for anything you can imagine, it's all about how many.
Luckily there are more than zero people that would never believe an AI in anything no matter how convincing it is in the worst case, but sadly the example you gave would be way more than zero people.
Re: (Score:2)
This strongly depends on which AI and what environment it's operating in. I do suspect that an AI could be trained to follow "organization policies", but I also suspect that with current AIs that would require LOTS of custom training.
Re: (Score:1)
I'm not an expert and who knows the future; ... uh ... governance.
But, The tricky thing is that you can't get one of these "Sum of all knowledge" LLMs we love so much and then train-in any sort of extra corporate policy stuff to a degree required by
You can control the inputs and filter outputs. Even with simple LLMs trained only on corporate policy. But it seems this approach is so far even still prone to mistakes. I had numerous times that an LLM seemed to refuse to help me code by triggering some sort
Re: (Score:2)
I'm not saying you could do that with one of the public versions of ChatGPT. But I think an LLM could be custom trained to do it. I also think it would take a *LOT* of time and effort. Probably enough to make this a silly idea.
So conflicted... (Score:2)
On the one hand, I am very familiar with how much garbage AI produces, especially when you ask it to edit articles with sources etc, which would not be as bad if it was obvious garbage, but it's not so it takes quite some effort to rifle through it.
On the other hand, I absolutely despise Wikipedia editors, so I don't know if AI is much worse. I was a regular contributor until I realised most of my edits were being removed for no good reason, so contributing by the average person was pointless. And then came
Re: (Score:2)
It used to be that we could write articles without having inline citations.
But then the rules became ever more strict about how information must be notable, verifiable and reliable.
Though that happened about 20 years ago.
Re: (Score:2)
Oh, yeah, I never added details without references. Still most things would not stick for more than a few months. So not immediate, I only realised after I noticed something important missing from an article that I was sure I had added, and then went back and looked at my contribution history and whether my additions were still there... Most were not (along with other parts of the articles missing or rewritten), it looked like I was wasting my time so gave up.
Fsck their fules. (Score:3)
"must be notable, verifiable and reliable."
For various definitions of those words. I and others wrote and edited a page about a specific 1990s TCP talk server codebase. Everything in it was verifiable given the author of the code wrote the majority of it, ditto reliable. Notable? Well the code was a Big Deal back in the mid to late 90s and the code in various forms gave rise to dozens of servers with possibly thousands of users spread across them in its hayday. But most wiki editors were probably still at s
Re: (Score:3)
NUTS does have a mention on the Talker article [wikipedia.org].
The content of the article is saved on Fandom [fandom.com]. It has (only) two sources.
But when you say that "the author of the code wrote the majority of it", I'm guessing you mean the article? The thing is that even if Einstein wrote the article about the theory of relativity, it wouldn't be accepted without reliable secondary sources. Which means not his own website.
Re: (Score:2)
A mention is hardly the same as an entire article. As for secondary sources, if the code is available on the download webpage and that webpage is used as the primary source then how can it not be reliable enough? What do they want, someone to go and start another page and cut and paste everything from the 1st then they're happy? Its a joke.
Re: (Score:3)
But when you say that "the author of the code wrote the majority of it", I'm guessing you mean the article? The thing is that even if Einstein wrote the article about the theory of relativity, it wouldn't be accepted without reliable secondary sources. Which means not his own website.
And for good reason. For every Einstein out there there are a thousand crazy people out there with their own crackpot theories showing how everything we know about every single field of science is wrong, who will gleefully write a fifty-page Wikipedia article telling how wonderful they are, how they are geniuses, how their theories explain everything from how UFOs achieve time travel to how to cure your cancer with distilled water, and how the god Vishnu or YHWH or Amon-Ra has blessed their effort, and they
Re: (Score:3)
I forgot to mention that when an article gets deleted, you can ask for it to get restored in a subpage of your user page. Even years after the deletion. It's actually straightforward and automatically granted as I discovered a few years ago.
If you can manage to prove its notability and find good sources, you can then submit it for review.
Re: (Score:2)
I bet they did it naive. "Which policies does the article violate?" with ChatGPT having as much idea of the different policies as average joe, which is none at all.
Put all the policy documents into the context and ChatGPT can give you the feedback you want. If the context is too small, you may need to think about if Wikipedia has too many policies ...
(You could still implement the check by allowing the bot to selectively looking up the policy pages).
Re: (Score:2)
Also again a 404media article. Slashdot should stop citing clickbait sites. In particular these that put a login wall in front of their articles.
Re: (Score:2)
A brief search of "wikipedia ai review" turned up six other sources for this story on the first page.
Re: (Score:2)
So somebody tried to let chatgtp run a business (Score:3)
What they found was that it would work well initially but every single time it would eventually just go crazy. Usually around the time that it needed to keep track of what had to be ordered for restocking after a period of time sometimes when new machines had to be put in new locations.
Eventually it would start hallucinating and not just screwing up the orders or the accounting but just going absolutely ape shit and refusing to do the work.
I suspect a general purpose llm is going to have a lot more problems than one built for purpose. But on the other hand llms are very expensive to run so the companies want general purpose ones so that they can have a lot of different customers.
There does still seem to be a pretty big hit on a lot of entry-level work though. Customer service especially. I know that companies I do business with have started using AI chatbots to replace their first level of support. Hilariously when you contact them you first hit a very basic chatbot and then if that doesn't work they escalate you to a more advanced chat bot. I don't know how many levels you need to get to a person since the fancy chat bots work for basic tasks like returns.
Give it a few years (Score:2)
ChatGPT et al are nowhere near ready to do any "heavy lifting" in Wikipedia.
But give it a few years and it will be.
The first "productive" will be high-quality author/editors using AI to assist with the grunt work of writing a high-quality draft. Things like finding possible references come to mind. That may already be happening and nobody knows it because the only "difference" AI is making is that established author/editors with reputations for producing high quality content already are doing are more pro
Re: (Score:3)
"Non AI" machine translation simply uses statistical analysis just like LLMs except it does it directly using standard programming techniques rather than as a 2nd order effect of the way an LLM neural net is built and trained.
Re: (Score:1)
But give it a few years and it will be.
Really? Because it seems to me that unlike most technology it gets harder and harder to get anything meaningful out of an LLM. The first ones I saw were impressive in the fact that they generated text that seemed to be going somewhere, it seemed to have points and purpose, but it took very little to tell that it, in fact did not.
Made up GPT-2 style example: Once upon a a pink deer frolicked through a strawberry field. Then as the princess said he burst into flames! "He's dead jim", said no one in partic
Will the AI also mark articles as "Not Notable" ? (Score:2)
Same Difference (Score:1)
ChatGPT making up things and incorrectly following policy would merely be in line with the current editorial pool, who ignores policy when it suits them and fabricates material if an article has any kind of political content.
stop using generative tools for authoritative work (Score:2)
People really want the thing that AI claims to offer. Unfortunately, AI falls flat for most use cases. People pick up a generative tool and expect it to do authoritative things. You are using the wrong tool for the job.