Twitter Opens Much of Its Source Code To the Global Community 70
Twitter blog: At Twitter 2.0, we believe that we have a responsibility, as the town square of the internet, to make our platform transparent. So today we are taking the first step in a new era of transparency and opening much of our source code to the global community.
On GitHub, you'll find two new repositories (main repo, ml repo) containing the source code for many parts of Twitter, including our recommendations algorithm, which controls the Tweets you see on the For You timeline. We're also sharing more information on our recommendation algorithm in this post on our Engineering Blog. For this release, we aimed for the highest possible degree of transparency, while excluding any code that would compromise user safety and privacy or the ability to protect our platform from bad actors, including undermining our efforts at combating child sexual exploitation and manipulation. Today's release also does not include the code that powers our ad recommendations.
We also took additional steps to ensure that user safety and privacy would be protected, including our decision not to release training data or model weights associated with the Twitter algorithm at this point. Ultimately, this is our first step to be more transparent in this way, and we plan to continue sharing more code that does not present a significant risk to Twitter or people on our platform.
On GitHub, you'll find two new repositories (main repo, ml repo) containing the source code for many parts of Twitter, including our recommendations algorithm, which controls the Tweets you see on the For You timeline. We're also sharing more information on our recommendation algorithm in this post on our Engineering Blog. For this release, we aimed for the highest possible degree of transparency, while excluding any code that would compromise user safety and privacy or the ability to protect our platform from bad actors, including undermining our efforts at combating child sexual exploitation and manipulation. Today's release also does not include the code that powers our ad recommendations.
We also took additional steps to ensure that user safety and privacy would be protected, including our decision not to release training data or model weights associated with the Twitter algorithm at this point. Ultimately, this is our first step to be more transparent in this way, and we plan to continue sharing more code that does not present a significant risk to Twitter or people on our platform.
Re: (Score:3)
Sounds like Elon just wants to make operating Twitter cheaper by having others update the codebase.
You can have read only repositories. It really seems a matter of transparency.
Re: (Score:3, Insightful)
There will be no shortage of feedback from people on how it can be improved. Those people not realizing they’re working for free.
Re: (Score:2)
There will be no shortage of feedback from people on how it can be improved. Those people not realizing they’re working for free.
maybe a handful, most will be looking the peer recognition and fame, some hoping for a job.
Re: (Score:2)
I can't even begin to imagine how much of the open source software that I use on a daily basis is only as good as it is due to the contributions of people who didn't get paid.
Re: (Score:2)
Somebody in-house would still have to approve anything that gets merged in, and the short-term possibility that somebody will find a zero-day and not report it is a considerable risk. If he knows what he's doing, he did the best security audit he could before release, so I expect dozens of obvious exploits within the next week.
Re:Cheaper (Score:5, Interesting)
The Fireplace Remains.... (Score:1)
Re: (Score:2)
Re: (Score:1)
That's based on the fallacy of finite bugs. You assume there's a small fixed number of security bugs in the code, and over time they will be discovered and patched.
Unfortunately that's not reality. Software is not static. New code constantly introduces new bugs. And existing bugs are not discovered and patched at a fast enough rate to stem the tide. The old saying "With enough eyes, all bugs are shallow" is a total myth. Look no further than Heartbleed for that. There will always be undiscovered
Re: (Score:1)
Says anonymous coward.
If you want to be taken seriously as not some sort of asshat troll, show yourself.
Re: (Score:2)
Hold on...you weren't clear about whether you were trying to spin that as a good, or bad thing.
Personally, I think he's just trying to take away peoples ability to screech about lack of transparency and potential conspiracy, in recommendations.
At any rate...it's just a recommendation algorithm...it's not going to require the contribution of the combined forces, of all humanity.
Re: (Score:1)
So fucking what? I'm sure anyone that owned the company would do the same.
Only those who think they're always right even when proven wrong over and over again would do something like that. When your ego gets so big no hat can ever cover the size of your head, it's time to step back and humble yourself.
Re: (Score:2)
So... anyone who's ever posted on social media, then.
So, no actual training data? (Score:2)
Re: (Score:2)
Of course not. It might validate the study that conservative views travel faster on Twitter. https://cdn.cms-twdigitalasset... [cms-twdigitalassets.com]
Re: (Score:3, Interesting)
Re: (Score:1)
The ease with which thousands of people can be laid off from these companies, really speaks to how bloated they are...and how inconsequential those peoples jobs are. I have no doubt the employees realize it too...but obviously can't admit it.
Re:So, no actual training data? (Score:5, Interesting)
Musk has said that Twitter was burning through money at a ferocious rate and was about four months from bankruptcy when he bought it. Now, it's close to break-even.
https://twitter.com/elonmusk/status/1639681386318536706 [twitter.com]
Musk thinks the current worth of Twitter is roughly half of what he paid for it. But it's also on track for making money rather than burning money, so in the long run it will be doing better than when he bought it.
obviously they weren't useless, in fact, they added value to the tune of $24bn, or more than 50% of Twitter's worth.
It would be just as fair to say that they obviously added to the cash burn rate and were dragging Twitter to its doom.
Also, it seems clear that Musk overpaid for Twitter. But he wanted to fix Twitter, rather than watch it die while making a competitor to it. He paid dearly to do this and I can only say "good on you" to him.
Even once Twitter starts making money, how long will it take to make it worth more than he paid for it? He's clearly not in this as a way to make money. I believe him that he wants to protect free speech online.
Re: (Score:3, Interesting)
The true technologists on Slashdot appreciate what he does.
Re: So, no actual training data? (Score:5, Interesting)
He's the CEO and majority owner of literally the only provider of manned spaceflight in the free world. That has to piss them off, particularly given that if he had taken no action at all, we'd still be relying on Russia, which would be a really bad thing right about now.
Re: (Score:2)
Musk has said that Twitter was burning through money at a ferocious rate and was about four months from bankruptcy when he bought it. Now, it's close to break-even.
Forgive me for not taking the holy word of a self appointed savour trying desperately to not look like a complete idiot for spending $44bn on Twitter as gospel.
Re: (Score:1)
So we should just believe everything Musk says?
I mean, that's literally your entire argument. That everything Musk says is true.
Have you considered listening to sources other than Musk? I mean, if you care about protecting free speech online, surely that's a bit pointless if you only ever listen to your one true god in Musk? Surely the whole point in protecting free speech online is allowing a broader plurality of views?
I'm not too surprised though, I mean, literally, Musk has shown since he started he's ha
Re: (Score:3)
The tweet you linked says "Titter was trending to lose ~$3B/year (revenue drop of ~$1.5B + debt servicing of ~$1.5B) and had $1B in cash, so only 4 months of money. Extremely dire situation.". Twitter was only in this situation after he bought it and drove everyone away. Their debt payments are only this high because he had Twitter take out massi
Re: (Score:3, Interesting)
The value wasn't in having warm bodies around...it was in advertising dollars. There was a "time of uncertainty"...so, advertisers started to bail.
There was a value blip....big fucking deal. Do you really think he's worried?
The overall deal was closed about 5 months ago. The "hope to fail" crowd won't acknowledge how small of a window of time they are basing their hopes of failure upon.
SpaceX and Tesla spent their time as "jokes" too.
Re: (Score:3)
Where are you seeing those stats?
Re: (Score:2, Informative)
The only "metric" claiming that is arbitrary claims by Musk and his fanboys. But this implies you clearly don't know what the word metric means.
Meanwhile, in the real world, even Twitter's staff left standing admit there's been an increase in child abuse materials and they just can't tackle it anymore:
https://www.bbc.co.uk/news/tec... [bbc.co.uk]
I guess you have some vested interest in keeping that quiet and lying about it. Enjoying Musk's support for access to kiddie porn on the platform are you?
At best he's buried th
A vey good first step, may others also do so (Score:5, Interesting)
Re: (Score:1)
Re:A vey good first step, may others also do so (Score:5, Interesting)
It doesn't appear to be complete transparency ...
Hence a first step.
Re: (Score:2)
You're assuming it's not the last step. Musk has an ornate ability for minimum appeasement.
Re: (Score:2)
They're releasing the code related to the "recommendations" algorithm...and in doing so, taking away peoples ability to claim that their recommendations are based on some hidden "conspiracy".
Re:A vey good first step, may others also do so (Score:4, Insightful)
A vey good first step, may others also do so. Transparency is necessary.
Assuming they actually opened the crucial parts of the recommendation algorithm, I expect the production code to quickly diverge. The problem is that there are too many people and organizations who will benefit from being able to game the system and thereby ensure that their tweets get widely recommended.
Heck, if I were running Twitter, I might publish incorrect recommendation code simply to waste the effort of Tweet "optimizers", and make the actual code more effective. I don't think Musk has done that, though.
Re: (Score:3)
Re: (Score:2)
Re: (Score:2, Informative)
The biggest problem is that despite the headline of open sourcing their source code, this is literally just the recommendation algorithm.
Specifically there's nothing here that tells us that this doesn't just provide base recommendation functionality, but that there's no other manipulation on top. It's perfectly plausible that whilst this does the basic recommendation, there's an entire layer of manipulation and boosting on top.
Given the algorithm is changing too, the question is whether this is their master
Re: (Score:2)
Re: (Score:2)
Elon Musk knows. He specifically requested this after Biden got more engagement than him during the Superbowl [businessinsider.com].
Yes, it actually resulted in people seeing not [theverge.com]
Re: (Score:2)
A radical idea comes to mind:
If you collect user data, you must publish the relevant code.
Really awesome (Score:5, Interesting)
Re: (Score:1)
Re: (Score:3, Interesting)
Pretty simlpe code (Score:5, Funny)
The recommendation code looks pretty simple:
if (tweet_author=="elonmusk") {timeline_position = 1};
Re: (Score:3, Informative)
How did you guess? You were almost there, here's the actual code, you probably just needed to Scala-ify your attempt and you'd probably have nailed it:
https://github.com/twitter/the... [github.com]
"author_is_elon", .getOrElse(AuthorIdFeature, None).contains(candidate.getOrElse(DDGStatsElonFeature, 0L))),
candidate =>
candidate
Re: (Score:2)
You can have that privilege on my social network if you spend $44bn.
Again? (Score:3)
Oh, voluntarily this time
Glow a little blinding (Score:2)
Hope they aren't expecting many contributions... (Score:2)
There aren't a lot of people that work with and contribute to a large project in Scala. A good chunk of them actually worked at Twitter of course. Not to mention all the Mesos hooks.
I know there is a lot talk about how outdated and brittle the Twitter thing is, but they fact that the company could be decimated to the degree it was and still stay up shows how well that stack can work.
In then end, best of luck replacing it with a hodgepodge of who knows what from Node, Java on top of some Kubernetes kludge.
A good start, but (Score:2)
Code doesn't matter *at all* for this sort of thing. Only data does.
To greatly simplify: for any pair of speaker and listener in this scenario, you have a DB entry that holds how likely the listener is to "approve of" the tweet, based on past retweets, likes, etc etc. That base value is then adjusted by "friend of a friend" factors, and so on. The core algorithm itself can literally be implemented in ten lines as a simple MUL loop of fractional weights (or a MAD'ed polynomial, etc), optionally recursed out
Town square (Score:2)
Dear Musk. Twitter is not the Town square. You're a privately owned shopping mall with security and a nutjob for an owner who wants to push people out of the shop every time he takes the podium.
On any other day (Score:1)
Interesting (Score:1)
Read through some of it. Quite interesting, Most of the code is as I expect however some of it looks quite innovative. Looks legit.
I am fascinated that they used a case statement with some dot products and sine functions when running their algos. The rest looks like vanillar coding.
Elon has many faults with twitter... (Score:1)