Slashdot Log In
Slashdot Posting Bug Infuriates Haggard Admins
Posted by
CmdrTaco
on Thu Nov 09, 2006 10:45 AM
from the this-is-never-good dept.
from the this-is-never-good dept.
Last night we crossed over 16,777,216 comments in the database. The wise amongst you might note that this number is 2^24, or in MySQLese an unsigned mediumint. Unfortunately, like 5 years ago we changed our primary keys in the comment table to unsigned int (32 bits, or 4.1 billion) but neglected to change the index that handles parents. We're awesome! Fixing is a simple ALTER TABLE statement... but on a table that is 16 million rows long, our system will take 3+ hours to do it, during which time there can be no posting. So today, we're disabling threading and will enable it again later tonight. Sorry for the inconvenience. We shall flog ourselves appropriately. Update: 11/10 12:52 GMT by J : It's fixed.
Related Stories
[+]
Facebook Goes To 64 Bit User IDs 144 comments
NewsCloud writes "Facebook has announced to developers that they are moving to a 64 bit user ID in November. At 32 bits, the current ID allows nearly 4.3 billion user accounts. Yet, despite having only 47 million users today, Facebook's move to 64 bits will allow it to have more than 18 quintillion (18,446,744,074,000,000,000) user accounts. Of course, there are currently only about 6.5 billion people in the world. Is Facebook setting their sights beyond Earth or just trying to avoid what happened when Slashdot ran out of space for comment IDs last year. Perhaps they are planning to implement personas."
[+]
Slashdot's Setup, Part 2- Software 151 comments
Today we have Part 2 in our exciting 2 part series about the infrastructure
that powers Slashdot. Last week Uriah told us
all about the
hardware powering the system. This week, Jamie McCarthy picks up
the story and tells us about the software... from pound to memcached to
mysql and more. Hit that link and read on.
This discussion has been archived.
No new comments can be posted.
The Fine Print: The following comments are owned by whoever posted them. We are not responsible for them in any way.
Full
Abbreviated
Hidden
Loading... please wait.
Only one thing to say (Score:5, Funny)
So does a first post ... (Score:5, Funny)
D'oh. (Score:5, Insightful)
And let this be a reminder to the kids - RTFM, twice!
I for one (Score:5, Funny)
Last post! (Score:5, Funny)
who we should REALLY blame (Score:5, Funny)
Thanks for breaking slashdot, jerk
Oh Noes! (Score:5, Funny)
Congrats taco (Score:5, Interesting)
Can anyone actually find it to see - I tried but could only get to 16777217 [slashdot.org], its likely to be in a journal or just a reply to an older article.
I sense a disturbance in the force (Score:5, Funny)
Better make it longer (Score:4, Funny)
So who's the killer? (Score:5, Insightful)
Re: Oh Noes! (Score:5, Funny)
I know what you mean. Y2K was supposed to put an end to civilization, but at least we'd have been able to post on slashdot.
seems strange (Score:5, Funny)
It's true that... (Score:5, Funny)
2^24 (Score:5, Funny)
Comment 16,777,216 does not exist (Score:5, Informative)
Some of you are asking which comment it was that got the cid 16,777,216. The answer is that none did. For redundancy, Slashdot is now running multiple-master replication which skips values for auto-increment [mysql.com]. Our db-1 assigns odd-numbered primary key IDs, and db-2 assigns even-numbered. Right now writes are going to db-1 so newly created rows will have only odd IDs.
The comment that got 2**24-1 was this one [slashdot.org], if anyone cares :)
Sorry about the inconvenience, everyone.
Digg? (Score:5, Funny)
Wait..sorry Commodore fans. I know it had better threading than Digg.
Haggard? (Score:5, Funny)
My Reply to the Funny Comment Above This (Score:5, Insightful)
You claim that the 16,777,216th comment would have broke it but I contest that actually the 16,777,217th comment poster would be the culprit. Since it should be able to handle that many comments if it is zero referenced, and it would actually be the one after that one that would break it. You laugh but these kinds of problems plague a lot of coders?
If you don't agree with me, please respond below and reference my comment ID.
16 million posts ... (Score:5, Funny)
Holy Chit! (Score:5, Funny)
And? (Score:5, Funny)
It's a joke, kids (Score:5, Funny)
Did somebody say... (Score:5, Funny)
Sounds Familiar (Score:5, Informative)
So was it fixed in Slash? (Score:5, Interesting)
Give Slashdot Subscriptions to the borkers (Score:5, Interesting)
If you can find the first guy who COULDN'T reply due to the limit, give him one too. He deserves something for his trouble.
EldavoJohn cid=16786251 reply (Score:5, Informative)
There's no telling which comment it is, because (16,777,217 + 2n) might not have been a reply, meaning it would come up correctly.
Dorks. (Score:4, Funny)
Reply to 16786251 (Score:5, Informative)
``You claim that the 16,777,216th comment would have broke it but I contest that actually the 16,777,217th comment poster would be the culprit. Since it should be able to handle that many comments if it is zero referenced''
Today is the first day I've had to type subjects. (Score:4, Interesting)
I certainly admit I wasn't thinking 0-based when I wrote that. The question is, though, should we blame the person who wrote the last valid comment (therefore ruining the fun for the rest of us), or whoever wrote the first broken comment?
Also, is everyone going to add the obligatory 'parent' link on their posts today?
[ Parent [slashdot.org] ] - [ Reply to this [slashdot.org] ]
Access to the Database? (Score:5, Interesting)
May i be the first to say... (Score:5, Funny)
I always wondered where Paula Bean ended up...
RE: 16777217 GET (Score:5, Funny)
Mod parent up (Score:5, Funny)
Why are all 16 million+ comments in a single table (Score:5, Interesting)
So why, pray, is this usage pattern not accounted for in the database design?
MOD parent up! (Score:4, Funny)
This was fortold a few months ago... (Score:4, Informative)
Take all the time you need (Score:5, Funny)
Slashed Eyeballs (Score:4, Insightful)
Open source - it's not just a buzzword, it's a way of life.
A real Slashdotter! (Score:5, Funny)
Now this is a real Slashdotter! This guy knows how to build an infinite computer!
24 bit? (Score:5, Insightful)
Re: Why are all 16 million+ comments in a single t (Score:5, Informative)
poot_rootbeer asks why all the comments are in one table, when the data access pattern is such that 90% of our hits are on only the most recent entries in that table.
The answer is that we used to do it this way but it's a huge pain. In 2000 we converted from having two tables for 'stories', recent and archived, and merged them together. The performance hit was not big, and it made the code so much simpler it was a no-brainer.
It's the database's job to cache properly whether we split the table or not, and the database does that just fine. The only performance problem could be when there is a rush of inserts, or updates to the same sets of rows, spanning both newer and older portions of the table, and that just doesn't happen.
If we did want to do this we wouldn't split the tables manually; the code complexity is too high a price to pay. In MySQL 5.0 we would use a MERGE [mysql.com] engine, which has issues of its own but would involve smaller changes to our code. That's still not worth it for us. What we're probably going to do is wait for MySQL 5.1 to get out of beta and then do some performance testing on tables partitioned [mysql.com] by date and see if that gains us anything. For example, a SELECT on our comments table could be limited with a WHERE clause to only retrieve rows with a date >= the discussion object's date, which for 90% of our queries MySQL 5.1 could optimize to only look at the most recent partition. If the gains turn out to be significant, then since partitioning involves very limited code changes, we'll probably do that. Generally speaking, though, database performance is not a problem for us. So far our main bottlenecks have been CPU and RAM on the webheads. As long as we don't do anything stupid our database performance has been fine, though, as today proves, we are quite capable of being stupid.
[ Parent [slashdot.org] ]
Let the flamewars begin... (Score:5, Funny)
Let the flamewars begin...
Unthreaded flame wars are much less enjoyable.
graphic artists (Score:5, Funny)
The number 2^24 is of interest to digital computer artists, as that is the number of unique colors combined in the commonly implemented "True Color" RGB8 space. That color space is looking pretty limiting in some respects, but that is truly a lot of unique colors when you think about it. A 16 megapixel image does not need to repeat any color used.
If all slashdot posts from the history of Slashdot were sorted into color bins,Once that were done, people could simply post their replies as a reference to existing posts. "Hey, #938D3A to you, buddy!" "Know what I think of that? #F2C2A9!"
Finally, a tale for my grandkids in the new era (Score:5, Funny)
Next, he tries Papa bear's integer, but proclaims "4 bytes is way too big for my little site, I'd just end up wasting so much."
Finally, he tries Mama bear's integer, and extols "3 bytes is just right," not noticing it was really the same as Papa Bear's bowl in disquise.
/. copyright question (Score:5, Interesting)
Regardless, while writing this post [slashdot.org] regarding why the
Comments on
If I own the copyright on the comments I've made, shouldn't I be able to rescind publication rights on them, and prevent
Or are publication rights, once granted, irrevocable?
Of course, I suppose asking questions when there's no way for people to hit reply is a specific form of vague insanity...still, I'm curious.
So how do I get to post #1? (Score:4, Informative)
+10 (Score:5, Insightful)
At least they didn’t try to make bullshit excuses. I respect them for being up front about the real nature of the issue.
"flogging definition" (Score:5, Interesting)