"I think a wise man once said, "you better check yo'self before you wreck yo'self" or something along those lines. ONTD is great because not only do they provide hours of fun, they also push our application and highlight the pressure points. No, ONTD didn't break LJ, and no they didn't break themselves (well, not really), what they did is point out that "hey! there might be a journal or community that has over 16 million comments! no, really!".
Here's the poop.
Problem: LJ was designed with a 24 bit limitation on number of comments -- in the database and in crucial parts of our commenting code. This may have been done for speed optimization, resource conservation, for the children, whatever. It was done with good intentions and now we have BAAAD intentions because the theoretical limit for a journal's comments is 16,777,216 (2^24). And ONTD just did *that*.
Solution?: Our crack team of crack teamsters is picking through code and releasing the 16 million limit (see here) as well as modifying the databases.
Risk?: We haven't deployed ANY of the new code to the website yet, and we've only started testing the alter statements on one of our secondary databases, but if things continue to test well, there is a SMAAAAALL risk that when we attempt to modify the rest of the databases, LJ could drop bits. All over the floor. But we're really trying to make sure that doesn't happen, and in spite of burr86 and dormando's help (hahaha, got you guys) tupshin, henrylyne, gariev, nicholaskurjan and mhwest will continue to work at getting rabid ONTD fans their due fix. It might be a bumpy weekend."