![[info]](http://l-stat.livejournal.com/img/userinfo.gif?v=92.1){kind=small}
![[info]](http://l-stat.livejournal.com/img/community.gif?v=92.1){kind=small}
http://www.danga.com/words/2004_mysqlco
Building a distributed filesystem for Fotobilder/LiveJournal (will be open source):
http://www.livejournal.com/~brad/200988
http://www.livejournal.com/~brad/201053
http://www.livejournal.com/~brad/201099
We just bought 2 machines with 16 250GB disks, so we'll soon have 8TB of storage. I imagine we'll get about 6TB of real storage out of that after redundancy. (thumbnails and scaled versions will only be on disk once, probably, since they can be recreated easily....)
Building a new load balancer for FotoBilder/LiveJournal, with special support for mixing efficient buffer of mod_perl requests and for efficiently serving large files (using sendfile(2)) from disk, so mod_perl doesn't have to do it:
http://www.livejournal.com/~brad/200794
The proxy works already w/ FotoBilder. Haven't put it into production yet, but we rebooted all our LiveJournal proxies into Debian testing w/ epoll.h headers so we could build IO::Epoll (which is a requirement for Perlbal). They were already running Linux 2.6 (for epoll)
April 27 2004, 16:42:03 UTC 8 years ago
April 27 2004, 17:08:47 UTC 8 years ago
April 28 2004, 00:29:49 UTC 8 years ago
One experience which prompted the change was a Penguin Computer database machine which is, will be or already was on it's second (or is it third?) trip back to them, this time for a refund. Failed to stay up for more than a few days. Failed memory tests after the first trip there and back.
Have been some issues with the SM machines over the last week, two Squid cache machines going down, but both look to be software issues.
I'm not the person who decides where the orders go but SM looks like the ongoing primary choice for a site which wants reliability out of the box (but tests anyway, of course).
April 28 2004, 23:07:31 UTC 8 years ago
April 27 2004, 18:07:57 UTC 8 years ago
Presentation
That presentation is really great, I work with a php/mysql DB every day, and we've had an explosion in traffic and users in the past year that's caused us to evolve and proceed much the way LJ did, it was really interesting to see some of the same thought processes played out elsewhere :) That and the sense of humor used in the presentation is very similar to our company culture. A company of 12+ people, all working mainly on the web (sound familiar?)Thanks for sharing!
April 27 2004, 18:39:22 UTC 8 years ago
April 28 2004, 23:06:05 UTC 8 years ago
April 28 2004, 09:28:57 UTC 8 years ago
P.S. Well, it might be a good idea to buy a SAN box for such amount of data. It's quite expensive but it's worth it.