I came upon an interesting post by Leonard Lin, a developer and (according to Linked In) former Yahoo!-er. It’s called Internet Asshattery, Armchair Scaling Experts Edition, and talks about issues with scaling a rapidly expanding application infrastructure.
Among the things I like about the article (including the word Asshattery, which is one of my favorite words, ever) is how he talks honestly and from experience the issues dealing with a rapidly growing site. You’ve got all sorts of constraints in terms of application, database, network, CPU, disk I/O etc., and like a gangly teenager, you hit different growth limitations at different and unexpected times.
I’ve also faced the challenges he describes, although from a slightly difference perspective, and I know exactly what he’s talking about. Things will come up that you can’t plan for, and there will be issues that you can plan for, but choose not to because of resource and time constraints.
Sometimes you’ve got to say “Yeah, this will probably bite me in the ass later, but what’s more important are the four things that are biting my ass right now.”
