Scaling Pinterest - From 0 to 10s of Billions of Page Views a Month in Two Years another day, another blog post linked by HackerNews.
There are lots of interesting points raised that I think hold a lot of merit. Some, that I’ve trying to better myself about recently which is why this one hit home.
- When you push something to the limit all these technologies fail in their own special way.
- Architecture is doing the right thing when growth can be handled by adding more of the same stuff.
It’s funny how simple those two statements are yet how difficult they can be to follow. Developing on and for Apache Accumulo, I will attest that the fewer moving pieces in any application, the easier it is to get to the bottom of the question “why is this slow??”
But, the most important point to me is this:
- It will fail. Keep it simple.
I’ve been trying hard lately to not get so caught up in the ever-present concern of is this the most optimal solution. It’s easy to sit on a problem, no matter how trivial, and just spin your wheels on it for 2x, 3x, 5x longer than you should. Some would say it’s a deficiency in Java developers (StatelessServerSessionManager and other absurd classnames) but, given TIMTOWTDI from Perl, I don’t think the argument holds much weight. I degress.
My personal goal as of late is just as simple as those 6 words. When you have a problem, think about it enough to make sure you understand the problem, and just do it! Now, I’m not advocating rushing into a problem as that’s going to cause you to shoehorn yourself into a situation where you’re constantly working around terrible initial assumptions, interfaces, etc (no, of course I’ve never done this myself… many times…). Trying to apply it, think of things this way: you can classify reasons you write software into one of two really generic buckets:
- You want to create something new
- You want to make something that was created better
Say you’re writing something new. A calendar. You design some data storage plan, a rudimentary API, and a grid-layout for show calendar events. Cool. You get 70-80% through implementing the API and then you realize; wait. What is more than one person uses this and they’re in different timezones?? Rewind, add in locale information. Keep going. Stop. It would be really nice to build in some native notification hooks. Desktop notifications? Email support? A sweet AJAX-y service that you could auto-embedded in your browser as you navigate and load in via Greasemonkey? Point being, you quickly find yourself completely overwhelmed with “things you have to do” and, ultimately, never ship a damn thing.
I’ll save you the excess of me writing another contrived example, but the same logic trap can catch you when trying to fix a problem. It’s more subtle, typically presenting itself via the word “rewrite”, but the end result is the same. If you lose sight of what you meant to create, you and your user end up empty handed.