Twitter's scaling issues

I saw this article on Twitter's scaling problems earlier on my mobile and immediately (of course) Twittered it. The gist is this: Twitter's ever-more-frequent downtime has to do with the fact that early on, they greatly underestimated the complex issue of one-to-one and many-to-one connections. "The issue is that group messaging is very difficult to achieve at a grand scale," Nik Cubrilovic points out, and "Twitter is unique in that it needs to parse a large number of messages and deliver them to multiple recipients, with each user having unique connections to other users." Basically, Twitter is doing something that has never been done before at this scale. "Every new Twitter user and every new connection results in an exponentially greater computational requirement."

Cubrilovic goes on to say:

Some of the best web applications are able to efficiently solve very complex problems to produce simple results for users (Eg. Google). The success of these applications is due to the innovative efforts by developers to solve large technical challenges, where they have often had to break new ground for solutions. For Twitter to reach a similar point of reliability they too will need a very comprehensive, ground-breaking solution.

Okay, let's stop there. At about the same time Twitter came online, Jaiku was unveiled by two former researchers from Nokia. Like Twitter, Jaiku was conceived from the ground up to work with mobile phones through SMS. Like FriendFeed, Jaiku pulled in activities from other services and called them "activity streams." And like JotSpot, Jaiku has been acquired by Google and has entered what looks to be a long period of reengineering, rescaling, and rebranding similar to JotSpot's. (JotSpot recently reemerged as Google Pages.)

I wonder if Google is even now working on the same scaling issues that are plaguing Twitter. They certainly have the expertise and the hardware to scale. I just hope it doesn't disappear into the Google maw the way Dodgeball did.

(n.b., Jaiku was founded by Jyri Engeström and Petteri Koponen. Jyri Engeström is the son of Yrjo Engeström, a name familiar to anyone with a passing interest in activity theory.)

(n.b., I have not bothered to discuss services such as FriendFeed and Pownce as competitors because they do not update via SMS. They do what they do decently, but SMS is a must for this space.)

Twitter At Scale: Will It Work?

Blogged with the Flock Browser