by Felix Francis
Twitter’s growing pains:
[Via Buzzworthy]

It’s hardly news that Twitter is experiencing growing pains, but a couple of items have appeared in recent days that shed some new light on just how bad they’re getting.
[More]

As mentioned below, some of the problems Twitter is having while trying to scale are rooted in its basic communication paradigm. It is much more complex than a system based on the telephone company. It is almost as if every phone call was a 5 or 6 person conference call.

Difficult to do with the best experts. But it sounds like Twitter was somewhat surprised by the direction its technology took and was not prepared for the type of growth it sustained. It is still a very small company and one that may not have had onboard all the engineering help it needed.

Twitter is, fundamentally, a messaging system. Twitter was not architected as a messaging system, however. For expediency’s sake, Twitter was built with technologies and practices that are more appropriate to a content management system. Over the last year and a half we’ve tried to make our system behave like a messaging system as much as possible, but that’s introduced a great deal of complexity and unpredictability. When we’re in crisis mode, adding more instrumentation to help us navigate the web of interdependencies in our current architecture is often our primary recourse. This is, clearly, not optimal.

Twitter broke ground on a new manner of using Web 2.0 tools. Time will tell if it is able to maintain its initial success.There are some very difficult problems that have to be solved. But there will be somebody who solves the scaling problem because this tool is just too useful.

Google was not the first search engine., just the best one so far. Web 2.0 works by allowing rapid prototyping of new tools as one works towards perfection. Twitter was able to accomplish a lot with really very little. It has hit a barrier now. It will be interesting to see how this problem get solved. It is not too unlikely that a user who is really knowledgeable will propose a solution.

We have kept an eye on the public discussions about what our architecture should be. Our favorite post from the community is by someone who’s actually tried to build a service similar to Twitter. Many of the best practices in scalability are inapplicable to the peculiar problem space of social messaging. Many off-the-shelf technologies that seem like intuitive fits do not, on closer inspection, meet our needs. We appreciate the creativity that the technical community has offered up in thinking about our issues, but our issues won’t be resolved in an afternoon’s blogging.

We’d like people to know that we’re motivated by the community discussion around our architecture. We’re immersed in ideas about improving our system, and we have a clear direction forward that takes into account many of the bright suggestions that have emerged from the community.

To those taking the time to blog about our architecture, I encourage you to check out our jobs page. If you want to make Twitter better, there’s no more direct way than getting involved in our engineering efforts. We love kicking around ideas, but code speaks louder than words.

That would be the Web 2.0 way.

Technorati Tags: Knowledge Creation, Social media, Web 2.0