The 'better' an open source project is technically, the more people use it. That's what I used to think. But lately I have gotten doubts: Why are some projects more successful than others? Are there technical reasons or is it something else? How do some get 10000 Github followers and others languish in obscurity? I was hoping there would be a nice, clear answer. I think there is - unfortunately not the one I was hoping for.

But why should we care at all how many people use a certain framework or library?

  • A project with more users will receive more bug reports from its users. Some of those users will contribute even more to the project, so it will get more patches and feature enhancements. It will be more battle-tested and grow faster.
  • More users, more tutorials, more blog posts, more Stackoverflow questions means the learning experience will be better.
  • The potential for the project creators, contributors and users to make a living off of it will increase. Let's take web application frameworks: How many jobs do you know that require "Pyramid" expertise? How many jobs are "Django" related? On the Python Job Board I count 6 mentions of Pyramid. For Django, I stopped counting at 30+ halfway down the page.

But in any one area, there are only so many developers available. Yes, the world is big enough for multiple projects in the same space, but in the end, software projects compete for developer attention. A few projects will dominate and have the biggest user base. These projects "win".

Let's take a closer look and compare the situation for some software libraries. Of course, if an open source project wants to be a serious contender, it should offer:

  • an okay feature-set
  • in a package that is not totally buggy
  • with a minimum of documentation (the days of "just read the source" are over.)
  • and a reasonable license (something like AGPL is an instant no-go for lots of people.)

Let's view these as necessary requirements (in the mathematical sense), but not sufficient ones.

Also, in this whole post I'm talking about software libraries - stuff meant to be used by other developers - but I'm excluding programming languages. The popularity of languages is strongly influenced by other, major factors. Take JavaScript, which rises steadily mostly because of a single fact: It is present in all browsers. Circumstances like that make it hard to draw useful general conclusions. Therefore, let's not talk about languages.

So, what makes a software project successful?

1) The one which is easier to use wins

For instance, Django is easier to use out of the box than Pyramid: It has a nice admin interface, the model and form validation libraries are already included and everything fits together nicely. jQuery supports the theory, too. Most will agree that it is more beginner-friendly than, say, Dojo or YUI.

But the theory does not hold, with two rather big counter-examples: First, Mercurial and Git. Some might find Git reasonably easy to use, but the majority of people that have used both find Mercurial easier to deal with. And yet, it didn't win the race against Git. Then there is Backbone and its competitors Knockout and Angular. Backbone is significantly more popular (just compare their Github followers or look at the projects that use Backbone) even though it's harder to use. Some might disagree with that last claim, so let's look at hard numbers: The poster application for JS frameworks - the ToDo app - requires way more code in Backbone than Angular or Knockout. And a personal data point: For a recent project I had to rewrite an old application and replaced Backbone with Angular. The JavaScript code size dropped from 2.5K lines of code to around 600. A large part of that can be attributed to the change in JS libraries.

Another theory then...

2) The faster runtime performance wins

We all know that Git has pretty much won the popularity contest against Mercurial. And it is somewhat faster than the latter, so that fits (quick googling gives: http://draketo.de/proj/hg-vs-git-server/test-results.html, http://xrunhprof.wordpress.com/2011/04/11/mercurial-vs-git-performance/, http://laserjock.wordpress.com/2008/05/09/bzr-git-and-hg-performance-on-the-linux-tree/ ).

People also like to say that Backbone is more lightweight and faster than the likes of Knockout and Ember. Jeremy Ashkenas, Backbone's creator, posted a benchmark to Hacker News showing how much faster it is than Ember (http://news.ycombinator.com/item?id=3616820). However, the benchmark was somewhat artificial, and you might notice a link between who created the benchmark and which framework is shown in a favorable light.

But still, the public perception is that Backbone has a faster runtime performance and it lets you tweak and optimize things better. So that's why it is more popular than Ember, Angular and so on, right?

But wait, what about Rails? For a long time, everybody's favorite Ruby web framework had a reputation of being slow. I am too lazy to come up with examples, but if you don't believe me, just google 'Rails slow' or 'Rails doesn't scale'. Here is a small benchmark: http://blog.curiasolutions.com/the-great-web-framework-shootout/. And yet Rails got hugely popular and it single-handedly catapulted Ruby into the group of mainstream languages. Or what about Pylons/Pyramid compared to Django? The benchmark above also shows Pyramid as faster than Django. Yet, Django's community is probably an order of magnitude larger than Pyramid's.

So this theory holds neither. What about the next one?

3) The more "hacker-like" one wins.

I don't have an exact definition of "hacker-like", so let me try to describe it: The software that let's you fiddle and tweak more. The software that is closer to the metal and uses less high-level magic. The software that might require you to write more low-level commands, but in turn, you know exactly what's happening.

Again we could come up with Git and Backbone in support of the theory. But again, the counter examples are similar: Rails and Django are both more monolithic and their structure is more rigid than Pylons/Pyramid. If you have special requirements, hacking and adapting a Pyramid project would be easier than bending Django into the shape you need. Yet, Rails and Django are far bigger.

You could also argue that YUI stays closer to "normal" Javascript as jQuery, which is almost like its own little mini-language on top of Javascript. Rumour has it that some fledgling web developers even skip the whole Javascript part and only learn to use jQuery. So jQuery is more removed from Javascript and more high-level, alright. That hasn't stopped it from getting hugely popular at all.

Getting impatient yet? Wondering when the heck I'll come to the point? Okay, here it is.

4) The one created earlier wins.

"Are you kidding me?" you're probably thinking right now. "So you're saying all that technical stuff doesn't matter? Ease-of-use? Runtime performance? Hackability?" Well, let's look at some facts:

  • Rails basically started the trend of modern MVC-frameworks in 2004. When it arrived, there was no other framework like it and it took the throne of most popular web framework by storm.
  • Git was released before Mercurial.
  • Backbone was the first of the new breed of Javascript frameworks. Angular, Knockout, Ember, Batman, you name it - they all came later.
  • Within the Python world, Django came first in July 2005, Turbogears second and Pylons third. Django has kept its lead in popularity ever since.
  • Twitter Bootstrap arrived on the scene one month before Zurb's Foundation. At this very moment, it has 48.800 followers on Github versus the 10.800 for the latter. To be fair, it seems that Bootstrap got inspired heavily by ZURB's work done before (http://news.ycombinator.com/item?id=4588826), but from a public release perspective, it came first.

Ah, but wait! If you followed the timelines closely back then, you might know that Git and Mercurial almost arrived at the same time. If we believe Wikipedia, then Git was released April 7th in 2005 whereas Mercurial's initial version was published April 19th. So 12 days are enough of a head start to lead to a huge success like Git's?

And what about YUI? Depending on what you count as public release date of jQuery, YUI even came before jQuery (Wikipedia claims John Resig presented jQuery in January 2006 at a BarCamp. It also says the initial release was August 26 in the same year. YUI came February 2006). The jQuery/YUI and Git/Mercurial timelines are not exactly supporting evidence for this theory.

I lied. "The earlier one wins" is not the point I was going to make. Yes, the release date is an important factor, but it is not the deciding one. However, it helps immensely.

So what is it then?

5) The one with more buzz wins.

Plain and simple. Call it PR, call it momentum, but the core message stays the same. Git didn't win the race because of its 12-day lead. In the beginning, it got more users because the name "Linus Torvalds" was behind it. Later, it got even more popular because of Github. Mercurial on the other hand - well, it didn't have its own Github. When Bitbucket arrived on the scene, it was already too late and the social pressure to put your project on Github was too high.

In the same vein, did you know that Tom Dale + Yehuda Katz < half-a-Jeremy? Wait, what? Tom and Yehuda are the creators of Ember, Jeremy is the head behind Backbone. If you add "Team Ember's" karma on Hacker News you don't even get half of Jeremy's. In other words, Jeremy is way more active on HN. I can hardly remember a discussion about Backbone where he didn't pop in and provide positive commentary. Probably he's just very passionate about his framework, but his words definitely help to spread the Backbone enthusiasm. What about the Angular team? I am not sure if they even have usernames on HN.

I could go an about all of the examples mentioned above and every single one of them fits this theory. I'll try to keep it short:

  • Rails? Pushed by DHH.
  • Django? Way better marketing than Pylons back then. Django's tagline: "The Web framework for perfectonists with deadlines". How smooth is that?!
  • Pyramid on the other hand? It doesn't even have its own domain but is hidden within the Pylons Project. (Sorry for that. I very much like Pyramid and it was the main choice for my side project. But in the PR department, it's lacking.)

jQuery and YUI? The former had John Resig constantly promoting it, whereas YUI was from a relatively unknown team within Yahoo!. And Yahoo never had the cool reputation that would have helped. Twitter's Bootstrap? The association with the name "Twitter" probably would have been enough to make it big - Twitter is young and hip after all (or used to be, back when Bootstrap was released). But there's more. Just look at Bootstrap: The project says it is "Designed and built with all the love in the world". There is something about this phrase that really annoys me, but it delivers a clear message: "We are young, feel-good programmers and not your standard corporate drones." They avoid semicolons in their javascript code, because that's how the cool kids write javascript these days - score another point for the young and hip image. One of the creators even has his blog on an xxx-domain. By now, you should realize one thing: They are really good at marketing and presenting themselves as the cool guys. Which, I'd argue, has helped their project's success immensely. (Just to make this clear: While I'm not a fan of being bludgeoned over the head with the oh-so-hip attitude, the Bootstrap guys have built something impressive. I have used Bootstrap for multiple projects and it was a huge time saver. Simply blank out the puddle of hipsterness they make you wade through.)

To be honest, this conclusion makes me somewhat sad. I would have preferred if the reasons for projects growing bigger and more successful were more technical. Alas, it seems that code plays the smaller role. The buzz, the presence in the blogosphere, the activity on HackerNews, the coolness factor seem to count more.

Summary / tl;dr

If you create or maintain an open source project and want to make it big: Generate buzz!

P.S.

On a more positive note: Instead of playing up the coolness factor of your project, you and your community should write good tutorials, blog posts, showcases and pump out regular releases. That way, the momentum you generate is not based on buzz for buzz's sake but has actual meaning. That's the right kind of buzz.





Further reading: