How to search for great programmers, Part 1

photo by super-structure	via PhotoRee

A running theme these days is that Github is the new resume and a great source for learning. I am skeptical of these claims. If Github is the new resume, then it is one that is potentially even more difficult to evaluate than before and will greatly limit the candidate pool and not necessarily to a high quality one. The bigger question is what role should Github play in the hiring process. After all, many if not most of the famous hero programmers out there (e.g., Brian Kernighan, Herb Sutter, Alex Stepanov) do not have Github profiles. Linus Torvalds does, but he seems to be an exception. A Github profile is also difficult to evaluate. Do we consider only superficial metrics such as commit/contribution frequency or dive deep into a random project? Either of those options do not seem particularly attractive. At most, they will provide an imperfect measure for enthusiasm. It seems to me if code samples is what we are after, then we should ask for code samples. A real resume provides a highly condensed means to evaluate for relevant experience and education that can be evaluated in seconds.

Who really drives high quality open source?

The current Cambrian explosion of software we are witnessing today is due in no small part to a number of great open source projects. Open source software has the lion's share of web browsers (Webkit, Firefox, and Chrome), web servers (Apache httpd and nginx), mobile and server OSses (Android and Linux), web frameworks (too many to summarize), distributed computing and deployment frameworks (Hadoop, Spark, Mesos), and databases/distributed data stores (PostgreSQL, MariaDB, Cassandra, Riak, Mongo). By "great" here, I mean a combination of widespread use and high quality as reflected (imperfectly) in terms of defect density. But what really drove those projects to the great success and influence we are witnessing now? Just as there is the rags to riches mythos, so there is the mythos of Joe Everybody contributing to open source. Consider an analogy with replication and research. In the research community, there is also the ideal that research results can be validated by replication of experimental results by others. However, in research, the incentives do not align to promote such replication activities since an academic cannot stake tenure on replication. I wonder if there is a similar gap between ideal and practice in open source organization and contribution. Another related question I would like to answer is what and how can one learn high quality software engineering practices from top open source projects.

How long does it take to compile a web browser, Part 1

In today's brave new world, people can build and deploy a web app very quickly. However, on the other side of the equation, systems software such as web browsers are not getting any simpler. The cost of the rich media experience in modern web apps quite frequently borne by systems software including browsers, servers, and plugins. Just eyeballing, in Chrome (default multi-process mode), each tab take 30-100 MB of RAM. Luckily, I am not one to keep many tabs open concurrently, but many, many people do. A rich web experience can quickly gobble up whatever amount of memory you can throw at it. There are many people who have compared memory usage for Chrome, Firefox, Safari, and Internet Explorer (TechPP, CNet, Tom's Hardware, DotNetPerls, and many more I am sure), but, here, I would like to take a look at another facet of web browser complexity: source code size, time to build, and size of debug executable. This post follows and expands upon the type of study I did on open source finance software.

Top 5 Open Source Quant Finance Projects

In the recent months, I have found the data on open source development projects to be quite fascinating. It turns out that most of the activity in open source quant finance code happens on Sourceforge. I pulled some of that data and charted them.