Who really drives high quality open source?

The current Cambrian explosion of software we are witnessing today is due in no small part to a number of great open source projects. Open source software has the lion's share of web browsers (Webkit, Firefox, and Chrome), web servers (Apache httpd and nginx), mobile and server OSses (Android and Linux), web frameworks (too many to summarize), distributed computing and deployment frameworks (Hadoop, Spark, Mesos), and databases/distributed data stores (PostgreSQL, MariaDB, Cassandra, Riak, Mongo). By "great" here, I mean a combination of widespread use and high quality as reflected (imperfectly) in terms of defect density. But what really drove those projects to the great success and influence we are witnessing now? Just as there is the rags to riches mythos, so there is the mythos of Joe Everybody contributing to open source. Consider an analogy with replication and research. In the research community, there is also the ideal that research results can be validated by replication of experimental results by others. However, in research, the incentives do not align to promote such replication activities since an academic cannot stake tenure on replication. I wonder if there is a similar gap between ideal and practice in open source organization and contribution. Another related question I would like to answer is what and how can one learn high quality software engineering practices from top open source projects.

What is New in Android Lollipop

It is exciting times with the new Android Lollipop (Android 5.0) release hitting a late model Android handset near you. It is apparently already available on Motorola Droid X and G and slated for Nexus phones on November 12th. There is a lot of interesting features in this release, from the switching on of the Android Runtime (ART) which enables ahead-of-time compilation of Android apps (instead of the more traditional Dalvik just-in-time compilation) to a new garbage collector.

ART was introduced in KitKat (Android 4.4) but it is now the default in Lollipop.

The Real Power Behind the Curtain (WhatsApp Edition)?

By now, you probably heard all about Facebook's $16B acquisition of WhatsApp. To put that in perspective, that is more than the total GDP of hundreds countries such as the likes of Iceland and ~30% the market cap of GM or ~10% that of Facebook itself. Consider how it was just a few years ago when Oracle bought the former behemoth Sun Microsystems (whose former campus is now Facebook's) for $7.4B in 2010. An interesting tidbit is behind the curtain is a modern, low-latency messaging backend powered by Erlang, a functional language optimized for concurrency and availability, which originally gained a following in the telecom community, especially from the likes of Ericsson. This is what ultimately powers the slick, reliable user experience of the messaging system that serves hundreds of millions around the world. Rick Reed, one of the engineers at Whatsapp, gave a talk on the use of Erlang at Whatsapp. One specialty of Erlang, which is also used by Facebook itself, is hot sliding, a kind of on-the-fly code module loading to enabling updating a system and pushing code without ever taking the system down.

Experimental Human Factors Take on Functional Programming

As a functional programmer, one often takes the productivity advantages of functional programming as an article of faith: Versus imperative and object-oriented counterparts, functional programs must be shorter, more robust, quicker to develop, and easier to maintain. Designers of programming languages and compilers harp on the supposed benefits of their languages and implementations. But when it comes down to it, where is the evidence? There is admittedly a lot of problems when forming a rigorous question and experiment to compare languages. An empirical study is even more difficult because there are really few large software systems that are implemented equivalently in multiple languages. Still, there turns out to be a few studies on the productivity and usability side of functional languages and mixed paradigm languages (e.g., Pankratius et al's "On the Benefits of Combining Functional and Imperative Programming for Multicore Software"). The results appear to be quite consistent: functional programs are shorter but they aren't any quicker to develop for even relatively skilled programmers. Performance is generally found to be on par. Moreover, fancy type systems have the disadvantage in that they could complicate the programmer's understanding of the program and prolong the debugging process.