How to search for great programmers, Part 2

Aline Lerner of Trialpay recently posted statistics on an experiment about resume review. The conclusion was that recruiters, engineers, and just about everyone score resumes all over the place and therefore resumes have weak signal value. The claim was that the strongest signal in the resume was the number of typos. Although the study seems extensive, I think there are a number of weaknesses in the experimental design. One weakness was acknowledged: the ground truth is the author's own subjective evaluation of the candidates. Another weakness was how the survey questions were somewhat misleading in the first place. The questionnaire asks "would you interview this candidate" and yet this was compared with the ground truth of "will the candidate perform well on the job or technical interview". As I alluded to in an earlier post in this series, the role of a resume is to help filter for red flags and to guide the formal interview, not to determine whether a candidate is a star performer by itself. The fact of the matter is, a resume is a self-reported synopsis of a candidate's track record. To evaluate a candidate, I would think track records are important, as is potential.

How to search for great programmers, Part 1

photo by super-structure	via PhotoRee

A running theme these days is that Github is the new resume and a great source for learning. I am skeptical of these claims. If Github is the new resume, then it is one that is potentially even more difficult to evaluate than before and will greatly limit the candidate pool and not necessarily to a high quality one. The bigger question is what role should Github play in the hiring process. After all, many if not most of the famous hero programmers out there (e.g., Brian Kernighan, Herb Sutter, Alex Stepanov) do not have Github profiles. Linus Torvalds does, but he seems to be an exception. A Github profile is also difficult to evaluate. Do we consider only superficial metrics such as commit/contribution frequency or dive deep into a random project? Either of those options do not seem particularly attractive. At most, they will provide an imperfect measure for enthusiasm. It seems to me if code samples is what we are after, then we should ask for code samples. A real resume provides a highly condensed means to evaluate for relevant experience and education that can be evaluated in seconds.