Every thing you need to know about a Programmer, you can learn from having them explain their own code to you.

I've been doing interviewing and hiring at our technology startup for almost a year now. When we began hiring, I found that I was basically the only person who would do it -- partly because I was the only person who displayed any sort of natural aptitude at judging another Engineer's technical and personal abilities.

It didn't take me long to create my own system which I thought was both fair to the interviewee (whom I empathized with, having been one for so long), and gave me a decent understanding of the candidate's qualities. With a little bit of intuition, and some trial-and-error, I evolved my system.

As the people I had interviewed and hired began to join me in the interviewing process, I got to notice how other people handled going from interviewee to interviewers, and how their outlook on a candidate's performance differed from mine.

This evolutionary process has lead me to solidify the following as my Golden Laws (tm) of hiring. Most people who are experienced with interviewing will not be surprised I think, but when a new person joins our team, I find that I need to explain these rules each time anew, so I thought I'd write them down.

1. Hiring is the single most important thing a company can do because it directly affects all other things that are done thereafter.

2. Hire only people you completely trust to fulfil a given role. Then completely trust those people.

3. Your job as an interviewer is NOT to hire "good" people -- that's an undecidable problem. Your job as interviewer is to REJECT substandard people.

4. The most critical personality trait to look for in a candidate is Honesty. All other important properties flow from that font. Dishonesty in an interview is the Kiss-of-Death.

5. Every thing you need to know about an Programmer, you can learn from having them explain their own code to you.

(And lastly, the Green Interviewer's Trap: Hiring is not a popularity contest. Most interviewees are a nice people.You are not required to hire nice people just because they're nice.)

The last law is the title of this entry because its perhaps the most controversial item. Yet I hold it to be the most essential truth I have discovered about hiring Programmers. So much so, I am going to let you in to my entire technical review process, and how it is centred around item 5:

The Sample

After reviewing the candidate's resume, and mentally making note of potential strengths and weaknesses, I always request a code sample from the candidate, in the following order of preference:

1. Open Source or otherwise Publicly Available Source

Open source is the best way to possibly find out how good a programmer is. A public SCM server has all history and contribution information regarding that project. You can graph a programmer's entire history with that code base to the most minute detail.

Mailing lists have discussions of technical matters, and help show how well the candidate gets along with other people.

Open Source is usually a work of passion, and outside the corrupting influence of bad company management, so you have already factored out the naked element of talent.

2. Redacted Source code taken from a company project (without violating any applicable laws, be careful)

Proprietary (but redacted in order to comply with confidentiality laws) source code is good because it shows what the candidate produces when given a real-world problem in real-world circumstances. Particularly it shows if they tend cut corners to meet deadlines or other practical constraints.

3. School Coursework (if a semi-recent graduate)

School coursework, if not too dated, is good because it shows what the candidate can do when asked to really apply their intellect to a serious problem. It's also an opportunity for the candidate to explain how his coding style has evolved over time, and why he would make different decisions today.

4. Toy Project selected by Candidate

Candidate-selected toy project is good because it shows what interest the candidate has if someone is not over their shoulder telling them what to do.

Do they choose something just difficult enough to meet your requirements? Do they choose something way too hard to complete in a reasonable amount of time? Do they report enjoying the exercise?

5. Toy Project selected by Me

One reason why I neglect this option to the last is that one must be respectful that candidates often are busy people, and they can and will turn down the interview because they feel they have better things to do than jump through your recruiting hoops.

6. Source written by Me

This is fun for showing off your work, and is realistic in the sense that programmers are routinely asked to do exactly that, however its rarely under such pressure situations. It's too easy to have a flustered programmer get thrown off by some minor detail, and thus you end up with incomplete reading of their true ability.

Either way, it is absolutely critical that the code is either hand-written by the candidate (or in cases where external code sources were used, clearly attributed as belonging to someone else), because we are going to ask them to explain its working to us. Clear attribution is critical because it is an unalterable part of our standard of honesty.

The Interview Prep

Once you have a sample of their code, you can of course review the code before the Interview to look at issues of Style, Correctness, Performance, Good Design. This is important. However the real purpose of reviewing the code is to generate questions for the candidate once he arrives to the interview. Ask yourself, "why did this person choose to make these decisions when writing the code?"

The Interview

When the Candidate arrives, I open the interview with a stupid joke or sarcasm. Partly because I often am as nervous as the candidate, and I want to break the ice. I also tell him that there are no "Right" answers in this interview, and no trick questions. It is important for the dignity of both parties to keep the interview cooperative, and to make it understood that you are both on the same side -- after you hire the candidate, you certainly will be counting on their cooperation and good will then!

Next I have a short list of simple, no-nonsense, general questions that I ask every candidate. For example "We are in the technology field, and as we both know, technology changes at a staggering rate. What methods do you use to keep on top of that change? Please be specific."

Next I have the list of questions generated from their sample code with me, and I move across the table to sit beside to the candidate with my laptop. I open their source code opened in a text editor (their preferred on if possible).

I ask the candidate to pretend that I am a Junior Engineer, and he must explain his code to me. As a curious person, I will stop them at various points to clarify the concerns I compiled during the Interview Prep, and put to rest any questions that happen to come into my mind as they are speaking.

What an observer can gather from this simple exercise is essentially everything you'll ever need to know about that person as a Programmer.

- A person who remembers even very old code is a person that was paying close attention when they wrote it, and understood the nuance of every line.

People who have trouble recalling what they wrote either didn't write it, or never understood it clearly when the wrote it.

- A person who speaks confidently and at a decent pace is a person at ease with their code and the decisions that code embodies. Even if the code is bad, a good programmer can justify their compromises in a rational manner.

People who speak in broad generalities, ramble incoherently, or carry off on tangents are more than just disorganized. They are trying to distract you from the fact that they don't know what they are talking about by burrying you in an avalache of verbage.

- A person who identifies their own mistakes and errors is more than just aware of their own mistakes, they are showing an honesty and desire to learn and improve. Similarly, people admit their don't know or understand some facet of the system.

A person who steadfastly insists on issues where you know him to be mistaken are not capable working with others, and should be rejected immediately.

- A person that can turn a complicated algorithm or technical trick into something easily understood by any simple engineer has not just a gift for teaching, but likely a deep and intuitive understanding of domain.

People who cannot relate technical matter without speaking entirely above your level may be brilliant, but could prove problematic if they have to work with others.

That said, even simple body language can tell you a lot about a candidate. One of the most dependable Engineers I've ever worked with failed horribly in trying to understand some source code I had written (possibly because I hadn't written it nearly as well as I thought I had!). Every time I asked him what a certain peice of code was doing, he politely answered "I don't know". What impressed me was the way in which he studied my code so closely, determined to fathom out my design through force of will. To this day his tenacity to reason through a problem completely is the hallmark of his good work with our company.

The wind-down

If I have come to a negative conclusion about the candidate, and decided not to continue, I ask the if he has any questions, then thank the person for their time, and ask them to leave.

If I have finished my walk-through on a positive note, I wrap the interview up with a brief explanation of my company, then invite the candidate to ask his own questions of us in a reversal of the interview process.

A complete and incisive set of questions for me to answer is a sign of a good candidate.

If I conclude that a person is technically competent, the next round of interviews include various members of our team, and are mostly about making sure that personalities match well, and that company culture is maintained.

If any one interviewer registers a "refuse" vote, then the candidate is refused.