Pair Programming for a job interview [closed]

Question

Our company has been thinking about scrapping our interview procedures and bringing each candidate in for a 4-5 hours sit-down with some of the programmers and just do some pair programming.

I like the idea in theory but I am not sure how you can really make it fair for each candidate. How would you rate them? Wouldn't their input really depend on what each programmer was working on that day?

Any thoughts on whether this is a good idea/bad idea or how to make it work is what I am kind of looking for here.

Cheers!

EDIT:

RESULT - AS requested

We are going to conduct the first steps of the interview the same as before. Phone followed by face to face. Instead of bringing them back for a third and final grilling, we are going to bring 3 developers back to sit with all 7 members of the team. We have decided to let the team decide who is then hired.

We have come to this conclusion for a couple of reasons. We believe this will empower the developers by giving them a choice who they are working. The second reason is group dynamic. We think it is really important to have a good group dynamic and it is hard to tell until after you hire a person if they will fit in or not.

So the end result is we are going to go ahead with the pair programming sessions but in a completely different way and for a completely different way than was originally intended.

Any thoughts or criticism of this approach is more than welcome!! (this edit is posted as an answer below so feel free to downvote if you feel this is not the best approach)

Answer 1

I hope you have a bunch of steps ahead of this one. For this to work you need an excellent resume and phone screen. You don't want to spend oodles of time on candidates that you shouldn't be talking to in the first place.

So you suggest an initial interview and possibly have the second interview as the pair programming session? – Ted Smith (1 min ago)

Yeah. You might even think of having a simple coding interview happen over the web using something like CoPilot.

Answer 2

Unless you use pair programming extensively in your real-world development, I'd be very hesitant to use this. I've met any number of high-quality professional developers who have mentioned a strong aversion to pair programming and whose skill would not be well-judged in such a process.

Answer 3

The easiest way is to give each person the same programmer to work with and the exact same piece of code.

The problem you're going to run into, is that hiring isn't like programming. There isn't a step by step process to lead to the right answer as to who to hire. (you can have multiple steps to make the decision easier). You have to evaluate each one on their strengths etc. and essentially make an educated guess as to which is the best one to hire. Sometimes you guess wrong.

The other thing about pair programming you're going to have to watch out for is the amount of time necessary to have each candidate at that stage go through that kind of a test. If I were looking for a job, I would be hesitant to go an interview at a company that would ask me to do that. Why? Because that is a lot of time, and if I am interviewing at multiple places, I could spend literally days just going to interviews for jobs I may not even get or want. Someplaces like Google or MS would be an exception, but most places are not like those two. (Not to mention the fact that if they are working on real code, you are essentially asking them to do someone's job for free).

Answer 4

I just had an interview with a San Francisco based company that prides itself on Agile methods/etc. I was to interview the CEO himself. I have about 20 years of experience in the industry, but have never pair programmed or developed using TDD approach. I was told it would be a "programming interview" but had not idea what to expect, and before we started the guy said that he thought that I may agree that all interviews should be done this way. (which in retrospect was nothing more than an arrogant statement).

Anyway, at the interview the exercise was to develop a class using TDD. It took me a second to adjust my thinking on the entire process, again since I had never pair programmed or done TDD. While I stumbled here and there I did ok in the end. but his reply was the I did not exhibit the aggressive back-and-forth nature that they require for their pair programming environment. Now, that could also have been an underhanded way of saying that "I didn't think you did great" kind of message.

Luckily I didn't need the job and to be honest the experience made me realize that I'd rather find a different career than having to be a software engineer that HAS to work in pairs, day in day out, when it came to developing code. Odd thing is that on occasion I have worked with another person on code simultaneously, so anything is possible.

In end I guess it was a good outcome since they didn't think I was a good fit and I didn't care for their working methods. But we would have came to the same conclusion had I talked a for a few minutes more about myself and had he given me a little more info on how they go about their work. Which is to say that there are other ways of finding a good fit candidate than putting them through the stress of pair programming with a complete stranger; bogus way to gauge competency imo.

Answer 5

As a personal anecdote, I got smacked around in an interview because of a technique like this. I had gone far in their interview process; passed the resume checks, the code submission and this was the face to face portion of the interview.

I was fresh out of university and had never pair programmed before nor done TDD. They sat me down to do a deck of card exercise and it flopped. Badly! I didn't understand why the interviewer was writing tests that seemed so dumb* (IE "return null;") and they didn't explain why and of course being foreign to TDD I didn't know what questions to ask. The end result was that it looked like I couldn't program my way out of a paper bag.

If you're going to do this type of exercise you need to cater to the interviewee because they're going to be in different spots with their aptitude. This means that you'll get different assessments that may not be based on actual talent and are thus going to be heavily biased.

**Now that I understand TDD, I do understand tests like this and how it's supposed to work, but man did that ever seem stupid at the time!*

Answer 6

I just had a pair programming interview a few days ago and to be honest, I don't really like it. I was notified of this a day just before the interview and then the interviewer told me that pair programming is what eventually I am going to do anyway in work. I went into the office and was paired up with someone who is a very senior software engineer. The company is in San Francisco and they are a well renowned company for pair programming, everyone pair programs in the office. At first it seemed to be fine, he explained about all the tools they used, their own unit testing framework that they build, and a bit of the project. He then basically wrote a bunch of unit tests and wanted me to work on the implementation to make it pass. Just as an FYI, the code base that already exists is huge, I would say 10k lines, it's not like a super complex project, but it is complex for someone to just step in and then write code without prior understanding of the class hierarchy etc. I find it really hard to believe that he expects someone to jump right away in a 10k line of source code that already exists. It just doesn't match for a pair programming interview, a smaller code base would help. I struggled a bit from navigating through the classes and going back and forth because I can't remember class names as I was overwhelmed by the amount of classes/code that already exists. To be honest, this really made me do horrible in the interview process. In the end I didn't feel really good about it. I haven't done pair programming before, mostly is just during assignments in my college year.

To me the power of pair programming can be harnessed if you're already proficient/comfortable with your pair, but is not really suitable for interview. Sometimes I would like to ask questions to my pair, but then I thought if I ask too much questions, then they would assume I were stupid and can't perform. If this was already on a real job, I wouldn't hesitate to ask, but in an interview it's hard.. you want to ask because your pair should help you out when you're stuck, but at the same time it's an interview, so you can't really ask much.

That is just my experience that I have from pair programming interview, my suggestion if you really want to do this:

1. be sure that you don't give the candidate to work with a large code base, work with a smaller one and therefore he/she can show his/her skills to the max
2. be up front with the candidate before pair programming interview, can you ask questions when you're stuck, should you be able to do this and that, what can't you do
3. be as detailed as possible

In the end, I wouldn't suggest it. It's hard to measure a candidate's performance in pair programming, and it might be biased as well.

Answer 7

One particular company uses a technique called extreme interviewing. For the extreme interview they will bring in say 30 developers and group them into 15 pairs. They will explain that they are looking for people who work well with others. That they will make a hiring decision based solely on their ability to work with others.

They will provide a problem for the pairs to solve. They will emphasis that they are not interested in the solution just each programmers ability to work with others. For each pair they will provide an observer of the pair. During the exercise (about 2 to 4 hours in duration), the observer will takes notes about a person ability to pair ... not the solution.

They are amazed how many programmers focus on solving the problem instead of collaborating. Of the 15 pairs, they will identify about 4 to 6 developers for a second interview. Those developers will be asked to come back and spend a week with the team (they get paid). After a week, they decide who to keep. Generally about half of them (2 to 3 developers).

When they are done, they have developers that are able to collaborate and after a week working with various pairs, the team has a strong indication who can effectively develop software. The process is both innovative and effective. They have had a high success rate with those they have hired.

Answer 8

I like this idea. However I think it might be difficult to do since it would require the candidate to have some knowledge of the project you would pair on with him. Also, 4 to 5 hours seems a bit long. What if you immediately see that it is not going to work out, are you going to sit through the whole session with the candidate?

Good question though. Stuff to think about.

Answer 9

Why not? Also, it's not like interviews are always (or ever) fair. You should evaluate the end results of the new approach against the traditional interview-based approach.

Also, a mini interview before the pair programming session might be good to keep from wasting the programmers' time with people who would be a bad fit.

Answer 10

From my limited experience, my feelings are mixed. I like the idea of pairing as part of an interview, esp. if the company uses pairing often, because it gives both a better feel for the fit. As a candidate, I've often gone through interviews where I sat in a room answering questions for a few hours, but afterward didn't have a good feel for what it would really be like to work in their environment. Pairing may be more beneficial than a random coding exercise, unless the interviewer is skilled at working someone through those. And I like being able to discuss technical stuff from both sides. And as a candidate, I'd rather interact with someone than just answer questions or solve code problems on my own.

But... as others have noted, the time needed can be an issue. I've gone through a couple days of pairing interviews and found some periods good, while others felt like a few hours were wasted: one because the developer wasn't working on something that lent itself to pairing (esp. given my background), the other because an env issue prevented much useful work for a while. If the job doesn't work out, it can be frustrating to have taken a day or two off work for this.

One place trying this approach wasn't sure if they should have someone outside the company working on a customer's project. They also worried that explaining the domain and work being done would take too long, though without that the candidate may not be able to contribute much. So they chose an open source project the employee was working on.

This seems to be a key point: there needs to be a well chosen task that the candidate can understand quickly and be able to contribute to. The latter part will depend somewhat on the candidate's skills. Also key would be the employee's ability to evaluate someone with this approach. Not everyone is great at normal interviewing, and that's probably more true of a pairing interview.

Also, if a company doesn't do much pairing then this kind of interview may not be as useful. There does seem benefit in seeing someone code (as Joel Spolsky notes), and this could be a good way to do that. But if pairing is not a typical part of the job, then perhaps a full pairing session isn't appropriate. Maybe a modified version.

I'd be curious what companies who have taken this approach think of the results. Reading some of the other answers to this question shows that it doesn't always seem ideal from the candidate's view.

Answer 11

To keep it fair, you'd have to make every participating staff member have a prepared problem to evaluate the candidate on. Preferably something taken form the real world in their company experience, but something that has already been resolved. This is a good chance to evaluate the knowledge on a problem and evaluate not just programming skills.

I hate it when too specific questions are answered. I had an interview once where a programmer was testing my knowledge of the STL which I used extensively and was trying to get me to answer that a custom allocator was needed. I had heard of them but never used them (esp in windows) and was made to feel dumb. IOW, avoid being judgmental.

So my point is, ask practical questions that aren't so much about testing programming knowledge as you can evaluate more qualitative personality and problem-solving approaches if you use the "pair programming" idea.

Good question!

Answer 12

Honestly, that sounds like a great idea, though Jason Punyon is certainly right that you should do a lot of weeding before you waste significant amounts of your developers' time on culls. You get a glimpse at an important metric out of it that's otherwise nearly unobtainable in interviewing: what someone's like to work with.

I don't think there's really any need to be concerned about it being "fair" based on the subject matter or trying to present consistent situations to different candidates, if you maintain the right evaluatory attitude -- that it isn't about whether they "got the right answer" or jumped through the right set of hoops, but what sort of effort, problem-solving, communication aptitude and flexibility they showed. You'd lose most of the benefit of the exercise by turning it into an artificial test, not to mention changing it from something that your developers can get some benefit from (or at least still get some work done during) to a massive waste of their time.

Answer 13

Joel Spolsky has an excellent Guerrilla Guide to Interviewing which talks about, amongst other things, programming tasks.

Trivia: Joel Spolsky is a co-founder of stackoverflow.com