Monday, October 17, 2005

What does you Recruitment Process say about you?

Having recently moved jobs, I have been generally unimpressed with the recruitment processes I have encountered. My particular bug-bear is the "technical assessment" questions and my second favorite "the personality profile".

From an eXtreme point of view I fully support the "test first" approach but as ever you must be careful what you are testing for. A lot of the technical tests take a long time and are extremely detailed and demand deep knowledge of the syntax and foibles of particular systems. But is that what you want to know about a candidate? Especially if they are going to be working as a consultant ? This is testing knowledge not application, and it is application which counts. I suppose an analogy is technical tests are like coverage tests, not functional tests: and to the eXtreme mind functional rule!

The "personality profile" supposedly addresses these short comings but again I fear they are testing the wrong thing. These are analogous to architectural reviews of a system -they inform you about the general shape of things, how you believe things hang together and you decide whether you believe its fit-for-purpose. But at no time is there any kind of measure (and I use that accurately, i.e. not a metric) of fit-for-purpose.

So what do I believe is the right way of doing this interview process? What do I see as the recruitment equivalent of functional testing?

Well as a test I might write it as, "I want a candidate who can understand a users needs, use development tools, discuss implementations, do test first, etc. so that they can deliver working systems in the context of our usual working environment".

And how do you implement this test? Pair Programming.

We used this technique very effectively for years at Connextra. The interview process goes like this:
  • Sift through CVs for reasonable candidates (I should blog on how to advertise too...)
  • Brief telephone interview to test whether they are genuine and have the basic skills necessary for the role. This could include a basic technical and syntax test, but should be brief and take no more than 10 minutes and able to be asked by anyone (i.e. not necessarily requiring technical knowledge)
  • If they pass that (and its amazing how many don't) invite them in for a Pair Programming session of about 30-60 minutes. If they freak and say no then you probably didn't really want them anyhow (never had this happen BTW)
  • If they pass the Pair Programming, then get into the company fit and position discussions, which is probably only 2 or 3 more phone calls or interviews
So how do they pass the functional test of Pair Programming? Well its definitely an instinct thing, but in the eXtreme tradition the Pairing process will expose a little of everything all the way down. You will within 10 minutes figure out if the candidate knows the technology, knows how to program, knows how to interact with other stakeholders in the system, and whether they have what it takes to work in your business and deliver value.

I recommend the 30-60 minute timing so you can get 2 people to do 30 minutes each with a candidate. The second one tries to understand what happened in the first 30 minutes through the candidate reiterating and demonstrating. This also helps avoid the single chooser problem.

There is an anxiety about Pair Programming in the interview process about it being a waste of resource. I would argue that it is in fact a valuable investment for two reasons. Firstly, it is the only truly effective way of testing if a candidate is entirely suitable for what you wish them to do. By testing the right thing early it reduces the costs of fixing mistakes later - which might include your company's reputation! Secondly, it involves your current employees directly in helping to choose their future colleagues - which can in itself be a motivational exercise.

Also, remember the interviews are a two-way street, and your company will be judged on the process of interview. In fact it is the only true measure a candidate gets of you. Think: What does your recruitment process say to the candidates?


At Monday, October 17, 2005 11:31:00 pm, Anonymous Anonymous said...

So why doesn't your recruitment department fix the problem?

At Tuesday, October 18, 2005 9:36:00 am, Blogger John S Nolan said...

My comments here are not about the Finetix process ini isolation - I am talking in my general experience recently.

Having said that, I've only been at Finetix 3 days and already there is action being taken with regard to my feedback and feedback from others in the company about recruiting. Good agile action too.

At Tuesday, October 18, 2005 11:32:00 am, Anonymous Anonymous said...

Nice picture - way cool dude...

At Wednesday, October 19, 2005 12:26:00 am, Anonymous Anonymous said...

Looks a bit gay though.

At Wednesday, October 19, 2005 9:48:00 am, Blogger John S Nolan said...

As a Little Britain fan I am tempt to claim to be "the only gay in the village"....but my wife would probably object :-)

Its my artistic tendencies showing through....

At Wednesday, October 19, 2005 11:57:00 pm, Anonymous Anonymous said...

defn a gay

At Thursday, October 20, 2005 12:00:00 am, Anonymous Anonymous said...

My god, so you have the audacity to admit you are a member of the eXtreme Tuesday Club (aka Geek club)?

I'd keep it quiet about it if I were you, otherwise you'll have not only all the gays after your ass but also all the geeks

At Thursday, October 20, 2005 10:13:00 am, Blogger John S Nolan said...

Audacity? "Fearless daring; intrepidity. Bold or insolent heedlessness of restraints, as of those imposed by prudence, propriety, or convention."

Never knew to being part of the XTC was so daring!

Having only been blogging for a week the conclusion the world seems to have of me is (a) i'm gay, and (b) i'm a geek. By this time next year I'll be a single lesbian mother from Kampala with an unfortunate speech impediment.

At Wednesday, November 02, 2005 10:08:00 am, Anonymous Anonymous said...

John, what code did you pair program on during interviews? Did you pair on production code or did you create examples for use in interview situations?

I've used a canned example in interviews and it worked well at quickly weeding out people who were unable to program, but it was too small to filter out "trainable" programmers, who knew things without really understanding them, from good programmers who understood the principles behind practices.

At Wednesday, November 02, 2005 10:37:00 am, Blogger John S Nolan said...

At Connextra we used production code. This has many advantages:
-the candidate gets to experience exactly what the environment and system is like
- you get to assess the candidate's domain knowledge or ability to understand the real domain of work
- you get to assess whether they have the skills required for your actual problem (technical and soft skills), as opposed to some artificial simulation of your problem

Part of the assessment is of course how far through a real task you can pair with them. Paolo Polce managed to Pair with one of the guys in his interview and actually released code in to the live system. We hired him, of course.

And interesting side benefit of this approach is that you get a free hour's work on your code base and your not entirely "wasting" the employees time who's Pairing with the candidate. I did used to joke that I could develop the product by continaully pairing interviewees with employees - hence only having to invest in half the resource. (I wasn't serious BTW)

The closer to the real thing you want them to do, the better the interview. Its an extreme approach - but that's me.

I would advocate this where possible. In consulting firms or defence systems there may be some issues with this approach, but the principle of trying to actually get them to do the job for an hour is the best approach.

At Wednesday, November 02, 2005 4:22:00 pm, Anonymous Anonymous said...

There'd be some problems with financial institutions as well, what with SOX legislation and all the complications that causes.

At Thursday, November 03, 2005 10:47:00 am, Blogger John S Nolan said...

SOX legislation? The Sarbanes-Oxley legislation in the US? I'm not sure how this would impact this approach?

As far as I understand it SOX is aimed at financial oversight and audit controls. Its impact on it is the necessity to maintain security, identity and auditability (see this article). Understandably, having someone work on your real system could be considered a threat to this, but as this will be done in a Pair with a resposnible employee that is covered by the company's SOX compliance, I would imagine this would actually be another benefit of this approach? (i.e. I would not advocate doing this "work on the system" unsupervised anyhow, but SOX would definitely bar it)

I am misunderstanding the impact of SOX?

I'm not a SOX expert but I do know that most software that is written non-paired in large organisations is rarely audited or reviewed for any loopholes or mechanisms that might neutralize a company's compliance with legislation such as SOX, or even basic security and access. IS this ever brought up in SOX discussions?

At Monday, November 21, 2005 11:22:00 am, Anonymous Anonymous said...


At Monday, November 21, 2005 11:41:00 am, Blogger John S Nolan said...

I'll take the Coltrane look-a-like as a huge compliment - I just wish I could play like him!


Post a Comment

<< Home