Tech Tests and Certainty
Tech tests in interviews: When it comes to choosing strategy, there is no obvious winner. Trying to evaluate a candidates’ technical experience in the relatively microscopic space of time you interact with them means you’re necessarily artificially limited.
Pick your poison.
Take Home Tests
Risk: favours the time rich.
Mitigation: have long interview cycles; structure the test to only be quick, etc.
I don’t rate take home tests.
Live pairing
Risk: stressful for candidates.
Mitigation: put in the pre-and-during work to de-stress; focus on non-coding skills; send them the exercise beforehand; let them use their own env, etc.
I rate this slightly above take home tests - for efficiency.
Code review discussion
Risk: doesn’t test coding skills.
Mitigation: accept testing coding skills isn’t helpful? 🤷♂️
It’s efficient, possibly less stressful than live pairing.
At the root of the need for a tech test are some assumptions: that doing this kind of technical evaluation with a candidate up-front is useful; that you can’t trust their CV; that you have to see them write code with your own eyes. If you can accept that a tech test is largely a waste of time, you’ll get a good sense of what effort to put into it. Use the time to evaluate the candidate’s competency, team working, communication etc.
But most importantly, the big assumption is that putting extra time + tasks into an interview scales your certainty about the candidate. Above a certain small amount of time spent in interviewing (1-2 hours) any increase in certainty from the process itself becomes minuscule.
If you want to raise certainty in your interview process:
1) Diversify your interview panel
2) Reduce your bias
3) Standardise and refine your rubric (with training!)
If you want to dig deep into coding talent: have an efficient interview, put in the work to give everyone a structured, supported probation, then be willing to fire people during their probation.
If you find you need to let someone go during probation, critically re-evaluate (inspect + adapt) your interview process with a focus on diversity, bias reduction, removing blind spots and efficiency.
There are no quick wins if you want quality and efficiency.
P.s. the two most important things to me in interviewing are: bias reduction and efficiency.
