Ask HN: What are some ways to avoid biases in the interviewing process?
3 comments
You could think of the bias as having two sides.
#1. You won't recognize the potential of a candidate application enough to do an actual interview.
#2. You won't recognize the potential of a candidate after interviewing them.
In my experience solving #1 is straightforward and does 80% of what you're wanting to do.
Thankfully, this was figured out by the NFL years ago. They call it the Rooney rule: you can't hire a head coach until you've interviewed a minority candidate.
That's it! The only requirement is that you perform an interview.
Thankfully, with Angel List, you don't need to work very hard to get diversity into your pipeline. Just keep clicking yes to profiles until you've said yes to some diverse candidates and they've responded that they want to be interviewed by you.
It's less a quota system and more of a marker for how much time you should spend sourcing candidates. Keep up that work until you meet this trivial requirement to have interviewed at least one candidate that meets your definition of diversity.
In my experience, the person who walks in the door is often both clearly the superior candidate and not being recruited very heavily. In other words, industry bias is so strong that your personal bias in the interviewing process is largely solved for you by the candidate being so strong.
I'm not saying not to work on your other biases, just pointing out that this is a high leverage place to start.
#1. You won't recognize the potential of a candidate application enough to do an actual interview.
#2. You won't recognize the potential of a candidate after interviewing them.
In my experience solving #1 is straightforward and does 80% of what you're wanting to do.
Thankfully, this was figured out by the NFL years ago. They call it the Rooney rule: you can't hire a head coach until you've interviewed a minority candidate.
That's it! The only requirement is that you perform an interview.
Thankfully, with Angel List, you don't need to work very hard to get diversity into your pipeline. Just keep clicking yes to profiles until you've said yes to some diverse candidates and they've responded that they want to be interviewed by you.
It's less a quota system and more of a marker for how much time you should spend sourcing candidates. Keep up that work until you meet this trivial requirement to have interviewed at least one candidate that meets your definition of diversity.
In my experience, the person who walks in the door is often both clearly the superior candidate and not being recruited very heavily. In other words, industry bias is so strong that your personal bias in the interviewing process is largely solved for you by the candidate being so strong.
I'm not saying not to work on your other biases, just pointing out that this is a high leverage place to start.
Avoiding platitudes will go a long way. Here's a tangentially-related example: Peter Thiel recently wrote (in Zero to One) that he decided not to invest in anyone who wore a suit.
Imagine if your hiring committee had a similar platitude. Is it illegal? Arguably no. Is it going to narrow candidates down based on any criteria that actually matters to the job? Probably not. I think catching yourself when you're tempted to use these kinds of heuristics is the key. Ask yourself, "are there any direct causal links between performance and the metric I'm measuring?" If not, you might not want to use that metric to make decisions.
[1] http://www.businessinsider.com/peter-thiel-hates-suits-2014-...
Imagine if your hiring committee had a similar platitude. Is it illegal? Arguably no. Is it going to narrow candidates down based on any criteria that actually matters to the job? Probably not. I think catching yourself when you're tempted to use these kinds of heuristics is the key. Ask yourself, "are there any direct causal links between performance and the metric I'm measuring?" If not, you might not want to use that metric to make decisions.
[1] http://www.businessinsider.com/peter-thiel-hates-suits-2014-...
You're citing this as an example of a batshit crazy practice people should not adopt, right?
Correct.
For dev/tech hiring, the answer is straightforward: deprecate person-to-person interviews in favor of work-sample testing. Take a simple project your team has built, carefully carve out some functionality from it, document it, and then have your candidates implement that functionality. The sample should be runnable in its incomplete state and should be accompanied by some record of what good output is.
Evaluate candidates on code that they write for your hiring process (not on what they've posted on Github).
Additional tips:
* Pre-interview every candidate with someone clueful whose job is to (a) sell the company to the candidate and (b) to over-answer questions about the interview process. A surprising amount of interview jitters come from candidates not having visibility into the process.
* Try to eliminate phone interviews entirely. They don't add value, but do amplify noise. Companies routinely evict candidates from the pipeline based on assessments of "confidence" or "passion" or "comfort" from phone interviews. Don't work for companies that do that.
* Never interview a candidate with more than one interviewer. Interviews are incredibly hostile experiences in the best of circumstances. Ganging up on candidates ensures that you can't possibly be in the best circumstance.
* Have lunch with candidates if you like, but don't make lunch a Q&A with the candidate. Train your team not to treat it that way. Be clear with the candidate when the metaphorical "red indicator light" goes on to say that they're being assessed, and when the light goes off so they can regroup. Assume even the most clueful and experienced candidate will need routine opportunities to regroup.
* Do not allow team members to contribute feedback based on "culture fit". Set aside that "culture fit" is often as not a fig leaf for a variety of discriminatory practices: it's also bad engineering. "Culture fit" allows any engineer to discard all the data you painstakingly collect on a candidate based solely on non-falsifiable gut-feeling assertions.
* Script your interviews. Your engineers will hate hate hate hate hate you for doing this (trust me, even if they start off thinking it's a good idea, they'll stop thinking that after they've done two of them), but you'll start generating actual data based on your candidates so you can make apples-apples comparisons and optimize your interviews instead of just occasionally changing the color of the rubber chicken you're waving around at candidates. Scripted interviews are also a forcing function for the development of objective standards for hiring, which will keep your team honest.
When I was at Matasano, we got minimal value out of campus interviewing, but a lot of value out of Internet outreach.
Things that make well-engineered interview processes tend to have a side effect of minimizing bias. The reason is that subconscious biases are already the biggest source of noise in your screening process anyways, so improving your screening will tend to require you to minimize bias.
Evaluate candidates on code that they write for your hiring process (not on what they've posted on Github).
Additional tips:
* Pre-interview every candidate with someone clueful whose job is to (a) sell the company to the candidate and (b) to over-answer questions about the interview process. A surprising amount of interview jitters come from candidates not having visibility into the process.
* Try to eliminate phone interviews entirely. They don't add value, but do amplify noise. Companies routinely evict candidates from the pipeline based on assessments of "confidence" or "passion" or "comfort" from phone interviews. Don't work for companies that do that.
* Never interview a candidate with more than one interviewer. Interviews are incredibly hostile experiences in the best of circumstances. Ganging up on candidates ensures that you can't possibly be in the best circumstance.
* Have lunch with candidates if you like, but don't make lunch a Q&A with the candidate. Train your team not to treat it that way. Be clear with the candidate when the metaphorical "red indicator light" goes on to say that they're being assessed, and when the light goes off so they can regroup. Assume even the most clueful and experienced candidate will need routine opportunities to regroup.
* Do not allow team members to contribute feedback based on "culture fit". Set aside that "culture fit" is often as not a fig leaf for a variety of discriminatory practices: it's also bad engineering. "Culture fit" allows any engineer to discard all the data you painstakingly collect on a candidate based solely on non-falsifiable gut-feeling assertions.
* Script your interviews. Your engineers will hate hate hate hate hate you for doing this (trust me, even if they start off thinking it's a good idea, they'll stop thinking that after they've done two of them), but you'll start generating actual data based on your candidates so you can make apples-apples comparisons and optimize your interviews instead of just occasionally changing the color of the rubber chicken you're waving around at candidates. Scripted interviews are also a forcing function for the development of objective standards for hiring, which will keep your team honest.
When I was at Matasano, we got minimal value out of campus interviewing, but a lot of value out of Internet outreach.
Things that make well-engineered interview processes tend to have a side effect of minimizing bias. The reason is that subconscious biases are already the biggest source of noise in your screening process anyways, so improving your screening will tend to require you to minimize bias.
I have to disagree with you on the "more than one interviewer" point. If you are only ever going to have your abilities assessed by one person, then you are 100% at the mercy of that one person's biases.
A single interviewer will almost always hire someone who is exactly like him or herself. That's pretty much the opposite of what you want to do if you want to increase diversity.
You're absolutely right that you shouldn't have a big panel, but two interviews with two people each shouldn't be overwhelming while still getting diversity in your hiring. If one person says "terrible x" and another person says "good x", then there's room for discussion about what particular things were good or bad, and on balance, whether the benefit of having that person is worth the risk.
A single interviewer will almost always hire someone who is exactly like him or herself. That's pretty much the opposite of what you want to do if you want to increase diversity.
You're absolutely right that you shouldn't have a big panel, but two interviews with two people each shouldn't be overwhelming while still getting diversity in your hiring. If one person says "terrible x" and another person says "good x", then there's room for discussion about what particular things were good or bad, and on balance, whether the benefit of having that person is worth the risk.
He means multiple interviewers in the room at a time. Adds to stress without adding additional information that couldn't be collected piecemeal with multiple 1:1 interviews.
[deleted]
I couldn't agree more about these items. I would attach a couple of caveats that are very situationally dependent.
1) I have found some value in phone screens if they are more about the selling of the company and the clarifying the process. The only factors I've ever found worth eliminating a candidate for at the phone screen level is if they do not have the tools/communication skills to move forward with the rest of the process and/or you catch them in outright lies about their resumes (though I highly discount resumes, I discount lying on them even more). Sometimes it is just too resource intensive to do the pre-interview any other way than a phone call, and I'd rather do the phone screen than not consider the candidate.
2) I completely agree with deprecating person-to-person interviews, but that is a very hard sell. If you want to accomplish that but are getting push back, the interview script process is a valuable tool for eliminating the interviews. It provides data to backup the argument and as no one will want to do scripted interviews they will become allies in getting rid of them.
3) Campus recruiting can be valuable for organizations that are prepared to take on the responsibility of hiring a new grad.
Having a data driven approach (ie scoring your process) is the most important advice.
1) I have found some value in phone screens if they are more about the selling of the company and the clarifying the process. The only factors I've ever found worth eliminating a candidate for at the phone screen level is if they do not have the tools/communication skills to move forward with the rest of the process and/or you catch them in outright lies about their resumes (though I highly discount resumes, I discount lying on them even more). Sometimes it is just too resource intensive to do the pre-interview any other way than a phone call, and I'd rather do the phone screen than not consider the candidate.
2) I completely agree with deprecating person-to-person interviews, but that is a very hard sell. If you want to accomplish that but are getting push back, the interview script process is a valuable tool for eliminating the interviews. It provides data to backup the argument and as no one will want to do scripted interviews they will become allies in getting rid of them.
3) Campus recruiting can be valuable for organizations that are prepared to take on the responsibility of hiring a new grad.
Having a data driven approach (ie scoring your process) is the most important advice.
If it's true only 7% of communication is verbal you are mislead by 93% when you do a phone screen.
That's really helpful Thomas. Many thanks for your detailed outline.
1. Make sure that the recruiting team is diverse. 2. Remove inherent biases in the interview process. 3. Implement a robust campus recruiting program that attracts young minds from all backgrounds
With regards to 2. what are some tools and process that can be used to avoid biases in hiring?
I'd like to know how you go about solving for it, if you have examples to share.
Many thanks.