Allow “Phone a Friend” during Technical Interviews?(n0tw0rthy.wordpress.com)
n0tw0rthy.wordpress.com
Allow “Phone a Friend” during Technical Interviews?
http://n0tw0rthy.wordpress.com/2013/02/04/allow-phone-a-friend-during-technical-interviews/
50 comments
This is definitely a really useful component to the interview, but I don't think it should be the only component. A lot of our jobs are doing straightforward things: "implement spec X, where X is pretty close to what's been done before." But occasionally we'll encounter a problem that requires us to think up an algorithm. It might be similar to what's out there, but it's not going to be a googleable solution. It's these cases that define a good engineer.
I'd liken our jobs to those of pilots. Most of the time, anyone can fly the plane: just punch in the flight plan to autopilot, and press 'A/P engage'. However, when a pilot is near cumulonimbus clouds with an engine failure, he is forced to step up to the plate. This is what pilots are paid for: to handle the crazy emergency situations that can arise when you're in the air. This, too, is what good engineers are paid for: handling the difficult problems that occasionally arise during development. If you're not hiring for this, you're hiring code monkeys.
I'd liken our jobs to those of pilots. Most of the time, anyone can fly the plane: just punch in the flight plan to autopilot, and press 'A/P engage'. However, when a pilot is near cumulonimbus clouds with an engine failure, he is forced to step up to the plate. This is what pilots are paid for: to handle the crazy emergency situations that can arise when you're in the air. This, too, is what good engineers are paid for: handling the difficult problems that occasionally arise during development. If you're not hiring for this, you're hiring code monkeys.
I agree this shouldn't be the only component overall, you need to see if he fits into your team but I feel like if he can solve a problem in a timely manner that I deem moderately difficult using google and some experience based tricks with me looking over their shoulder then I can judge if they are a good engineer. Everyone I've hired using this method has never been fired for incompetence or issues with their skill. My issue is judging peoples personality and team fit. I still have yet to find a solid solution to this and asking Microsoft interview questions or how linked lists in C work isn't going to do that either. People are often overly nice/fit for the first few weeks/months but then go downhill from there as their true personalities show. I see no way to judge this other than calling contacts/past employers and even that doesn't always work.
This is pretty much exactly how I got hired.
In my case, I had to show design capabilities in addition to code and being creative didn't mean just applying hacks found on the internet. But my boss had to actually see if I can get through with building the project using a combination of existing knowledge and the ability to find solutions.
My challenge was to build a discussion forum from scratch with user registration and a reasonably attractive UI. So that's PHP + Photoshop + CSS/HTML/JS
In my case, I had to show design capabilities in addition to code and being creative didn't mean just applying hacks found on the internet. But my boss had to actually see if I can get through with building the project using a combination of existing knowledge and the ability to find solutions.
My challenge was to build a discussion forum from scratch with user registration and a reasonably attractive UI. So that's PHP + Photoshop + CSS/HTML/JS
That sounds like an awfully big project for a job interview. If someone asked me to do this, I would probably install phpbb.
To be fair this was after the initial screener interview and I did have a couple of hours to get it done. And we're not talking all the features (certainly not of phpBB) or anything.
3 Tables "posts", "replies", "users" (replies was a closure table), topics index, topic view page, a form page for registration and one for the profile page. JS form validation and a simple layout via CSS and HTML5.
It was more of a Twitter-lite (a very poor man's Twitter-lite)
3 Tables "posts", "replies", "users" (replies was a closure table), topics index, topic view page, a form page for registration and one for the profile page. JS form validation and a simple layout via CSS and HTML5.
It was more of a Twitter-lite (a very poor man's Twitter-lite)
This. Technical interviews are important but their purpose should not be to check whether you know a particular function/API whatever. Their purpose should be to take a real world problem and show the interview how you resolved it. Sure, remembering specific details of a technical subject is always handy but just because I don't remember the function that does task 'x' in language 'y', doesn't mean that I cannot use it.
We have candidates that make it through the first cut of "culture fit" interviews do an open ended coding project with some of our real data.
What they do and how long they take to do it is totally up to them. They can call a friend, use online resources, and even contact anyone at the company with questions. They present their project at their second interview.
It has been a great way to gauge technical, problem solving, and presentation skills as well as interest in our company.
What they do and how long they take to do it is totally up to them. They can call a friend, use online resources, and even contact anyone at the company with questions. They present their project at their second interview.
It has been a great way to gauge technical, problem solving, and presentation skills as well as interest in our company.
This is a tough one, because I can see two sides of this pretty easily. In one sense, I can understand what makes this so valuable to an employer trying to evaluate a prospective employee. On the other sense, I'm reluctant to do free coding with production-level potential.
Have you seen any resistance from applicants to this method?
Have you seen any resistance from applicants to this method?
We have already done the culture fit interviews. We only extend the opportunity to work on the project to people we want to hire if their technical skills meet our expectations.
If that isn't clear to a candidate when they are offered the project, or if they still are reluctant, then they aren't a good fit for us anyways.
If that isn't clear to a candidate when they are offered the project, or if they still are reluctant, then they aren't a good fit for us anyways.
A balanced approach is use trial work as a level 2 filter and pay the applicant contract rates for a day / week.
If the person turns out to be a no hire, you managed to avoid an expensive false positive.
If the person turns out to be a no hire, you managed to avoid an expensive false positive.
All this interview crap is just that -- crap. It's a poorly performing, bureaucratic cover-your-ass piece of bullshit window-dressing that should largely go away. It's the interviewee crafting a word-picture of themselves that might as well be fictional while the interviewers are crafting an imaginary picture of the interviewee that might as well be fictional. The only way to figure out what it's like to work with somebody is to work with them.
EDIT: People have to communicate as part of their jobs. An interview is fine for determining that, because that's what it is. If you think it will help you determine how good they are to work with, or how well someone codes, you're living in a fantasy land.
EDIT: People have to communicate as part of their jobs. An interview is fine for determining that, because that's what it is. If you think it will help you determine how good they are to work with, or how well someone codes, you're living in a fantasy land.
Testing a candidate's ability to express themselves is a good idea, but there are probably simpler ways of doing it than phoning a friend. If you wish to test communication skills, ask them an open-ended question like "how does a browser/OS/compiler/database work?".
I wouldn't like put one of my friends in such a position when answering an interview question. Nor would I like to be (partly) responsible for a friend's performance in theirs.
Anyway, surely a candidate that can do the question on their own is better than one that can't. And surely the friend that can answer the question is better than the candidate?
A technical interview question shouldn't be about specific knowledge of particular problems (although bad ones often are, usually having "Aha!" moments). They should test areas of knowledge: reversing a linked list is checking if they understand how pointers work, something which is not really possible to learn in 5 minutes of Googling, even if they can then answer the question. Questions should test general technical problem solving skills, which are not easily Googled, even if a specific instance is.
Finally, if a candidate requires knowledge of a particular API (e.g. the order of parameters to a function), then the interviewer should tell them. If the interviewer doesn't know, then they must acknowledge that it's not really important, and allow the candidate to just pseudocode it.
I wouldn't like put one of my friends in such a position when answering an interview question. Nor would I like to be (partly) responsible for a friend's performance in theirs.
Anyway, surely a candidate that can do the question on their own is better than one that can't. And surely the friend that can answer the question is better than the candidate?
A technical interview question shouldn't be about specific knowledge of particular problems (although bad ones often are, usually having "Aha!" moments). They should test areas of knowledge: reversing a linked list is checking if they understand how pointers work, something which is not really possible to learn in 5 minutes of Googling, even if they can then answer the question. Questions should test general technical problem solving skills, which are not easily Googled, even if a specific instance is.
Finally, if a candidate requires knowledge of a particular API (e.g. the order of parameters to a function), then the interviewer should tell them. If the interviewer doesn't know, then they must acknowledge that it's not really important, and allow the candidate to just pseudocode it.
I agree. Generally, open ended questions give you a pretty good idea of a candidate's aptitudes. I had a guy come to interview who'd supposedly got a CS degree with first-class honours from a fairly well-ranked British uni. He couldn't tell me what functional programming was, which I thought was pretty peculiar.
I tried to call the uni to verify his degree but they wouldn't confirm it without consent (Data Protection Act), and I wasn't going to employ him (for several reasons), so I didn't take it any further.
I tried to call the uni to verify his degree but they wouldn't confirm it without consent (Data Protection Act), and I wasn't going to employ him (for several reasons), so I didn't take it any further.
I don't think not knowing functional programming is peculiar at all. When I went to school, the dominant language was Java. Aside from a few courses, some of which covered functional programming, every other course and project was written in Java. It'd be very easy for me to imagine a good graduate making it through University and not remembering functional programming at the end.
Not that he didn't know it. He didn't know what it was.
I agree. Generally, open ended questions give you a pretty good idea of a candidate's aptitudes.
I would still have to test their ability to code something, as I know how good I am (if I may say so) at blagging my way through open ended questions.
I would still have to test their ability to code something, as I know how good I am (if I may say so) at blagging my way through open ended questions.
That what practical tests are for. Throw a problem to solve, leave the programmer for an hour and see what he'll produce, while having normal access to the online world.
But you don't want to check encyclopedic knowledge during interview talk. You want to check how well he understands what he's doing, what is his approach to problems, what kind of problems did he solve in the past and how he did it, and lot of other things that don't require the only one correct answer (and cannot even be answered by one).
But you don't want to check encyclopedic knowledge during interview talk. You want to check how well he understands what he's doing, what is his approach to problems, what kind of problems did he solve in the past and how he did it, and lot of other things that don't require the only one correct answer (and cannot even be answered by one).
I let candidate ask me questions. If can't remember it either, I'll just allow the assumption that a method exists, as long as I know what it's doing (and it's not abstracting away anything core to the question I'm asking). In the same vein, I also explicitly allow pseudocode.
This is my vote. Open up a dialog between interviewer and interviewee. Everyone can use a hint sometimes and it gives a chance to have a technical discussion.
That being said lately I worked hiring contractors. We give them a week long paid trial assignment (paid if they pass). Half the people I hire I don't even have a resume for. This is really the best way to hire people.
That being said lately I worked hiring contractors. We give them a week long paid trial assignment (paid if they pass). Half the people I hire I don't even have a resume for. This is really the best way to hire people.
No, the best way would be paid even if they fail. Making someone work for you for a week for free is horrific.
Yes. This appears to be massively exploitative. They should never get to the point of hire if they're too unqualified to be paid.
I happen to agree but I have no say in this. If it mitigates this at all, the trial assignment is not actual work. It's been solved many times and the candidates who are good do it in 2 days, not much different than the time it takes to do an extensive round of interviews.
I've done the "google it" approach, so here's my .02
We talk to a lot of college grads. Clasically, colleges don't seem to teach a lick of web development. Just old-fashioned comp-sci uselessness. We know these kids are bright, but they're not going to be able to develop full stack out of the gate, usually, without some aide. We were all like that once.
What I want to see from a bright young talent is whether or not they know where to go to get the answer, and how quickly they can use it, and how quickly they can grok it. It doesn't take an incredibly long amount of time to start becoming useful to web dev with all the resources out there. I'm OK if they don't know some arcane bit of CSS, like that white-space: nowrap exists. They can and will find that information very quickly, and it's not difficult to remember, retain, or understand.
If I'm interviewing a candidate with a proclaimed 8 years experience in the industry, for a position of mid-to-senior level developer, they really shouldn't need that extra help. They should be shipping-ready (given the time needed to understand the idiosyncrasies of your company's style).
I've found that this approach has netted us a good deal of sleeper candidates that, on paper and on a white board, may not look like much, but end up being killer, driven developers.
We talk to a lot of college grads. Clasically, colleges don't seem to teach a lick of web development. Just old-fashioned comp-sci uselessness. We know these kids are bright, but they're not going to be able to develop full stack out of the gate, usually, without some aide. We were all like that once.
What I want to see from a bright young talent is whether or not they know where to go to get the answer, and how quickly they can use it, and how quickly they can grok it. It doesn't take an incredibly long amount of time to start becoming useful to web dev with all the resources out there. I'm OK if they don't know some arcane bit of CSS, like that white-space: nowrap exists. They can and will find that information very quickly, and it's not difficult to remember, retain, or understand.
If I'm interviewing a candidate with a proclaimed 8 years experience in the industry, for a position of mid-to-senior level developer, they really shouldn't need that extra help. They should be shipping-ready (given the time needed to understand the idiosyncrasies of your company's style).
I've found that this approach has netted us a good deal of sleeper candidates that, on paper and on a white board, may not look like much, but end up being killer, driven developers.
> Just old-fashioned comp-sci uselessness.
The reason they are able to pick up "web development" so quickly is because of this old-fashioned comp-sci education.
Web dev isn't a thing, it's just another platform to learn. At its core are basic CS concepts.
Sure, you won't be a super expert on day one, but it's just like picking up iOS development. I'd rather have someone who could pick up iOS dev in a day than someone that only does it and took months to learn.
The reason they are able to pick up "web development" so quickly is because of this old-fashioned comp-sci education.
Web dev isn't a thing, it's just another platform to learn. At its core are basic CS concepts.
Sure, you won't be a super expert on day one, but it's just like picking up iOS development. I'd rather have someone who could pick up iOS dev in a day than someone that only does it and took months to learn.
> Just old-fashioned comp-sci uselessness
Everything you're saying is reasonable but this phrase reminded me, one of my favorite profs used to say, "Knowing about computational complexity won't ever get you a job, but it will keep you from being fired." Several times I've started a job and discovered my predecessor had coded himself into a corner because he didn't see the combinatorial explosion on the other side of what he was doing.
Everything you're saying is reasonable but this phrase reminded me, one of my favorite profs used to say, "Knowing about computational complexity won't ever get you a job, but it will keep you from being fired." Several times I've started a job and discovered my predecessor had coded himself into a corner because he didn't see the combinatorial explosion on the other side of what he was doing.
Were your predecessors predominantly CS grads or not?
Not.
What was their background?
Hard saying in much detail; after all, I showed up after they left. Factually at least twice they were highschoolers or college dropouts. In another case he had a history degree, and a business degree in another. I now work for the NRAO, so I can tell you any level of education in a physics or math track is not necessarily sufficient to learn how to program--the physicists in particular seem wholly unable to perceive code quality or understand what might affect performance and what wouldn't. This, of course, doesn't stop them from writing their own code and giving opinions about what's wrong with your code, sight unseen. ("I think your program is calling malloc and free incorrectly."--scientist, about my Java code).
> Just old-fashioned comp-sci uselessness.
Yet so many interview tests tend to have a focus on CS backgrounds. I just went through a few weeks of interviewing and some of the questions asked were really there to see if you knew your CS stuff. For some companies the questions were at least relevant to the industry and tasks they were trying to accomplish, but some were so far out there it just didn't make sense.
The company I finally decided to accept at had a sane interviewing process, one that had challenging enough questions to show that you know what you are talking about - and when there was one lesser known method I wanted to use, but couldn't remember the syntax, they allowed me to look it up real quick. In the end, I was able to teach the interviewer about this method, and it helped me complete the task in a far more efficient manner.
Yet so many interview tests tend to have a focus on CS backgrounds. I just went through a few weeks of interviewing and some of the questions asked were really there to see if you knew your CS stuff. For some companies the questions were at least relevant to the industry and tasks they were trying to accomplish, but some were so far out there it just didn't make sense.
The company I finally decided to accept at had a sane interviewing process, one that had challenging enough questions to show that you know what you are talking about - and when there was one lesser known method I wanted to use, but couldn't remember the syntax, they allowed me to look it up real quick. In the end, I was able to teach the interviewer about this method, and it helped me complete the task in a far more efficient manner.
I can see the appeal, if you're hiring for someone who can, eventually, figure out how to do what you want them to. However, if you're hiring for people with a common knowledge base (i.e. computer scientists), who will be required to think up novel solutions to problems that cannot be found online, then you're not going to get the right kind of candidate (and should, instead, be offering double what you would have to that friend who got the answer in 5 min, over the phone, with all the communication issues that presents, to what took the original interviewer 10 or 20+ minutes to figure out they couldn't do by themselves).
In short, if you're trying to hire someone to do work that's been done elsewhere many times before, then this is probably fine. But I don't expect the most innovative companies to ever want to filter for google skills.
In short, if you're trying to hire someone to do work that's been done elsewhere many times before, then this is probably fine. But I don't expect the most innovative companies to ever want to filter for google skills.
An interesting question is if Google itself would consider "google skills" an advantage!
I think this is a great idea in principle. Being able to learn and understand the solution quickly on the internet is a very valuable skill.
At the same time, there is deeper stuff that really isn't suited to this approach. If I were conducting an interview, I'd set the candidate a task the evening before their interview that involved something I knew they had know experience in, to see how well they can delve into and understand new tools.
A face to face interview is a great way to really understand how good a candidate's core knowledge is. Do they understand data structures and how to manipulate them? Do they understand algorithms and their complexity? Sure, you can look this stuff up, but it's the kind of knowledge you need to have before you even start.
At the same time, there is deeper stuff that really isn't suited to this approach. If I were conducting an interview, I'd set the candidate a task the evening before their interview that involved something I knew they had know experience in, to see how well they can delve into and understand new tools.
A face to face interview is a great way to really understand how good a candidate's core knowledge is. Do they understand data structures and how to manipulate them? Do they understand algorithms and their complexity? Sure, you can look this stuff up, but it's the kind of knowledge you need to have before you even start.
Only if the interviewers are allowed to know the name and number of this friend :) Why bother with the person who can't answer the question when you can go directly to the goto guy? (and I say this as someone who's only ever been the interviewee).
It’s a shame that this post can even be deemed heretical when in fact it’s the normal mechanism for completing tasks by the vast preponderance of developers for overcoming technical challenges.
Understanding the ways in which technical people find answers, and can fully grok and socialize those answers internally with options is invaluable in a technical interview.
Understanding the ways in which technical people find answers, and can fully grok and socialize those answers internally with options is invaluable in a technical interview.
The applicant can already do this, in asking the right sorts of questions from the interviewer to help them solve whatever rut they've found themselves in.
I think the phone-a-friend possibility is a lot more interesting because it does force the candidate to verbalize the problem and interpret verbal feedback. If you make them do it in front of you, you'll also get to see how they interact with other people under pressure and also how their friend treats their asking of the question will be revealing as well.
I don't see the point to this.
The purpose of any phone screen I've done is that it's simple enough that no self respecting programmer would have trouble with the questions, ie they are of the fizzbuzz variety.
The point being that if you need to "phone a friend" then you've probably already failed the interview.
The purpose of any phone screen I've done is that it's simple enough that no self respecting programmer would have trouble with the questions, ie they are of the fizzbuzz variety.
The point being that if you need to "phone a friend" then you've probably already failed the interview.
If you allow the candidate to do what they'd do at their desk you can ask more than trivial questions. Certainly, if they have to ping google to figure out how to write fizzbuzz, that's a red flag, but it depends what the person spec is.
How many positions would you estimate (I'm an academic so have no experience) where encyclopedic and gap-less knowledge is required to accomplish a task? I envision a technical interview as an opportunity to make sure the candidate can actually do the task you need them to - the screening of non-programmers should be done beforehand.
How many positions would you estimate (I'm an academic so have no experience) where encyclopedic and gap-less knowledge is required to accomplish a task? I envision a technical interview as an opportunity to make sure the candidate can actually do the task you need them to - the screening of non-programmers should be done beforehand.
> I envision a technical interview as an opportunity to make sure the candidate can actually do the task you need them to - the screening of non-programmers should be done beforehand.
Ahh I see where we disagree, I might have read the blog post wrong. To me a phone interview is the actual screen. I'd never do an actual interview over the phone that was the basis for the final hire/no hire.
If the OP means phone interview as the actual interview rather than a phone screen then yes, I'd certainly relax my position somewhat.
usually for phone screens I'll open a shared google doc site and get the user to do a problem like fizz buzz.
Ahh I see where we disagree, I might have read the blog post wrong. To me a phone interview is the actual screen. I'd never do an actual interview over the phone that was the basis for the final hire/no hire.
If the OP means phone interview as the actual interview rather than a phone screen then yes, I'd certainly relax my position somewhat.
usually for phone screens I'll open a shared google doc site and get the user to do a problem like fizz buzz.
I think that being able to "google it" is a technical skill by itself especialy for developers. For example, you are getting an exception in your code. How do you take that specific exception msg and figure out what to do with it. Do you just copy/paste the exact err msg in google and hope for a miracle ? Do you combine that err msg with some other keyword to get a more suited result ? Or do you just not use the err msg but google for the specific process you are trying to implement ? Depending on what you google, you could get different answers.
I think that if a developer knows how and where to look for the resources to solve a problem, he is worth a shot. I am sure if he has to look up "C++ syntax" when being interviewd for "experience C++ developer", then thats bad. You get the idea.
I think that if a developer knows how and where to look for the resources to solve a problem, he is worth a shot. I am sure if he has to look up "C++ syntax" when being interviewd for "experience C++ developer", then thats bad. You get the idea.
I would hire the friend since he can actually solve problems.
It doesn't seem likely to me that the friend wants the job.
If you think this might be great for your hiring process, think about the questions you ask. In some cases it might be that you ask questions that hardly show candidates knowledge.
For instance, average and worst case complexity of various algos (usually sorting). This is kind of a question you might want a call/gooogle and I believe is common one. But I don't think it does make any difference in real-world practice whether you know (and can recall in a stressful situation) what is quick sort's worst case complexity and in what case it happens. If I need that, I will google it and find it less than 5mins. Obviously, awareness of worst-case is necessary but not memorizing for each algorithm.
For instance, average and worst case complexity of various algos (usually sorting). This is kind of a question you might want a call/gooogle and I believe is common one. But I don't think it does make any difference in real-world practice whether you know (and can recall in a stressful situation) what is quick sort's worst case complexity and in what case it happens. If I need that, I will google it and find it less than 5mins. Obviously, awareness of worst-case is necessary but not memorizing for each algorithm.
I do variant number 2 all the time. I'm not interested in pop trivia. If someone wants to consult C++ STL docs, or unix manpages, or python library docs, that's fine. They just have to tell me they're doing it. Relying on what someone has memorized is silly.
The thing about interviews or any spoken communication. A lot of people just get really nervous. Even for the most basic questions.
What is your name?, "Fuck let me call in a life line".
What is your name?, "Fuck let me call in a life line".
I think it's hokey to structure interviews on the model of a TV game show. Much better to see what they consider doing when they don't know the answer to a question, or suggest an approach to them in the moment to see their skills in finding information.
I just hired someone for a job who's first response was to copy and paste the error code into Google and view a stack overflow post with a similar problem. From there he took what the response on stack overflow said and crafted a solid solution to the issue. It took him about an hour and I saw that he had to Google a few functions (Python language) but he showed he knew the resources he needed and the basics to coding solutions to problems.
I don't care if you know how to create a linked list in C on a piece of paper or why we have round manhole covers instead of square, this isn't useful in the real world, I want to see if you can solve a real problem and the method you go about it.