On Hiring: How Not to Annoy Developers(fredwu.me)
fredwu.me
On Hiring: How Not to Annoy Developers
http://fredwu.me/post/16510145575/on-hiring-how-not-to-annoy-developers
11 comments
the other suggestion on the post - pair programming is a good one. The interviewer itself who is engaging the candidate in pair programming has to be really good to step back and let the candidate bring in more suggestions and code.
Only off side is if the candidate is not comfortable working together with one more person.
Only off side is if the candidate is not comfortable working together with one more person.
Well doing it on a white board in the same room as the interviewer allows you to see how some one approaches a problem - which is more important than the quondam task of coding.
[deleted]
Except, most people asking white board questions are asking for code.
> allows you to see how some one approaches a problem
I've never jumped to a whiteboard. Rather, I've researched the problem. If they really wanted to see how you'd approach a problem, they'd give you the problem before the interview, and then when you come in, ask you to explain it, allowing you to use the whiteboard.
> allows you to see how some one approaches a problem
I've never jumped to a whiteboard. Rather, I've researched the problem. If they really wanted to see how you'd approach a problem, they'd give you the problem before the interview, and then when you come in, ask you to explain it, allowing you to use the whiteboard.
> I've never jumped to a whiteboard.
I've also never had to reverse a BST in place outside of an interview setting.
I personally like technical questions in interviews. I've had some good experiences with them, but that may be because I have always been a good "test-taker". I feel like I understand what tests (and interviewers) are looking for and can tailor my response accordingly.
For an interview, I've learned that the question isn't "come up with the best solution". The question is really "demonstrate how you approach this problem". I've learned to vocally explain that I'm starting with a naive approach. Then I explain how I'm considering the constraints for optimization. I state out loud any patterns or similarities between the optimizations I am trying to make and ones I've made in the past. (ie, "To make this O(logn), we need to find ways to eliminate half the data set each iteration. Is this similar to a BST at all?"). I pursue paths I know might not be correct, and backtrack when I see the fault in the idea. The key is to narrate for the interviewer.
Maybe I've had good interviewers, but 7 of the last 8 interviews I've had resulted in job or internship offers and I felt the interviewer was able to get an accurate picture of my abilities.
I've also never had to reverse a BST in place outside of an interview setting.
I personally like technical questions in interviews. I've had some good experiences with them, but that may be because I have always been a good "test-taker". I feel like I understand what tests (and interviewers) are looking for and can tailor my response accordingly.
For an interview, I've learned that the question isn't "come up with the best solution". The question is really "demonstrate how you approach this problem". I've learned to vocally explain that I'm starting with a naive approach. Then I explain how I'm considering the constraints for optimization. I state out loud any patterns or similarities between the optimizations I am trying to make and ones I've made in the past. (ie, "To make this O(logn), we need to find ways to eliminate half the data set each iteration. Is this similar to a BST at all?"). I pursue paths I know might not be correct, and backtrack when I see the fault in the idea. The key is to narrate for the interviewer.
Maybe I've had good interviewers, but 7 of the last 8 interviews I've had resulted in job or internship offers and I felt the interviewer was able to get an accurate picture of my abilities.
Right I agree, asking someone to drag you along through there thought process while they are trying to solve problem is not natural. It's akin to saying, hey give a public speech while you are writing an algorithm. How many times do we here people say hold on I am concentrating. Most people opt to solve a problem, refine it, and then say OK here is what I did, now that I have had time to think about it and write it down.
Different strokes for different people I often sketch out my ideas and possible solutions on a white board or on some sheets of A1 flipchart paper laid out on a spare desk near me.
I disagree, a fair majority of developers fall within the INTP or INTJ classification of Myers-briggs. both of which like to analyze a situation before they would consider answering. Specifically, the INTJ classification is filtered heavily at the whiteboard because of their Judging trait. INTJ's know what they don't know, and if they don't they are going to stop to find a resource that can provide that portion of the answer. Due to the INT part of their profile, they are not going to look at anyone in that room as a potential authority for that answer and therefore they will feel research is needed. This can cause a situation in which an INTJ can basically lock-up. They know they don't know a portion of the answer, and they have none of the natural authorities to look to, to resolve the answer so that they can move on. This can happen on something as simple as forgetting an API method or the modulus operator.
No matter how much interviewers say they are just looking for solutioning, they are not being honest with themselves because when they see code that is not their style or they see someone make stupid mistakes (which we all do) on the whiteboard, it embeds in the mind. It such an overpowering signal that it outweighs all other evidence. An INT already knows this and would therefore prefer to present finished code, not how you get there, which finished code is all that really matters, you can walk them through how they arrived at the finished code, and they will have a far more competent answer because they are not trying to code and drag people along with them at the same time.
Even if a developer makes 20 mistakes, and uses the compiler/debugger to catch them, then fixes them, if the code that they are going to check in is always clean, is his process bad? on the whiteboard it is, but for me personally if said hypothetical coder never checked in a mistake, he would be my top pick. There are just too many failure points for me to recommend the whiteboard. I believe that they provide no use in an interview and I believe they hurt companies more that employees.
Organizations that utilize the whiteboard are doing themselves a disservice because they are setting introverted people up in a situation where a portion of them are naturally going to fail, in a situation that send such a strong false signal that all other evidence is negated. Then some of them are complaining that they cannot find talent. If a company is whiteboarding then they are filtering INTJ's out no two ways about it. As such, they are filtering out a lot of people that combine technical skills with creativity.
No matter how much interviewers say they are just looking for solutioning, they are not being honest with themselves because when they see code that is not their style or they see someone make stupid mistakes (which we all do) on the whiteboard, it embeds in the mind. It such an overpowering signal that it outweighs all other evidence. An INT already knows this and would therefore prefer to present finished code, not how you get there, which finished code is all that really matters, you can walk them through how they arrived at the finished code, and they will have a far more competent answer because they are not trying to code and drag people along with them at the same time.
Even if a developer makes 20 mistakes, and uses the compiler/debugger to catch them, then fixes them, if the code that they are going to check in is always clean, is his process bad? on the whiteboard it is, but for me personally if said hypothetical coder never checked in a mistake, he would be my top pick. There are just too many failure points for me to recommend the whiteboard. I believe that they provide no use in an interview and I believe they hurt companies more that employees.
Organizations that utilize the whiteboard are doing themselves a disservice because they are setting introverted people up in a situation where a portion of them are naturally going to fail, in a situation that send such a strong false signal that all other evidence is negated. Then some of them are complaining that they cannot find talent. If a company is whiteboarding then they are filtering INTJ's out no two ways about it. As such, they are filtering out a lot of people that combine technical skills with creativity.
"allows you to see how some one approaches a problem"
Although the whiteboard is good for some purposes, the author specifically mentions "brainfuck" coding questions.
Having them do those on the whiteboard really only tests how the think on their feet in high pressure situations... Something not really required of 99% of coders.
Although the whiteboard is good for some purposes, the author specifically mentions "brainfuck" coding questions.
Having them do those on the whiteboard really only tests how the think on their feet in high pressure situations... Something not really required of 99% of coders.
While that may be true, if you're really top talent as he's suggesting he is, you probably aren't part of that 99% either. And that's the problem I have with the article in general. It has a very prima donna attitude to it. As in: "I'm so good you need to soften the interview for me and the fact I'm demanding that is proof enough that I'm that good."
That's not what I meant.
White-boarding in a high pressure situation is simply a different "skill"... I think it's fair to say that most coders are "internal" thinkers / introverts, that do their best work when they have a chance to quietly think about a problem.
On the whiteboard, you're asking them to do the exact opposite.
Eliminating the "brainfuck" white-board questions doesn't mean you need to soften the interview. You can actually give a more substantial take-home problem to your final round of candidates.
White-boarding in a high pressure situation is simply a different "skill"... I think it's fair to say that most coders are "internal" thinkers / introverts, that do their best work when they have a chance to quietly think about a problem.
On the whiteboard, you're asking them to do the exact opposite.
Eliminating the "brainfuck" white-board questions doesn't mean you need to soften the interview. You can actually give a more substantial take-home problem to your final round of candidates.
Well, brainfuck questions in general aren't all that useful, whether you do them on the board or take it home. But if you can't do basic stuff on a whiteboard or a sheet of paper, that's a red flag to me.
But, I'm biased. I would never do a take home interview shrug
But, I'm biased. I would never do a take home interview shrug
The original post, and my comment, mention nothing about "basic stuff" on the whiteboard.
Yes anyone who has aspirations to be a professional in this industry really should be happy to take a problem and come up with some outline solutions/approaches on the fly.
Though as my first job was at a RnD Organisation this was expected from everyone from the shop guys to the Boss (the then president of the mechanicals) I might have a higher expectaion than most :-)
Though as my first job was at a RnD Organisation this was expected from everyone from the shop guys to the Boss (the then president of the mechanicals) I might have a higher expectaion than most :-)
"pair programming is an excellent way for both parties to get a sense of what it is like to work together." So true and if you can convince a developer to do this, power to you. We are an early stage startup and we are two for two with this method and have carved out a strong team.
I would argue that high salary requirements can be negotiated if you can frame a strong vision. Developers are not investment bankers and often are not doing this for the money alone. Often, good developers are looking for a place where their voice has as much weight as anyone else and where they can grow with company. We wouldn't have it any other way.
I would argue that high salary requirements can be negotiated if you can frame a strong vision. Developers are not investment bankers and often are not doing this for the money alone. Often, good developers are looking for a place where their voice has as much weight as anyone else and where they can grow with company. We wouldn't have it any other way.
In case anyone is interested, I've written another piece on how to attract technical co-founders:
http://news.ycombinator.com/item?id=3521476
http://news.ycombinator.com/item?id=3521476
Hi Fred. I would love to hear about examples of small companies that got it right. Have any stood out? What hiring techniques would make you happy?
Hi Richard,
To me personally, not requiring on-the-spot coding tests would make me happy. Compare the following two scenarios:
Company A: During the interview, I was asked to write a short ruby method to solve a problem on the whiteboard. I didn't really do well because I couldn't remember some APIs. Though in the end I still got the job, so I don't really know how effective the whiteboard coding test was.
Company B: During the interview, I was asked to explain the coding logic to solving a problem in Backbone JS on the whiteboard. I was much more comfortable with that, as I didn't have to remember exact APIs or methods to use. During this process, the interviewers got a good sense of my technical ability not just in Backbone JS, but also in JavaScript and MVC in general.
To me personally, not requiring on-the-spot coding tests would make me happy. Compare the following two scenarios:
Company A: During the interview, I was asked to write a short ruby method to solve a problem on the whiteboard. I didn't really do well because I couldn't remember some APIs. Though in the end I still got the job, so I don't really know how effective the whiteboard coding test was.
Company B: During the interview, I was asked to explain the coding logic to solving a problem in Backbone JS on the whiteboard. I was much more comfortable with that, as I didn't have to remember exact APIs or methods to use. During this process, the interviewers got a good sense of my technical ability not just in Backbone JS, but also in JavaScript and MVC in general.
Are both fundamentally not the same thing? I understand that company A may have made you a bit uncomfortable, but perhaps their intent was the same as company B's? That is, to see how you logic out a problem. You did say that you got the job in the end, so clearly the point wasn't to see if you knew some APIs by memory as a make-or-break situation.
There's also some benefits that can be gained by company A's method ostensibly over company B's. You get to see their response to the discomfort of not remembering an API. You get to understand their problem solving abilities, not in terms of code, but in terms of interpersonal relationships. In a recent article I read, someone had suggested offering a code prompt and then challenging the applicant to find the bug on completion (even if there wasn't one), simply to see how they respond when faced with that challenge.
Naturally it may not be perfect, it's just hard to really quantify which one is "better" if they're arriving at the same end -- a presumably competent engineer being hired. Are companies not interested in hiring people with both interpersonal skills and technical ability, after all?
There's also some benefits that can be gained by company A's method ostensibly over company B's. You get to see their response to the discomfort of not remembering an API. You get to understand their problem solving abilities, not in terms of code, but in terms of interpersonal relationships. In a recent article I read, someone had suggested offering a code prompt and then challenging the applicant to find the bug on completion (even if there wasn't one), simply to see how they respond when faced with that challenge.
Naturally it may not be perfect, it's just hard to really quantify which one is "better" if they're arriving at the same end -- a presumably competent engineer being hired. Are companies not interested in hiring people with both interpersonal skills and technical ability, after all?
Fair point. :) However, I would argue that in company A's scenario, the discomfort I got from not remembering the APIs would never happen in real world situations where API documentation and Google/StackOverflow are readily available.
Ideally, if one does really well in whiteboard situations then great. But similar to how some engineers give terrible public talks, we can't discount the ones that do get nervous in those situations.
Ideally, if one does really well in whiteboard situations then great. But similar to how some engineers give terrible public talks, we can't discount the ones that do get nervous in those situations.
Having never experienced it, can someone tell me what the point of having the applicant code on a whiteboard instead of on a computer is (Maybe with screen-sharing so the interviewer doesn't have to be looking over the applicant's shoulder)?
It really depends what you are testing for, but using a whiteboard lets you get away from the details of things like syntax errors and language particulars and let's you focus on solving a problem at a much higher level of abstraction. Basically it comes down to, do you want to see if the person can write good Ruby code (use a computer) or to you want to see if a person can come up with clever abstract algorithms to solve hard problems (use a whiteboard).
But I come up with all my clever abstract algorithms in the shower - usually while I am thinking of something else. I don't think being "clever" on demand is the mark of a good developer. I would prefer they look at some of my available open source code - that gives a much better reflection of my abilities.
Last fall, I had an interview where they asked me to reverse a linked-list in python and to filter out every unique items of a list. I failed because I did not remember the API by heart and they tried to impress me with a python one-liner that would do the job for me. Not so sure if the interview was for me or for them... Complete waste of time.
They wouldn't allow you to substitute your own implementation? That's rather odd. I'd be more interested in whether you knew how to traverse a linked list than I would be in any knowledge of the Python API.
I don't think reversing a link-list and removing unique items requires any special Python knowledge. No magical API needs to be used, just the basics.
A white board interview shouldn't exclude naive implementations. After the naive implementation has been done it's reasonable to ask for less naive approaches.
Also the continual whining about white board interviews is tiresome. It's a fact of life that exists in almost any tech interview you do. Practice, get your friends to bust your balls before the fact, write itoa on a white board or detect a loop in a singly linked list and move on with your life.
At some point if you're going to get paid you need to demonstrate your ability, reality says the potential employer decides what type of proof is required.
A white board interview shouldn't exclude naive implementations. After the naive implementation has been done it's reasonable to ask for less naive approaches.
Also the continual whining about white board interviews is tiresome. It's a fact of life that exists in almost any tech interview you do. Practice, get your friends to bust your balls before the fact, write itoa on a white board or detect a loop in a singly linked list and move on with your life.
At some point if you're going to get paid you need to demonstrate your ability, reality says the potential employer decides what type of proof is required.
I did some pseudo-code to show them how I would do it. They kept telling me to code it. As for the filtering, I did my own implementation with a loop but I guess the ultimate goal of that question was to see if I knew the one-liner..
I've never understood the whole emphasize on testing technical skills during the interview phase. In most countries it's either easy to fire someone, or you have legal probation period.
One month of employment should be enough to find out if someone has the practical skills they've expressed in the interview, and catch the 1% that can bullshit their way through.
Of course that 1% becomes a lot bigger number if the interviewer doesn't know what they are talking about. But that's not a problem you can solve with tests.
One month of employment should be enough to find out if someone has the practical skills they've expressed in the interview, and catch the 1% that can bullshit their way through.
Of course that 1% becomes a lot bigger number if the interviewer doesn't know what they are talking about. But that's not a problem you can solve with tests.
Some people are good at talking about coding (or pen testing, or sysadmining, or architecture, or...) but when you put them in front of a blank black screen with a cursor they're incapable of making forward progress.
It's five figures cheaper to ask everyone to FizzBuzz than it is to fire someone even after a week, and God help you if they're in a protected class.
It's five figures cheaper to ask everyone to FizzBuzz than it is to fire someone even after a week, and God help you if they're in a protected class.
Sadly, I've been that person. I read pg essays, blogs about programming and watched various videos about programming for about 3 years before finally deciding to leave my relatively comfortable small-business as an EFL cram-school partner and start actually coding stuff. People have been absolutely amazed by how little I know compared to how much I sound like I know.
Fizzbuzz wouldn't have filtered me out of getting my first (and current) real software job, but any sort of Google/MS-style interview would have. It may not be the wisest thing to admit to on the internet, but I'm basically in the opposite boat of your general target on HN: I'm fine with selling, but limited (non-EFL) skills to sell!
That, along with seeing numerous senior iOS dev applicants struggle both with Fizzbuzz and with Macs in general, has convinced me that it's more than worth it to test candidates without code portfolios.
Fizzbuzz wouldn't have filtered me out of getting my first (and current) real software job, but any sort of Google/MS-style interview would have. It may not be the wisest thing to admit to on the internet, but I'm basically in the opposite boat of your general target on HN: I'm fine with selling, but limited (non-EFL) skills to sell!
That, along with seeing numerous senior iOS dev applicants struggle both with Fizzbuzz and with Macs in general, has convinced me that it's more than worth it to test candidates without code portfolios.
The opposite, of course, is true. I've never been in a situation in the real world where, given a problem, I jumped immediately to a white board and felt I had to come up with a solution while being tested by people I didn't know. I have, however, had the joy of being given questions that could literally not happen, with requirements that cannot be met, all with the intent on seeing how I can think. Oh, and with no time to think, either.
If you really want to test how someone thinks over questions like that, actually give them time. Give them the questions before they show up for the interview. Then go over their solutions. They aren't under pressure, they give you a thoughtful answer, and you get a better sense of what they can do in the real world.
If you really want to test how someone thinks over questions like that, actually give them time. Give them the questions before they show up for the interview. Then go over their solutions. They aren't under pressure, they give you a thoughtful answer, and you get a better sense of what they can do in the real world.
Just last night I took an online test as a prescreen for a job I'm interested in. I guess this firm is C++ heavy so, of course, the test was entirely on C++. No algorithms, no problem solving. 30 questions with a max of 3 minutes per question.
I know enough C++ to write a decent GUI for a client (I mostly code in Qt, which is what hooked my resume in the first place). This test covered every possible wacky thing in the language, stuff I have no idea what to do with and is probably never even used in this company's day-to-day work. Nested namespaces? Template specialization?
Why do they do this? I'm totally discouraged from continuing onward with these people. I know I can do a perfectly fine job with this group. I think I'm going to take C++ off my resume and just give up looking for work in that arena.
I know enough C++ to write a decent GUI for a client (I mostly code in Qt, which is what hooked my resume in the first place). This test covered every possible wacky thing in the language, stuff I have no idea what to do with and is probably never even used in this company's day-to-day work. Nested namespaces? Template specialization?
Why do they do this? I'm totally discouraged from continuing onward with these people. I know I can do a perfectly fine job with this group. I think I'm going to take C++ off my resume and just give up looking for work in that arena.
In the US, it's easy to do a contract-to-hire. I know at least a few companies that bring in most of their new people this way.
I've always admired the 37Signals approach of having potential hires do a short contract.
I've always admired the 37Signals approach of having potential hires do a short contract.
"The technical questions should focus on the logic behind a candidate’s solutions, not what functions or libraries a candidate may or may not remember from the API documentation."
Finally someone said it. I'm an Asp.Net dev by trade, yet I don't even use or know 80% of the available functionality. Yet, interviewers are very adamant on you knowing this.
What it seems is that they don't understand that a lot of solutions can be solved purely with logic, not by some specific function, library, etc provided by the framework. I don't need to know the full scope of Asp.Net because I'm able to hack up a quick and logically sound solution. The thought process from A to B to C is vastly important compared to the API. In layman's speaking, Batman's brain is much more useful than his utility belt.
And let say you do need those items to complete a task, Google is always there to help. I sometimes wish that they would test out your Googling skills instead of asking you a framework question.
Finally someone said it. I'm an Asp.Net dev by trade, yet I don't even use or know 80% of the available functionality. Yet, interviewers are very adamant on you knowing this.
What it seems is that they don't understand that a lot of solutions can be solved purely with logic, not by some specific function, library, etc provided by the framework. I don't need to know the full scope of Asp.Net because I'm able to hack up a quick and logically sound solution. The thought process from A to B to C is vastly important compared to the API. In layman's speaking, Batman's brain is much more useful than his utility belt.
And let say you do need those items to complete a task, Google is always there to help. I sometimes wish that they would test out your Googling skills instead of asking you a framework question.
I recently interviewed with a startup. They had all 6 engineers interview me one by one. Each interview had a whiteboard/tech question component. At no point in time was I restricted to a language or expected to use an API. The questions measured an understanding of algorithms (data structures and big-O notation), an understanding of operating systems (concurrency, memory allocation, etc.) and object oriented programming.
There was even a gross bithack, which I really struggled with. After a half hour the interviewer was laughing, saying "I love this problem because everyone just goes bananas with it". It wasn't to make me look stupid, it was to see how I approached a problem that was apparently unsolvable.
With each question I was free to ask any questions, whether it was a clarification on pthreads or a question about problem constraints. I did not feel like my "off the cuff" knowledge of programming was being tested, but rather my experience and understanding of programming.
My only complaint with the interview was that it was long (11am - 7pm with a lunch break). That was mostly due to the fact they wanted me to meet with the entire team one at a time, as well as the fact that I was only in town for the day.
There was even a gross bithack, which I really struggled with. After a half hour the interviewer was laughing, saying "I love this problem because everyone just goes bananas with it". It wasn't to make me look stupid, it was to see how I approached a problem that was apparently unsolvable.
With each question I was free to ask any questions, whether it was a clarification on pthreads or a question about problem constraints. I did not feel like my "off the cuff" knowledge of programming was being tested, but rather my experience and understanding of programming.
My only complaint with the interview was that it was long (11am - 7pm with a lunch break). That was mostly due to the fact they wanted me to meet with the entire team one at a time, as well as the fact that I was only in town for the day.
Since going on the job market in November, I've been exercising an experimental policy: Interview with basically anyone (basic constraints apply - has to actually be a job I can do, a language I can speak, etc). It's been enlightening, especially because I'm at best a middling engineer (not a 'rockstar').
Virtually every company I've interviewed with has had tremendously awful HR, committing dozens of enormous mistakes that would have led me to abandon any interest in working for them (if not for my above-stated rule). I've learned a lot of interesting things from this experiment, but I think the biggest two are these:
A lot of companies screw up hiring from the HR side, but in general, the ones who mess up more than once tend to mess up a lot.
And in general, the ones with big money and big HR departments seem to be worse at interviewing and hiring than the tiny scrappy ones that can't even afford an HR department.
Just to give a few illustrative examples from the maybe 50 or so companies I've dealt with so far:
One big local developer scheduled a day-long interview that spanned lunch (starting around 11:30am and ending at 5pm), and knew that I had to drive an hour+ in traffic to get there. There was no lunch scheduled, they had me sit in a room and interview all day, even after I pointed out how little sense this made to the hiring director while I was on-site. They did not reimburse me for parking at their facility. This on-site was after a long series of successful phone screens. After the on-site, they told me personally that they would contact me with an offer, and then proceeded to completely ignore me for 2 months before a person I had never met before finally responded with something cryptic about an internal reorganization.
Another up-and-coming developer repeatedly made basic scheduling mistakes that made it near impossible to actually interview with them. Their hiring manager failed to e-mail me to notify me that a director would be calling in the early morning and also failed to tell me who he was, so when I got an unsolicited call from a stranger in the middle of a night's sleep, I asked if I could return his call, and he said 'no, I'm too busy'. This pattern was reinforced when they scheduled my on-site interview to begin 45 minutes after my flight landed (in a city where it took over an hour of cab travel just to reach their office). They compounded this by having me wait an hour and a half in the lobby before actually starting the interview, and having me interviewed by employees who clearly had not even been told my name.
The technical director of one well-funded studio contacted me out of the blue to ask if I was on the market and solicit my resume. After providing it, he sent me their 'engineering pre-screen', a series of around 10 moderately complex engineering and math problems that took me around 8 hours to complete (and probably would have taken someone more skilled at least 4 hours). A week after I submitted the completed screen, the technical director responded with a long diatribe about how my solutions to simple algorithmic problems should have used specific SIMD instruction sets or optimized for corner cases that were not specified, and commented that my lack of certain domain-specific skills made me an unsuitable candidate for a job he had told me nothing about.
In general, it feels like the average company hiring developers doesn't have a clue how to do it. It fills me with terror to think of how many potentially awesome candidates are falling at the wayside due to simple mistakes, and the idea that those great candidates end up at mediocre companies only because of HR mistakes is a rough one. Worse still, it's extremely rare for a company's HR department to solicit feedback on the hiring process - even companies that make me offers tend not to ask how it went. It seems likely that they do not make any effort to improve.
Virtually every company I've interviewed with has had tremendously awful HR, committing dozens of enormous mistakes that would have led me to abandon any interest in working for them (if not for my above-stated rule). I've learned a lot of interesting things from this experiment, but I think the biggest two are these:
A lot of companies screw up hiring from the HR side, but in general, the ones who mess up more than once tend to mess up a lot.
And in general, the ones with big money and big HR departments seem to be worse at interviewing and hiring than the tiny scrappy ones that can't even afford an HR department.
Just to give a few illustrative examples from the maybe 50 or so companies I've dealt with so far:
One big local developer scheduled a day-long interview that spanned lunch (starting around 11:30am and ending at 5pm), and knew that I had to drive an hour+ in traffic to get there. There was no lunch scheduled, they had me sit in a room and interview all day, even after I pointed out how little sense this made to the hiring director while I was on-site. They did not reimburse me for parking at their facility. This on-site was after a long series of successful phone screens. After the on-site, they told me personally that they would contact me with an offer, and then proceeded to completely ignore me for 2 months before a person I had never met before finally responded with something cryptic about an internal reorganization.
Another up-and-coming developer repeatedly made basic scheduling mistakes that made it near impossible to actually interview with them. Their hiring manager failed to e-mail me to notify me that a director would be calling in the early morning and also failed to tell me who he was, so when I got an unsolicited call from a stranger in the middle of a night's sleep, I asked if I could return his call, and he said 'no, I'm too busy'. This pattern was reinforced when they scheduled my on-site interview to begin 45 minutes after my flight landed (in a city where it took over an hour of cab travel just to reach their office). They compounded this by having me wait an hour and a half in the lobby before actually starting the interview, and having me interviewed by employees who clearly had not even been told my name.
The technical director of one well-funded studio contacted me out of the blue to ask if I was on the market and solicit my resume. After providing it, he sent me their 'engineering pre-screen', a series of around 10 moderately complex engineering and math problems that took me around 8 hours to complete (and probably would have taken someone more skilled at least 4 hours). A week after I submitted the completed screen, the technical director responded with a long diatribe about how my solutions to simple algorithmic problems should have used specific SIMD instruction sets or optimized for corner cases that were not specified, and commented that my lack of certain domain-specific skills made me an unsuitable candidate for a job he had told me nothing about.
In general, it feels like the average company hiring developers doesn't have a clue how to do it. It fills me with terror to think of how many potentially awesome candidates are falling at the wayside due to simple mistakes, and the idea that those great candidates end up at mediocre companies only because of HR mistakes is a rough one. Worse still, it's extremely rare for a company's HR department to solicit feedback on the hiring process - even companies that make me offers tend not to ask how it went. It seems likely that they do not make any effort to improve.
Whenever I hear about how horrible some company does something, I always think "Opportunity!" Just not sucking at something puts your scrappy little company ahead of your more impressive competitors.
I agree with this, I am a vocal advocate of getting rid of the whiteboard. These tests reflect the skills far more than a whiteboard in an interview. That being said, you should only ask developers that are in the final round of hiring to do this, if it is a filter to even get in the door for an interview then you are going to filter out a lot of good candidates. Developers are smart, more importantly most of them are good at simple math. They know hiring is a numbers game and are not going to waste time on a project, that does not even guarantee an interview. I am not a big fan of asking people to do stuff for free, because it is akin to saying here do some work, because we don't know how to spot good talent. It's not the message you want to be impressing on developers. As such, if they are in the final rounds, you have at least some rapport built with them, and can explain why it is important that they perform the task, and they get the confirmation that they are within sight of the goal, before they are asked to expend the extra energy.