Ask HN: Is it typical to have a coding project for interview?
12 comments
What kinds of projects are these? Are they asking you to do work for them, or are they asking for you to demonstrate that you can do work for them?
Tipoffs for the former: you're building features that they don't have, or fixing bugs that they currently do have, working alongside one of their team members, &c.
Tipoffs for the latter: you're working on a toy problem, filling out the skeleton of an application, building some kind of system that they won't ever deploy.
The latter kind of project is pretty reasonable, assuming the rest of the interview process is calibrated to account for how much time you took. For instance, if you're doing 3-10 hours of coding (10 hours is a lot, by the way) just to get a phone screen, or to go to the first of several rounds of onsite interviews, that's an unreasonable ask. But on the other hand, if you're being asked to spend a couple hours doing in the comfort of your own home what you would have to do onsite anyways in a real interview, that's very reasonable.
Tipoffs for the former: you're building features that they don't have, or fixing bugs that they currently do have, working alongside one of their team members, &c.
Tipoffs for the latter: you're working on a toy problem, filling out the skeleton of an application, building some kind of system that they won't ever deploy.
The latter kind of project is pretty reasonable, assuming the rest of the interview process is calibrated to account for how much time you took. For instance, if you're doing 3-10 hours of coding (10 hours is a lot, by the way) just to get a phone screen, or to go to the first of several rounds of onsite interviews, that's an unreasonable ask. But on the other hand, if you're being asked to spend a couple hours doing in the comfort of your own home what you would have to do onsite anyways in a real interview, that's very reasonable.
I was looking for a job last year and interviewed at multiple places (https://matt.sh/searching-2013), so I think I have a recent multi-employer perspective here.
The biggest flaw in the entire current "standard accepted interview procedure" (read: cargo cult hiring process) is: they don't care about the interviewee's time. The secondary flaw is: the tech people designing interviewing processes don't understand the nature of "work," "thought," or what makes valuable employees, well, valuable.
Companies enjoy using coding tests (sometimes lengthy) as a simple filter, but they don't really care about it. Every company I did a "coding test" for accepted me for an interview, but at no point did any of them ever mention or talk about my submission. I could have plagiarized every submission whole from some random site online. They'd never find out since they never asked me to talk through the work. One company, after 10 hours of my working on their exercise, even told me "sorry, we sent you the wrong one. do this other one instead."
If you ask companies why they have their interview processes set up the way they do, they'll probably just say "because that's how company Y does it too." They want to be part of a successful pattern without understanding the underlying meaning. It's like a sales person using a dozen buzzword acronyms without understanding how anything fits together.
But, if they do want to continue handing out 10 hour coding tests, society has a generally accepted way of handling issues like this: pay for time. If there is a power imbalance (employer vs. interviewee), the more powerful party should refrain from exploiting the weaker party without compensation.
If coding tests really get annoying during your search, just launch a startup to destroy all the companies with bad hiring practices. No fury like an interviewee scorned.
The biggest flaw in the entire current "standard accepted interview procedure" (read: cargo cult hiring process) is: they don't care about the interviewee's time. The secondary flaw is: the tech people designing interviewing processes don't understand the nature of "work," "thought," or what makes valuable employees, well, valuable.
Companies enjoy using coding tests (sometimes lengthy) as a simple filter, but they don't really care about it. Every company I did a "coding test" for accepted me for an interview, but at no point did any of them ever mention or talk about my submission. I could have plagiarized every submission whole from some random site online. They'd never find out since they never asked me to talk through the work. One company, after 10 hours of my working on their exercise, even told me "sorry, we sent you the wrong one. do this other one instead."
If you ask companies why they have their interview processes set up the way they do, they'll probably just say "because that's how company Y does it too." They want to be part of a successful pattern without understanding the underlying meaning. It's like a sales person using a dozen buzzword acronyms without understanding how anything fits together.
But, if they do want to continue handing out 10 hour coding tests, society has a generally accepted way of handling issues like this: pay for time. If there is a power imbalance (employer vs. interviewee), the more powerful party should refrain from exploiting the weaker party without compensation.
If coding tests really get annoying during your search, just launch a startup to destroy all the companies with bad hiring practices. No fury like an interviewee scorned.
Yes, I agree strongly that this is an issue. We do a series of work-sample challenges at Matasano, some of which involve significant coding (reverse-engineering a protocol and then building an attack tool for it). The process would not work at all if we didn't have a rubric for evaluating them mechanically --- the person who evaluates the challenge doesn't even need to know who the candidate is.
In fact, now that you mention this, I think I'll try to make sure that becomes a rule --- we'll try to blind challenge grading.
Work-sample testing is very, very powerful, but there are important nuances to actually executing them.
In fact, now that you mention this, I think I'll try to make sure that becomes a rule --- we'll try to blind challenge grading.
Work-sample testing is very, very powerful, but there are important nuances to actually executing them.
I've been flat-out refusing pre-interview coding assignments or tests. It's very disrespectful of a candidate's time.
I used to do every test, but the conversion rate was so low that I no longer bother.
I currently have a job and I'm only spending 1-2 hours a week looking. Why should I waste 1-5 hours for the chance to MAYBE get an interview? Will I spend my limited time on every stupid test, or on the guy who respects me as a professional?
If you're that serious about try-before-you-buy hiring, make it a consulting project and pay me for my time.
I used to do every test, but the conversion rate was so low that I no longer bother.
I currently have a job and I'm only spending 1-2 hours a week looking. Why should I waste 1-5 hours for the chance to MAYBE get an interview? Will I spend my limited time on every stupid test, or on the guy who respects me as a professional?
If you're that serious about try-before-you-buy hiring, make it a consulting project and pay me for my time.
These pre-interview assignments actually end up saving everyone a lot of time in my experience, and go further to respect the candidate (letting them use their own editors, highlight their own strengths, etc.) Try-before-you-buy is actually the entire point of an interview, you're just not going to escape that unless you are already close with the interviewers.
My company has a coding practical. It is designed to be done in about an hour or so. http://cloudspace.com/hiring/ (description at the bottom).
Ultimately it's going to come down to a harsh reality: how bad do you want that particular job? If you really want a particular job and they tell you that they won't hire anyone who doesn't wear a funny hat to the interview, you'll be browsing Amazon's hat section (unless you are pre-equipped with funny hats, in which case you may be an exceptional hire).
Companies equally have to make a decision although they may not realize it. If their interview process seems more painful than necessary, a well-qualified person (who is in demand) may choose to apply somewhere else.
Ultimately it's going to come down to a harsh reality: how bad do you want that particular job? If you really want a particular job and they tell you that they won't hire anyone who doesn't wear a funny hat to the interview, you'll be browsing Amazon's hat section (unless you are pre-equipped with funny hats, in which case you may be an exceptional hire).
Companies equally have to make a decision although they may not realize it. If their interview process seems more painful than necessary, a well-qualified person (who is in demand) may choose to apply somewhere else.
I've chosen not to take jobs because their interview process wasn't rigorous enough.
Working with good developers is one of the single biggest job satisfaction metrics I've found and I don't know of any interview process that works repeatedly better than having people create code at home for review. So I view it as a red flag if companies don't ask for this.
That said 5 hours is too much. 10 is overbearing.
Working with good developers is one of the single biggest job satisfaction metrics I've found and I don't know of any interview process that works repeatedly better than having people create code at home for review. So I view it as a red flag if companies don't ask for this.
That said 5 hours is too much. 10 is overbearing.
If their interview process seems more painful than necessary, a well-qualified person (who is in demand) may choose to apply somewhere else.
Things are great for job candidates now, but I worry what happens if this attitude you describe gets applied if/when things go bad, like they have in the past and might again.
Things are great for job candidates now, but I worry what happens if this attitude you describe gets applied if/when things go bad, like they have in the past and might again.
I fell like it is common, not sure I would say it is typical. I avoid it at all costs. Usually, if I complete the project the and the host company doesn't like it, I don't get any feedback on the code and my time gets mostly wasted. If companies would give some degree of feedback on the code I'd be more into it. "No" isn't the kind of feedback that I value.
I try to stick closer to companies that have a balanced interview process and a smaller live coding challenge, think 20-45 min can really suss out a person's technical level anyways.
Just my $.02
I try to stick closer to companies that have a balanced interview process and a smaller live coding challenge, think 20-45 min can really suss out a person's technical level anyways.
Just my $.02
I've done a lot of work in this area (including quais-rigorous experimentation with hiring pipeline processes) and I have found that live coding challenges are anti-correlated with good hires. I was surprised by this as I quite like them.
What people like us, who do like them, don't realize is how much stress it puts on people who don't like them. This stress does not usually have any real world correlation with the job you are hiring for, so it severely biases against candidates who would be great but fail due to stress. It also biases towards people who can come up with glib fast solutions, which also frequently isn't correlated with being able to solve long complex problems.
What people like us, who do like them, don't realize is how much stress it puts on people who don't like them. This stress does not usually have any real world correlation with the job you are hiring for, so it severely biases against candidates who would be great but fail due to stress. It also biases towards people who can come up with glib fast solutions, which also frequently isn't correlated with being able to solve long complex problems.
It's getting much more common among places that care about getting good developers. I've implemented this practice myself where I'm involved in hiring, although when it occurs in the process has varied. It's always been after the initial phone screen, but before/after on-site has varied. I tend to prefer it after, e.g., phone screen, short on-site to suss out major compatibility issues, take-home drill(s), then longer on-site if the code passes muster.
You can find out a lot about how a person thinks about programming with exercises that take actual time. Twenty minutes with FizzBuzz (which I still get on occasion) doesn't speak towards anything beyond "can monkey type?"
I've made it a rule to provide fairly detailed feedback if I get to the point of giving someone an exercise. Not all places do that, and I think it's too bad. If someone goes to the trouble of spending their free time on this I feel I owe them at least some of mine.
You can find out a lot about how a person thinks about programming with exercises that take actual time. Twenty minutes with FizzBuzz (which I still get on occasion) doesn't speak towards anything beyond "can monkey type?"
I've made it a rule to provide fairly detailed feedback if I get to the point of giving someone an exercise. Not all places do that, and I think it's too bad. If someone goes to the trouble of spending their free time on this I feel I owe them at least some of mine.
When you got hired as an interviewer, did they make you interview a couple of people at home before a phone screening? You know, to check out your interviewing skills and to see if you're good or not.
Also, do you do this with your doctors and lawyers too? Give them homework to see if they are worth the money?
Also, do you do this with your doctors and lawyers too? Give them homework to see if they are worth the money?
As I stated, my take-home tests are post-phone screen. The idea of hiring a programmer without seeing non-trivial code is stupid. If a dev has a body of publicly-available work IMO a take-home test might not be necessary, but it depends on the nature of those works.
For doctors and lawyers there are known sources of publicly-available information regarding their history, background, cases, etc. and the referral system is more formalized than for developers.
For doctors and lawyers there are known sources of publicly-available information regarding their history, background, cases, etc. and the referral system is more formalized than for developers.
At a start up I applied to last time I was seriously looking, they asked me to implement a program to play hangman. It was a fun puzzle; certainly didn't take me 3 hours but it did eat a chunk of time. I don't object to this kind of thing, within reason. As tptacek and others caution, they shouldn't be farming out actual work, though.
In my opinion, the amount of time and effort invested by the candidate should be matched with equal effort by the prospective employer. If you ask me to spend 10 hours proving myself to you with a coding sample, you have better be prepared to spend 10 hours evaluating me and selling me on your company and the benefits of working there. Alternatively, you can pay me for my time, as a gesture showing that you respect that my time has value to me.
I won't waste time coding toy programs if I don't think the employer is truly interested. And I can't gauge that until a phone interview has happened. So any company that requires a code challenge prior to a voice interview is effectively blacklisted.
I won't waste time coding toy programs if I don't think the employer is truly interested. And I can't gauge that until a phone interview has happened. So any company that requires a code challenge prior to a voice interview is effectively blacklisted.
10 hours is too much, but otherwise it serves as a nice screen.
NB: They should be investing as much time in you, not just using this as the first step in order to see who has the most time to waste.
NB: They should be investing as much time in you, not just using this as the first step in order to see who has the most time to waste.
We usually ask a candidate to built some toy application and describe his process rationale to use. But only after we had an actual interview. Depending on level of experience this will take him/her somewhere between 1 and 6 hours (if we suspect it will take longer we usually don't consider the candidate anymore).
Not really intended as a hiring criterium, more like a monitoring tool for actual experience, modus operandi, etc.
Not really intended as a hiring criterium, more like a monitoring tool for actual experience, modus operandi, etc.
This type of tests are often used to filter mindless drones working in monotonous companies. If you have the patience to work with no pay on an imaginary problem for 10 hours straight, then you're a perfect fit.
If you value your sanity, avoid companies that ask for it. Although they do seem to increase in numbers at an alarming rate.
If you value your sanity, avoid companies that ask for it. Although they do seem to increase in numbers at an alarming rate.
Yes it's fairly common. Its effectiveness as a screening tool is debatable, but yes I've encountered them quite a bit. 3-10 hours sounds longer than normal. Most are a couple hours tops.
Yes, it's fairly common. Some friends of mine at Amazon had to face this sort of projects as well.
Is that typical? And what will you do when they ask you to do that? Thanks.