I don't care about your code challenge(blog.mattbasta.com)
blog.mattbasta.com
I don't care about your code challenge
http://blog.mattbasta.com/i_dont_care_about_your_code_challenge.html
37 comments
Hahaha good going. Interesting story, thanks for sharing.
> I got pretty frustrated by the two "Senior" devs who seemed to take great pride in tearing my code and my decisions apart.
Two possibilities:
a) You got a glimpse of how your life on the job would look like.
b) Just like qualified specialists can be terrible interviewees, they can be terrible interviewers. They just screwed up.
Two possibilities:
a) You got a glimpse of how your life on the job would look like.
b) Just like qualified specialists can be terrible interviewees, they can be terrible interviewers. They just screwed up.
c) Push the interviewer to defend their decisions, and see how they react. If you can't defend your decision to intelligent people with strong opinions (!== assholes automatically), you won't make a good developer
[deleted]
Can resonate with the author and you. Had similar experiences.
> I basically said, "If standards are so important to you, then why are you holding people who don't even work for your company to a higher standard than those who actually do?"
Well, you want to hire people who are better than you, not worse than you. Hiring bar at a good company should go up, not down, as it grows.
Well, you want to hire people who are better than you, not worse than you. Hiring bar at a good company should go up, not down, as it grows.
Maybe they had a different definition of "validate." Did you ask your interlocutors what they did to validate their sites?
I'm amused that you take so personally the concept of a professional who's not familiar with your competencies trying to evaluate your ability. Which by the way is more than just pumping out code, it also involves defending difficult decisions to your peers. If you can't handle a little pressure from a code review, find a new career.
There's a difference between evaluating abilities and trying to engage him in a pissing contest. Interviews should determine if the candidate will deliver what he's being paid for, nitpicking and flamewars can be saved for future coffee break entertainment. "It's not my solution therefore it's shit" is not what code review should be about.
I think the problem can be that many times there aren't difficult decisions to be made to solve most coding challenges. Getting grilled over trivialities or obvious decisions makes the interviewer seem either incompetent or like a micro-manager, neither of which would make me seem happy to work with them.
Many years ago (wow...almost 10 now...I'm so old...) I was given a phone screen and at the end of the screen I was given a question to code a solution to and email it in. Since this was for a job across the country, I figure what the hell.
The problem was - A man has to get a fox, a chicken, and a sack of corn across a river. He has a rowboat, and it can only carry him and one other thing. If the fox and the chicken are left together, the fox will eat the chicken. If the chicken and the corn is left together, the chicken will eat the corn. Write a program that will help the man make the correct decision and explain the operational complexity.
I was given an hour to come up with a solution. I worked on it about half-an-hour coming up with the algorithm. And then for some reason my brain keyed in on the "write a program" part of the question. So I wrote the following in C++ (from memory so may not be exact) -
Must have been an OK answer because I got flown up and ended up working for them for 4 years (before a series of bad jobs that have finally lead back to something good).
The problem was - A man has to get a fox, a chicken, and a sack of corn across a river. He has a rowboat, and it can only carry him and one other thing. If the fox and the chicken are left together, the fox will eat the chicken. If the chicken and the corn is left together, the chicken will eat the corn. Write a program that will help the man make the correct decision and explain the operational complexity.
I was given an hour to come up with a solution. I worked on it about half-an-hour coming up with the algorithm. And then for some reason my brain keyed in on the "write a program" part of the question. So I wrote the following in C++ (from memory so may not be exact) -
int main(int argc, char** argv) {
std::cout
<< "Man takes chicken, leaves fox and corn, and returns" << std::endl
<< "Man takes fox, leaves corn, and returns with chicken" << std::endl
<< "Man leaves chicken and takes corn, leaves corn with fox, and returns" << std::endl
<< "Man takes chicken back across" << std::endl;
return 0;
}
Must have been an OK answer because I got flown up and ended up working for them for 4 years (before a series of bad jobs that have finally lead back to something good).
You have no idea how the "challenge" was completed. A candidate could have rent-a-coder-ed the project, refactored it a bit, and slapped their name on it.
Sure, if you ask no follow-up questions after the challenge is completed. However, with a few well-placed questions about the implementation and some inspection of the code, it'll be quite clear whether you're talking to the author or not.
We once gave a (non-web-dev) programming "challenge" (I thought it was a joke/insult given the candidate's claimed experience) to a senior dev candidate. Mind you, this was on-site. I even tested it by putting our newly hired grad to the test and he aced it within 20 minutes. We gave the candidate an hour to complete it (and added a 30 minute extension when he was struggling).
The challenge involved developing something against an SDK (interacting with a camera) and we gave him internet access and the SDK examples to work off of. In the end, he produced something not quite what we were asking, but he was sure to claim it as his work. Although it wasn't ideal, I was prepared to say that he "passed" the assignment, because he was close and I was sympathetic to factors of stress and nervousness.
As he was toured around the facility, I inspected the code and saw that he was completely passing off one of the SDK examples as his own work (diffs showed only two variable names had been changed). The reason his code didn't quite do what we wanted, is that he wasn't even able to tweak the SDK example to do what we asked. He was a complete fraud and never mind technical ability, we couldn't have dishonesty like that on the team.
Before that interview I thought those kind of challenges were a joke, but after this I saw their value. Of course, their value depends highly on how their are implemented and targeted, but dismiss their usefulness at your own peril.
Sure, if you ask no follow-up questions after the challenge is completed. However, with a few well-placed questions about the implementation and some inspection of the code, it'll be quite clear whether you're talking to the author or not.
We once gave a (non-web-dev) programming "challenge" (I thought it was a joke/insult given the candidate's claimed experience) to a senior dev candidate. Mind you, this was on-site. I even tested it by putting our newly hired grad to the test and he aced it within 20 minutes. We gave the candidate an hour to complete it (and added a 30 minute extension when he was struggling).
The challenge involved developing something against an SDK (interacting with a camera) and we gave him internet access and the SDK examples to work off of. In the end, he produced something not quite what we were asking, but he was sure to claim it as his work. Although it wasn't ideal, I was prepared to say that he "passed" the assignment, because he was close and I was sympathetic to factors of stress and nervousness.
As he was toured around the facility, I inspected the code and saw that he was completely passing off one of the SDK examples as his own work (diffs showed only two variable names had been changed). The reason his code didn't quite do what we wanted, is that he wasn't even able to tweak the SDK example to do what we asked. He was a complete fraud and never mind technical ability, we couldn't have dishonesty like that on the team.
Before that interview I thought those kind of challenges were a joke, but after this I saw their value. Of course, their value depends highly on how their are implemented and targeted, but dismiss their usefulness at your own peril.
This is a lopsided view. You don't indicate how long your "newly hired grad" was working there, whether the "SDK" was an in-house SDK or a third-party one, what the candidate's resume skills were in comparison to the kind of programming around the SDK your test was designed for, etc.
These are all relevant details, because except for the most trivial of SDKs and their appropriate domains, a thoughtful, skilled developer might spend an entire hour simply reading through the SDK and/or its documentation to get a handle on its capabilities.
So I don't get that your story is an anecdote in support of the potential usefulness of code challenges. Instead, what I gather from your story is that your company prefers candidates who practice cowboy, shoot-from-the-hip programming instead of thoughtfully approaching a new, unfamiliar SDK and pondering how it might be used to solve problems related to the domain it was built for, then applying this thoughtful process to provide a solution to a problem. In fact, if I'm given a non-trivial SDK and told "write a program that does X using this SDK; you have one hour", my response might be something very similar to the above, directed at the person giving the "assignment" out, followed by a polite but firm indication that the company doesn't meet my requirements.
These are all relevant details, because except for the most trivial of SDKs and their appropriate domains, a thoughtful, skilled developer might spend an entire hour simply reading through the SDK and/or its documentation to get a handle on its capabilities.
So I don't get that your story is an anecdote in support of the potential usefulness of code challenges. Instead, what I gather from your story is that your company prefers candidates who practice cowboy, shoot-from-the-hip programming instead of thoughtfully approaching a new, unfamiliar SDK and pondering how it might be used to solve problems related to the domain it was built for, then applying this thoughtful process to provide a solution to a problem. In fact, if I'm given a non-trivial SDK and told "write a program that does X using this SDK; you have one hour", my response might be something very similar to the above, directed at the person giving the "assignment" out, followed by a polite but firm indication that the company doesn't meet my requirements.
>You don't indicate how long your "newly hired grad" was working there, whether the "SDK" was an in-house SDK or a third-party one, what the candidate's resume skills were in comparison to the kind of programming around the SDK your test was designed for, etc.
I wanted to avoid writing a novel, so I left out some of those details you're interested in. Trust me, when this "test" was first suggested, I was wary of it myself and did everything in my power to make it very achievable. That's why I considered it a joke, it was not nearly difficult enough to assess the level of seniority this candidate was claiming. This was someone with 10+ years of development experience, in the same field, experience with the language and environment we used. On paper and even on the phone, he made himself sound like the perfect candidate. In fact, before the on-site interview, I thought he had it in the bag and was excited to get some help on the team.
The SDKs in question were for a camera (to get images from the camera) and OpenCV, which is well known in imaging circles. The test was to receive a video stream from the camera (there was an example showing how to do this and I made it clear I didn't mind if he made that example his starting point) and to subtract consecutive images to create an image representing the "difference" between the two images. Not a meaningful result and one could barely call it image processing, but it would show us the candidates abilities to interact with multiple well-documented SDKs to accomplish a very simple task. If someone who claims to have years of experience with image processing can't accomplish subtracting two consecutive images and drawing the result to the screen, given all the tools to do this, there's a big problem.
Our new grad was mostly a tester and an occasional developer. He had less than a year of experience. He completed in 20 minutes without supervision.
>Instead, what I gather from your story is that your company prefers candidates who practice cowboy, shoot-from-the-hip programming instead of thoughtfully approaching a new, unfamiliar SDK and pondering how it might be used to solve problems related to the domain it was built for, then applying this thoughtful process to provide a solution to a problem. In fact, if I'm given a non-trivial SDK and told "write a program that does X using this SDK; you have one hour", my response might be something very similar to the above, directed at the person giving the "assignment" out, followed by a polite but firm indication that the company doesn't meet my requirements.
While I am fully aware that there are severe limitations to the usefulness of a test like this, in this situation, it saved us from a terrible hire (this test wasn't the only indication that he wasn't who he claimed to be).
In all honesty, you sound like a fairly thorny/difficult person to work with and in your hypothetical scenario where the test caused you spurn the position, it would have inadvertently done its job.
I wanted to avoid writing a novel, so I left out some of those details you're interested in. Trust me, when this "test" was first suggested, I was wary of it myself and did everything in my power to make it very achievable. That's why I considered it a joke, it was not nearly difficult enough to assess the level of seniority this candidate was claiming. This was someone with 10+ years of development experience, in the same field, experience with the language and environment we used. On paper and even on the phone, he made himself sound like the perfect candidate. In fact, before the on-site interview, I thought he had it in the bag and was excited to get some help on the team.
The SDKs in question were for a camera (to get images from the camera) and OpenCV, which is well known in imaging circles. The test was to receive a video stream from the camera (there was an example showing how to do this and I made it clear I didn't mind if he made that example his starting point) and to subtract consecutive images to create an image representing the "difference" between the two images. Not a meaningful result and one could barely call it image processing, but it would show us the candidates abilities to interact with multiple well-documented SDKs to accomplish a very simple task. If someone who claims to have years of experience with image processing can't accomplish subtracting two consecutive images and drawing the result to the screen, given all the tools to do this, there's a big problem.
Our new grad was mostly a tester and an occasional developer. He had less than a year of experience. He completed in 20 minutes without supervision.
>Instead, what I gather from your story is that your company prefers candidates who practice cowboy, shoot-from-the-hip programming instead of thoughtfully approaching a new, unfamiliar SDK and pondering how it might be used to solve problems related to the domain it was built for, then applying this thoughtful process to provide a solution to a problem. In fact, if I'm given a non-trivial SDK and told "write a program that does X using this SDK; you have one hour", my response might be something very similar to the above, directed at the person giving the "assignment" out, followed by a polite but firm indication that the company doesn't meet my requirements.
While I am fully aware that there are severe limitations to the usefulness of a test like this, in this situation, it saved us from a terrible hire (this test wasn't the only indication that he wasn't who he claimed to be).
In all honesty, you sound like a fairly thorny/difficult person to work with and in your hypothetical scenario where the test caused you spurn the position, it would have inadvertently done its job.
I'm often surprised by what people will take umbrage at. It's one thing to decide that it's a seller's market, and that therefore you'll interview somewhere where they'll buy you lunch and offer you a six figure salary and generous share allocation after nothing more than a cozy chat. But it's quite another to become actively offended by the effrontery of a company demanding that you demonstrate the very skills they're hoping to hire you to apply on a daily basis!
What are people worried about? I have to say I can't help but assume the worst. Because what have they been doing for their career as a professional programmer, if when asked to actually write a program, they can't actually do it? Even with an example right in front of them?
Don't forget that in this particular example we have a supposed senior programmer! Somebody applying for this sort of job should, I feel, be able to actually program. Even if they've never seen the library before. In fact good candidates should be barely fazed by an unknown language! What is the "senior" bit for, if not this? At the very least, perhaps the candidate could admit that they seem to have found themselves interviewing for the wrong sort of role, and one that makes better use of their other skills would be more suitable? That might be a better tactic than storming off in a huff.
But I've never stormed out of an interview in a huff - so what do I know.
What are people worried about? I have to say I can't help but assume the worst. Because what have they been doing for their career as a professional programmer, if when asked to actually write a program, they can't actually do it? Even with an example right in front of them?
Don't forget that in this particular example we have a supposed senior programmer! Somebody applying for this sort of job should, I feel, be able to actually program. Even if they've never seen the library before. In fact good candidates should be barely fazed by an unknown language! What is the "senior" bit for, if not this? At the very least, perhaps the candidate could admit that they seem to have found themselves interviewing for the wrong sort of role, and one that makes better use of their other skills would be more suitable? That might be a better tactic than storming off in a huff.
But I've never stormed out of an interview in a huff - so what do I know.
There's a reasonable expectation to have to demonstrate one's skills in an interview. Whether any given test or set of questions actually is valid for doing that is another matter entirely.
Put another way: it's just as unreasonable for a company to expect a developer's skills to be measured by contrived tests or to demand they spend any unpaid time at all doing "work" (e.g. projects), as it is for a candidate to expect a company to effectively kiss his butt and lay out red carpet just because he decided to apply.
Put another way: it's just as unreasonable for a company to expect a developer's skills to be measured by contrived tests or to demand they spend any unpaid time at all doing "work" (e.g. projects), as it is for a candidate to expect a company to effectively kiss his butt and lay out red carpet just because he decided to apply.
I agree that there certainly are problems that would make a good "code challenge" question, but I'd argue that there are better tools for weeding out poor candidates. I also think that there's a big difference between on-site challenges and "here's what we'd like to have you do, call us back when you've got something" challenges.
Even "small" programming assignments take a lot of time. Several years ago, I applied for a job as a developer for a financial trading system. The take-home assignment was to build a small trading system that received buy and sell orders from the command line, and executed trades when the price matched. The input was just: buy/sell, price and number of shares.
This functionality is pretty simple, but it still took many hours to code. My feeling was that they thought it would perhaps take an hour to do. It was fun and interesting to do, but it took much longer than I thought. I think the companies giving the challenges should have their own engineers do the task, so they get a good handle on how much they are asking from the applicant. And of course it also depends on if it is a buyer's or seller's market (jobwise that is, not in terms of my "trading" system). If there is a lot of competition for developers, maybe they should give too big projects.
This functionality is pretty simple, but it still took many hours to code. My feeling was that they thought it would perhaps take an hour to do. It was fun and interesting to do, but it took much longer than I thought. I think the companies giving the challenges should have their own engineers do the task, so they get a good handle on how much they are asking from the applicant. And of course it also depends on if it is a buyer's or seller's market (jobwise that is, not in terms of my "trading" system). If there is a lot of competition for developers, maybe they should give too big projects.
Sigh - yet another hiring process complaint post. As an employer hopefully I can shed some light here.
a) If you are trying to hire 5 engineers, say, you need to talk to ~25 candidates at least. It's a competitive market. b) If you aim to talk to 25 candidates in a short period of time, you need to have SOME standard way to evaluate candidates if you don't want to just throw darts on the board. Being sloppy with process is bad for companies and the engineers and a waste of everyone's time. c) In order to create a standard process, you have a very limited # of options to work from. I've seen comments like "do a co-working day to get a real sense" or "do a contract and see them work with you daily" positioned as realistic options. They are not. If an engineer has an existing full-time job they don't have the time to do co-working days or contracts. d) The other options are 1) whiteboard interviews 2) coding challenges 3) some combination thereof. e) Lots of candidates hate whiteboard interviews. Lots of candidates hate coding challenges too. Neither is a perfect system but it's better than doing nothing or asking for co-working time. f) We use coding challenges at the top end but we time-box it to 3 hours. If someone asked us to pay for the time, we would happily do so. g) Our coding challenges have 3 different problems so candidates can pick whichever aligns best to skills. One problem is JavaScript-heavy and 2 others are Ruby/Rails-heavy. h) Several engineers on the team solved the coding challenges on a timed basis before we put them out to candidates. Doing otherwise would be stupid/unfair. i) In the few cases where we've been able to do 1-2 co-working days, we have offered to pay for the candidate's time. Not full freight consulting dollars but something that indicates our seriousness. j) In this case, the OP got caught by a dumb coding challenge to build a full website. But the headline paints with a broad brush because coding challenges by themselves are not evil. They are a tool that can be used well or poorly. That's all.
a) If you are trying to hire 5 engineers, say, you need to talk to ~25 candidates at least. It's a competitive market. b) If you aim to talk to 25 candidates in a short period of time, you need to have SOME standard way to evaluate candidates if you don't want to just throw darts on the board. Being sloppy with process is bad for companies and the engineers and a waste of everyone's time. c) In order to create a standard process, you have a very limited # of options to work from. I've seen comments like "do a co-working day to get a real sense" or "do a contract and see them work with you daily" positioned as realistic options. They are not. If an engineer has an existing full-time job they don't have the time to do co-working days or contracts. d) The other options are 1) whiteboard interviews 2) coding challenges 3) some combination thereof. e) Lots of candidates hate whiteboard interviews. Lots of candidates hate coding challenges too. Neither is a perfect system but it's better than doing nothing or asking for co-working time. f) We use coding challenges at the top end but we time-box it to 3 hours. If someone asked us to pay for the time, we would happily do so. g) Our coding challenges have 3 different problems so candidates can pick whichever aligns best to skills. One problem is JavaScript-heavy and 2 others are Ruby/Rails-heavy. h) Several engineers on the team solved the coding challenges on a timed basis before we put them out to candidates. Doing otherwise would be stupid/unfair. i) In the few cases where we've been able to do 1-2 co-working days, we have offered to pay for the candidate's time. Not full freight consulting dollars but something that indicates our seriousness. j) In this case, the OP got caught by a dumb coding challenge to build a full website. But the headline paints with a broad brush because coding challenges by themselves are not evil. They are a tool that can be used well or poorly. That's all.
Remember, as an employer, you are getting more out of the employee than they are getting out of you. Otherwise you don't make a profit. I understand why you want to do it as quickly and cheaply as possible. But seriously, fuck you. How about a better answer. Unless the person just came out of school, don't treat then as though they just came out of school.
Thank you for the gratuitous profanity :)
1. You're dead wrong if you think employers are always getting more out of the employee. It's a mutually beneficial relationship and a free, highly competitive market in SF. If poorly treated, candidates bail and current team members do too.
2. We aren't talking about profits at all. This confuses and conflates eventual profits with employee salaries.
3. Nothing that I said indicates that we treat them like they came out of school. If you have a better alternative, I am all ears.
4. We don't want to do it quickly and cheaply. We want to do it as best as possible under our constraints. We're a small startup.
Good luck to you as well and thanks for flipping the bird. So mature!
1. You're dead wrong if you think employers are always getting more out of the employee. It's a mutually beneficial relationship and a free, highly competitive market in SF. If poorly treated, candidates bail and current team members do too.
2. We aren't talking about profits at all. This confuses and conflates eventual profits with employee salaries.
3. Nothing that I said indicates that we treat them like they came out of school. If you have a better alternative, I am all ears.
4. We don't want to do it quickly and cheaply. We want to do it as best as possible under our constraints. We're a small startup.
Good luck to you as well and thanks for flipping the bird. So mature!
Please rework this explanation so that the points you are (trying) to make are emphasised rather than obfuscated. Then get back to me and I'll consider whether working with you is an option for me.
Sorry I got the formatting screwed up (if that's indeed what you are referring to). Will try to fix.
I wasn't trying to get you to work with me but thanks anyways!
I wasn't trying to get you to work with me but thanks anyways!
Fair enough, and sorry for my snarky response. I know you weren't trying to get to work with me, but I felt the need to let you know that you needed to present your message properly. The audience here are programmers, often very good programmers, people who take pride in organizing their thoughts and their programs with optimum logic and efficiently. Your message looked as if it might be interesting, but it was ruined by the formatting (yes, that was the problem).
I think the "25 to get 5" is aggressive. My experience is close to "25 to get 1".
Standard processes are important, but they need to respect everyone's time.
Standard processes are important, but they need to respect everyone's time.
I was talking about 25 onsite conversations leading to 10 offers and maybe 3-5 acceptances. In order to have 25 onsite conversations companies likely have to talk to 125-150+ total individuals via phone/email/networking events. That's at the low end once all the processes are dialed in.
I agree that companies have to be respectful of candidates' time. People do talk to their friends before they engage with companies or accept offers and word can get around fast if companies mistreat candidates (as it should).
I agree that companies have to be respectful of candidates' time. People do talk to their friends before they engage with companies or accept offers and word can get around fast if companies mistreat candidates (as it should).
Ahhh - gotcha. I think our experiences are similar.
While I agree that the request to build, deploy and open source the code of a website is a bit too much, I think you are exaggerating a little bit on the why's you wouldn't do it.
"I'm not equipped to do your challenge" - really?
"Engineers aren't different" - again, really?
Rhetorical question: you don't want the job, do you?
I am not sure if the issue is with the hiring process, or with the candidates' ego here. Candidates. Plural. As the article's author doesn't seem to be the only one.
Apparently there is a lot of people that believe they are rockstars and they don't need the job, the job needs them. Challenges like this help to figure out non technical skills as well, like: how committed a candidate is; how much he wants that job.
I'd rather hire a junior that is committed than a senior that is not.
"I'm not equipped to do your challenge" - really?
"Engineers aren't different" - again, really?
Rhetorical question: you don't want the job, do you?
I am not sure if the issue is with the hiring process, or with the candidates' ego here. Candidates. Plural. As the article's author doesn't seem to be the only one.
Apparently there is a lot of people that believe they are rockstars and they don't need the job, the job needs them. Challenges like this help to figure out non technical skills as well, like: how committed a candidate is; how much he wants that job.
I'd rather hire a junior that is committed than a senior that is not.
You're not supposed to be committed to a job.
When I work for a company, I want to make sure that whatever I am working on will help the company succeed. I want to be part of that success. This is commitment.
I am not there just to get the pay check for my 9 to 5 hours and move on to another company because they offered me extra $5 bucks.
If only modern companies had a similar view toward their employees. The fact is, whatever your particular situation, most companies don't return the exuberant loyalty you have for them. Although some have managers who are skilled enough salesmen to make you believe otherwise.
I agree and that is the main reason that made me move on from previous companies that I've worked for. However, I never used this fact as an excuse to care less about the work I am doing.
Hope you own the company, or are higher up in it.
Your boss may like you, but your company couldn't care less if you were hit by a bus tomorrow. Sure, you'd get a mention and a small office party with stale cookies and cheap pizza, but you'd be forgotten within the year.
Your boss may like you, but your company couldn't care less if you were hit by a bus tomorrow. Sure, you'd get a mention and a small office party with stale cookies and cheap pizza, but you'd be forgotten within the year.
That only works if the company shares that commitment and wants to share the rewards of that success. In my (admittedly not small-startup) experience, that is not often the case.
This is 2013. You gotta look out for Number One.
There are a lot of jobs going around these days. Employers no longer have the luxury to treat prospective hires poorly. This is the employer's problem, not the candidates.
Personally, I've asked potential hires for work product demonstrations. However, two crucial differences: 1) I paid them for their time, and 2) the task was on a real part of our infrastructure, the code they produced eventually being used in production. Everyone was happy with this situation.
Personally, I've asked potential hires for work product demonstrations. However, two crucial differences: 1) I paid them for their time, and 2) the task was on a real part of our infrastructure, the code they produced eventually being used in production. Everyone was happy with this situation.
I'd say that's a much more fair way to go about it. If you're going to ask someone to do work for you, that should be a paying gig, and it should be a task worth doing. If you're looking for a free sample a public repo or a task that fits within the parameters of a normal interview time would be better alternatives.
Depends on your definition of committed. If being committed to you means taking the better part of a day, weekend, etc. to exhaustively go through a "code challenge" for one job opportunity, that's one thing. For the average job seeker, if you've got ten recruiters that you're in contact with and one of those ten want you to dedicate that much time to their position and the other nine want to put you on the phone with an engineer and bring you on-site, how attractive is that one job? For someone already working more than full-time, probably not very.
I think the long and short of it is that being committed to a job has nothing to do with whether you'll tolerate a code challenge. Being "committed enough" to finishing a task to unlock the gate to actually speaking with a company's engineering team is completely orthogonal to how much you care about what you're paid to do when you have a job.
I think the long and short of it is that being committed to a job has nothing to do with whether you'll tolerate a code challenge. Being "committed enough" to finishing a task to unlock the gate to actually speaking with a company's engineering team is completely orthogonal to how much you care about what you're paid to do when you have a job.
I'm sorry, but an employer gets more out of the employee than the employee gets out of the employer. Otherwise the company would not be making a profit. But you know, maths.
Apparently there is a lot of people that believe they are rockstars and they don't need the job, the job needs them.
Here's a novel idea: both the candidate and the job need each other. If the job didn't need a candidate, then why are you looking to hire someone new?
Humility goes both ways. So does the arrogance. You reap what you sow.
I'd rather hire a junior that is committed than a senior that is not.
As long as you're prepared to deal with the difference between the output of a junior and the output of a senior, then you're okay. Word of advice, though: you'd best hold on to your own seniors.
Here's a novel idea: both the candidate and the job need each other. If the job didn't need a candidate, then why are you looking to hire someone new?
Humility goes both ways. So does the arrogance. You reap what you sow.
I'd rather hire a junior that is committed than a senior that is not.
As long as you're prepared to deal with the difference between the output of a junior and the output of a senior, then you're okay. Word of advice, though: you'd best hold on to your own seniors.
The worst thing about code-challenges, is they never give feed back.
We're sorry, but what you produced is not good enough. We're not going to tell you what you missed.
At least technology tells you what you missed. If you have a standard, have it documented so that I'm not firing a bullet into the dark. Chances are I'm just as good as you; but you won't know because I haven't seen your coding style. You're on my project, you code like me. I'm on yours, you better believe I'll code like you.
And then there is the times they like to dismiss you for it but have some other reason and use it as a cover. So you worry about your code not being perfect and you develop a ritual to validate yourself before it goes through a compiler. Yep, I write my code three(avg) times now. 10 lines, over and over until it's a effing poem. It's called ocd.
Coding challenges, pfft, worthless. There is only one company that I did a coding challenge for in the past year and it's because they were really awesome people. - I'm here to make a product and work with a team. I'm not here to explain shit to you just so you can tell me to look up the documentation. If I take the time to explain my code to you, you better well explain why you think something is better rather then telling me to google it.
fufff, I needed that.
We're sorry, but what you produced is not good enough. We're not going to tell you what you missed.
At least technology tells you what you missed. If you have a standard, have it documented so that I'm not firing a bullet into the dark. Chances are I'm just as good as you; but you won't know because I haven't seen your coding style. You're on my project, you code like me. I'm on yours, you better believe I'll code like you.
And then there is the times they like to dismiss you for it but have some other reason and use it as a cover. So you worry about your code not being perfect and you develop a ritual to validate yourself before it goes through a compiler. Yep, I write my code three(avg) times now. 10 lines, over and over until it's a effing poem. It's called ocd.
Coding challenges, pfft, worthless. There is only one company that I did a coding challenge for in the past year and it's because they were really awesome people. - I'm here to make a product and work with a team. I'm not here to explain shit to you just so you can tell me to look up the documentation. If I take the time to explain my code to you, you better well explain why you think something is better rather then telling me to google it.
fufff, I needed that.
> The worst thing about code-challenges, is they never give feed back. We're sorry, but what you produced is not good enough. We're not going to tell you what you missed.
This is most interviews in general, and I hate that. When interviewing candidates I try to give actionable feedback to each of them, regardless of whether it results in whiny emails back (and it often does). There was only one case in recent memory where I couldn't because he'd gotten literally nothing right while bragging about how fun it was to write thousand-line functions. Eek.
This is most interviews in general, and I hate that. When interviewing candidates I try to give actionable feedback to each of them, regardless of whether it results in whiny emails back (and it often does). There was only one case in recent memory where I couldn't because he'd gotten literally nothing right while bragging about how fun it was to write thousand-line functions. Eek.
Building a full production grade website seems extreme, but giving a prebuilt website to a front end candidate and asking them to add a few simple pages or features is basically fizzbuzz type vetting. I also like it when coding "challenges" are timeboxed to say 1-2 hours. This keeps the time expectations reasonable for both the interviewer who has to read a bunch of candidates' challenges and the interviewee who has to do them for (potentially) a bunch of gigs.
The type of coding challenge a company asks (interview questions also) can be one of the most important data points to use to judge a company's culture...
The type of coding challenge a company asks (interview questions also) can be one of the most important data points to use to judge a company's culture...
Yes exactly, this is overboard. We have "take home test" (I hate that term but haven't come up with anything better) for both our front-end and back-end candidates. They're
(a) Short - 45 minutes to an hour
(b) Only as complex as necessary (for the front end we provide and end-point serving JSON, for example, so they don't have to worry about that)
(c) Real problems we've run into, distilled down into a form appropriate for an interview
(d) Made up of problems that we were having candidates do on the whiteboard before we developed the at-home test, meaning they're not super complex
I think that at-home coding is the perfect second step to an interview (after a phone screen). It's more realistic than whiteboard coding. You have the internet at your fingertips. You don't have the pressure of an interviewer sitting there. You have syntax checking. From my side, I don't have to waste 30 - 60 minutes on a bad candidate. And, assuming the code is good, reviewing it provides an awesome first hour of an in person interview. And the problems I give you to take home would have been given to you in person anyway, so you're not spending any more time than you would have if you'd just come in for an interview.
The issue here is that the tests he's being given are bad examples of a take home test, not that take home tests are bad ipso facto.
(a) Short - 45 minutes to an hour
(b) Only as complex as necessary (for the front end we provide and end-point serving JSON, for example, so they don't have to worry about that)
(c) Real problems we've run into, distilled down into a form appropriate for an interview
(d) Made up of problems that we were having candidates do on the whiteboard before we developed the at-home test, meaning they're not super complex
I think that at-home coding is the perfect second step to an interview (after a phone screen). It's more realistic than whiteboard coding. You have the internet at your fingertips. You don't have the pressure of an interviewer sitting there. You have syntax checking. From my side, I don't have to waste 30 - 60 minutes on a bad candidate. And, assuming the code is good, reviewing it provides an awesome first hour of an in person interview. And the problems I give you to take home would have been given to you in person anyway, so you're not spending any more time than you would have if you'd just come in for an interview.
The issue here is that the tests he's being given are bad examples of a take home test, not that take home tests are bad ipso facto.
Have you (or people on your team) done the task yourselves? It's easy to underestimate the effort required.
Yes absolutely. They're all problems that we've solved in the process of building out our product. In addition to that, as I mentioned, we originally gave them as white board questions, so we're well aware of the issues that come up when solving the problems and the effort required to resolve the issues.
Our front-end test is a bit longer than the back-end test with a tradeoff of a shorter in person interview, but the front-end test shouldn't take more than 90 minutes.
One of our problems is a small matching algorithm. Another is a thread safety problem. The front-end test consists of pulling JSON data from a server, formatting it, and posting a single action back to the server. It sounds like we're in a minority though. Reading these other posts it seems like a lot of the tests are very onerous (build a small trading system? that is a ton of work).
Our front-end test is a bit longer than the back-end test with a tradeoff of a shorter in person interview, but the front-end test shouldn't take more than 90 minutes.
One of our problems is a small matching algorithm. Another is a thread safety problem. The front-end test consists of pulling JSON data from a server, formatting it, and posting a single action back to the server. It sounds like we're in a minority though. Reading these other posts it seems like a lot of the tests are very onerous (build a small trading system? that is a ton of work).
> They're all problems that we've solved in the process of building out our product
Have you gotten feedback on the questions? It's easy to say that it's a valid question, but when you're close to the problem, it's easy to make a mistake since you might encounter a specific sort of problem that others might not be nearly as familiar with.
The difference to me I think is that yours are very specific, and "build a small trading system" can be very general. If it's timeboxed, there might be only so much that you can do--i.e. you might not be expected to complete the task. You want to test someone's knowledge, without being too 'in depth'. A candidate that can solve a problem on his own in 30 minutes isn't necessarily better than a candidate who can solve the problem in 10 minutes with the assistance of Google.
0.02
Have you gotten feedback on the questions? It's easy to say that it's a valid question, but when you're close to the problem, it's easy to make a mistake since you might encounter a specific sort of problem that others might not be nearly as familiar with.
The difference to me I think is that yours are very specific, and "build a small trading system" can be very general. If it's timeboxed, there might be only so much that you can do--i.e. you might not be expected to complete the task. You want to test someone's knowledge, without being too 'in depth'. A candidate that can solve a problem on his own in 30 minutes isn't necessarily better than a candidate who can solve the problem in 10 minutes with the assistance of Google.
0.02
Thanks for the feedback, I'll definitely take a look at getting an outside opinion on our takehome test
Pro tip: these tests are filtering mechanism on only one candidate funnel. Go figure out a way to get in through another funnel. Impress someone at a meetup, send a targetted email. If you can't distinguish yourself from the masses, you are the masses.
Personally, I wouldn't bother. Asinine requirements are usually the product of asinine leadership and toxic environment. Just find another company to apply to.
Totally agree. I am so busy creating and making real products, that I just don't have time for silly quizzes or challenges. If you like my work, hire me. If you don't, then don't waste my time. I was seeking employment in the Bay area briefly and did three code tests. Absolute waste of time that didn't reflect my skill, diligence or anything for that matter. After I realized that these techniques were the norm, I quit looking. I have way better things to do with my time.
His complaints seem to be that this method of take-home-project is not as good as the method of hiring the candidate to work in the office temporarily for week's trial. That is true, but this is still superior to a typical interview. (Also not all candidates and not all employers can spare a week to do a trial.)
Nevertheless you should never agree to complete a full production project without being paid.
Nevertheless you should never agree to complete a full production project without being paid.
Or decode the source code upon the full payment.
Is it? I "interviewed" at one YC funded startup in particular and they asked me to build a trivially small and simple js application (even though I applied for a DevOps position, but I guess devops is different for everyone...). The only communication up to this point was over e-mail and I sent a resume.
I built it. It worked. Then they said my application was not aesthetic enough and said thank you for your time.
The whole thing seemed backwards...
In my opinion (as someone who has been going through the process for a few years now throughout undergrad with internships and now looking for a full time job at the end of my masters), very few companies do the "coding" problem right.
I don't want to work for you if you ask me to write a nested for loop and regurgitate the overhead involved in searching a b-tree. I also don't want to work for you if you ask me to build a application with no pre-defined criteria.
In my opinion...the best coding question you can ask me is fizzbuzz types of questions or pseudo-code for something complex (to see if the candidate can think through the steps of solving the complex problem without having to waste the time solving it).
So far not a single company has done the below.
I built it. It worked. Then they said my application was not aesthetic enough and said thank you for your time.
The whole thing seemed backwards...
In my opinion (as someone who has been going through the process for a few years now throughout undergrad with internships and now looking for a full time job at the end of my masters), very few companies do the "coding" problem right.
I don't want to work for you if you ask me to write a nested for loop and regurgitate the overhead involved in searching a b-tree. I also don't want to work for you if you ask me to build a application with no pre-defined criteria.
In my opinion...the best coding question you can ask me is fizzbuzz types of questions or pseudo-code for something complex (to see if the candidate can think through the steps of solving the complex problem without having to waste the time solving it).
So far not a single company has done the below.
Pretty sure I interviewed at the same place, with the same reaction. I wasn't too impressed.
If a company asked me that, I'd tell them thanks but no thanks and end the interview. Ask me a few questions, have me solve your riddles, see if I'd work well with your team/company. But building an entire production ready site? Ain't nobody got time for that.
Pretty sure no interviewer ever asked for a production ready site, this is just hyperbole from you and the author of this article.
If I were a front end developer, I would much prefer to be asked to write a quick, tailored to fit within the time allotment (e.g. one hour) web site rather than answering obscure coding puzzles that probably have little relevance with what I'll actually be doing at the company.
As the interviewer, I would phrase it as: "Build as much as you can in one hour, with the following functionalities in order of decreased priorities: login, cart, etc...".
Then I would judge you on how much of it you built, how solid each of the features are, etc. In my book, you can choose to go broad (build a lot of features, albeit superficially) or broad (only a few features, but well designed, tested, clean code).
If I were a front end developer, I would much prefer to be asked to write a quick, tailored to fit within the time allotment (e.g. one hour) web site rather than answering obscure coding puzzles that probably have little relevance with what I'll actually be doing at the company.
As the interviewer, I would phrase it as: "Build as much as you can in one hour, with the following functionalities in order of decreased priorities: login, cart, etc...".
Then I would judge you on how much of it you built, how solid each of the features are, etc. In my book, you can choose to go broad (build a lot of features, albeit superficially) or broad (only a few features, but well designed, tested, clean code).
> Pretty sure no interviewer ever asked for a production ready site this is just hyperbole from you and the author of this article.
How can you be so sure? Were you there? You're essentially calling the author a liar without even pretending to have access to more data than that author. Thats just rude.
If you know something that the rest of us don't that would lead you to make such a forceful assertion, please share.
How can you be so sure? Were you there? You're essentially calling the author a liar without even pretending to have access to more data than that author. Thats just rude.
If you know something that the rest of us don't that would lead you to make such a forceful assertion, please share.
> Pretty sure no interviewer ever asked for a production ready site, this is just hyperbole from you and the author of this article.
I'm sad to say that this is untrue, and that in writing this article I have not exaggerated the requirements in the least. I'm not going to name the company, but the requirements of the challenge indicate that it should be something you would be comfortable putting in production.
The "challenge" is a repo on the company's github. I'm sure you could probably easily find it with a bit of sleuthing.
I'm sad to say that this is untrue, and that in writing this article I have not exaggerated the requirements in the least. I'm not going to name the company, but the requirements of the challenge indicate that it should be something you would be comfortable putting in production.
The "challenge" is a repo on the company's github. I'm sure you could probably easily find it with a bit of sleuthing.
I never claimed this happened to me, I was speaking hypothetically to the authors situation.
I have had companies ask for an idea or some sort of feature to develop for the company and refused because it was essentially a "do some work for us and we might hire you" deal.
I have had companies ask for an idea or some sort of feature to develop for the company and refused because it was essentially a "do some work for us and we might hire you" deal.
I wouldn't end the interview then and there. Instead, I'd pull out a copy of my generic freelance contract, fill in my billable rate for the project after reviewing the scope, and end the interview after they refused to sign it.
My opinion is that companies that think unpaid "code challenges" are a good idea don't respect my time enough as an individual to be worth it to work for. It's a signal that they're either going to have an awful work/life balance or that they are filled with unjustifiably arrogant "rockstars" in a caustic environment of petty elitism (or both). The latter signal applies especially if the company's product is some web "application" for the young-20-something set.
My opinion is that companies that think unpaid "code challenges" are a good idea don't respect my time enough as an individual to be worth it to work for. It's a signal that they're either going to have an awful work/life balance or that they are filled with unjustifiably arrogant "rockstars" in a caustic environment of petty elitism (or both). The latter signal applies especially if the company's product is some web "application" for the young-20-something set.
I happen to really like coding challenges. I much prefer them to a "technical interview" situation, which I find very stressful--trying to answer difficult questions in real time.
However, the coding challenges I've done for applications have always been in the vein of "solve this interesting puzzle and show us how you did it", not "do some boring day-to-day work, but do it for free".
However, the coding challenges I've done for applications have always been in the vein of "solve this interesting puzzle and show us how you did it", not "do some boring day-to-day work, but do it for free".
This exactly.
With the exception of my current employer, who asked me to make a small static site with some JS oddities (about 6 hours total), they've all been fairly simple challenges just to get a grip on my thinking process, not really have me do the routine stuff.
With the exception of my current employer, who asked me to make a small static site with some JS oddities (about 6 hours total), they've all been fairly simple challenges just to get a grip on my thinking process, not really have me do the routine stuff.
I wonder whether the same companies that insist that job candidates spend a day of their lives on a "code challenge" are the same ones complaining about good developers being impossible to find.
On the other hand, as someone currently looking for my first dev job, I'd /much/ rather have a take home code challenge, where I can work at my usual pace, with my usual tools, without stress than have to do a code challenge while someone hovers over my shoulder. The take homes have been time intensive - I would say the average one has taken about 15 hours to do, but I feel they much better reflect my ability to code than real time nervous coding.
Your perspective changes when you need to find that extra 15 hours of time while also working full time. Apply to 2-3 companies at once, and all of a sudden 30-45 hours are added on to your normal work week.
Every time this comes up, I find it quite depressing how everyone becomes so self-centered. Half the comments here are along the lines of: I'm good at X, but bad at Y, so X is a better way to judge programmers than Y and if you do Y, your process is broken because I'm awesome despite being bad at Y.
Another small fraction of the comments are: I don't care if you can't do X despite being good or won't do Y because it's a big hassle - the bug is a feature and it filters out people who are not into our instasnapsocialmobileresponsiveapp. Get real guys, no one's that into you, some people are just desperate. Pretending that you're some prestigious company where everyone wants to work doesn't make it so - you have to earn it and if you get there, you won't have to pretend because you'd get so many resumes that you'll be forced come up with a quick way to be selective.
The secret to hiring is just like what pg said about the secret to VC investing - get better at distinguishing good programmers from bad ones so that you don't need to waste so much of everyone's time to make a decision. Being indecisive, after being clearly presented with enough information is a sign of incompetence.
Another small fraction of the comments are: I don't care if you can't do X despite being good or won't do Y because it's a big hassle - the bug is a feature and it filters out people who are not into our instasnapsocialmobileresponsiveapp. Get real guys, no one's that into you, some people are just desperate. Pretending that you're some prestigious company where everyone wants to work doesn't make it so - you have to earn it and if you get there, you won't have to pretend because you'd get so many resumes that you'll be forced come up with a quick way to be selective.
The secret to hiring is just like what pg said about the secret to VC investing - get better at distinguishing good programmers from bad ones so that you don't need to waste so much of everyone's time to make a decision. Being indecisive, after being clearly presented with enough information is a sign of incompetence.
We used to ask job candidates to do something similar.
Since then, publishing code samples became the normal thing to do, so we simply require code samples instead, and give out the coding task only when they have nothing at all to share with us.
If you're in the job market, publishing some sample code, even if it's just a throwaway toy instead of, say, a full blown OSS project, can be a great tool for people to evaluate you as a candidate.
Since then, publishing code samples became the normal thing to do, so we simply require code samples instead, and give out the coding task only when they have nothing at all to share with us.
If you're in the job market, publishing some sample code, even if it's just a throwaway toy instead of, say, a full blown OSS project, can be a great tool for people to evaluate you as a candidate.
I would never go to an interview which wanted to present me with a code challenge, and I don't have publicly available code repositories for you to browse.
What I do have is years of experience building mission critical applications that move millions of dollars per day in transactions, and I have a solid track record of being easy to work with. I am good at what I do and I work hard to be an asset to any company that employs me, and I work hard to be enjoyable to work with.
If you can't be good enough to figure that out from a conversation and looking at my track record, why would I work for you?
BTW, I haven't had to look for a job in over 13 years - although I've worked for several companies during that time.
What I do have is years of experience building mission critical applications that move millions of dollars per day in transactions, and I have a solid track record of being easy to work with. I am good at what I do and I work hard to be an asset to any company that employs me, and I work hard to be enjoyable to work with.
If you can't be good enough to figure that out from a conversation and looking at my track record, why would I work for you?
BTW, I haven't had to look for a job in over 13 years - although I've worked for several companies during that time.
tl;dr: if companies know that they are going to miss quality candidates with a code challenge, and are ok with that, it's ok to have one. If candidates are ok with giving some companies the miss because they won't put up with a challenge, that's ok too. And code challenges are good for some types of positions and not others.
I agree that companies are excluding people such as yourself with these coding challenges. That's their right, just as it is your right to not apply to any companies that require such challenges.
If someone is finding good candidates with a coding challenge, that's great. If someone is finding good jobs while avoiding coding challenges, that's great too.
I will say that there's some nuance implied in your comment that simply isn't present in any of the github/coding challenge blog posts that have popped up on HN over the last few days.
Whether a coding challenge is appropriate doesn't just depend on the company or the candidate--it also depends on the position! It's far easier to test junior/mid level skills than senior skills in a 1-2 hour task. Geez, some of the skills you mention above would be hard to test in a 1-2 day task.
When you are looking for esoteric, softer, or more senior skills (requirements gathering, or architecture), you need other ways of evaluating candidates beyond a code challenge. Things like what you suggest: conversations, references, etc. Of course, there are some "senior" folks out there who aren't great coders even though they talk a good game, and if coding is what the position requires, a code challenge is fine.
I agree that companies are excluding people such as yourself with these coding challenges. That's their right, just as it is your right to not apply to any companies that require such challenges.
If someone is finding good candidates with a coding challenge, that's great. If someone is finding good jobs while avoiding coding challenges, that's great too.
I will say that there's some nuance implied in your comment that simply isn't present in any of the github/coding challenge blog posts that have popped up on HN over the last few days.
Whether a coding challenge is appropriate doesn't just depend on the company or the candidate--it also depends on the position! It's far easier to test junior/mid level skills than senior skills in a 1-2 hour task. Geez, some of the skills you mention above would be hard to test in a 1-2 day task.
When you are looking for esoteric, softer, or more senior skills (requirements gathering, or architecture), you need other ways of evaluating candidates beyond a code challenge. Things like what you suggest: conversations, references, etc. Of course, there are some "senior" folks out there who aren't great coders even though they talk a good game, and if coding is what the position requires, a code challenge is fine.
The problem is that many people in our industry lie or stretch the truth a bit. Some of them are even good at bullshitting about it.
I work with several people who have over a decade of experience with "mission critical" applications and do not have the commensurate skills.
I work with several people who have over a decade of experience with "mission critical" applications and do not have the commensurate skills.
> What I do have is years of experience building mission critical applications that move millions of dollars per day in transactions, and I have a solid track record of being easy to work with.
> If you can't be good enough to figure that out from a conversation and looking at my track record, why would I work for you?
Just because you can get something done, that doesn't mean you did it by writing well-thought-out, sanely structured, maintainable, readable, useful code. I mean, you probably aren't bad if you're bragging about yourself like this, but who really knows until the code is visible?
The language and environment we work in isn't known for high-quality coders. Quite the opposite. Filtering out the wheat from the chaff is basically a requirement, even for the people that seem good after a phone screen and an in-person interview.
> If you can't be good enough to figure that out from a conversation and looking at my track record, why would I work for you?
Just because you can get something done, that doesn't mean you did it by writing well-thought-out, sanely structured, maintainable, readable, useful code. I mean, you probably aren't bad if you're bragging about yourself like this, but who really knows until the code is visible?
The language and environment we work in isn't known for high-quality coders. Quite the opposite. Filtering out the wheat from the chaff is basically a requirement, even for the people that seem good after a phone screen and an in-person interview.
"Just because you can get something done, that doesn't mean you did it by writing well-thought-out, sanely structured, maintainable, readable, useful code."
At no point did I claim to 'just get things done'. YOU said that. What I wrote was that I have years of experience building mission critical applications (which, of course, I would not claim if I could not prove).
It's not hard to prove or disprove if someone knows what they are talking about. I view 'coding challenges' as a sign that the company is not confident in their ability to assess people. It's a red flag to me. It's an amateur move. It tells me a LOT about the company from the inside, and it's not good.
'Coding challenges' are for companies who don't understand the skill set they are hiring for. I've hired plenty of devs over the years as well - we've had some bad ass programmers who worked for us, and I never needed a coding challenge to figure out who to hire.
At no point did I claim to 'just get things done'. YOU said that. What I wrote was that I have years of experience building mission critical applications (which, of course, I would not claim if I could not prove).
It's not hard to prove or disprove if someone knows what they are talking about. I view 'coding challenges' as a sign that the company is not confident in their ability to assess people. It's a red flag to me. It's an amateur move. It tells me a LOT about the company from the inside, and it's not good.
'Coding challenges' are for companies who don't understand the skill set they are hiring for. I've hired plenty of devs over the years as well - we've had some bad ass programmers who worked for us, and I never needed a coding challenge to figure out who to hire.
I understand the point the author's trying to make.
For me, personally, this would be the better choice to judge my skills - and probably would lead to better chances for me to get the job. I suck at these braindead trick questions that are considered 'clever'. I'm not comfortable around a whiteboard only, being grilled by one or more other people while thinking.
I'm not slow, not really anyway. But I'm faster and better the more I know about the stuff at hand. I'll be good on the job, I'll suck solving math puzzles that some guy just read about in a book that ranks high on Amazon's list of code puzzle collections..
So - I'm glad that there's both. The author obviously would've preferred to show his skills live, in a discussion. This guy here would be happy to hand you a (small, mind you - I agree with the time-sink argument) finished project and discuss _that_ afterwards.
For me, personally, this would be the better choice to judge my skills - and probably would lead to better chances for me to get the job. I suck at these braindead trick questions that are considered 'clever'. I'm not comfortable around a whiteboard only, being grilled by one or more other people while thinking.
I'm not slow, not really anyway. But I'm faster and better the more I know about the stuff at hand. I'll be good on the job, I'll suck solving math puzzles that some guy just read about in a book that ranks high on Amazon's list of code puzzle collections..
So - I'm glad that there's both. The author obviously would've preferred to show his skills live, in a discussion. This guy here would be happy to hand you a (small, mind you - I agree with the time-sink argument) finished project and discuss _that_ afterwards.
I agree and disagree. I am a candidate and prefer coding challenges provided they are reasonable. Meaning, if you are wanting to check the person's ability in x, check the person ability in x, not y. So for example give a starter code with part that would showcase person talent in x is left out. Yes, this involves prior effort on the part of the company to come up with this exercise and on the person. I don't quite vouch for code samples, because it is very difficult to understand where they came up with the ideas and who wrote the code, and if it was a multi-person project. In the former case, it is a very controlled experiment and one you designed, so one can judge much better.
The single most important point is that the process is candidate unfriendly. You can get away with being candidate unfriendly during college hiring. Maybe. If you're not hiring at a top CS school. You can't get away with it when you are trying to reel in a 10x superstar. Those types of candidates need to be wooed. If you make someone do too much work in the process, you'll only wind up with candidates who hate their jobs, not love their jobs.
That's not to say don't make sure people are smart. And the idea of proving coding skills is good too. But the manner has to be consistent with respecting the time and skills of an expert.
That's not to say don't make sure people are smart. And the idea of proving coding skills is good too. But the manner has to be consistent with respecting the time and skills of an expert.
The code challenge is usually just one part of the interview process to see if you can produce a quality product within a certain amount of time. It's not designed to tell every single thing you outlined in your post. It's meant to provide some signal before the actual on-site interview occurs. The on-site interview is expensive on the company's part, it is easier to filter out people who can't solve a simple full-stack solution. Companies who ask things like these are looking for people who understand both front and back ends. In short, just knowing either front or backend but not both, is not enough for the company.
We all know why they are asking candidates to do this. The question is whether this is a reasonable way to go about getting this information.
My work speaks for itself and I am more than happy to share working codes samples.
I had someone offer me a paid test. I refused because I am too busy.
By requiring coding tests, employers and managers may be accidentally selecting only desperate candidates.
I had someone offer me a paid test. I refused because I am too busy.
By requiring coding tests, employers and managers may be accidentally selecting only desperate candidates.
To my mind the ideal challenge is no more than 4 hours. You make an appointment. At the agreed time, the hiring company emails you the requirements. If you have clarification question, they remain available throughout the challenge time to answer them. From a hiring point of view the challenge should have some vague areas in the requirements in order to test whether the candidate is a coder or a developer who asks decent clarification questions. The challenge requires no special tools to do, either JS in a browser or Python or something like that. The exact requirments of the dev environment are specified at the time the appointment is made, i.e. if you want the person to develop in Python 2.6.3 or to use JS with backbone.js, then the candidate can get set up in a few minutes and be ready.
I consider 4 hours to be a max. 45 minutes is a bit short IMHO and does not allow for including unit tests. Also, I believe that a good challenge should ask for a complete set of unit tests as well.
But a full working MVP is way overboard and is too complex for the hirer to make meaningful decisions based on the outcome.
Any challenge should be followed up by an interview in which questions focus on the solution as developed by the candidate. The why of choices and the intent are important factors, probably more important than the actual code.
I consider 4 hours to be a max. 45 minutes is a bit short IMHO and does not allow for including unit tests. Also, I believe that a good challenge should ask for a complete set of unit tests as well.
But a full working MVP is way overboard and is too complex for the hirer to make meaningful decisions based on the outcome.
Any challenge should be followed up by an interview in which questions focus on the solution as developed by the candidate. The why of choices and the intent are important factors, probably more important than the actual code.
This is essentially what we do.
Write a program to fetch some data from S3 using these credentials we made you, perform some analysis on them (algorithm question, generally with recursion) and then display the final output. And write a test for it.
We provide a stubbed-out Rails project on Github so they don't have to know all the details about S3.
With this we test three things:
1) Can they quickly find missing info on Google/Stack Overflow (S3 stubs)
2) Can they handle recursion? (Algorithm question)
3) Can they write a test?
Then we go over their submission on Skype. It's worked pretty well for us.
Write a program to fetch some data from S3 using these credentials we made you, perform some analysis on them (algorithm question, generally with recursion) and then display the final output. And write a test for it.
We provide a stubbed-out Rails project on Github so they don't have to know all the details about S3.
With this we test three things:
1) Can they quickly find missing info on Google/Stack Overflow (S3 stubs)
2) Can they handle recursion? (Algorithm question)
3) Can they write a test?
Then we go over their submission on Skype. It's worked pretty well for us.
I love the idea of these challenges, but as a recent SF transplant\new job seeker, I'm absolutely appalled at how awful the execution usually is:
1. Company A sends me an exercise with slapped together and near nonsensical instructions, no browser compatibility requirements, and a link to a data API that doesn't resolve. Keep in mind this is an established company offering big bucks. After getting them to send me a link to a working API, submit perfectly functioning implementation with squeaky-clean code (~2 hour exercise). Receive absolutely no fucking feedback whatsoever - I only know that I was explicitly rejected because the 3rd party recruiter eventually told me. Still would love to know what they took issue with (maybe they were offended when I asked for clarification on their washy challenge?).
2. Company B sends me a ~5 hour Ruby coding exercise for a JavaScript position.
In both cases, a complete lack of respect for my time has been demonstrated. How naive of me to think that the quality of companies out here would be better. I'm thinking of calling quits on getting a new and exciting job out here already. Don't get me started with the other pieces of interview "process" of encountered. Yeesh.
1. Company A sends me an exercise with slapped together and near nonsensical instructions, no browser compatibility requirements, and a link to a data API that doesn't resolve. Keep in mind this is an established company offering big bucks. After getting them to send me a link to a working API, submit perfectly functioning implementation with squeaky-clean code (~2 hour exercise). Receive absolutely no fucking feedback whatsoever - I only know that I was explicitly rejected because the 3rd party recruiter eventually told me. Still would love to know what they took issue with (maybe they were offended when I asked for clarification on their washy challenge?).
2. Company B sends me a ~5 hour Ruby coding exercise for a JavaScript position.
In both cases, a complete lack of respect for my time has been demonstrated. How naive of me to think that the quality of companies out here would be better. I'm thinking of calling quits on getting a new and exciting job out here already. Don't get me started with the other pieces of interview "process" of encountered. Yeesh.
For the most part, I'm happy to do most of the coding challenges I've been asked to do. I've been lucky that in most of the challenges I've been able to learn something about how to make a certain algorithm that I hadn't done before, of I've enjoyed the feedback about using method x vs. method y.
However, I've had two coding challenges which I did not like. In the first was just too time consuming (more than 3 hours) and I stopped working on it because I felt it was taking too much time, and the second was on-site without an internet connection where when I drew a blank on using setInterval, and couldn't remember document.getElementsByClass (I was trying getElementByClass) I was unable to look-up the answer, which basically meant I had to sit there wracking my brain trying to remember and completely failed a test which would have been a walk in the park otherwise.
At the same time, the two companies that I've chosen to work for have not asked for coding challenges. My current employer did look at my GitHub profile, I don't think my previous employer did.
I think that supplying sample code should be enough for most employers, hopefully there is a way to confirm the applicant wrote the code they are taking credit for (I've had somebody claim to build one of my sites before, so it's happened. I got a bunch of e-mails by people asking me for a reference).
Another good way for businesses to decide on a coding question (if they decide to do one at all) is to take questions from larger successful technology businesses. I had one company ask me to complete one of the questions from Spotify https://www.spotify.com/au/jobs/, it didn't take very long at all (may 40 minutes) and if it is good enough for Spotify, it is probably good enough for your company.
However, I've had two coding challenges which I did not like. In the first was just too time consuming (more than 3 hours) and I stopped working on it because I felt it was taking too much time, and the second was on-site without an internet connection where when I drew a blank on using setInterval, and couldn't remember document.getElementsByClass (I was trying getElementByClass) I was unable to look-up the answer, which basically meant I had to sit there wracking my brain trying to remember and completely failed a test which would have been a walk in the park otherwise.
At the same time, the two companies that I've chosen to work for have not asked for coding challenges. My current employer did look at my GitHub profile, I don't think my previous employer did.
I think that supplying sample code should be enough for most employers, hopefully there is a way to confirm the applicant wrote the code they are taking credit for (I've had somebody claim to build one of my sites before, so it's happened. I got a bunch of e-mails by people asking me for a reference).
Another good way for businesses to decide on a coding question (if they decide to do one at all) is to take questions from larger successful technology businesses. I had one company ask me to complete one of the questions from Spotify https://www.spotify.com/au/jobs/, it didn't take very long at all (may 40 minutes) and if it is good enough for Spotify, it is probably good enough for your company.
If you use a coding challenge as part of your hiring process, please take the time to give feedback to candidates.
I recently completed a code challenge, and I was told by the recruiter that there were "problems with it," but he didn't get into any specifics because he was just passing on the message, and no one else has called either. I look at it as a dodged bullet, since it's a sign that they don't respect my time. I expect that had I gotten the job, they wouldn't have respected my free time or created a good work/life balance. Regardless, it has left a bad taste in my mouth. After I've spent about 12 hours of my free time (in addition to my 40 hr/week job), the least they could do is give me 15 minutes of their time to call or write out some comments about it.
I recently completed a code challenge, and I was told by the recruiter that there were "problems with it," but he didn't get into any specifics because he was just passing on the message, and no one else has called either. I look at it as a dodged bullet, since it's a sign that they don't respect my time. I expect that had I gotten the job, they wouldn't have respected my free time or created a good work/life balance. Regardless, it has left a bad taste in my mouth. After I've spent about 12 hours of my free time (in addition to my 40 hr/week job), the least they could do is give me 15 minutes of their time to call or write out some comments about it.
Many companies butcher their interview processes, so it's not surprising to hear this story. That said, OP's refusal of code challenges all together seems every bit as silly as being asked to jump through the hoops described.
If we were to take a moment to think about the purpose of interviews at a high level:
Initial Interviews (Phone/HR/etc)
- Determine mutual interest - Estimate likelihood of match - Check for red flags
Follow-up Interviews (With team/Code Challenges/etc)
- Confirm initial impressions - Evaluate finer points of fit (skills, etc) - Determine final interest
Each of the interview goals above apply to both the interviewer and interviewee. Failures happen when either side tries to monkey with this formula too much.
If your job is to write code and be an engineer, it's not outlandish to think that you'll be asked to write code as part of your evaluation for a new position. It's also not too much to expect that as an interviewee, you should be properly sold on the job by the interviewer before being asked to invest time to prove your skills. (Both sides should estimate high likelihood of match before confirming impressions)
If the company's preference is to hire full-stack engineers into front-end jobs, fine — more power to them. But those expectations should be included way up front in the process, so candidates like OP can mark a "red flag" and opt out of additional interviews.
TL;DR: Properly setting expectations in the interview process is ultra-important, but rarely practiced.
Note: I'm the founder of Mighty Spring (https://www.mightyspring.com), an anonymous, passive job discovery site. Part of our service is to help our clients (companies) hire more efficiently by coaching them up on interview process improvements like this. We're in private beta now, but send out new candidate invites regularly.
If we were to take a moment to think about the purpose of interviews at a high level:
Initial Interviews (Phone/HR/etc)
- Determine mutual interest - Estimate likelihood of match - Check for red flags
Follow-up Interviews (With team/Code Challenges/etc)
- Confirm initial impressions - Evaluate finer points of fit (skills, etc) - Determine final interest
Each of the interview goals above apply to both the interviewer and interviewee. Failures happen when either side tries to monkey with this formula too much.
If your job is to write code and be an engineer, it's not outlandish to think that you'll be asked to write code as part of your evaluation for a new position. It's also not too much to expect that as an interviewee, you should be properly sold on the job by the interviewer before being asked to invest time to prove your skills. (Both sides should estimate high likelihood of match before confirming impressions)
If the company's preference is to hire full-stack engineers into front-end jobs, fine — more power to them. But those expectations should be included way up front in the process, so candidates like OP can mark a "red flag" and opt out of additional interviews.
TL;DR: Properly setting expectations in the interview process is ultra-important, but rarely practiced.
Note: I'm the founder of Mighty Spring (https://www.mightyspring.com), an anonymous, passive job discovery site. Part of our service is to help our clients (companies) hire more efficiently by coaching them up on interview process improvements like this. We're in private beta now, but send out new candidate invites regularly.
Your td;dr: should go at the top.
Having started working at a company that asked me to do one of these, it was different specifically in that it was paid. This means it's not a waste of anyone's time. A time-constrained, feature-rich project shows a number of things about a programmer including how they handle pressure, how they prioritize their tasks, whether their estimates are reasonable, and whether they know about what you're hiring them for.
I get approached all the time by recruiters who supposedly love my work and want me to interview with their clients. It's getting harder to tell them that I'm not interested in a job. My consulting is doing okay and I'm actually starting to hire out some work. Why would I give that up, for a dice roll with a good chance of losing on whether or not my new boss turns out to be an asshole?
Good point. Getting to get back into consulting for that reason.
If you are going to do a code challenge, and take time out of your life to work on something without pay, the very least a company should do in return is provide feedback, good or bad. too many times I've done a code challenge and it's never even mentioned in follow up interviews. And if I don't pass the challenge, I should learn something from my time spent, right?
Lately I have been solving coding challenges. For the sake of solving them only. I submit them, wait for "good job!" or "nice work!" and move on to the next challenge--at the next company. After a few months or years of doing this, I may decide to advance to technical phone screens on shared documents. I'll try that for a while. After that who knows...
I understand the desire for a challenge, but you might consider not submitting them. Effort goes into advancing a candidate's application at each stage, so someone out there is wasting their time so you can get your "good job!" boost.
It's a tough market. It can take years to hone your problem-solving skills. And years after that to hone your technical-phone-screen-on-shared-document skills. This takes deliberate practice. When I'm ready to advance to the next stage, I will.
There is a serious point: companies often aren't forthcoming about day-to-day working conditions. But the response you get to your puzzle solving activities can be telling.
There is a serious point: companies often aren't forthcoming about day-to-day working conditions. But the response you get to your puzzle solving activities can be telling.
[deleted]
i don't think every candidate should be interviewed the same way, and i don't think these processes should be set in stone.
if you look great on paper (e.g. have some nice name brands on your resume, are a minor celebrity in some relevant community, or come in through a strong internal referral), there's no need for stuff like this.
however, if you don't fall into one of these categories, until there's a reliable way to tell if you're good from your resume, having an OPTIONAL coding challenge is a great way to get your foot in the door.
coding challenges shouldn't be about making people jump through arbitrary procedural hoops. they should be about making the (very broken) eng hiring process more meritocratic for people who'd never get their foot in the door otherwise.
if you look great on paper (e.g. have some nice name brands on your resume, are a minor celebrity in some relevant community, or come in through a strong internal referral), there's no need for stuff like this.
however, if you don't fall into one of these categories, until there's a reliable way to tell if you're good from your resume, having an OPTIONAL coding challenge is a great way to get your foot in the door.
coding challenges shouldn't be about making people jump through arbitrary procedural hoops. they should be about making the (very broken) eng hiring process more meritocratic for people who'd never get their foot in the door otherwise.
If you're being hired to code, expect to be asked to code in an interview process.
not many other fields of work where that analog exists, so I call BS. What's the personalized test given to CPAs? Sales? Marketers? You ask about past experience, accomplishments, etc..but rarely put them under the gun to produce on the spot.
This problem is typically solved through certification. As none exists in our world, we have to improvise.
If someone asked me to write an iOS app for them for free as a test I'd offer them to download some of my work from the App Store. Otherwise, no.
I don't care about code challenges either, I just always assumed that people don't care that I don't care.
I have to say, that headline font is awful and the link highlighting on the blog's front page is worse. There is a bit of oddness in declaring yourself a front-end person by talking about not needing to be tested as a back-end person when you're applying for a front-end job yet having a blog with a such a jarring front-end.
I hope one day we can get to the point where we discuss the content of a post on HN, and not cast subjective opinion about its presentation.
I don't think there is anything wrong with the headline font. I'm not too fond of the link highlighting used on the front page, but it's fine. Really. And you can be a "front end person" and have very, very little to do with design these days. Entire webapps are made on the front-end.
I don't think there is anything wrong with the headline font. I'm not too fond of the link highlighting used on the front page, but it's fine. Really. And you can be a "front end person" and have very, very little to do with design these days. Entire webapps are made on the front-end.
Presentation is always important. It doesn't matter if it's a scientific paper or blog, presenting in a clear and intelligible way is critical to communication in all forms.
As for the content:
I've dealt with numerous developers who only want to do one thing, only backend, only front-end, only design. I've constantly run into shenanigans when working with such people in a fast paced dynamic environment. "I can't work on the frontend till the backend api is finished", "I can't work on the backend till the frontend data requirements are defined", "I can't work on the frontend till the UI/UX are defined". It's a never ending requirements dependency argument.
More capable full stack people are able to mock or fake the bits that are as of yet undefined allowing them to continue working on their core piece while gaining a better understand of the system as a whole. This allows them to contribute to the product definition at all levels instead of just living in their own little piece of it, waiting for definition from outside.
As for the content:
I've dealt with numerous developers who only want to do one thing, only backend, only front-end, only design. I've constantly run into shenanigans when working with such people in a fast paced dynamic environment. "I can't work on the frontend till the backend api is finished", "I can't work on the backend till the frontend data requirements are defined", "I can't work on the frontend till the UI/UX are defined". It's a never ending requirements dependency argument.
More capable full stack people are able to mock or fake the bits that are as of yet undefined allowing them to continue working on their core piece while gaining a better understand of the system as a whole. This allows them to contribute to the product definition at all levels instead of just living in their own little piece of it, waiting for definition from outside.
So, I decided to treat your awful comment like you decided to treat the original article.
> I have to
That's not true. You don't have to reply. No one is requiring you do that. You've already started off by lying. This means anything else you might want to contribute is awful and worthless.
> say
You didn't. You typed it out. I'm pretty sure you didn't "say" anything. And now you are trying to be confusing. That's just awful.
> , that headline font is awful
Well, that's your opinion. Your opinion is worthless because you haven't validated your credentials.
> and the link highlighting on the blog's front page is worse.
So, not the page we were directed to. Did you happen to check out his Facebook page, too? Did you decide to comment on his awful dinner choice last night?
> There is a bit of oddness
This whole sentence is odd. In fact, that sentence is far worse than anything the OP did. You would say it's awful. In this case, you'd be right, because it's a shit sentence.
> in declaring yourself a front-end person by talking about not needing to be tested as a back-end person
That's not what he's doing. He's saying he's a front-end person, and was tested for back-end work. He's not saying he's a front-end person by talking about not needing to be tested as a back-end person.
Those are two completely different things. If you don't immediately understand why, I apologize, but you wrote the awful and confusing sentence.
> when you're applying for a front-end job yet having a blog with a such a jarring front-end.
What's jarring about it? Surely you aren't speaking of the design. That would be a designer, not a front-end person. And we've established he's not apply for a designer's position.
Unless you are suggesting all designers should be expert programmers? Because that's the only way what you said remotely makes any sense.
Awful, just awful.
> I have to
That's not true. You don't have to reply. No one is requiring you do that. You've already started off by lying. This means anything else you might want to contribute is awful and worthless.
> say
You didn't. You typed it out. I'm pretty sure you didn't "say" anything. And now you are trying to be confusing. That's just awful.
> , that headline font is awful
Well, that's your opinion. Your opinion is worthless because you haven't validated your credentials.
> and the link highlighting on the blog's front page is worse.
So, not the page we were directed to. Did you happen to check out his Facebook page, too? Did you decide to comment on his awful dinner choice last night?
> There is a bit of oddness
This whole sentence is odd. In fact, that sentence is far worse than anything the OP did. You would say it's awful. In this case, you'd be right, because it's a shit sentence.
> in declaring yourself a front-end person by talking about not needing to be tested as a back-end person
That's not what he's doing. He's saying he's a front-end person, and was tested for back-end work. He's not saying he's a front-end person by talking about not needing to be tested as a back-end person.
Those are two completely different things. If you don't immediately understand why, I apologize, but you wrote the awful and confusing sentence.
> when you're applying for a front-end job yet having a blog with a such a jarring front-end.
What's jarring about it? Surely you aren't speaking of the design. That would be a designer, not a front-end person. And we've established he's not apply for a designer's position.
Unless you are suggesting all designers should be expert programmers? Because that's the only way what you said remotely makes any sense.
Awful, just awful.
I told FB the same thing, too!
Though it really based on what lever you are at. This kinda tests are always for entry-mid level recruitment I guess.
I agree with author 100 percent.
Here's the gist of how it went down:
1) first part was the "personality" interview. Two "HR" people blew smoke up my ass for about 30 minutes about how awesome my site and my work was, then asked me to do a "code challenge". I agreed - I mean, how bad could it be?
2) The code challenge. I was given a PSD with a straight forward design. Asked to code it, make sure it worked in legacy browsers, publish on a server and show them the code on GitHub
3) Passed and then was asked to come back and explain some of my coding decisions. I was told this was more for show and I was already in effect hired, I just had to pass this last hurdle. It turned out to be a very confrontational, inquisition type of an interview. Not at all the type of laid back code review I'm used it.
I got pretty frustrated by the two "Senior" devs who seemed to take great pride in tearing my code and my decisions apart. After my first heated exchange with one of the devs, I actually asked them how many of the sites listed in the gallery on their site they actually tested and validated the code on. They said "All of them, why do you ask?" I told them I actually ran all of their sites through the W3C validation site and none of them passed. Every single one had over 60 errors on it. I basically said, "If standards are so important to you, then why are you holding people who don't even work for your company to a higher standard than those who actually do?"
There was a long silence and then I finally spoke up and said, "Clearly, this company has its priorities in the wrong order. Thank you for time, but I'm not interested." and got up and walked out.
I got several worried calls from the company recruiter, but by then I was so pissed at how much of my time they had wasted, and vowed never to do something like it again.