Enough with the Programming Puzzles(linkedin.com)
linkedin.com
Enough with the Programming Puzzles
https://www.linkedin.com/pulse/enough-programming-puzzles-ron-cox
18 comments
It seems that other professions have solved this. Nobody is asking lawyers to run a trial on a whiteboard or solve a whodunit mystery in 2 hours. And I don't think that licensing process for lawyers guarantees anything other than the right to practice law (there are pretty bad licensed lawyers out there). Do construction workers draw diagrams of installing drywall in their hiring process? Do plumbers get challenged with the back of cereal box trace-the-pipe puzzle?
There are jobs, which have a similar hiring process. Performance artists actually need to perform to get hired but they do perform their actual work. I.e. they are not playing the anthem of Cambodia on a toy recorder in order to be hired as a death metal band drummer.
There are jobs, which have a similar hiring process. Performance artists actually need to perform to get hired but they do perform their actual work. I.e. they are not playing the anthem of Cambodia on a toy recorder in order to be hired as a death metal band drummer.
>I.e. they are not playing the anthem of Cambodia on a toy recorder in order to be hired as a death metal band drummer.
Think about it this way: they are not playing a full playlist on an actual stage with a soundcheck and live audience either. Compared to the real gig they are performing a smaller task in an artificially simplified environment. It is actually very similar to a coding interview.
People often forget that puzzles are not a thing into itself, it's a tool for discovering candidate's style and a springboard for further discussion. I'm not interested in a solution itself as much as in an ability to discuss it, explain it & improve it -- because those are the things that are being done in a workplace on a daily basis.
Think about it this way: they are not playing a full playlist on an actual stage with a soundcheck and live audience either. Compared to the real gig they are performing a smaller task in an artificially simplified environment. It is actually very similar to a coding interview.
People often forget that puzzles are not a thing into itself, it's a tool for discovering candidate's style and a springboard for further discussion. I'm not interested in a solution itself as much as in an ability to discuss it, explain it & improve it -- because those are the things that are being done in a workplace on a daily basis.
People also forget the goal of the hiring process - finding somebody who can get the work done. I am in the game industry and, luckily, not everybody is doing whiteboard coding and puzzles here, but, out of my anecdotal experience, studios that do it also release pretty crappy games. It could be that springboarding discussion is not selecting very good programmers. It could also be that the management who enforces these practices is not competent to run the studio. I cannot tell either way, just my experience.
Out of curiosity, what do the studios that don't release crappy games do?
Math and general CS (in the sense of "What is IEEE representation of 1.f?" not "The difference between Red-Black and B- trees?") questions. Past experience questions.
You can flip this around.
Do you want to work for a company that does what everyone else does without a clear well-defined rationale for it, or do you want to work for a company that has a deep understanding of what it needs from its employees, and has worked out the most efficient way to get it?
Good companies run a continuous improvement cycle on hiring practices. Bad companies do the same shit over and over and make the same mistakes again and again.
So I would be surprised if poor hiring didn't correlate with poor results in other areas.
Do you want to work for a company that does what everyone else does without a clear well-defined rationale for it, or do you want to work for a company that has a deep understanding of what it needs from its employees, and has worked out the most efficient way to get it?
Good companies run a continuous improvement cycle on hiring practices. Bad companies do the same shit over and over and make the same mistakes again and again.
So I would be surprised if poor hiring didn't correlate with poor results in other areas.
Why are so many of them given as take home / pre interview screens then?
Actually, many firms do set puzzles, normally by asking the candidate to create a brief based on materials provided. It takes far more than a couple hours. Bar exams, depending on where you are, also sometimes incorporate this sort of thing.
The California bar exam includes two separate performance tests of (IIRC) two- to three hours each. You're given a packet of "facts" and legal-reference materials. Your assignment is to write a motion for a court case; a memo to the client; etc. (Starting in July 2017 it'll be cut to one performance test of 90 minutes [0].)
[0] http://abovethelaw.com/2015/07/california-bar-exam-cut-from-...
[0] http://abovethelaw.com/2015/07/california-bar-exam-cut-from-...
In my day, the multistate also had a "practical" exam option that involved creating a brief from a small pile of evidences. It only took a few hours.
In those other jobs get hired a lot :
And yes, dev are spoiled in the US, so recruiting technics move fast, trying to adapt. But in France, they are not, and we have the same old recruiting technics. And it's not better. Lot of bad dev are recruited.
1 - by social networking;
2 - on diploma.
- non programmers. And dev will likely not be found in the same social circles than those people;
- programmers. Which tend to use a technical solution to all problems.
And yes, dev are spoiled in the US, so recruiting technics move fast, trying to adapt. But in France, they are not, and we have the same old recruiting technics. And it's not better. Lot of bad dev are recruited.
When I freelance, they just ask if I can get the job done.
When I get hired, they want me to do dumb tests.
The only difference for them is that they can stop paying me right away when I freelance, but when I'm an employee they have to pay me for at least 2 weeks to go (at least in Germany were we have 6 months probation time on new jobs, where you can be fired easily)
I made many people I worked for happy, but many people who gave me tests wouldn't even consider me as employee.
I don't know if I would have been a bad employee for the testing-companies, but I never had issues with employers or customers who hired me, so I have the feeling the companies who didn't hire me were missing out.
When I get hired, they want me to do dumb tests.
The only difference for them is that they can stop paying me right away when I freelance, but when I'm an employee they have to pay me for at least 2 weeks to go (at least in Germany were we have 6 months probation time on new jobs, where you can be fired easily)
I made many people I worked for happy, but many people who gave me tests wouldn't even consider me as employee.
I don't know if I would have been a bad employee for the testing-companies, but I never had issues with employers or customers who hired me, so I have the feeling the companies who didn't hire me were missing out.
Lawyers do not investigate crimes; solving a whodunnit is a terrible comparison.
Plumbers, unlike computer programmers, are typically hired purely by referral or a small amount of marketing. Again, you make a terrible analogy.
A plumber got hired because
A) The risk of a bad plumber is less threatening than the leaking faucet I have right now.
B) a friend said they were a good plumber.
Since JavaScript refactoring is not as pressing as a leaking tap the hirer can take their time and subject you to a rigour out process in order to ensure that the millions of pounds of intellectual property you have access to will be handled in the correct manner.
I agree with the top voted response; the perception is engineers are beginning to whine about everything.
Not you fault, you are just one part of the sum but that sum is sounding more like a spoiled brat than ever before.
Plumbers, unlike computer programmers, are typically hired purely by referral or a small amount of marketing. Again, you make a terrible analogy.
A plumber got hired because
A) The risk of a bad plumber is less threatening than the leaking faucet I have right now.
B) a friend said they were a good plumber.
Since JavaScript refactoring is not as pressing as a leaking tap the hirer can take their time and subject you to a rigour out process in order to ensure that the millions of pounds of intellectual property you have access to will be handled in the correct manner.
I agree with the top voted response; the perception is engineers are beginning to whine about everything.
Not you fault, you are just one part of the sum but that sum is sounding more like a spoiled brat than ever before.
By your complains about my terrible analogies (which they are, intentionally) I am guessing you are one of the few programmers who actually does all the stuff from the interviews on the job?
No. I freelance and don't have to do any of that.
However, when I was seeking permanent employment I understood that it was not my circus and not my monkey's. I had to jump through the hoops. The same for coding as it was product management as it was for consultancy.
Rather than complain, why not invent a startup that acts as a universal badge of coding ability and is widely accepted as the industry standard for benchmarking technological ability...
The post just screams (once again) "I am a coder you should simply accept my genius and leave me in peace to work."
Law of averages just tells you that there is a bell curve of ability...at least some of the individuals on Hacker News are precisely the individuals that white boarding and puzzles are designed to catch.
However, when I was seeking permanent employment I understood that it was not my circus and not my monkey's. I had to jump through the hoops. The same for coding as it was product management as it was for consultancy.
Rather than complain, why not invent a startup that acts as a universal badge of coding ability and is widely accepted as the industry standard for benchmarking technological ability...
The post just screams (once again) "I am a coder you should simply accept my genius and leave me in peace to work."
Law of averages just tells you that there is a bell curve of ability...at least some of the individuals on Hacker News are precisely the individuals that white boarding and puzzles are designed to catch.
The issue is that many people already have a substantial proof of their coding ability in form of completed work. The rational approach would be to evaluate that work and, as safety, test that they have actually performed it. Whiteboarding them catches nothing other than inability to perform unusual tasks under stress.
As as we know, engineers are never in stressful situations so no point testing it.
Production never goes down, critical errors don't arise and time-pressure tasks are rare.
Production never goes down, critical errors don't arise and time-pressure tasks are rare.
People do not behave consistently under stress so such a testing value is close to 0. It's a fight or flight response. If "fight" rolled up on the interview there is no guarantee it will roll up next time your production goes down. You can just as well toss a coin.
...and yet soldiers in WW1 and WW2 were subjected to stressful situations lasting years, some of which manifested in mental trauma but many of which actually behaved in the manner required.
My point is; you are neither psychologist nor professional human resources executive search.
Your reasoning for not conducting testing interviews is flawed and can be challenged at every juncture precisely because this is not your area of expertise.
It's not mine either but I am not advocating sweeping policy changes across an entire professional landscape.
If you feel there is a better route; then write up, test it, get it peer reviewed and implement it.
My point is; you are neither psychologist nor professional human resources executive search.
Your reasoning for not conducting testing interviews is flawed and can be challenged at every juncture precisely because this is not your area of expertise.
It's not mine either but I am not advocating sweeping policy changes across an entire professional landscape.
If you feel there is a better route; then write up, test it, get it peer reviewed and implement it.
You can certainly challenge anybody's reasoning just as well as you can challenge anybody in court over any matter. But it requires more than typing first thing that comes into your mind for such a challenge to stick, jfyi.
But not in this case, the first thing that came to mind was a sufficient rebuttal.
Since you are constantly referencing the legal system you should remember it is the prosecutors role to prove the hypothesis. It is not on the defence to disprove it.
Merely rebutt sufficiently.
You have not made a single compelling argument for not testing coders with the current model.
Since you are constantly referencing the legal system you should remember it is the prosecutors role to prove the hypothesis. It is not on the defence to disprove it.
Merely rebutt sufficiently.
You have not made a single compelling argument for not testing coders with the current model.
[deleted]
> We don't like writing code on the whiteboard, we don't like mind teasers, we don't like programming puzzles, and so on.
I love all of those things. In fact, dare I say I like them more than the job itself:
(0) The given task is usually short, self-contained, and has a single best solution that can be found purely by logical reasoning.
(1) There are no requirements of the form “you have to use this obviously misdesigned technology”. All one has to do is come up with data structures and algorithms.
(2) I won't piss anyone off for trying to do things in the most elegant way possible.
What's not to love?
I love all of those things. In fact, dare I say I like them more than the job itself:
(0) The given task is usually short, self-contained, and has a single best solution that can be found purely by logical reasoning.
(1) There are no requirements of the form “you have to use this obviously misdesigned technology”. All one has to do is come up with data structures and algorithms.
(2) I won't piss anyone off for trying to do things in the most elegant way possible.
What's not to love?
We should get together and just give each other such puzzles. You even index from zero. You're hired!
I suggest we work on a blackboard instead of on a whiteboard. I think they are nicer.
I suggest we work on a blackboard instead of on a whiteboard. I think they are nicer.
> We should get together and just give each other such puzzles.
Would be nice.
> I suggest we work on a blackboard instead of on a whiteboard. I think they are nicer.
Using blackboard and chalk is indeed nicer. The unpleasantness comes afterwards, when you have to wash the chalk off of your clothes.
Would be nice.
> I suggest we work on a blackboard instead of on a whiteboard. I think they are nicer.
Using blackboard and chalk is indeed nicer. The unpleasantness comes afterwards, when you have to wash the chalk off of your clothes.
We realize that all these exercises are pretty pointless an unrelated to the work we are doing. Is that really acting like a spoiled kid?
Those puzzles might serve an ancillary goal of finding people willing to conform, are good at doing what they are told and have the ability to do unconventional programming tasks on a whim.
All of those are characteristics that the company might be looking for.
In short those tests aren't checking for just aptitude, they are checking for the personality characteristics of people willing to do them and tolerate the absurdity - like a cult.
I once applied as a JavaScript developer to a local company and they gave me a test in Cake PHP. This was after a face-to-face interview with the owner.
I finished most of it but their was one thing I couldn't figure out. His response via email was along the lines of " You really should learn this stuff". This was after answering a bunch of granular questions about JavaScript and passing those preliminary questions.
But I guess its my fault and I "really should learn this stuff" (j/k)
:/
All of those are characteristics that the company might be looking for.
In short those tests aren't checking for just aptitude, they are checking for the personality characteristics of people willing to do them and tolerate the absurdity - like a cult.
I once applied as a JavaScript developer to a local company and they gave me a test in Cake PHP. This was after a face-to-face interview with the owner.
I finished most of it but their was one thing I couldn't figure out. His response via email was along the lines of " You really should learn this stuff". This was after answering a bunch of granular questions about JavaScript and passing those preliminary questions.
But I guess its my fault and I "really should learn this stuff" (j/k)
:/
These tests are probably designed to check your motivation, not your skill.
If Google is asking questions about manhole covers, you better study about them before an interview. A candidate who can answer such a question would more likely want to badly work for Google. That is probably a better predictor for a good workplace performance than raw skill.
If Google is asking questions about manhole covers, you better study about them before an interview. A candidate who can answer such a question would more likely want to badly work for Google. That is probably a better predictor for a good workplace performance than raw skill.
It's a good predictor for wanting to work at Google.
It's in no way a good predictor for being able to do good work at Google - or anywhere else.
It's in no way a good predictor for being able to do good work at Google - or anywhere else.
Isn't that what my university degrees were supposed to be for dammit!
/ha ha only serious
/ha ha only serious
Isn't it difficult to reject these puzzles on principle without sounding either bitter or like you simply can't solve them? I can't imagine a single case where this is going to cause a potential employer to re examine these exercises. They'll simply chock you up to being unqualified.
I reckon most jobs simply require you to not be a knob for an hour while someone asks you tricky questions about your worst behaviors. How often, day to day, are you required to explain something bad about yourself in a constructive way? Yet it is as utterly common as programming puzzles.
My supposition is that if that's where the wind is blowing, I'd had damn better face my sail that way. I hate silly problem solving puzzles as much as anyone, but not a shit lot you can do about it when they're the ones holding the keys.
I reckon most jobs simply require you to not be a knob for an hour while someone asks you tricky questions about your worst behaviors. How often, day to day, are you required to explain something bad about yourself in a constructive way? Yet it is as utterly common as programming puzzles.
My supposition is that if that's where the wind is blowing, I'd had damn better face my sail that way. I hate silly problem solving puzzles as much as anyone, but not a shit lot you can do about it when they're the ones holding the keys.
Well, I guess you could start your own company. You could even hire anyone who comes in the door claiming to be able to code, although what you'll do with a dozen people who can't even FizzBuzz is beyond me.
I'm actually in the middle of a bunch of interviews right now. The most ridiculous interview process I encountered so far was one that was supposed to be 7 (!!) steps long (that usually takes 3-4 weeks by their own estimates), beginning with a 45 minute HackerRank challenge, then a technical project that was said to take 2-5 hours on average (but the time given was a whole week, so I have my doubts about that estimate), followed by up to 4 Skype calls, and finally an onsite.
The HackerRank challenge consisted of two relatively trivial multiple choice problems, a simple FizzBuzz-level coding problem, and a programming puzzle. And I do mean programming puzzle. I consider it a puzzle in the sense that it had very little to do with any classical CS domain I'm aware of (although that may just be an artifact of my own ignorance, but to someone unaware of that domain like myself, it as might as well just be a puzzle) and certainly wasn't anything one would ever encounter in front-end development (I was applying as a front-end engineer). I later found the problem online and here's a thread discussing it for those interested (the company I interviewed with wasn't Google though): http://stackoverflow.com/questions/28967020/check-if-there-e...
I finished the 2 MC questions and first coding problem within the first 5 minutes, but I wasn't able to finish the programming puzzle in the time remaining (For what it's worth, I was on the right track with my approach, but it took me a lot of trial and error to arrive at that approach, so I didn't have enough time left to finish the implementation). I got my rejection email two days later.
I really question the wisdom of using a programming puzzle like this as a filter in the first step of your interview process (or really, in any step of the interview process for that matter). You're effectively discarding candidates solely based on their inability to solve an esoteric programming puzzle under some arbitrary time constraint.
Although at the end of the day, I'm not that disappointed in this early rejection. If this programming puzzle was a sign of things to come in the next 6 steps of their interview process, I think I may have dodged a bullet. That could have been so many hours of my life that I'd never get back. Good riddance I say.
The HackerRank challenge consisted of two relatively trivial multiple choice problems, a simple FizzBuzz-level coding problem, and a programming puzzle. And I do mean programming puzzle. I consider it a puzzle in the sense that it had very little to do with any classical CS domain I'm aware of (although that may just be an artifact of my own ignorance, but to someone unaware of that domain like myself, it as might as well just be a puzzle) and certainly wasn't anything one would ever encounter in front-end development (I was applying as a front-end engineer). I later found the problem online and here's a thread discussing it for those interested (the company I interviewed with wasn't Google though): http://stackoverflow.com/questions/28967020/check-if-there-e...
I finished the 2 MC questions and first coding problem within the first 5 minutes, but I wasn't able to finish the programming puzzle in the time remaining (For what it's worth, I was on the right track with my approach, but it took me a lot of trial and error to arrive at that approach, so I didn't have enough time left to finish the implementation). I got my rejection email two days later.
I really question the wisdom of using a programming puzzle like this as a filter in the first step of your interview process (or really, in any step of the interview process for that matter). You're effectively discarding candidates solely based on their inability to solve an esoteric programming puzzle under some arbitrary time constraint.
Although at the end of the day, I'm not that disappointed in this early rejection. If this programming puzzle was a sign of things to come in the next 6 steps of their interview process, I think I may have dodged a bullet. That could have been so many hours of my life that I'd never get back. Good riddance I say.
You dismiss this as "esoteric", but that's the whole point, the purpose of a puzzle like this is for you to be unable to map it to something you've done previously or to something you learned in school, and instead to force you to invent an algorithm on your own.
It is the opposite of what the author of the article complains about - "academic", "targeted at cs grads" - it goes against that. It is an attempt to test problem-solving ability by removing knowledge from the equation. Knowledge can then be tested separately. Fresh CS grads are not going to do better on this problem than a guy who graduated 10 years ago.
In fact, this is the kind of problem given to children (who can't be expected to have CS knowledge, and you don't want to give a big advantage to those who happen to have some) in programming competitions. I would put the difficulty at either easy problem for high schoolers or difficult problem for elementary schoolers (this doesn't mean it's super-easy, these problems have to be relatively hard or the smartest kids would all solve everything).
I am not sure why so many developers dismiss the value of this and are in fact often actively hostile to any suggestion that it might be ONE of the indicators of programming ability.
I do agree that the time constraint was probably too strict and favors people who've solved stuff like this before, mainly due to them knowing they should try to solve this with a pen and paper or in their head, then just code the solution once they've found it. Trial and error through code is a huge waste of time and this was probably your problem.
It is the opposite of what the author of the article complains about - "academic", "targeted at cs grads" - it goes against that. It is an attempt to test problem-solving ability by removing knowledge from the equation. Knowledge can then be tested separately. Fresh CS grads are not going to do better on this problem than a guy who graduated 10 years ago.
In fact, this is the kind of problem given to children (who can't be expected to have CS knowledge, and you don't want to give a big advantage to those who happen to have some) in programming competitions. I would put the difficulty at either easy problem for high schoolers or difficult problem for elementary schoolers (this doesn't mean it's super-easy, these problems have to be relatively hard or the smartest kids would all solve everything).
I am not sure why so many developers dismiss the value of this and are in fact often actively hostile to any suggestion that it might be ONE of the indicators of programming ability.
I do agree that the time constraint was probably too strict and favors people who've solved stuff like this before, mainly due to them knowing they should try to solve this with a pen and paper or in their head, then just code the solution once they've found it. Trial and error through code is a huge waste of time and this was probably your problem.
> I am not sure why so many developers dismiss the value of this and are in fact often actively hostile to any suggestion that it might be ONE of the indicators of programming ability.
No disagreement there. I certainly wasn't trying to dismiss the ability to come up with novel solutions to problems never encountered as an indicator for programming ability and general intelligence completely. However, I certainly don't believe it's a good enough indicator to use as the sole basis for simply rejecting otherwise perfectly capable candidates before you've even talked to them.
I'm also of the opinion that interviews should strive to assess how well candidates can perform on the specific position you're hiring for (that is the main purpose of interviews, is it not?). In my own case (I was applying for a front-end engineer), and in the case of most other engineering positions, I don't believe puzzle solving is a very useful indicator for that purpose, because an overwhelming majority of engineering tasks involve applying the scientific method and engineering knowledge to solve variations of well-known engineering problems in their specific fields.
EDIT: Another point that I forgot to mention: If you are in fact interested in a candidate's ability to solve novel problems they've never encountered, at least do a better job of ensuring the problem you give them is actually novel. I found the StackOverflow thread for the problem they gave me in about 30 seconds of Googling. By not ensuring the problem novel enough that searching for it won't do you any good, you're going to be disproportionately passing dishonest candidates who would simply cheat to get through this part of your process. Certainly that can't be the result you're looking for.
No disagreement there. I certainly wasn't trying to dismiss the ability to come up with novel solutions to problems never encountered as an indicator for programming ability and general intelligence completely. However, I certainly don't believe it's a good enough indicator to use as the sole basis for simply rejecting otherwise perfectly capable candidates before you've even talked to them.
I'm also of the opinion that interviews should strive to assess how well candidates can perform on the specific position you're hiring for (that is the main purpose of interviews, is it not?). In my own case (I was applying for a front-end engineer), and in the case of most other engineering positions, I don't believe puzzle solving is a very useful indicator for that purpose, because an overwhelming majority of engineering tasks involve applying the scientific method and engineering knowledge to solve variations of well-known engineering problems in their specific fields.
EDIT: Another point that I forgot to mention: If you are in fact interested in a candidate's ability to solve novel problems they've never encountered, at least do a better job of ensuring the problem you give them is actually novel. I found the StackOverflow thread for the problem they gave me in about 30 seconds of Googling. By not ensuring the problem novel enough that searching for it won't do you any good, you're going to be disproportionately passing dishonest candidates who would simply cheat to get through this part of your process. Certainly that can't be the result you're looking for.
Regarding the first, I think they basically do this to test how "programming smart" someone is. Like my boss tells me when I interview, "determine potential". However, I agree that this shouldn't disqualify someone for a more technical kind of job. For me it is more of a "if he is really smart we should hire him despite the fact he is perhaps unqualified, he will easily learn" thing. The disqualifying questions are easier than this one.
Regarding the second, I agree completely. This problem is good because it doesn't give an advantage to rote memorization of basic CS from college (unlike asking about red-black trees etc), but when stolen from the internet and reused becomes bad because it gives an advantage to something worse - googling questions and memorizing answers.
I assume (or hope) Google switches up the problems they use all the time and hopefully doesn't use automated testing (because I care how someone thinks, not just whether they produce the solution in X minutes), but since you interviewed at a different company perhaps it is a cargo cult effect: Google has some of the smartest programmers and they asked this, so obviously we should also ask the same thing! Cargo culting is extremely common in IT in less-established companies, my workplace is not immune to it either. Pick things a super-successful company does that require almost zero investment, effort or true change, and copy those. It is ridiculous.
Of course, the things these companies really do right, they are too much effort, let's just copy them in superficial ways and hope for the best!
Regarding the second, I agree completely. This problem is good because it doesn't give an advantage to rote memorization of basic CS from college (unlike asking about red-black trees etc), but when stolen from the internet and reused becomes bad because it gives an advantage to something worse - googling questions and memorizing answers.
I assume (or hope) Google switches up the problems they use all the time and hopefully doesn't use automated testing (because I care how someone thinks, not just whether they produce the solution in X minutes), but since you interviewed at a different company perhaps it is a cargo cult effect: Google has some of the smartest programmers and they asked this, so obviously we should also ask the same thing! Cargo culting is extremely common in IT in less-established companies, my workplace is not immune to it either. Pick things a super-successful company does that require almost zero investment, effort or true change, and copy those. It is ridiculous.
Of course, the things these companies really do right, they are too much effort, let's just copy them in superficial ways and hope for the best!
One of the problems with coding challenges is that I suspect you miss out on a lot of potential candidates. I did a round of interviews recently and one wanted me to do an online test and then a puzzle.
I did the online test, received the puzzle, and in the intervening period I was interviewed and offered a job by someone else. When I rang the first place to tell them the guy was very upset because it had taken so long to find someone who could pass the test!
Seems to me if you're that desperate for candidates then don't assume you're the only company in the world.
I did the online test, received the puzzle, and in the intervening period I was interviewed and offered a job by someone else. When I rang the first place to tell them the guy was very upset because it had taken so long to find someone who could pass the test!
Seems to me if you're that desperate for candidates then don't assume you're the only company in the world.
I would have to largely agree with this based on my own experience, and I've been on both sides of the fence.
The best interviews I've had was when I was invited to come work with the team after a few meet and greets. It allows all parties to relax a little and get a taste of what working together would be like.
The best interviews I've had was when I was invited to come work with the team after a few meet and greets. It allows all parties to relax a little and get a taste of what working together would be like.
Do people applying for other roles get treated to tests like this? Seems peculiar to IT afaict. Having worked in IT for > 25 years, I would have thought my track record would speak for itself, but still I'm treated as though I've been lying and cheating my way through my career. It's not like just asking questions, either: it's a test of basic aptitude and (like the author) I'm at the stage of refusing to do them.
There's assessment centers for most decent entry level management jobs which usually involve grouping tasks by importance and team oriented tasks. They can last a day, sometimes two. Typical consulting gigs have case studies and market size guesstimating (the dreaded how many lawnmowers in NYC etc.)
Typical office secretary jobs here require a work sample so you'll get a typical task for the job and have to do it on the spot (write a letter to someone, translate a text). My current job (academia) required a somewhat simple test to see if I have a decent understanding of method/statistics and the programming test was answering a couple of questions :P
In logistics you usually have to reason through a couple of typical work situations.
After entry level...it's usually just an interview and references.
Typical office secretary jobs here require a work sample so you'll get a typical task for the job and have to do it on the spot (write a letter to someone, translate a text). My current job (academia) required a somewhat simple test to see if I have a decent understanding of method/statistics and the programming test was answering a couple of questions :P
In logistics you usually have to reason through a couple of typical work situations.
After entry level...it's usually just an interview and references.
I guess that's encouraging, kind of. I've always felt like this whole thing was just an extension of "don't trust the IT guy", a viewpoint a one-time employer was happy to articulate loud and often ... in front of the IT guys.
I once had an agent almost freak out over the phone because he'd seen my open-source project and couldn't believe I'd created it myself. "It had to come from somewhere", I told him. "Why shouldn't it be me as much as any other random stranger on the phone?" Seems like even when I presented them with impressive, real-life work, they still didn't want to trust me :)
I once had an agent almost freak out over the phone because he'd seen my open-source project and couldn't believe I'd created it myself. "It had to come from somewhere", I told him. "Why shouldn't it be me as much as any other random stranger on the phone?" Seems like even when I presented them with impressive, real-life work, they still didn't want to trust me :)
After entry level...it's usually just an interview and references.
Except it's not, that's the problem. But this is not a matter of incompetence on the part of the employer, but of deliberate and calculated policy.
Except it's not, that's the problem. But this is not a matter of incompetence on the part of the employer, but of deliberate and calculated policy.
Just to clarify. I was referring to the non-programming jobs. So for an entry level management job you'll do the assessment center and jump through all sorts of hoops. For your next job it's usually a basic discussion. It'll get more rigid again higher up as there will be a vetting process etc.
But beyond entry level there's usually no skill tests in the domains I know outside of IT. I think that's the main point. Most people could accept all sorts of programming and possibly even logic tests for entry level IT jobs but after you've shipped software it becomes a bit ridiculous in many cases.
A friend of mine is a seller in a big firm and was asked to guess how many golf balls exist in the world and other such questions.
I don't know what the solution to hiring is, but I don't think it is as simple as just talking to people.
While I have not been involved in too many new hires, I have known many people that come across as knowledgeable in conversation but produce pretty terrible code.
I think coding challenges provide some value, but it can be a but much to expect candidates to spend hour's of their time early in the process
I think coding challenges provide some value, but it can be a but much to expect candidates to spend hour's of their time early in the process
Surely there's a middle-ground between puzzles designed to trip you up, and puzzles designed to show you have some baseline chops. There's a lot of people who can't walk their talk.
At one place I work at, we're trying to get some senior-level PHP devs... and we're getting some applicants that can't even conceptually do FizzBuzz...
At one place I work at, we're trying to get some senior-level PHP devs... and we're getting some applicants that can't even conceptually do FizzBuzz...
Its not usually that difficult to work out if people know what they are talking about.
Some people are really good at bullshitting their way through stuff. You need to make them actually do stuff to discover they flounder.
And since you can't have them do anything meaningful in an interview (there isn't enough time), anything you ask them to do will be a silly "programming puzzle".
And since you can't have them do anything meaningful in an interview (there isn't enough time), anything you ask them to do will be a silly "programming puzzle".
As an interviewer I am upfront with our hiring process and that part of it is we ask candidates to do a "exercise" after the first face to face meeting. This exercise is crafted to be representative of the job we are trying to fill and is followed up by a second face to face code review session of the code.
The key of the exercise isn't necessarily to complete the whole task (though often people do as the exercises are interesting) but to provide a discussion piece for the code review session. Candidates are told this so that they don't feel like they have to put too much work into it (2 hours is suggested).
The code review session is key, walking through someones code and discussing decisions and trade offs. It provides a layered insight into their smarts, communication skills and thought processes. It is the closest I've gotten to actually working with someone in the interview process.
And isn't that what both parties are looking for?
The key of the exercise isn't necessarily to complete the whole task (though often people do as the exercises are interesting) but to provide a discussion piece for the code review session. Candidates are told this so that they don't feel like they have to put too much work into it (2 hours is suggested).
The code review session is key, walking through someones code and discussing decisions and trade offs. It provides a layered insight into their smarts, communication skills and thought processes. It is the closest I've gotten to actually working with someone in the interview process.
And isn't that what both parties are looking for?
So I guess we know the answer to the "How would you react if assigned a task you didn't enjoy?" question. A: Rather than spend two hours getting the job done, an hour is spent doing a write-up on why the task wasn't done.
A job, any job, will involve thousands of hours with at least a few hundred of them on tasks you really do not appreciate. A candidate not willing to suffer for a couple hours during the interview probably wouldn't have lasted very long anyway.
A job, any job, will involve thousands of hours with at least a few hundred of them on tasks you really do not appreciate. A candidate not willing to suffer for a couple hours during the interview probably wouldn't have lasted very long anyway.
> A job, any job, will involve thousands of hours with at least a few hundred of them on tasks you really do not appreciate.
But you get paid to do those tasks. And remember that those 2 hours are an estimate, which means it'll probably take longer. Then multiply that by the number of interviews you have and you can quickly see why it becomes unfair rather than 'lazy'.
But you get paid to do those tasks. And remember that those 2 hours are an estimate, which means it'll probably take longer. Then multiply that by the number of interviews you have and you can quickly see why it becomes unfair rather than 'lazy'.
Sometimes you have to work on spec.
Typical LinkedIn "thought leader", didn't Google et al abandon brainteasers over a year ago?
The article isn't talking about brain teasers. Google did indeed banish them, but now they've switched to 4-5 interviews where you're asked a short coding problem.
The problems themselves aren't difficult and given the time constraint, you can fit the answer in the height of one whiteboard. The trick is that you have to be fast, optimal and correct (i.e. syntax counts). From my experience, you should be able to complete each problem in about 20 minutes - i.e. a good enough solution that the interviewer moves on.
The article is right: there are books and books on doing these toy problems with array manipulation, tree traversals, sort/search and so on. If you did a TopCoder SRM every night for a few months, you would probably ace most of these interviews.
But this is the issue: the interviews basically test how well you prepared. They don't simulate a work environment and I don't think I was ever asked any serious questions about my work experience/research beyond chit-chat. Given that I was applying for a specialist computer vision role, that was a bit surprising.
The problems themselves aren't difficult and given the time constraint, you can fit the answer in the height of one whiteboard. The trick is that you have to be fast, optimal and correct (i.e. syntax counts). From my experience, you should be able to complete each problem in about 20 minutes - i.e. a good enough solution that the interviewer moves on.
The article is right: there are books and books on doing these toy problems with array manipulation, tree traversals, sort/search and so on. If you did a TopCoder SRM every night for a few months, you would probably ace most of these interviews.
But this is the issue: the interviews basically test how well you prepared. They don't simulate a work environment and I don't think I was ever asked any serious questions about my work experience/research beyond chit-chat. Given that I was applying for a specialist computer vision role, that was a bit surprising.
I wonder if the author didn't know enough about the company to know if he wanted to do an online coding test, then how was he certain that the content wasn't relevant to what the company was looking for?
I think a candidate needs to have both algorithmic and software design skills. Companies put a lot of emphasis on the former skill, ignoring the latter, which imo should equally be tested.
Well done for replying back with a reasoned response for not participating. I have done this at a few places. I hope if enough people do it people will get rid of these interview techniques.
If you want to find out if a programmer can actually do the job, then it seems to me that the way forward is to have them do something that actually resembles the job.
In the high-end cooking world, it's not uncommon to do one of two things when looking to hire a new cook:
1) Make me a dish. Present the chef with the kitchen, and possibly a specific main ingredient(s), and have them make the best dish they can with that ingredient. Sometimes this is also under some kind of time constraint, to test pressure under fire, but doesn't have to be (great chefs might take weeks to develop a new recipe after all).
2) The "stage". The kitchen takes on the chef temporarily for a short period, where they will actually work on different portions of the line, so that the employer can see both how they perform in real time and where their strengths lie. This can be as little as a single shift, or up to a month or more at some very, very high end kitchens like Thomas Keller's. Crucially as well, this is generally paid (it'd actually be illegal otherwise), so the prospective hiree can actually afford to dedicate that kind of time.
Now some of this is similar to stuff that some tech companies do already. But I think they often miss crucial elements that makes this actually work in a way that's not painful for both parties.
The "make me a dish" method for instance, superficially resembles the infamous "sit down at our computer and code a thing" tests I've read of many times. But there are key differences. Tooling is a pain point in programming that doesn't so much exist in cooking (in fact part of this test can be making sure the prospective cook is familiar with all the basic tools). Sitting in at a foreign editor/IDE/language etc. is likely to be more stressful than is really accurate to how one can expect a new coder to perform in the real world. The simple solution is "bring your own knives." Let the coder use their own machine, or at least give them time to set up a basic version of their favorite tools.
The other thing about "make me a dish" that's missing from programming equivalents that is I think most important is creativity. "Make me a dish" isn't about just proving you can cook, it's about proving you can create. And isn't that something we want from great programmers as well? Generally when I see and read about these tasks their very proscribed: there's a specific spec/demand/puzzle, and often with the expectation it will be solved in a particular way. One test task I was given once actually even specified what data format I was supposed to use to communicate between back and front ends. What about a mini-hackathon/jam approach instead? Give me a subject or a data source and a bit of time, and see what I can come with?
The "stage" as well is something I think is underconsidered with tech companies. Companies spend thousands of dollars on hiring practices and recruiters and advertising, but they can't just do a paid trial period instead? It makes no sense. If you think you've got a promising candidate, just take them on for a couple weeks, point them at a relatively low-hanging fruit task on the issues board, and see how they do.
Alternately, maybe just accept a fucking risk now and again. If the candidate seems to know what they're talking about in the interview, maybe just hire the fucking person, and if they suck, fire them. This is the same stricture and risk 90% of hiring practices are under, and somehow the economy hasn't collapsed yet so I am gonna tentatively suggest it seems to work.
Sometimes the path of least resistance is the easier, cheaper way.
In the high-end cooking world, it's not uncommon to do one of two things when looking to hire a new cook:
1) Make me a dish. Present the chef with the kitchen, and possibly a specific main ingredient(s), and have them make the best dish they can with that ingredient. Sometimes this is also under some kind of time constraint, to test pressure under fire, but doesn't have to be (great chefs might take weeks to develop a new recipe after all).
2) The "stage". The kitchen takes on the chef temporarily for a short period, where they will actually work on different portions of the line, so that the employer can see both how they perform in real time and where their strengths lie. This can be as little as a single shift, or up to a month or more at some very, very high end kitchens like Thomas Keller's. Crucially as well, this is generally paid (it'd actually be illegal otherwise), so the prospective hiree can actually afford to dedicate that kind of time.
Now some of this is similar to stuff that some tech companies do already. But I think they often miss crucial elements that makes this actually work in a way that's not painful for both parties.
The "make me a dish" method for instance, superficially resembles the infamous "sit down at our computer and code a thing" tests I've read of many times. But there are key differences. Tooling is a pain point in programming that doesn't so much exist in cooking (in fact part of this test can be making sure the prospective cook is familiar with all the basic tools). Sitting in at a foreign editor/IDE/language etc. is likely to be more stressful than is really accurate to how one can expect a new coder to perform in the real world. The simple solution is "bring your own knives." Let the coder use their own machine, or at least give them time to set up a basic version of their favorite tools.
The other thing about "make me a dish" that's missing from programming equivalents that is I think most important is creativity. "Make me a dish" isn't about just proving you can cook, it's about proving you can create. And isn't that something we want from great programmers as well? Generally when I see and read about these tasks their very proscribed: there's a specific spec/demand/puzzle, and often with the expectation it will be solved in a particular way. One test task I was given once actually even specified what data format I was supposed to use to communicate between back and front ends. What about a mini-hackathon/jam approach instead? Give me a subject or a data source and a bit of time, and see what I can come with?
The "stage" as well is something I think is underconsidered with tech companies. Companies spend thousands of dollars on hiring practices and recruiters and advertising, but they can't just do a paid trial period instead? It makes no sense. If you think you've got a promising candidate, just take them on for a couple weeks, point them at a relatively low-hanging fruit task on the issues board, and see how they do.
Alternately, maybe just accept a fucking risk now and again. If the candidate seems to know what they're talking about in the interview, maybe just hire the fucking person, and if they suck, fire them. This is the same stricture and risk 90% of hiring practices are under, and somehow the economy hasn't collapsed yet so I am gonna tentatively suggest it seems to work.
Sometimes the path of least resistance is the easier, cheaper way.
One problem with "take on people for a trial period" is that it's more risky for the candidate too. If you happen to be unemployed then doing a trial-work-period is obviously an improvement on not working at all, but otherwise I'm going to be reluctant to quit my current job for an "after a couple of months we may say sorry, didn't work out" setup. (That's different from the current standard '3-6 months is a probationary period' because right now the assumption is that unless you actually screw up you get the job, so it's not an in-practice risk for a competent employee. With a more definite "trial period" approach the employer is going to be more likely to say 'no' at the end -- otherwise the "pick who to trial" question is still just as high-stakes as "who do we hire" was and they haven't gained anything from the change.)
That's a good point, and one I didn't think about. Cooking is, on the whole, a much more unstable industry in terms of work experience and generally the kinds of restaurants that even do a stage are the kinds it's even worth taking the risk on even if you are employed.
One possible solution might be to make it a remote contract. "Work on this bug/commit/sample project at your leisure, commit by no later X date". Then moonlighting applicants or even just people who don't want to risk being off the search during the trial period can still participate on the same playing field, though they would potentially still be disadvantaged compared to unemployed applicants. I don't think though that a little bias there is entirely a bad idea personally, though you could possibly argue the merits either way.
One possible solution might be to make it a remote contract. "Work on this bug/commit/sample project at your leisure, commit by no later X date". Then moonlighting applicants or even just people who don't want to risk being off the search during the trial period can still participate on the same playing field, though they would potentially still be disadvantaged compared to unemployed applicants. I don't think though that a little bias there is entirely a bad idea personally, though you could possibly argue the merits either way.
> If the candidate seems to know what they're talking about in the interview, maybe just hire the fucking person, and if they suck, fire them.
In some countries, employment laws are so crazy, that it's almost impossible to fire someone willy nilly because they "suck". You have to give them written warnings, re-"skill" them, move them to different departments, etc. It's such a pain in the ass and then they still take you to court for unfair dismissal.
In some countries, employment laws are so crazy, that it's almost impossible to fire someone willy nilly because they "suck". You have to give them written warnings, re-"skill" them, move them to different departments, etc. It's such a pain in the ass and then they still take you to court for unfair dismissal.
Even in countries with labor laws, people usually have a 3 to 6 months trial periods where they can be fired at will.
I think the true deterrent is not the law, but that it is rather expensive to train someone for 2 months then have to fire him/her, and find someone else.
I think the true deterrent is not the law, but that it is rather expensive to train someone for 2 months then have to fire him/her, and find someone else.
In Germany, new employees typically have a trial period where they can be fired on short notice. Usually 3-6 months, depending on the wording of the contract. Dismissal protection only kicks in after the trial period.
Do you really need to test whether a candidate is technically competent if they have a degree from a well respected university like MIT, etc? You already know he has the discipline and academic requisites.
Interviewing someone at that point, I'd expect the employer would be more interested to see if they're a good fit for the company culture and not waste time on academic trivialities.
Interviewing someone at that point, I'd expect the employer would be more interested to see if they're a good fit for the company culture and not waste time on academic trivialities.
Well, Laszlo Bock claims Google looked at the data and found zero correlation between having a degree from a well respected university and being useful at Google
http://mobile.nytimes.com/2014/02/23/opinion/sunday/friedman...
So no, using a degree isn't useful for hiring engineers apparently
http://mobile.nytimes.com/2014/02/23/opinion/sunday/friedman...
So no, using a degree isn't useful for hiring engineers apparently
The common response to various claims that Google make based on their HR data is: "But they only looked at the people who succesfully got hired at Google".
Which seems a fair criticism to me. Google's hiring process may be as wacky and broken as any other corp, but they're a big name, and will attract a lot of very good applicants.
So "People who got succesfully got hired at Google, and don't have a prestigious degree, do about as well as people who got hired at Google and did have a prestigious degree."
If you're on the outside looking to get a job, you'd probably want to know what percentage of Google employees had prestigous degrees, and what other resume filling achievements the non-prestigious degree crowd had before basing any life decisions on this data point.
Which seems a fair criticism to me. Google's hiring process may be as wacky and broken as any other corp, but they're a big name, and will attract a lot of very good applicants.
So "People who got succesfully got hired at Google, and don't have a prestigious degree, do about as well as people who got hired at Google and did have a prestigious degree."
If you're on the outside looking to get a job, you'd probably want to know what percentage of Google employees had prestigous degrees, and what other resume filling achievements the non-prestigious degree crowd had before basing any life decisions on this data point.
That's pretty weird though.
Is using a degree useful for hiring medical doctors, mechanical engineers, actuaries? It's really hard to believe that only Computer Science as a field has a problem where it's churning out people who are technically incapable of programming to such a degree that you need to explicitly test them in interviews through "puzzles".
Is using a degree useful for hiring medical doctors, mechanical engineers, actuaries? It's really hard to believe that only Computer Science as a field has a problem where it's churning out people who are technically incapable of programming to such a degree that you need to explicitly test them in interviews through "puzzles".
Maybe they all have that problem and no one has ever bother to actually check the data?
I know for me and interviewing programmers my personal experience is the degree doesn't matter. I've met plenty of degreed "engineers" who couldn't program for shit from schools like MIT and Harvard. I've also met some amazing programmers from those same places. I don't know what make the difference except possible love of the activity. Not sure how to asses that in an interview or whether it correlates better with outcomes.
I know for me and interviewing programmers my personal experience is the degree doesn't matter. I've met plenty of degreed "engineers" who couldn't program for shit from schools like MIT and Harvard. I've also met some amazing programmers from those same places. I don't know what make the difference except possible love of the activity. Not sure how to asses that in an interview or whether it correlates better with outcomes.
For medical doctors, having a degree just means you get to spend years in a residency training program under supervision. SEs also have internships, but they usually want someone they don't need to train from 'scratch'.
I have met some people with degrees, from extremely respected institutions, who I would not let organise the coffee in a startup let alone be involved in serious business tasks.
Likewise, some of the best engineers I have met were self-taught and curious.
Likewise, some of the best engineers I have met were self-taught and curious.
We don't like writing code on the whiteboard, we don't like mind teasers, we don't like programming puzzles, and so on. We complain about everything.
Well, hiring is a difficult problem, and nobody has a universal solution yet. But the truth is that if I want to get a job, I will need eat the humble pie and get out there and try to perform as good as I can to whatever I am asked. If I don't, somebody else will.
However, remember that interview is a two way process, if I don't like the employer's process, then maybe I will not even interview for them. In this case though, I cannot expect them to give me an offer, just because.