Senior devs. Is anyone else insulted by coding exams?
394 comments
I've interviewed hundreds of programmers, and really, you cannot trust pretty much anyone.
There's so many people that think they can code, or just say they do that really don't, that some sort of basic coding test is necessary if the job requires it.
Our hiring rate for some roles is 200:1, but we pay handsomely for those.
Think it the other way around, interviews are a two way process, you're also interviewing the company, and the way they conduct the interviews, in many ways also tells you what they value. What they're like.
Take it as an opportunity to get to know each other. Bring this up in a constructive manner. It may make all the difference and provide a stronger signal to both of you, you learn how do they react to feedback and at the same time you show that you're capable of critical thinking and care enough about your craft to call bullshit when you see it.
There's so many people that think they can code, or just say they do that really don't, that some sort of basic coding test is necessary if the job requires it.
Our hiring rate for some roles is 200:1, but we pay handsomely for those.
Think it the other way around, interviews are a two way process, you're also interviewing the company, and the way they conduct the interviews, in many ways also tells you what they value. What they're like.
Take it as an opportunity to get to know each other. Bring this up in a constructive manner. It may make all the difference and provide a stronger signal to both of you, you learn how do they react to feedback and at the same time you show that you're capable of critical thinking and care enough about your craft to call bullshit when you see it.
> Also, hiring managers... Do you find these kind of tests useful? Why? How?
Former hiring manager.
It's unfortunate because it is a bit insulting. However, there are some people who lie. They lie about being able to code. I don't want to hire a senior dev who I then have to fire shortly thereafter.
A simple, short coding test that any senior dev should be able to do quickly in their language of choice or a longer project that they get paid for are my two favorite options to filter out folks who say they are senior devs but can't code.
Former hiring manager.
It's unfortunate because it is a bit insulting. However, there are some people who lie. They lie about being able to code. I don't want to hire a senior dev who I then have to fire shortly thereafter.
A simple, short coding test that any senior dev should be able to do quickly in their language of choice or a longer project that they get paid for are my two favorite options to filter out folks who say they are senior devs but can't code.
I got the chance to peek behind the scenes once while assisting with the hiring for a startup. It was painful to see how many folks applied who had zero chance of making it. As in, they wouldn't be able to actually write a "Hello World" program, let alone FizzBuzz or the types of exams you're mentioning here. The number of those kinds of applications was up in the 80% range of all application submitted.
And HR (when you have them) are not able to filter them out properly, since all it takes is dose of buzzwords and sufficient confidence to get past them.
So, yeah, code tests are usually stupid and insulting to experienced developers. But that's because they're needed to filter out the stupid.
And HR (when you have them) are not able to filter them out properly, since all it takes is dose of buzzwords and sufficient confidence to get past them.
So, yeah, code tests are usually stupid and insulting to experienced developers. But that's because they're needed to filter out the stupid.
> Do you find these kind of tests useful?
I’ve hired dozens of Sr. engineers and given coding tests to almost all of them (after a 2nd round interview, never earlier).
However, my tests are time-boxes (1hr) and are about greenfielding an app (like a ToDo app) to see how they approach a problem rather than programming knowledge.
> Why?
About 33-50% of devs that pass the two interviews fail the coding test. Of the ones that fail, they’re great at interviewing but probably oversold their skills. Some are really poor at time management. Some are really poor at simplifying problems. And some are really poor at following directions.
It’s possible that they were on teams that carried their weight or possible they were exaggerating their capabilities or prior responsibilities.
The opposite has happened too where I’ve been blown away by the code quality and approach.
But overall it’s been a very helpful datapoint in hiring.
I’ve hired dozens of Sr. engineers and given coding tests to almost all of them (after a 2nd round interview, never earlier).
However, my tests are time-boxes (1hr) and are about greenfielding an app (like a ToDo app) to see how they approach a problem rather than programming knowledge.
> Why?
About 33-50% of devs that pass the two interviews fail the coding test. Of the ones that fail, they’re great at interviewing but probably oversold their skills. Some are really poor at time management. Some are really poor at simplifying problems. And some are really poor at following directions.
It’s possible that they were on teams that carried their weight or possible they were exaggerating their capabilities or prior responsibilities.
The opposite has happened too where I’ve been blown away by the code quality and approach.
But overall it’s been a very helpful datapoint in hiring.
This industry has no professional standards or relevant certification - what does a senior developer mean ?
You say you have relevant experience - will you show them your work so they can review it ?
I have a friend that has >7 years of experience - which was spent on obsolete stack in huge corporate contexts - you put him on a new project to bootstrap a generic app and he's basically a junior/mid with some debugging experience and knowledge of SCRUM.
And this was made obvious to him by multiple interviews/tasks he had to do - after which he basically had to spend 6 months getting up to date to be able to work on modern projects.
You say you have relevant experience - will you show them your work so they can review it ?
I have a friend that has >7 years of experience - which was spent on obsolete stack in huge corporate contexts - you put him on a new project to bootstrap a generic app and he's basically a junior/mid with some debugging experience and knowledge of SCRUM.
And this was made obvious to him by multiple interviews/tasks he had to do - after which he basically had to spend 6 months getting up to date to be able to work on modern projects.
Insulted? Never. Annoyed? Many times, yes. It really depends on the type of test. If they give me the "this should take 4 hours" project, I usually tell them to take a hike because that's very disrespectful of my time and the 4 hours is complete bullshit if one is trying to deliver a perfect result. Coding puzzles are extremely annoying, but I can usually get through them.
The best coding tests are those that demonstrate you know how to code applications relevant to the job; e.g. a UI component or an API endpoint that takes a parameter and body and returns a filtered result. Then the test is followed up with discussions around what else could be done with it in the real world (auth, performance enhancements, etc.). Seemingly this would be the easiest and most rational path for both interviewer and interviewee, so I'm not sure why the industry is still stuck on those stupid puzzle exams.
"We can't find any good devs". No, your hiring method sucks and you're eliminating good devs because of it.
The best coding tests are those that demonstrate you know how to code applications relevant to the job; e.g. a UI component or an API endpoint that takes a parameter and body and returns a filtered result. Then the test is followed up with discussions around what else could be done with it in the real world (auth, performance enhancements, etc.). Seemingly this would be the easiest and most rational path for both interviewer and interviewee, so I'm not sure why the industry is still stuck on those stupid puzzle exams.
"We can't find any good devs". No, your hiring method sucks and you're eliminating good devs because of it.
I am coding for 40+ years...
My experience: * out of college people remember theory, cannot code. * mid-level engineers start to forget theory, can code, begin system-wide design. * senior engineers can design, forget how to code.
If the position of senior engineer expects hands-on coding (mine does), coding exercise during interview is quite revealing (sadly). Personally I do not believe in engineers who cannot code.
My experience: * out of college people remember theory, cannot code. * mid-level engineers start to forget theory, can code, begin system-wide design. * senior engineers can design, forget how to code.
If the position of senior engineer expects hands-on coding (mine does), coding exercise during interview is quite revealing (sadly). Personally I do not believe in engineers who cannot code.
I'm a Senior Dev. I dabbled in some coding in Jr. High, but go more involved in my early 20's. I'm now 60.
I've interviewed many engineers. My process involves asking open-ended question about important topics, and let them explain.
Write down about 20 or 30 tidbits about the language you expect them to know. Include a range of simple, medium, hard, and, hard-core expert stuff. See where they fail, or where they struggle to explain or discuss.
You can't "practice test" your way into that, and it's always obvious where someone is knowledgable, and where they struggle, or don't know anything.
Further, I ask probe them for their 'problem solving' approaches. How do you learn new things? Name some of your favorite books on programming/development/design.
Oh, that last one, that may date me, cause I come from a time when they had these things called books. We would buy them, and read them. I've found so many developers that just say "oh, I google for it". Sure, that's great, and I google the crap out of things also.. but, here is the fundamental issue with "learning" that way.
"You only get the answer to the question you knew to ask."
The other answer, the one that says "oh, there is a totally different, but superior approach" you learn from reading and getting a solid background on the capabilities of the language/library/module/environment. Books (and online courses or comprehensive material is equivalent) teach you that stuff.
The best programmer I ever hired read books. She (yes, she) was far younger, and unlike her peers, had discovered that reading and keeping some books in your library is valuable.
My advice, ask them what they read. If they "just google it", kick them to the curb. "Next".
I've interviewed many engineers. My process involves asking open-ended question about important topics, and let them explain.
Write down about 20 or 30 tidbits about the language you expect them to know. Include a range of simple, medium, hard, and, hard-core expert stuff. See where they fail, or where they struggle to explain or discuss.
You can't "practice test" your way into that, and it's always obvious where someone is knowledgable, and where they struggle, or don't know anything.
Further, I ask probe them for their 'problem solving' approaches. How do you learn new things? Name some of your favorite books on programming/development/design.
Oh, that last one, that may date me, cause I come from a time when they had these things called books. We would buy them, and read them. I've found so many developers that just say "oh, I google for it". Sure, that's great, and I google the crap out of things also.. but, here is the fundamental issue with "learning" that way.
"You only get the answer to the question you knew to ask."
The other answer, the one that says "oh, there is a totally different, but superior approach" you learn from reading and getting a solid background on the capabilities of the language/library/module/environment. Books (and online courses or comprehensive material is equivalent) teach you that stuff.
The best programmer I ever hired read books. She (yes, she) was far younger, and unlike her peers, had discovered that reading and keeping some books in your library is valuable.
My advice, ask them what they read. If they "just google it", kick them to the curb. "Next".
One thing that I always thought would really help, which "senior developers" as a group seem to be largely against, would be a basic competency test like the medical board exam or law bar exam. Do it once, and demonstrate that you can know some minimum standard amount of knowledge and can do the bare minimum of software development. It wouldn't test every development skill out there. It wouldn't test particular language competency. It wouldn't test for business domain knowledge. And it would not replace the interview or prove that you were the right fit for any particular job. But it could weed out 90% of the fakers who literally cannot program or know nothing about software that all the hiring managers in these threads are complaining about. It would help to prevent our current mess where every company in the world has to waste time doing FizzBuzz again and again for every candidate.
I don't understand why we are against this basic, minimal gate. We have this unreasonable fear of regulation and gatekeeping in our profession, which is ultimately to everyone's collective detriment. I'm not calling for 20,000 pages of federal regulations on software engineering. A minimal gate. Nobody benefits from the current world where anyone with no knowledge or skills can simply claim to be a software engineer. Nobody except the fakers.
I don't understand why we are against this basic, minimal gate. We have this unreasonable fear of regulation and gatekeeping in our profession, which is ultimately to everyone's collective detriment. I'm not calling for 20,000 pages of federal regulations on software engineering. A minimal gate. Nobody benefits from the current world where anyone with no knowledge or skills can simply claim to be a software engineer. Nobody except the fakers.
As a senior engineer I have a very comfortable good paying job. When a tech recruiter or a job listing entices me to an interview they have a lot to overcome to get me to switch jobs. The recruiters especially often frame this as "Company x is so excited to talk to me!" and that I am going to be a great fit.
At this point it seems like the company wants me more than I want them. Then they give me a coding test and they have completely sunk any of the inertia needed to get me to leave my current job. Add to this hours and hours of interviews and I am like "Nope, I will keep my cushy job. Thank you."
It's even worse when these companies make me do a coding test before even talking to me. Companies need to get me interested and excited to work for them before I am going to spend my valuable time jumping through their hoops.
At this point it seems like the company wants me more than I want them. Then they give me a coding test and they have completely sunk any of the inertia needed to get me to leave my current job. Add to this hours and hours of interviews and I am like "Nope, I will keep my cushy job. Thank you."
It's even worse when these companies make me do a coding test before even talking to me. Companies need to get me interested and excited to work for them before I am going to spend my valuable time jumping through their hoops.
I have seen candidates present as a senior dev on paper, but clearly aren't senior dev based on interview performance or code. Without testing the code, there is a risk of hiring a senior dev (and passing on a more qualified one).
The disconnect between what is done day to day and the types of things asked for in an interview is because some candidates may refuse to do what could be seen as work product for the hiring company. There are companies out there that have taken this further and give a long "interview" that is actually work product. Making sure that the code being written is clearly demonstration / contrived examples reduces that potential conflict.
So yes, I do find coding tests useful as it allows me to put my best foot forward as a candidate demonstrating that I can write clean and well crafted code. Likewise, on the hiring side of the table, coding tests can help separate the candidates who don't care about the interview or will be a net negative after hiring from the ones that are willing to put in some effort.
Excessively long code projects are to be discouraged though - there is too much time that can be consumed by them and it biases to people who have an excess of time available to work on them.
The disconnect between what is done day to day and the types of things asked for in an interview is because some candidates may refuse to do what could be seen as work product for the hiring company. There are companies out there that have taken this further and give a long "interview" that is actually work product. Making sure that the code being written is clearly demonstration / contrived examples reduces that potential conflict.
So yes, I do find coding tests useful as it allows me to put my best foot forward as a candidate demonstrating that I can write clean and well crafted code. Likewise, on the hiring side of the table, coding tests can help separate the candidates who don't care about the interview or will be a net negative after hiring from the ones that are willing to put in some effort.
Excessively long code projects are to be discouraged though - there is too much time that can be consumed by them and it biases to people who have an excess of time available to work on them.
Actually I'm also insulted by coding exams, and even more so by the entire hiring process. Anyone who has worked two or three decades in our field like you and I have knows that the key skill needed today is the ability to learn quickly. In our information rich field, knowing how to develop a solution to a particular problem often reduces to knowing how to search and utilize information effectively. Effectiveness comes from experience and isn't easily measured by assessing our knowledge of this or that programming language or algorithm. Current hiring practices are often mediocre and managed by people with limited understanding of the nature of what we do as coders. You are right in feeling that you should avoid working for these kinds of places.
Nobody who really works on software works on these kind of bugs.
That's kind of the point. Coding tests are minimal test cases. They're the sort of thing you wish you'd get when someone reports a problem - a distilled reproducible example of something so you can focus on the problem and only the problem.
That's how coding tests should be. Tests with huge blocks of "real world" code that doesn't add anything to the problem you need to solve are much worse.
no programmer is going to write anything useful for you
This is also kind of the point. Coding tests should not produce useful code. They're designed to assess someone's ability without asking them to do proper work. If you end up doing real work in a test situation without getting properly paid for it then you're being scammed.
The problem is often not the test itself, but the failure to do the groundwork to explain why I should take the test. I have a policy that I will never take a coding test before talking to someone from the company - I don't want to waste a few hours of my time on a role I wouldn't accept even if it was offered. I need to talk to the company just to sound out whether the role sounds appropriate, whether the company sounds like somewhere I want to join, and whether or not they're willing to put up with me. If I sound like a reasonable prospect from my resume then the company should be willing to invest 30 minutes on a Zoom call for me to answer their basic questions, and for them answer a few questions I might have too, before I commit to a more indepth series of interviews. If they won't do that then they're not serious about employing me...
That's kind of the point. Coding tests are minimal test cases. They're the sort of thing you wish you'd get when someone reports a problem - a distilled reproducible example of something so you can focus on the problem and only the problem.
That's how coding tests should be. Tests with huge blocks of "real world" code that doesn't add anything to the problem you need to solve are much worse.
no programmer is going to write anything useful for you
This is also kind of the point. Coding tests should not produce useful code. They're designed to assess someone's ability without asking them to do proper work. If you end up doing real work in a test situation without getting properly paid for it then you're being scammed.
The problem is often not the test itself, but the failure to do the groundwork to explain why I should take the test. I have a policy that I will never take a coding test before talking to someone from the company - I don't want to waste a few hours of my time on a role I wouldn't accept even if it was offered. I need to talk to the company just to sound out whether the role sounds appropriate, whether the company sounds like somewhere I want to join, and whether or not they're willing to put up with me. If I sound like a reasonable prospect from my resume then the company should be willing to invest 30 minutes on a Zoom call for me to answer their basic questions, and for them answer a few questions I might have too, before I commit to a more indepth series of interviews. If they won't do that then they're not serious about employing me...
As a hiring manager, they are very useful.
As an applicant, you should consider it a good thing. You work in one of the few fields where it is possible to easily and quickly test whether someone is incompetent. If it weren’t for the tests you would be in direct competition with legions of bullshitters.
As an applicant, you should consider it a good thing. You work in one of the few fields where it is possible to easily and quickly test whether someone is incompetent. If it weren’t for the tests you would be in direct competition with legions of bullshitters.
As a candidate, coding exams are really useful for identifying companies I don't want to work for.
As a hirer, they're useful for not very much at all. They don't tell you if someone can produce working software, they don't give any insight into how someone thinks or how quickly they can learn, and they will rule out many candidates who are better than most of those they rule in.
They might help you whittle very large candidate lists down to smaller ones cheaply but if you have that problem you should be using a different approach.
With code exams as part of your hiring process over time you will skew your hiring towards presentees who have crammed and rote learned and are there for the wrong reasons.
Hire people you think are smart, try very hard to stamp on your unconscious biases, aim for a broad variety of experience levels and types in your team, give them all the support and help you can and make sure that you quickly fire people who turn out to have misled you about their skills.
As a hirer, they're useful for not very much at all. They don't tell you if someone can produce working software, they don't give any insight into how someone thinks or how quickly they can learn, and they will rule out many candidates who are better than most of those they rule in.
They might help you whittle very large candidate lists down to smaller ones cheaply but if you have that problem you should be using a different approach.
With code exams as part of your hiring process over time you will skew your hiring towards presentees who have crammed and rote learned and are there for the wrong reasons.
Hire people you think are smart, try very hard to stamp on your unconscious biases, aim for a broad variety of experience levels and types in your team, give them all the support and help you can and make sure that you quickly fire people who turn out to have misled you about their skills.
Senior dev here, yes these tests are pretty useless for experienced hires. They only test a fraction of the skills a senior dev should have. Senior developers don't work on problems that can be defined in a paragraph, coded and finished in a few hours. Their projects will often involve a very large, complex, aging system sometimes with documentation and tests. They are given assignments immediately, have to dig into code and data with little help or supervision. Find the spot in these million lines of code where I can add 10 more lines. Their experience allows them to learn large code bases, manage risk, fill gaps in knowledge, proactively communicate with others and much more.
For interviews I've done code or design walkthroughs with the candidate. Have them open up their code and talk about it. And also I'll show some of our code, and have them read and explain it. Lots of good questions and discussion will come out of it.
For interviews I've done code or design walkthroughs with the candidate. Have them open up their code and talk about it. And also I'll show some of our code, and have them read and explain it. Lots of good questions and discussion will come out of it.
This will probably make me come off as a diva, but I don't want to interview for companies that don't want to hire me for my abilities. I'm not interested in being just another index in the developers array. If that is what you are looking for, then you are quite bluntly looking for someone else.
I have an internet search engine on my resume. I have implemented a C compiler. Don't give me FizzBuzz.
I have an internet search engine on my resume. I have implemented a C compiler. Don't give me FizzBuzz.
>Is anyone else insulted by coding exams?
Yes, I refuse to take them. Fortunately I've built a pretty good network and reputation over the years so I get hired over a lunch meeting. I haven't done a serious cold code interview in years.
The last one I did was around 10 years ago and I kinda wanted to do iPhone app development full time. I had several apps on the Apple App Store. They were live/downloadable and the published by was obvious it was me. One was in the top 10 list of its category at the time. The company still wanted me to "write a quick Twitter client app by tomorrow." I did it and they kept asking for more. I finally told them I wasn't interested anymore. There wasn't an HR department or anything, it was a small startup with funding. That company is out of business now. I'd have to be pretty desperate to do that again.
I've been on both sides of the hiring table. I suspect most good senior devs won't put up with these type of shenanigans, so but requiring this sort of stuff, you're limiting your candidate pool to poor or mediocre or rising star that will leave in a couple of years at best. You should think of a question that any senior developer would know but someone who's never coded wouldn't. Something really simple like, "when declaring a method, what keyword do you use to denote that it will return nothing?"
I say this every time this comes up, but the cargo cult is strong.
Yes, I refuse to take them. Fortunately I've built a pretty good network and reputation over the years so I get hired over a lunch meeting. I haven't done a serious cold code interview in years.
The last one I did was around 10 years ago and I kinda wanted to do iPhone app development full time. I had several apps on the Apple App Store. They were live/downloadable and the published by was obvious it was me. One was in the top 10 list of its category at the time. The company still wanted me to "write a quick Twitter client app by tomorrow." I did it and they kept asking for more. I finally told them I wasn't interested anymore. There wasn't an HR department or anything, it was a small startup with funding. That company is out of business now. I'd have to be pretty desperate to do that again.
I've been on both sides of the hiring table. I suspect most good senior devs won't put up with these type of shenanigans, so but requiring this sort of stuff, you're limiting your candidate pool to poor or mediocre or rising star that will leave in a couple of years at best. You should think of a question that any senior developer would know but someone who's never coded wouldn't. Something really simple like, "when declaring a method, what keyword do you use to denote that it will return nothing?"
I say this every time this comes up, but the cargo cult is strong.
It depends on the exam. I would be at least somewhat insulted at this point to deal with l33tcode, brain teasers, and CS questions. A simple take home project can be okay depending on what it is. If it's a generic "JavaScript" quiz, I'd just walk out. Although this is tooting my own horn, I have enough of a background, enough references, and enough code on GitHub that the idea that I could somehow be lying to a company is more of a conspiracy theory than anything else.
I'm an individual contributor, but I've helped conduct quite a few interviews at my current place (where we have coding tests) and in previous places (where we haven't had them).
I totally understand where you're coming from. The coding test I had to do at my current place took me few hours to do. The company is great, and I'm happy at the place, however, I would be lying if I didn't think twice when I got told I'd have to do a coding test after the initial screening. Certainly, I did feel insulted. The reason I ended up taking it was because it was very obviously just a test (i.e. they weren't trying to get free labour out of me). In the end, it all worked out, and I'm glad I did end up doing it.
On the flip side, sadly I've worked with a lot of people who just don't know how to code. Coding exercising aren't perfect either, but it certainly weeds out a whole lot of bad hires. I've worked with individuals who have essentially become "the documentation person". This is fine if you're in an enterprise, but in a small 3/4 person team, that one person holds back the rest a lot.
I think having a _set_ coding exercise that doesn't get free labour out of the candidate, it can help a lot towards having a stronger team. As a candidate, do be careful whose coding exercise you end up doing. There are still quite a few companies that try to get free labour out of candidates.
So yes, I think it is insulting unfortunately, but it is a necessary evil to build better teams, and avoid those who are, quite frankly, lying and are going to hold the team back.
I totally understand where you're coming from. The coding test I had to do at my current place took me few hours to do. The company is great, and I'm happy at the place, however, I would be lying if I didn't think twice when I got told I'd have to do a coding test after the initial screening. Certainly, I did feel insulted. The reason I ended up taking it was because it was very obviously just a test (i.e. they weren't trying to get free labour out of me). In the end, it all worked out, and I'm glad I did end up doing it.
On the flip side, sadly I've worked with a lot of people who just don't know how to code. Coding exercising aren't perfect either, but it certainly weeds out a whole lot of bad hires. I've worked with individuals who have essentially become "the documentation person". This is fine if you're in an enterprise, but in a small 3/4 person team, that one person holds back the rest a lot.
I think having a _set_ coding exercise that doesn't get free labour out of the candidate, it can help a lot towards having a stronger team. As a candidate, do be careful whose coding exercise you end up doing. There are still quite a few companies that try to get free labour out of candidates.
So yes, I think it is insulting unfortunately, but it is a necessary evil to build better teams, and avoid those who are, quite frankly, lying and are going to hold the team back.
Interviewing people sucks. It's so hard to gauge the competence of a person in 45 minutes. Good code tests are a decent, standardized way of getting a read on a person. They are more fair and suffer less from the "confidence = competence" fallacy.
My experience has been that good coding challenges correlate strongly with good team members. It absolutely filters out good candidates on occasion, but it also filters out ones that can't code too.
From my perspective as a candidate. I just pay attention to what the recruiter says their interview process will be, and brush up on relevant skills.
My experience has been that good coding challenges correlate strongly with good team members. It absolutely filters out good candidates on occasion, but it also filters out ones that can't code too.
From my perspective as a candidate. I just pay attention to what the recruiter says their interview process will be, and brush up on relevant skills.
Ugh, so many bad practices being defended in the comments here.
I've been on both sides of the table:
- As an interviewee, my experience is that companies that use coding exams are not places I would want to work - companies that use these exams have a poor hiring process or any technical interviewers themselves either are not proficient with the technologies they are interviewing for or cannot discuss the technologies effectively. Any of those cases is a big red flag for me.
- As an interviewer, there are two problems with coding exams: either they are so short that they can only test CS knowledge instead of the actual required skill (which is translating the business problem domain into code) or they are so long they impose too great a burden on the interviewee. ("Oh, don't spend more than 4-8 hours on this project" is not an acceptable burden in my book.)
Further, the hiring manager is not the one that should be evaluating the technical ability of candidates. (Well, unless the hiring manager is hiring Employee #2 for a start-up, but that should be more about trust and hiring someone with a track-record of success.) The technical staff should be evaluating the technical capability of the candidates, and the technical staff doing the interviewing should have both the technical knowledge and communication ability to easily screen any candidate in 30-45 minutes of casual conversation, tops. The hiring manager should be looking for team fit, communication ability, and other soft skill abilities the candidates have.
Anecdotally the businesses I've encountered in the Upper Midwest that do actual interviews instead of coding exams all have solid developers as employees. The businesses that use coding exams have the second-rate (and lower) developers writing code.
No other field in engineering, barring positions requiring the professional engineer's exam, requires candidates to go through an engineering-specific exam. The employment contracts that are drawn up usually include a 90-day probationary period in which the hire can be fired again for non-performance. I don't see why software positions should be treated any differently, since that hiring process has worked (and worked well) for other engineering fields.
I've been on both sides of the table:
- As an interviewee, my experience is that companies that use coding exams are not places I would want to work - companies that use these exams have a poor hiring process or any technical interviewers themselves either are not proficient with the technologies they are interviewing for or cannot discuss the technologies effectively. Any of those cases is a big red flag for me.
- As an interviewer, there are two problems with coding exams: either they are so short that they can only test CS knowledge instead of the actual required skill (which is translating the business problem domain into code) or they are so long they impose too great a burden on the interviewee. ("Oh, don't spend more than 4-8 hours on this project" is not an acceptable burden in my book.)
Further, the hiring manager is not the one that should be evaluating the technical ability of candidates. (Well, unless the hiring manager is hiring Employee #2 for a start-up, but that should be more about trust and hiring someone with a track-record of success.) The technical staff should be evaluating the technical capability of the candidates, and the technical staff doing the interviewing should have both the technical knowledge and communication ability to easily screen any candidate in 30-45 minutes of casual conversation, tops. The hiring manager should be looking for team fit, communication ability, and other soft skill abilities the candidates have.
Anecdotally the businesses I've encountered in the Upper Midwest that do actual interviews instead of coding exams all have solid developers as employees. The businesses that use coding exams have the second-rate (and lower) developers writing code.
No other field in engineering, barring positions requiring the professional engineer's exam, requires candidates to go through an engineering-specific exam. The employment contracts that are drawn up usually include a 90-day probationary period in which the hire can be fired again for non-performance. I don't see why software positions should be treated any differently, since that hiring process has worked (and worked well) for other engineering fields.
If coding has taught me just one thing its.... keep your ego in a box.
If you need a job, do the fucking exam. if you dont, tell them to pound sand. Its as simple as the dynamic of power.
If you need a job, do the fucking exam. if you dont, tell them to pound sand. Its as simple as the dynamic of power.
> It makes me not want to work at places that do this.
I won't work places that do this. It's inconsiderate of my (highly valuable) time, and it makes me think that the company has trust issues.
You'll especially feel this way when you look at The hiring process for CTO[1], which is more conversational. I believe senior engineer interviews should be something like this, but with more heavy technical questions.
[1] - https://www.karllhughes.com/posts/cto-interview-questions
I won't work places that do this. It's inconsiderate of my (highly valuable) time, and it makes me think that the company has trust issues.
You'll especially feel this way when you look at The hiring process for CTO[1], which is more conversational. I believe senior engineer interviews should be something like this, but with more heavy technical questions.
[1] - https://www.karllhughes.com/posts/cto-interview-questions
Personally, I find the algos interview to be the bullshit one. I've lost track of the number of people we've interviewed and hired that do well on it and then once they are working for us go ahead and design unnecessarily distributed systems (i.e. split some functionality across two or more services when there wasn't a scaling requirement that necessitated splitting things up) and thought nothing of all the extra CPU and memory that was now going to be used on serialization/deserialization between those two services and that's before you consider to all the added incidental complexity from (1) separate deploys, (2) separate on-call shifts, (3) separate logs and metrics, (4) more complicated migrations of interfaces between them, (5) increased complexity in testing that the services work correctly together, etc.
In the last 7.5 years in my current role, I have yet to apply much if anything I needed to study for the algos interview for this role. I get that some adjacent roles in the company might occasionally need this stuff, but it's used rarely enough that it's a better skillset to be able to find the algo you need when you need it then study to be able to conjure one on demand only to have that knowledge atrophy while on the job.
In the last 7.5 years in my current role, I have yet to apply much if anything I needed to study for the algos interview for this role. I get that some adjacent roles in the company might occasionally need this stuff, but it's used rarely enough that it's a better skillset to be able to find the algo you need when you need it then study to be able to conjure one on demand only to have that knowledge atrophy while on the job.
It's insulting when a company is the one to reach out to someone who is senior, and already employed, but then added to some pipeline where you are expected to first jump through hoops like technical quizzes. Demonstrate you've done "preparation" for interview, asked why you want to work for company X, etc.
Smaller companies shouldn't delude themselves they are some FAANG type company with huge comp that thousands of people want to work for.
Smaller companies shouldn't delude themselves they are some FAANG type company with huge comp that thousands of people want to work for.
The tests are close to worthless. It is easy to check if someone truly did what they claim on their resume by asking questions about projects they worked on. Caught way more people misrepresenting on their capabilities by doing that but you have to have people doing the interview who has both breadth and depth in multiple areas.
I rarely see interviewer(s) asking questions to try and discover the person level of knowledge but instead ask questions showing theirs.
I rarely see interviewer(s) asking questions to try and discover the person level of knowledge but instead ask questions showing theirs.
We had a conversation about this a short while back in a local Slack channel, and my personal opinion is that I have never been fond of coding as an exercise for a technical interview, especially if it is a whiteboarding exercise in front of others or a timed event. Due to their simplicity, they offer little beyond basic syntax or entry level abilities if you're looking at a short timespan, and anything more complex is probably prohibitive if you're interviewing multiple candidates, because the code has to be reviewed.
Take home coding exercises are a bit better, but someone still has to review it, and who has that kind of time if they are interviewing a lot of potential candidates?
Instead, if a coding exercise is necessary, I would lean toward having the candidate perform a code review on a known bad piece of code, looking to see how they spot inefficiencies, and then have a separate technical conversation, akin to refining a project that someone would work on. It would be more real-world, and I think you could gauge their thinking process better, and still get an idea of the complex thought processes involved with senior development.
Take home coding exercises are a bit better, but someone still has to review it, and who has that kind of time if they are interviewing a lot of potential candidates?
Instead, if a coding exercise is necessary, I would lean toward having the candidate perform a code review on a known bad piece of code, looking to see how they spot inefficiencies, and then have a separate technical conversation, akin to refining a project that someone would work on. It would be more real-world, and I think you could gauge their thinking process better, and still get an idea of the complex thought processes involved with senior development.
I find that coding tests are a huge red flag. I usually stop any conversation as soon as I know it will be required even for freelance side jobs.
As hiring manager I do not disagree with basic tests for junior roles but for senior roles it definitely does not assess where the value is, coding is only part of the equation. I believe that a short call with a peer from the team to review a merge request/architecture decision is far more interesting.
As hiring manager I do not disagree with basic tests for junior roles but for senior roles it definitely does not assess where the value is, coding is only part of the equation. I believe that a short call with a peer from the team to review a merge request/architecture decision is far more interesting.
As a rule they aren't insulting – but as with anything there are good and bad implementations.
I've been on both sides of the fence, as both an applicant and the hiring manager. I personally hate "leetcode" nonsense, "find-the-bug"-style quizzes or algorithmic white board challenges. I'd consider going through with them as an applicant, depending on the organisation, but I don't think they're useful tools and it would be a bit of a red flag that suggested they didn't have a great hiring process. But I realise that people have different views on this and try not to take it personally.
We have settled on using a compact take-home coding exercise, which is a pretty representative example of the sort of thing we do day-to-day. There are obvious downsides to this, in that it requires a few hours of commitment, but we've tried to be as flexible and light-touch with it as possible. My experience is that roughly 50% of people totally flunk it – and this is post-first-interview, after we've had a decent chat and decided that they can communicate and have the right level of apparent expertise. And our analysis isn't that harsh – it's basically "did you write code that I could read and which does something in the ballpark of the spec". It's pretty surprising to see so many don't hit that bar.
I am very much aware of the requirement to be respectful of everyone's time, so I figure this is probably fairer than having people waste effort continuing with the process when they've no chance. Any approach you take is going to have a set of tradeoffs, and all you can really do is make sure that whatever process you put in place is checking for the things you want.
I've been on both sides of the fence, as both an applicant and the hiring manager. I personally hate "leetcode" nonsense, "find-the-bug"-style quizzes or algorithmic white board challenges. I'd consider going through with them as an applicant, depending on the organisation, but I don't think they're useful tools and it would be a bit of a red flag that suggested they didn't have a great hiring process. But I realise that people have different views on this and try not to take it personally.
We have settled on using a compact take-home coding exercise, which is a pretty representative example of the sort of thing we do day-to-day. There are obvious downsides to this, in that it requires a few hours of commitment, but we've tried to be as flexible and light-touch with it as possible. My experience is that roughly 50% of people totally flunk it – and this is post-first-interview, after we've had a decent chat and decided that they can communicate and have the right level of apparent expertise. And our analysis isn't that harsh – it's basically "did you write code that I could read and which does something in the ballpark of the spec". It's pretty surprising to see so many don't hit that bar.
I am very much aware of the requirement to be respectful of everyone's time, so I figure this is probably fairer than having people waste effort continuing with the process when they've no chance. Any approach you take is going to have a set of tradeoffs, and all you can really do is make sure that whatever process you put in place is checking for the things you want.
I've written JS and shipped numerous apps in it over the last 20 years and let me tell ya, if anyone worked for me and wrote code like I saw on those tests, I'd fire them.
My favorite was a sort question that had an obvious bug. Both sides of the sort had the same test (so no sorting would happen). But the question wasn't about that... it was about which input would tell you the code was wrong. This is most ridiculous way to diagnose or fix a bug that I've heard of.
Nobody who really works on software works on these kind of bugs. And this diagnostic technique of arbitrarily working backwards to force a multiple choice solution is probably the worst method of programming that I've ever come across. No wonder so many companies are hiring shitty programmers and writing garbage code.
Without an actual problem and an understanding of the problem domain, no programmer is going to write anything useful for you. Frankly, I'm frustrated by this method of interviewing and selecting candidates. It makes me not want to work at places that do this.
Anyone else have a similar experience? Also, hiring managers... Do you find these kind of tests useful? Why? How?