Aaron Swartz: How I Hire Programmers(aaronsw.com)
aaronsw.com
Aaron Swartz: How I Hire Programmers
http://www.aaronsw.com/weblog/hiring
98 comments
I'm not aware of all the details of Aaron's career, but I was not under the impression that he has been in a position to hire that many people. Knowing who he was hiring for what size organizations and in what capacity would be useful information.
You're right, that would be useful information. However, Aaron was one of the reddit founders and I imagine gained a few pennies from that. Perhaps the project he's hired for is openlibrary.org?
From http://openlibrary.org/about/people: "Aaron Swartz, Former Project Leader. The Open Library wouldn't exist without Aaron. He wrote the backbone of the system you see today. He has also worked on specs like RSS, startups like Reddit.com, and software projects like web.py. He worked from San Francisco to architect the site, put together the team, and attempt to keep things organized, and we couldn't have come this far without his crucial expertise."
From http://openlibrary.org/about/people: "Aaron Swartz, Former Project Leader. The Open Library wouldn't exist without Aaron. He wrote the backbone of the system you see today. He has also worked on specs like RSS, startups like Reddit.com, and software projects like web.py. He worked from San Francisco to architect the site, put together the team, and attempt to keep things organized, and we couldn't have come this far without his crucial expertise."
"Gained a few pennies" goes without doubt, but that doesn't necessarily translate into hiring. For instance, 'our very own' PG has probably hired fewer people than many people here, although I'm sure he's interviewed way more than most of us have.
Busy guy... In the past year, he also cofounded "Progressive Change Campaign Committee". Which amoung other things, has done great work on health care reform this past summer/fall.
The best "interview" I've ever had consisted of me spending an entire day hanging out with the team. We essentially pretended that that it was my first day working there. They asked me to keep coming back.
Even if they hadn't hired me I would have been happy to do it. It was enjoyable and I learned a lot of interesting stuff. I know some people would say it's unfair to ask this, but I really wish it would become the industry standard method.
Even if they hadn't hired me I would have been happy to do it. It was enjoyable and I learned a lot of interesting stuff. I know some people would say it's unfair to ask this, but I really wish it would become the industry standard method.
What I don't understand about this kind of time-intensive stuff is that everyone seems to think good programmers don't need another job, they already have one. As such, they tend to leave on their own terms and don't have periods of extended unemployment proportional to mediocre programmers.
Maybe that's true, maybe it isn't, but at least personally speaking, I always do my interviews with PTO and only quit after I get a job offer I like. I'd never burn PTO for something like that unless it was a truly extraordinary opportunity, and don't better programmers have enough options that they wouldn't have to do so?
Maybe that's true, maybe it isn't, but at least personally speaking, I always do my interviews with PTO and only quit after I get a job offer I like. I'd never burn PTO for something like that unless it was a truly extraordinary opportunity, and don't better programmers have enough options that they wouldn't have to do so?
Did they have you sign an NDA?
It seems like it'd be hard, using this tactic, to protect trade secrets in companies where that's important.
It seems like it'd be hard, using this tactic, to protect trade secrets in companies where that's important.
I did not sign an NDA. Good point though. This company truly did have very sensitive IP to protect (unlike most companies IMHO). The CTO chaperoned me the entire day and made sure I didn't see/touch/hear anything too sensitive.
Facebook makes everyone who walks in the door sign an NDA, including visitors and people coming in for interviews. It literally takes 15 seconds to do.
Really? How can you read a legal document that quickly?
If I was signing something that could get me sued in the future, I'd make darn sure that I read it thoroughly before signing!
That's just common sense:)
If I was signing something that could get me sued in the future, I'd make darn sure that I read it thoroughly before signing!
That's just common sense:)
Few people read contracts they sign any more. In most of our (U.S., at least) culture, people no longer view a contract as an agreement between party A and party B, but as a paperwork step that has to be signed as a formality. I think most people don't even read their own employment contracts; they just sign however many places it calls for and move on.
With EULAs and click-through "contracts" devaluing the term, the average person probably signs hundreds of contracts a year, and reads none of them.
With EULAs and click-through "contracts" devaluing the term, the average person probably signs hundreds of contracts a year, and reads none of them.
I mostly agree with you, but I think it also depends on the types of contracts we're talking about. I definitely read all 20+ pages of my apartment lease before I signed it, but not a word of the iTunes EULA when upgrading it.
You must spend a lot of time reading legal documents then. Damn near everything involves me signing something saying that I can either get sued or can't sue over something.
I don't think it's that many. This year I signed a new credit card contract, contract extension with my landlord, mobile phone contract, some online service direct debit signup and a car rental paperwork. That's it - and I read all of that. I'm not sure a typical person ever goes over 10 things to sign a year... It's not that many and it really pays off to read them before you sign.
It's a half-page NDA. You can read it in about 2 minutes.
So it literally takes 15 seconds to do?
Secrecy always harms something else. Most obvious example is closed source being less convenient for users to work with when they have to peek inside (e.g. bugs in closed source drivers).
Also, I doubt any secret that can e leaned in a day is worth a lot.
Also, I doubt any secret that can e leaned in a day is worth a lot.
This works well as a second interview, in my experience (and yes, use an NDA). It's still helpful to have a first interview, specifically as a reality check to weed out people for whom the day would be a waste of everybody's time.
Did they phone-screen you before they asked you to come over? I can't imagine a company asking each of its candidates to come spend an entire day with them without a prior filtering of sorts..
I came recommended by someone inside the company, as almost everyone else there did.
I think this is a terrible way to interview. I've encountered, again and again, candidates who appear to be very intelligent, you can have a great discussion with them, terrific rapport, they understand everything, they give interesting suggestions, and when you ask them to write a simple piece of code they absolutely cannot do it. Not because they're stressed etc., you can actually see they can't write code. Sometimes the incongruity between their confident appearance and convincing talk versus their inability to write code is mind-boggling.
Now I consider any interview process incomplete and utterly unreliable if it doesn't include writing actual code, however simple and straightforward.
Now I consider any interview process incomplete and utterly unreliable if it doesn't include writing actual code, however simple and straightforward.
He was pretty clear that he looks at their code. Go back and read the third paragraph.
The place I'm currently at interviews in a very similar matter and we're currently batting 3 for 3 on new hires over the last year or so. I've never been a fan of the "run puzzles at them until they fall over" method. Google's hiring methods gave me a big fat headache and made me run as fast as I could from there.
I recently interviewed at another company that does a variation on this by sending you a coding challenge and you send the results back within 2 hours (you text them at a certain time and then they send the challenge). I liked this approach as well.
Both have their pros and cons but each of them will show you how someone codes. The hardest part for us has been figuring out if their style/approach to problem solving meshes with the team. That's still hard to determine in an interview because it's something of a personality thing We've worked our way through those issues, though.
The place I'm currently at interviews in a very similar matter and we're currently batting 3 for 3 on new hires over the last year or so. I've never been a fan of the "run puzzles at them until they fall over" method. Google's hiring methods gave me a big fat headache and made me run as fast as I could from there.
I recently interviewed at another company that does a variation on this by sending you a coding challenge and you send the results back within 2 hours (you text them at a certain time and then they send the challenge). I liked this approach as well.
Both have their pros and cons but each of them will show you how someone codes. The hardest part for us has been figuring out if their style/approach to problem solving meshes with the team. That's still hard to determine in an interview because it's something of a personality thing We've worked our way through those issues, though.
What sort of programming problems are you giving them? Did these candidates have experience with other projects and were they able to fluidly explain their involvement with them?
It must be pretty rare to find a person who can describe a complex project in detail but is unable to write any code during an interview.
It must be pretty rare to find a person who can describe a complex project in detail but is unable to write any code during an interview.
I thought it was rare too until I started interviewing a lot.
Yes, these candidates would typically be able to explain their involvement with previous projects, to give a technical overview of what they'd done. They would totally pass my bullshit filter. I thought I had a well-tuned bullshit filter. I was wrong.
The programming problems are typically very simple, an order of magnitude simpler than an algorithm or design question that might be a part of the same interview. I don't want to give specific examples, but maybe implement one of a standard algorithms on an array or a tree (I'm not talking balanced trees, way simpler). Maybe do something that requires a nested loop and some bit-twiddling, or careful thinking about entries in a two-dimensional array, or a bit of smarts to hold some sparse data - but again, nearly trivial from the algorithmic point of view. There are many possibilities.
What you want to check is whether the candidate can think through their own code, whether they know to check for edge cases, whether they think of test cases, how they deal with off-by-1 possibilities, how they debug their own code, do they have a reasonably clear style.
What you tend to see when a candidate fails such an assignment is people who don't remember the mask to fill a byte with 1's, who can't get a nested loop right because they can't seem to keep two counters in their head at the same time, who don't have the instinct to check for an off-by-1, who can't trace through their own code visually with sample arguments, etc. etc. Sometimes they had just impressed you a lot a few minutes before with a nice technical summary of what they did in their previous project.
Yes, these candidates would typically be able to explain their involvement with previous projects, to give a technical overview of what they'd done. They would totally pass my bullshit filter. I thought I had a well-tuned bullshit filter. I was wrong.
The programming problems are typically very simple, an order of magnitude simpler than an algorithm or design question that might be a part of the same interview. I don't want to give specific examples, but maybe implement one of a standard algorithms on an array or a tree (I'm not talking balanced trees, way simpler). Maybe do something that requires a nested loop and some bit-twiddling, or careful thinking about entries in a two-dimensional array, or a bit of smarts to hold some sparse data - but again, nearly trivial from the algorithmic point of view. There are many possibilities.
What you want to check is whether the candidate can think through their own code, whether they know to check for edge cases, whether they think of test cases, how they deal with off-by-1 possibilities, how they debug their own code, do they have a reasonably clear style.
What you tend to see when a candidate fails such an assignment is people who don't remember the mask to fill a byte with 1's, who can't get a nested loop right because they can't seem to keep two counters in their head at the same time, who don't have the instinct to check for an off-by-1, who can't trace through their own code visually with sample arguments, etc. etc. Sometimes they had just impressed you a lot a few minutes before with a nice technical summary of what they did in their previous project.
He mentions in the article that the interviewees DO write code in his process
No, he said that he would ask for a code sample or demo. There doesn't seem to be any programming involved during the interview.
"I ask them to do part of the job. Usually this means picking some fairly separable piece we need and asking them to write it."
I have no recollection of reading this paragraph when I first read the submission. Maybe I skipped it somehow. Anyway, this doesn't make it clear whether this is done face-to-face as part of the interview, or offline. I also doubt it's practical.
Sorry, you're right.
I don't think that paragraph and the proceeding one was there when I originally read the post. Seems it's been edited since it was submitted.
I don't think that paragraph and the proceeding one was there when I originally read the post. Seems it's been edited since it was submitted.
Both you and Anatoly are right: the paragraphs were added later, based on comments here and to the blog itself.
See this thread in the comments on the blog (from the author): "The comments here and on Hacker News reminded me I left out a step. I’ve added it." (http://www.aaronsw.com/weblog/hiring#c9)
I just didn't want you two thinking you were crazy or somehow accidentally skipped two paragraphs.
See this thread in the comments on the blog (from the author): "The comments here and on Hacker News reminded me I left out a step. I’ve added it." (http://www.aaronsw.com/weblog/hiring#c9)
I just didn't want you two thinking you were crazy or somehow accidentally skipped two paragraphs.
Thanks, much appreciated.
I think this is a great technique. To all those who think programming IS a job done under pressure, really think hard about your specific job and your specific situation. We all like to glorify our careers (to the point of fooling ourselves) and imagine scenarios where something mission critical breaks and it HAS to be fixed in five minutes (something like you'd see on the show 24).
But has that ever happened at your job? If so, instead of hiring high pressure programmers, could you take steps to eliminate the high pressure scenarios, e.g., unit testing, back up systems, roll back plans?
I know everyone will have counterexamples of this high pressure thing that happens.. but is it really typical for your job, really?
But has that ever happened at your job? If so, instead of hiring high pressure programmers, could you take steps to eliminate the high pressure scenarios, e.g., unit testing, back up systems, roll back plans?
I know everyone will have counterexamples of this high pressure thing that happens.. but is it really typical for your job, really?
My favorite hiring process I've been a part of was with a AI startup. The initial phase was a list of maybe a dozen questions in an email. Fairly open-ended questions about how I perceived aspects of intelligence, ontology, and general philosophical concepts. Then a day at the company talking with the founder and the team, walking through their concept, long two way discussions about it, and a discussion about the direction of AI in general. Didn't get into a lot of coding details, it was assumed if I could participate in the conversation at a sufficient level that I'd be able to work the rest of it out. Which was true. Great process, completely appropriate for that situation.
Something else to consider ...
There are a lot of developers that "get stuff done" without accomplishing anything of use to the company.
There are a lot of developers that "get stuff done" without accomplishing anything of use to the company.
I think that's mostly their manager's fault.
Part of "get stuff done" is that you manage yourself and communicate with your manager well enough to ensure you accomplish something of use to the team.
Interviewers should understand your contribution to the company isn't in your control, but your contribution to the team was.
Interviewers should understand your contribution to the company isn't in your control, but your contribution to the team was.
Inspired no doubt by Joel Spolsky's famous (in some circles, anyway) article: http://www.joelonsoftware.com/articles/GuerrillaInterviewing...
Their objectives are basically the same. The difference is in the ways of proving the "getting things done" part.
I never assumed the programming questions in an interview were really about getting the right answer in ten minutes or however long you spend on it, but more about giving the interviewee an opportunity to explain their thought process and problem solving approach to the interviewer. I don't agree that asking interviewees to solve programming problems in an interview is useless, but obviously looking at code samples from other work is great, too.
Their objectives are basically the same. The difference is in the ways of proving the "getting things done" part.
I never assumed the programming questions in an interview were really about getting the right answer in ten minutes or however long you spend on it, but more about giving the interviewee an opportunity to explain their thought process and problem solving approach to the interviewer. I don't agree that asking interviewees to solve programming problems in an interview is useless, but obviously looking at code samples from other work is great, too.
>"I never assumed the programming questions in an interview were really about getting the right answer in ten minutes or however long you spend on it, but more about giving the interviewee an opportunity to explain their thought process and problem solving approach to the interviewer."
That's what they say. But do you know anyone who failed the questions and still got hired? My experience has been that even when I nail the algorithmic questions, I miss a few trivia questions and don't get called back. Nuance cannot be replicated at corporate scale.
That's what they say. But do you know anyone who failed the questions and still got hired? My experience has been that even when I nail the algorithmic questions, I miss a few trivia questions and don't get called back. Nuance cannot be replicated at corporate scale.
Well, I can say on the flip side I've been involved with hiring decisions where the interviewee did not nail every single answer, but we hired anyhow.
It should be pointed out that I structure the interviews to try to avoid that case anyhow. If your interviewee nails everything, it either means that you've got a class-A programmer, or your interview difficulty tops out too soon. I've become a big believer in gradated interviews. I've got a pretty good SQL series that starts out where the answer is literally "SELECT * FROM some_table" (though I find it occasionally trips people up to simply omit the where clause) and goes up from there on this little two-table schema. I've got some good HTML generation series that starts out with a simple "can you dump out a table from a 2D array or equivalent structure?" (since I try to do language-agnostic interviews) and go up through "do you know about escaping" and "can you handle optional parameters" and so on. Over time I've been selecting for the series that produce more gradation.
It should be pointed out that I structure the interviews to try to avoid that case anyhow. If your interviewee nails everything, it either means that you've got a class-A programmer, or your interview difficulty tops out too soon. I've become a big believer in gradated interviews. I've got a pretty good SQL series that starts out where the answer is literally "SELECT * FROM some_table" (though I find it occasionally trips people up to simply omit the where clause) and goes up from there on this little two-table schema. I've got some good HTML generation series that starts out with a simple "can you dump out a table from a 2D array or equivalent structure?" (since I try to do language-agnostic interviews) and go up through "do you know about escaping" and "can you handle optional parameters" and so on. Over time I've been selecting for the series that produce more gradation.
Potentially worthless anecdote: I mostly botched an algorithm for finding Pythagorean triplets in an interview recently, and still got called back. (For round 2 I brought a nice big cup of coffee ;-)
I don't have the experience to know how well it works in the general case, though. Anyway, I don't think Aaron is arguing against interviewers using the technique wrong, but rather that it doesn't correlate at all with "good hires", even when used more intelligently. I don't think that's true.
I don't have the experience to know how well it works in the general case, though. Anyway, I don't think Aaron is arguing against interviewers using the technique wrong, but rather that it doesn't correlate at all with "good hires", even when used more intelligently. I don't think that's true.
I'm surprised nobody has talked about pairing interviews in the comments yet. I spent years clumsily hiring developers crapshoot-style. I shared the misconceptions that people get so nervous during technical interviews that it obviates their usefulness and that it doesn't really matter if a candidate writes code or not during their interview. My current employer, Pivotal Labs, hired me after a first interview that consists of interview-y conversation and a short pairing exercise followed by a second full day of pairing with the team on an actual project.
I know pairing is sometimes controversial around these parts, but for interview purposes its benefits are pretty evident.
I know pairing is sometimes controversial around these parts, but for interview purposes its benefits are pretty evident.
On one hand, I have always looked forward to and enjoyed Aaron's writing, and was really curious about what he had to say in this post.
On the other hand, this is a subject near and dear to my own heart, and while I appreciate a lot of his methods, I don't think his model scales well. In order for it to work the way he wants, he has to conduct the interviews himself.
A few thoughts...
Programming isn’t typically a job done under pressure, so seeing how people perform when nervous is pretty useless.
Not my experience. Actually, quite the contrary.
The single biggest difference I have ever seen (over and over again) between a good programmer and a great programmer is how they respond under pressure. Given enough time, lots of programmers can code something that works. The problem is that there are many occasions where there simply isn't enough time. Can you hit a deadline? Can you stay awake all night if you have to? Can you resolve this problem that's holding everything and everyone else up? Can you settle down that user or customer who's up in arms? Can you figure out what wrong decision was made 4 steps ago that is causing big problems now? And can you do it now?
Understanding how well someone performs under pressure (and whether or not that makes them nervous) is hardly useless. I can't think of too many more things that are useful to know.
So I just request a code sample and a demo and see whether it looks good.
There are too many cases where this doesn't work at all. What if they've never contributed to an open source project? (Whether or not you like this has nothing to do with their skill as a programmer and is another subject). What if they're prohibited from sharing someone else's proprietary property? What if they're bound by someone else's (illogical) shop standards? What if they salvaged someone else's shitty architecture with great workaround code? If they wrote something as part of a team, which part was theirs? How much of the analysis, design, architecture, testing, and deployment did they do? For that matter, how much of the actual programming did they do? Lots of posers share something they "worked on" but could never build in a hundred years.
To find out whether someone’s smart, I just have a casual conversation with them.
That's it? You make judgements on the fly? Using what metrics?
The idea of a learning from a casual conversation makes a lot of sense. But what is your plan? Ten people having ten conversations will come up with ten different assessments of the same candidate. Unless there is some kind of standard and a plan going into the meeting. How will you know they are smart? How will you know they can get things done? How will you know they can work with others? Without a plan to answer these questions, you're just waving your hands.
I suspect OP already has a plan of attack for his casual conversations but it's hard to say because he never says anything about it.
Ask them what they’ve been thinking about and probe them about it.
Again, this may or may not be a good litmus test. I know programmers with IQs of 160 who have written tons of brilliant code. If I asked tham what they've been thinking about, the answer could be "who will win the Notre Dame game" or "will I get laid tonight" just as easily as it can be about something truly cerebral.
Finally, I figure out whether I can work with someone just by hanging out with them for a bit.
Again, you can learn a lot about someone else this way, but sooner or later, you'll need some kind of standard and metric for this method to be replicable. Before you start, you must be able to say, "This is when I will know what I need to know." Until then this is just one guy's hand waving.
I’m amazed that so many companies use such silly hiring methods instead.
I'm amazed that so many companies use such silly methods for so many other things as well. But there are just as many who hire very well. I know many great programmers employed and still delivering great product 15 or 20 years with the same company. The same companies that seldom make bad hires. They use a lot of OP's strategies. But they also combine these strategies into a comprehensive system that works no matter who does the hiring.
I like OP's thinking. I just don't see how it will work well once his company becomes too big for him to do every interview himself. Systemitizing his methods will make it scale.
On the other hand, this is a subject near and dear to my own heart, and while I appreciate a lot of his methods, I don't think his model scales well. In order for it to work the way he wants, he has to conduct the interviews himself.
A few thoughts...
Programming isn’t typically a job done under pressure, so seeing how people perform when nervous is pretty useless.
Not my experience. Actually, quite the contrary.
The single biggest difference I have ever seen (over and over again) between a good programmer and a great programmer is how they respond under pressure. Given enough time, lots of programmers can code something that works. The problem is that there are many occasions where there simply isn't enough time. Can you hit a deadline? Can you stay awake all night if you have to? Can you resolve this problem that's holding everything and everyone else up? Can you settle down that user or customer who's up in arms? Can you figure out what wrong decision was made 4 steps ago that is causing big problems now? And can you do it now?
Understanding how well someone performs under pressure (and whether or not that makes them nervous) is hardly useless. I can't think of too many more things that are useful to know.
So I just request a code sample and a demo and see whether it looks good.
There are too many cases where this doesn't work at all. What if they've never contributed to an open source project? (Whether or not you like this has nothing to do with their skill as a programmer and is another subject). What if they're prohibited from sharing someone else's proprietary property? What if they're bound by someone else's (illogical) shop standards? What if they salvaged someone else's shitty architecture with great workaround code? If they wrote something as part of a team, which part was theirs? How much of the analysis, design, architecture, testing, and deployment did they do? For that matter, how much of the actual programming did they do? Lots of posers share something they "worked on" but could never build in a hundred years.
To find out whether someone’s smart, I just have a casual conversation with them.
That's it? You make judgements on the fly? Using what metrics?
The idea of a learning from a casual conversation makes a lot of sense. But what is your plan? Ten people having ten conversations will come up with ten different assessments of the same candidate. Unless there is some kind of standard and a plan going into the meeting. How will you know they are smart? How will you know they can get things done? How will you know they can work with others? Without a plan to answer these questions, you're just waving your hands.
I suspect OP already has a plan of attack for his casual conversations but it's hard to say because he never says anything about it.
Ask them what they’ve been thinking about and probe them about it.
Again, this may or may not be a good litmus test. I know programmers with IQs of 160 who have written tons of brilliant code. If I asked tham what they've been thinking about, the answer could be "who will win the Notre Dame game" or "will I get laid tonight" just as easily as it can be about something truly cerebral.
Finally, I figure out whether I can work with someone just by hanging out with them for a bit.
Again, you can learn a lot about someone else this way, but sooner or later, you'll need some kind of standard and metric for this method to be replicable. Before you start, you must be able to say, "This is when I will know what I need to know." Until then this is just one guy's hand waving.
I’m amazed that so many companies use such silly hiring methods instead.
I'm amazed that so many companies use such silly methods for so many other things as well. But there are just as many who hire very well. I know many great programmers employed and still delivering great product 15 or 20 years with the same company. The same companies that seldom make bad hires. They use a lot of OP's strategies. But they also combine these strategies into a comprehensive system that works no matter who does the hiring.
I like OP's thinking. I just don't see how it will work well once his company becomes too big for him to do every interview himself. Systemitizing his methods will make it scale.
"The single biggest difference I have ever seen (over and over again) between a good programmer and a great programmer is how they respond under pressure."
There is a huge difference between being under pressure and the unique pressure of performing in an interview. The former is being able to handle a longer term fear of failure. The latter is dealing with a short-term fear that activates the primitive brain's "fight or flight" instinct.
There is a HUGE difference, and I would never trust anything I got from the latter style interview.
There is a huge difference between being under pressure and the unique pressure of performing in an interview. The former is being able to handle a longer term fear of failure. The latter is dealing with a short-term fear that activates the primitive brain's "fight or flight" instinct.
There is a HUGE difference, and I would never trust anything I got from the latter style interview.
I'd also like to add that the interview question type of pressure is of the type "I have to start moving my mouth within 4 seconds or I'll look bad."
All-night overtime or short deadline pressures are completely different beasts.
All-night overtime or short deadline pressures are completely different beasts.
>> Programming isn’t typically a job done under pressure, so seeing how people perform when nervous is pretty useless.
> Not my experience. Actually, quite the contrary.
> The single biggest difference I have ever seen (over and over again) between a good programmer and a great programmer is how they respond under pressure. Given enough time, lots of programmers can code something that works. The problem is that there are many occasions where there simply isn't enough time. Can you hit a deadline? Can you stay awake all night if you have to? Can you resolve this problem that's holding everything and everyone else up? Can you settle down that user or customer who's up in arms? Can you figure out what wrong decision was made 4 steps ago that is causing big problems now? And can you do it now?
> Understanding how well someone performs under pressure (and whether or not that makes them nervous) is hardly useless. I can't think of too many more things that are useful to know.
I'm pretty sure your examples conflate different sorts of pressure, and I don't think how you perform for one reflects how you'll perform for all. I've been in all those situations and the physiological responses are different.
There's never been a time on the job that I've felt the sort of pressure I have in an interview. In fact, if that were routine, I'm pretty sure my friends would have burned out of software development long ago. Moreover, if my job routinely required last second cowboy-style programming, which is sure to lead to bad design and bugs, I'd get a new job.
I'd say that, for most software positions, if it makes sense to hire programmers whose success depends on performing well under the same physiological conditions as are normal during an interview, something's broken in your processes.
Disclosure: As an ex-basketball player, crunch time pressure is extremely similar to interview pressure, and I handle both exceedingly well. I've always felt a little fraudulent since I come off better in interviews than some programmers who are more talented than I am.
> Not my experience. Actually, quite the contrary.
> The single biggest difference I have ever seen (over and over again) between a good programmer and a great programmer is how they respond under pressure. Given enough time, lots of programmers can code something that works. The problem is that there are many occasions where there simply isn't enough time. Can you hit a deadline? Can you stay awake all night if you have to? Can you resolve this problem that's holding everything and everyone else up? Can you settle down that user or customer who's up in arms? Can you figure out what wrong decision was made 4 steps ago that is causing big problems now? And can you do it now?
> Understanding how well someone performs under pressure (and whether or not that makes them nervous) is hardly useless. I can't think of too many more things that are useful to know.
I'm pretty sure your examples conflate different sorts of pressure, and I don't think how you perform for one reflects how you'll perform for all. I've been in all those situations and the physiological responses are different.
There's never been a time on the job that I've felt the sort of pressure I have in an interview. In fact, if that were routine, I'm pretty sure my friends would have burned out of software development long ago. Moreover, if my job routinely required last second cowboy-style programming, which is sure to lead to bad design and bugs, I'd get a new job.
I'd say that, for most software positions, if it makes sense to hire programmers whose success depends on performing well under the same physiological conditions as are normal during an interview, something's broken in your processes.
Disclosure: As an ex-basketball player, crunch time pressure is extremely similar to interview pressure, and I handle both exceedingly well. I've always felt a little fraudulent since I come off better in interviews than some programmers who are more talented than I am.
I can generally do interviews, but for quite a few years I had problems with burning out easily. I had to take a long vacation every year, etc.
I found the reason in the end. I wasn't extra stress sensitive. I just couldn't relax during stressed periods -- I was "on" 24/7.
Mediation solved that.
Edit: This was a bit personal, but I'll let it be. It might help someone. (AA, etc.)
I found the reason in the end. I wasn't extra stress sensitive. I just couldn't relax during stressed periods -- I was "on" 24/7.
Mediation solved that.
Edit: This was a bit personal, but I'll let it be. It might help someone. (AA, etc.)
s/Mediation/Meditation/
Sigh.
Sigh.
I know it's probably an exaggeration, but IQs of 160 are the 99.997th percentile so there should be something like 10,000 people in the united states of all ages with 160+ IQ's. Suggesting those are the type of people you want to recruit is mostly a waste of time because there are just not that many people let alone people with that IQ let alone people with that IQ who are looking for programming jobs.
Hmm,
Try as we might, IQs don't actually follow a bell curve. There are many more on the high and low ends than would be expected by looking at the curvature in the middle. Whether this is a measurement error or something else, we don't know.
Also, because of the unique position of the USA in the world (and especially places like Silicon Valley, Cambridge, Pasadena, New York) an awful lot of the right hand side of the bell curve for nations like China, India, Russia, etc. are currently living here, and often working for tech companies.
Additionally, when you do happen to get extremely brilliant people excited and working on the same thing as you in the same organization, the effects are magical. I've experienced nothing else like it.
Finally, when you're in Silicon Valley, you get the sense that there are maybe a couple thousand serious players who, if you look close, are directly involved in or are supporting most of the major efforts. It is a small world -- it seems like it's about two degrees of separation, give or take. You really get the sense that the right kind of people are the crucial ingredient that fires the whole engine of innovation here.
IQ is probably one of the less important characteristics in most positions in most companies. But it's still an ok measure for something important.
But much better to have people who are not merely good intellectual generalists (high 'g') but who are actually alarmingly good at the actual things you need to do in your company. There are as many times more of these as there are distinct things to do.
Try as we might, IQs don't actually follow a bell curve. There are many more on the high and low ends than would be expected by looking at the curvature in the middle. Whether this is a measurement error or something else, we don't know.
Also, because of the unique position of the USA in the world (and especially places like Silicon Valley, Cambridge, Pasadena, New York) an awful lot of the right hand side of the bell curve for nations like China, India, Russia, etc. are currently living here, and often working for tech companies.
Additionally, when you do happen to get extremely brilliant people excited and working on the same thing as you in the same organization, the effects are magical. I've experienced nothing else like it.
Finally, when you're in Silicon Valley, you get the sense that there are maybe a couple thousand serious players who, if you look close, are directly involved in or are supporting most of the major efforts. It is a small world -- it seems like it's about two degrees of separation, give or take. You really get the sense that the right kind of people are the crucial ingredient that fires the whole engine of innovation here.
IQ is probably one of the less important characteristics in most positions in most companies. But it's still an ok measure for something important.
But much better to have people who are not merely good intellectual generalists (high 'g') but who are actually alarmingly good at the actual things you need to do in your company. There are as many times more of these as there are distinct things to do.
"Intelligence Quotient"
The value is defined by a bell curve.
The value is defined by a bell curve.
No. No. No.
The IQ test was invented by Alfred Binet as a way of identifying people who were having trouble in school because they were mentally slow. For this he came up with a large number of different questions that exercised the brain in various ways, and figured out how well an average kid would do. When he gave the test to a real kid he would take their performance and figure out a "mental age" that they performed at. Their intelligence quotient was then defined as 100 * (mental age) / (physical age).
The development of IQ tests aimed at adults which are defined based on a bell curve was a later innovation. The name was kept simply because it was then well-known.
The IQ test was invented by Alfred Binet as a way of identifying people who were having trouble in school because they were mentally slow. For this he came up with a large number of different questions that exercised the brain in various ways, and figured out how well an average kid would do. When he gave the test to a real kid he would take their performance and figure out a "mental age" that they performed at. Their intelligence quotient was then defined as 100 * (mental age) / (physical age).
The development of IQ tests aimed at adults which are defined based on a bell curve was a later innovation. The name was kept simply because it was then well-known.
The value is defined by your test results. The tests generate scores that are roughly normally distributed up to about an IQ of 130 (two standard deviations), but then have many more people at high IQs than a bell curve would predict.
Most IQ tests can't even measure above 155 or so, anyway, and they obviously can't measure below 0. So it's a little silly to talk about them being normally distributed.
Most IQ tests can't even measure above 155 or so, anyway, and they obviously can't measure below 0. So it's a little silly to talk about them being normally distributed.
I could just as well have replied to any of the other comments below this talking about IQ, but I wanted to make sure people know that 1) IQ is not a perfect measure of one's intelligence, nor even necessarily the type of intelligence needing for programming, and 2) IQ tests are far from perfect in even measuring IQ. I don't know my IQ because I've scored anywhere from 120 to 200 on IQ tests.
As a programmer with an IQ of ~125, I'm doubtful that I would want to be a programmer if my IQ was 160+. I have a hard time believing that there are many professional programmers with IQs that high.
Having an IQ of 160 or above is not a life of sitting on clouds thinking deep thoughts about how string theory is obviously wrong. You run especially hard into the nurture trap so often discussed here.
That is, hearing,"You're so smart," so many times, from teachers, other students, coworkers, bosses, etc., cultivates a sense of elitism and a shock when something is actually difficult.
You're also bound by the same emotions as most other people. If you're a particularly aggressive or irritable person, imagine being surrounded by people who keep making mistakes because they're stupid. If you turn it outward, you're impatient at best and a bully at worst. If you turn it inward, you're stressed out, anxious, and depressed.
All of that said, being a professional programmer was the most satisfying thing I've done. Unfortunately, I moved on precisely because of the above paragraph -- I had managers I felt were particularly dim.
That is, hearing,"You're so smart," so many times, from teachers, other students, coworkers, bosses, etc., cultivates a sense of elitism and a shock when something is actually difficult.
You're also bound by the same emotions as most other people. If you're a particularly aggressive or irritable person, imagine being surrounded by people who keep making mistakes because they're stupid. If you turn it outward, you're impatient at best and a bully at worst. If you turn it inward, you're stressed out, anxious, and depressed.
All of that said, being a professional programmer was the most satisfying thing I've done. Unfortunately, I moved on precisely because of the above paragraph -- I had managers I felt were particularly dim.
Always worth mentioning: "The Inverse Power of Praise" http://nymag.com/news/features/27840/
I do not appreciate compliments anymore. I haven't for a long time. I have a complicated view I don't really wish to explain right now (maybe I'll write a blog post later) about why compliments are almost insulting, especially insulting by those people who care about "you" and who give them often (Sorry for that passive voice).
My thoughts all stemmed from the idea that compliments make people worse at what they do. At BEST they make the person continue at the same speed with a boost of moral. Imho.
My thoughts all stemmed from the idea that compliments make people worse at what they do. At BEST they make the person continue at the same speed with a boost of moral. Imho.
Anecdote: while the environment is probably a contributing factor, the 25-30 teenagers and adults I know with IQs of 130-160 (gifted children's program) basically make the same career and life choices as others in the same socio-economic group, with the exception of post-graduate degrees, which they mostly avoided.
What do you imagine you would want to be?
IQ has fat tails. There are many more people with IQs over 160 than the bell curve would predict.
I'm curious. Do you know any hiring method which scales well while maintaining high standards? I have yet to encounter any company where the quality of hires didn't decrease as the company grew.
My company is up to ~40 programmers in my regional office, and our method appears to scale well enough for now.
We let everyone who asks take a very relaxed programming test. Simple (if you know basic algorithms, at least) problem, 4 hours deadline, work from home or in one of our free offices, programming language up to the interviewee. The delivered program is then rated by a few of our programmers who give some comments and a thumbs up or down. If thumbs are generally up, the candidate goes to the interview. The ones who get to the interview almost always end up hired.
So, basically, we have 3 tasks:
* Answer emails, send out tests, receive deliveries, and schedule interviews or say "sorry" to each candidate: This is the most stressful, so we rotate that every once in a while.
* Going over code and commenting: Enough people consider this a somewhat fun break from normal work that we don't have any problems scaling this.
* Interviewing: This has to require the R&D leader and at least one senior programmer. Since the people who get this far are almost always hired, it takes up no more of their time than it has to.
We let everyone who asks take a very relaxed programming test. Simple (if you know basic algorithms, at least) problem, 4 hours deadline, work from home or in one of our free offices, programming language up to the interviewee. The delivered program is then rated by a few of our programmers who give some comments and a thumbs up or down. If thumbs are generally up, the candidate goes to the interview. The ones who get to the interview almost always end up hired.
So, basically, we have 3 tasks:
* Answer emails, send out tests, receive deliveries, and schedule interviews or say "sorry" to each candidate: This is the most stressful, so we rotate that every once in a while.
* Going over code and commenting: Enough people consider this a somewhat fun break from normal work that we don't have any problems scaling this.
* Interviewing: This has to require the R&D leader and at least one senior programmer. Since the people who get this far are almost always hired, it takes up no more of their time than it has to.
Why don't you think this technique will scale? It asks three questions: Are they smart? Have they done good stuff in the past? Do you think you can work with them? These seem like really simple questions anyone can answer and in my experience competent people almost always answer them the same way.
Why don't you think this technique will scale?
Because you have provided no mechanism to insure consistent results no matter who is conducting the assessment.
If you and I each assessed the same candidate using your methods, we could very easily come up with 2 very different assessments. And that's the trap you'll fall into as soon as you're not conducting every interview.
All 3 of your questions are important. How do you know when you have the answer?
What is the definition of "smart"? "good stuff in the past"? "works well with others"? We all have different definitions.
You may think someone is smart because they can code Algorithm X 5 different ways in 7 minutes. But I may think they're not so smart because they piss off the user trying to determine what the algorithm should do.
You may love something they have done in the past while I hate it. For a thousand different reasons.
Can they code a bubble sort in 2 different languages in 15 minutes? Can they do an iteration with an early exit, based upon an easily determined condition? How much can they speed up a piece of code with multiple obvious opportunities? Whether or not you'd ever use these particular questions, they are much more likely to give questions like yours the consistent yes/no answer you want.
I applaud your desire to avoid the sins of the past. You lay out a sensible approach. What's also needed is the application of some standards and metrics to give a higher probability of consistent results. That's all.
Because you have provided no mechanism to insure consistent results no matter who is conducting the assessment.
If you and I each assessed the same candidate using your methods, we could very easily come up with 2 very different assessments. And that's the trap you'll fall into as soon as you're not conducting every interview.
All 3 of your questions are important. How do you know when you have the answer?
What is the definition of "smart"? "good stuff in the past"? "works well with others"? We all have different definitions.
You may think someone is smart because they can code Algorithm X 5 different ways in 7 minutes. But I may think they're not so smart because they piss off the user trying to determine what the algorithm should do.
You may love something they have done in the past while I hate it. For a thousand different reasons.
Can they code a bubble sort in 2 different languages in 15 minutes? Can they do an iteration with an early exit, based upon an easily determined condition? How much can they speed up a piece of code with multiple obvious opportunities? Whether or not you'd ever use these particular questions, they are much more likely to give questions like yours the consistent yes/no answer you want.
I applaud your desire to avoid the sins of the past. You lay out a sensible approach. What's also needed is the application of some standards and metrics to give a higher probability of consistent results. That's all.
Since I wrote something questioning Aaron above, I'll take the other side with this one:
> If you and I each assessed the same candidate using your methods, we could very easily come up with 2 very different assessments. And that's the trap you'll fall into as soon as you're not conducting every interview.
...
> What's also needed is the application of some standards and metrics to give a higher probability of consistent results.
I don't think that's necessarily true. If you're Google or IBM, it might be, but if you're hiring for a smaller firm, it may very well be best to get someone that is a good fit for that particular culture.
Which actually sort of ties into my other post: I think your point about scaling is important to consider, but it only kicks in at a certain size, and I have a sneaking suspicion that Aaron has not handled hiring for, say, Accenture (nor would he want to).
I don't think hiring is ever going to be a one-size-fits-all, cookie-cutter, "standards and metrics" sort of business as long as you're talking about hiring people.
> If you and I each assessed the same candidate using your methods, we could very easily come up with 2 very different assessments. And that's the trap you'll fall into as soon as you're not conducting every interview.
...
> What's also needed is the application of some standards and metrics to give a higher probability of consistent results.
I don't think that's necessarily true. If you're Google or IBM, it might be, but if you're hiring for a smaller firm, it may very well be best to get someone that is a good fit for that particular culture.
Which actually sort of ties into my other post: I think your point about scaling is important to consider, but it only kicks in at a certain size, and I have a sneaking suspicion that Aaron has not handled hiring for, say, Accenture (nor would he want to).
I don't think hiring is ever going to be a one-size-fits-all, cookie-cutter, "standards and metrics" sort of business as long as you're talking about hiring people.
"The single biggest difference I have ever seen (over and over again) between a good programmer and a great programmer is how they respond under pressure."
As many have pointed out already, there's a big difference between the pressure of an interview where you've got a couple of minutes to either win the job or completely blow it and a deadline where you usually have several days to plan and prepare.
I would also point out that if management is constantly springing short, do-or-die deadlines on workers then there's something very wrong with their planning.
As many have pointed out already, there's a big difference between the pressure of an interview where you've got a couple of minutes to either win the job or completely blow it and a deadline where you usually have several days to plan and prepare.
I would also point out that if management is constantly springing short, do-or-die deadlines on workers then there's something very wrong with their planning.
* IQs of 160...what they've been thinking about..."will I get laid tonight"*
This is also a good test of common sense: if they answer "will I get laid" rather than "monads", you know they don't have any.
This is also a good test of common sense: if they answer "will I get laid" rather than "monads", you know they don't have any.
I often hear people assert that engineering isn't done under pressure, so an interviewee's performance under pressure doesn't matter. I don't think that's necessarily true: lots of systems have to be designed for life-and-death situations, web sites often have major crises/bugs, every team faces last minute deadlines, etc.
I wouldn't want anyone on my team who fell apart under stress. Performance under pressure isn't the only factor, but it is a factor.
That said, if anyone has a zero-stress engineering job, please tell me.
I wouldn't want anyone on my team who fell apart under stress. Performance under pressure isn't the only factor, but it is a factor.
That said, if anyone has a zero-stress engineering job, please tell me.
But this is a different kind of stress. Interviews are usually performed under "social" stress - people normally get nervous when meeting someone new, probably for evolutionary reasons. This "short term" stress is very different from the stress you get because of a deadline or a critical bug.
Behavioral interviewing techniques are supposed to be designed to induce stress well above and beyond nervousness when meeting someone new. The stress you feel from a deadline or a critical bug is also social stress. The deadline/bug are not able to care whether they are dealt with. The humans who expect you to deal with them are the source of the stress, whether they are your boss, management from other departments, or if you're running the show, your customers.
I've never really felt it was that different, but for the sake of discussion, how would you simulate the non-social variety of stress in a hiring situation?
You could request the candidate to retrieve an assignment from a password-protected website and require a solution within X hours after retrieval.
This does put on some pressure - there is a deadline, after all - but people who are shy but confident in their coding abilities may be less anxious. This would also solve the common "but I wouldn't code without an IDE/Google/Wikipedia" complaint.
This does put on some pressure - there is a deadline, after all - but people who are shy but confident in their coding abilities may be less anxious. This would also solve the common "but I wouldn't code without an IDE/Google/Wikipedia" complaint.
Having experienced it myself, I can vouch that such an difference exist. When forced to write a complicated piece of code with a bunch of strange people breathing down my neck and watching every step I take, my productivity tends to slacken off when I hit a coders equivalent of writer’s block. It just doesn’t come out as naturally as it used to. In contrast, same task in a more familiar and comfortable setting looks like a piece of cake even if deadline is drawing close.
Coder's block is a good way to describe it.
I've had phone interviews where I've been asked to solve a logic puzzle with pen/paper while explaining my approach. Solving a problem you've never thought about while being put on the spot turns out to be incredibly difficult. These sorts of problems require deep concentration and quiet yet it tends to be awkward to have silence between strangers. The situation really inhibits serious problem solving.
I've had phone interviews where I've been asked to solve a logic puzzle with pen/paper while explaining my approach. Solving a problem you've never thought about while being put on the spot turns out to be incredibly difficult. These sorts of problems require deep concentration and quiet yet it tends to be awkward to have silence between strangers. The situation really inhibits serious problem solving.
"Many brilliant people can seem delightful in a one-hour conversation, but their eccentricities become grating after a couple hours."
That is so true. Most people can't keep up their "interview face" for very long. It's tiring. At some point, if you put them at ease, they will revert to their normal self. That is when you'll get to see what they're really like. I've seen some pretty amazing transformations when waiting for this second phase of the interview.
That is so true. Most people can't keep up their "interview face" for very long. It's tiring. At some point, if you put them at ease, they will revert to their normal self. That is when you'll get to see what they're really like. I've seen some pretty amazing transformations when waiting for this second phase of the interview.
this is all pretty good advice, especially the reviewing past [code] experiences and getting a sense for what it'd be like to work with the person.
i like staunch's "spend a day with the team" method a lot, though talking about strengths/weaknesses and role playing certain scenarios is also a good way to get a sense for potential clashes ("tests are for monkeys" or what not)
what i didn't understand was the "smart" part. i don't think it's possible to get a sense for whether someone is smart--especially technical smarts as opposed to, say, leadership or empathy smarts--simply by talking with someone. in fact, the whole concept of smart is exactly what pressure-ful, fanciful interview questions are trying to get at. i'm not even sure "smart" is a single or useful concept.
specific interview questions do try to determine technical skills or thinking process...but it a chat over coffee, a single, brief encounter, any different? in a good way?
i'm nitpicking here, but i felt that the solution to smarts--assuming we define smart to something useful--is kind of weak.
i like staunch's "spend a day with the team" method a lot, though talking about strengths/weaknesses and role playing certain scenarios is also a good way to get a sense for potential clashes ("tests are for monkeys" or what not)
what i didn't understand was the "smart" part. i don't think it's possible to get a sense for whether someone is smart--especially technical smarts as opposed to, say, leadership or empathy smarts--simply by talking with someone. in fact, the whole concept of smart is exactly what pressure-ful, fanciful interview questions are trying to get at. i'm not even sure "smart" is a single or useful concept.
specific interview questions do try to determine technical skills or thinking process...but it a chat over coffee, a single, brief encounter, any different? in a good way?
i'm nitpicking here, but i felt that the solution to smarts--assuming we define smart to something useful--is kind of weak.
I suspect the "being smart" is in terms that the employer can relate to - do they think about things in the kind of depth that I like / appreciate / can work with.
From my personal experience, I've dealt with smart people and smart people - one kind knows / understands a lot of things in a broad field, the other kind knows a lot about things but is almost completely devoid of common sense. I'd prefer the first kind in any situation where I or my friends would have to deal with them over the medium to long term. I'm pretty sure this would be reflected in all areas worth measuring for a company too.
So trying to define smart as a concrete measure may be missing the point, as you will actually have to work with the person, and what you feel is what tends to form your opinion / bias and affects your future interaction in such a way that it generally tends to become a self-fulfilling prophecy.
From my personal experience, I've dealt with smart people and smart people - one kind knows / understands a lot of things in a broad field, the other kind knows a lot about things but is almost completely devoid of common sense. I'd prefer the first kind in any situation where I or my friends would have to deal with them over the medium to long term. I'm pretty sure this would be reflected in all areas worth measuring for a company too.
So trying to define smart as a concrete measure may be missing the point, as you will actually have to work with the person, and what you feel is what tends to form your opinion / bias and affects your future interaction in such a way that it generally tends to become a self-fulfilling prophecy.
tl;dr version
Smart, affable, productive people with demonstrated ability make great employees.
I'm not interested so much in who to hire -- I think that's a no brainer, frankly. I'm more interested in where to find these great people to hire. Really. We are having hell finding great developers.
Smart, affable, productive people with demonstrated ability make great employees.
I'm not interested so much in who to hire -- I think that's a no brainer, frankly. I'm more interested in where to find these great people to hire. Really. We are having hell finding great developers.
I'd imagine that googling for bug reports that one has been involved in on either side would be a good heuristic for problem solving skills and sociability. At the very least, you'll avoid hiring anyone who can be identified by googling for "bugzilla asshole".
Great advice - I like the fact that Aaron keeps it real simple. I also like how he doesn't like questions along the lines of "counting the number of piano tuners in Chicago". I've been witness to interviews with similar questions and I'm always left wondering as to what the point was.
That said, I'm not sure if all of his advice can be followed by a reasonably medium sized company/organization where hiring decisions are made by a committee, rather than one person.
As a side note, I think that reading a resume can actually be a good starting point to understand the candidate's background.
That said, I'm not sure if all of his advice can be followed by a reasonably medium sized company/organization where hiring decisions are made by a committee, rather than one person.
As a side note, I think that reading a resume can actually be a good starting point to understand the candidate's background.
This thread has some decent discussion about the case-study type questions: http://news.ycombinator.com/item?id=940527
I've personally felt that asking obscure tech trivia questions is meaningless. There is no use in asking someone a question, the answer to which can be obtained by a quick Internet search.
Edit: Grammar
I'm not a big fan of trivia, but there are basic details you will necessarily learn over time just by working with any technology, and not knowing these is damning. E.g., asking how arguments are passed is a quick shibboleth to catch the guy who thinks clicking "print to file" entitles him to put PostScript on his résumé (a colleague has actually witnessed this).
the problem is that these questions can devolve into a (you know) waving contest in which the interviewer is, instead of interviewing, satisfying his own ego by asking obscure questions that sever no purpose other than to inflate the interviewers perception of his own intelligence. This is particularly dangerous in technology professions where their is a high propensity of intellectual self-conceit.
I understand the low hanging fruit, I just witnessed and interview the other day for a Java web developer, and one of the questions was describe a singleton pattern to which the interviewee could not respond. Given that there has been a history of singleton / Java / Bad things web applications problem, not understanding the pattern is a significant issue and I did thing it was an appropriate question to use as a filter, but asking a web developer deep questions about class-loader nuances seems like overkill to me and I have seen that done as well.
I understand the low hanging fruit, I just witnessed and interview the other day for a Java web developer, and one of the questions was describe a singleton pattern to which the interviewee could not respond. Given that there has been a history of singleton / Java / Bad things web applications problem, not understanding the pattern is a significant issue and I did thing it was an appropriate question to use as a filter, but asking a web developer deep questions about class-loader nuances seems like overkill to me and I have seen that done as well.
I always loved to ask a single super obscure question that almost nobody would know the answer to. I didn't expect, or desire the 'correct' answer, I wanted to see how they handled the fact that they were asked something they almost assuredly didn't know. Did they say 'I dunno' and just shrug? Did they make some shit up? Did they try to work it out? That reaction to the uncertain/unknown is, I think, a pretty important part of figuring out if they'll be good to work with - who doesn't run into something unknown on a fairly common basis?
You should also check if the person is reliable (i.e. competent, responsible, hard worker and isn’t an obnoxious diva). In a lot of situations you get people who seem fairly smart, yet make an effort to bring their personal problems to the workplace. This can be extremely counter productive (http://www.reddit.com/r/reddit.com/comments/1octb/reddit_cof...).
That's the second time you've posted that link on HN. Do you have something against Aaron?
Yes, he is an annoying self-anointed expert. This ruins what could be an interesting debate (on any topic). It is ironic that a bad employee somehow becomes an expert in hiring people.
I wouldn’t mind if people stop posting drivel from his site.
I wouldn’t mind if people stop posting drivel from his site.
Perhaps you should look past the author to the content. There are some interesting ideas in there and it seems to have spawned some good discussion.
I think this a terrific interviewing method, but only if you have the metacognition to actually know why you didn't like a candidate, and the courage to tell them how they failed.
If you don't commit to that communication, you'll be an unsettling and depressing interviewer. You can't just play part of the process by the book, it really has to be all or nothing.
If you don't commit to that communication, you'll be an unsettling and depressing interviewer. You can't just play part of the process by the book, it really has to be all or nothing.
An interviewer has no obligation whatsoever to tell a candidate why he or she "failed" an interview. In fact, I'd say you'd be hard pressed to find one who would tell, for legal reasons. Companies hate being sued, so they're not going to tell you anything that might potentially give someone a reason to sue them.
Perhaps off-topic, I was a bit put off at his definition of a friend. I wouldn't want to befriend someone who keeps me around to solve his problems for free while I'm at work.
Being that I've only interviewed at big-ish companies and in Japan (where companies ape the large companies uncritically), I haven't had a chance to see some of these techniques :/
Being that I've only interviewed at big-ish companies and in Japan (where companies ape the large companies uncritically), I haven't had a chance to see some of these techniques :/
[deleted]
I thought he had a great point on the technical questions some of the best developers that I have see, don't spend their time studying every trick question for the array of technologies they use, just so, on the off chance that if it comes up in an interview, they we be spot on. It is crazy, and it reflect poorly on the ability of the interviewer to competently understand the task they are trying to accomplish, which is hire a good developer that integrates into the team well, not a jeopardy contestant.