Use Project-based Interviews Instead of “GitHub”(ejohn.org)
ejohn.org
Use Project-based Interviews Instead of “GitHub”
http://ejohn.org/blog/project-based-interviews/
14 comments
After the work is submitted the following day, if it looks good then call them in for an interview and ask them what were the weak points of the solution. If the developer is good, they will have worked it over in their head for the next day or two, and will have an answer ready.
Asking questions about the work produced is a good way to identify people who cheated. Ask questions about why and what was your intent with this.
Asking questions about the work produced is a good way to identify people who cheated. Ask questions about why and what was your intent with this.
Yep; we did that too. The flow was something like:
- Phone screen with our HR rep to figure out if you're even remotely what we're looking for - Takehome challenge to make sure you can code - Engineer reviews the challenge and, if good, calls you to discuss the challenge and discuss your experience in depth. - If good, come onsite to meet the team, do a couple 1-hour use-anything challenges, and answer a few higher level long-form questions. - Offer.
- Phone screen with our HR rep to figure out if you're even remotely what we're looking for - Takehome challenge to make sure you can code - Engineer reviews the challenge and, if good, calls you to discuss the challenge and discuss your experience in depth. - If good, come onsite to meet the team, do a couple 1-hour use-anything challenges, and answer a few higher level long-form questions. - Offer.
I think it's stupid to reject people who don't have an active Github account. Lots of very talented programmers work on proprietary code, especially outside the narrow niche of advertising-driven products where companies tend to produce closed-source software.
Or working on their small startups which might not have time to open source components. It's easier when you have more scale, time and freedom.
I have been approached with the project-interview and it left a bad taste in my mouth leading to me ultimately turning down the interview. Getting a candidate to do work for you in their free time for no compensation is perhaps not the best way to attract and find good talent.
If you want to hire me but want to see some real-world work first, give me an 8 hour contract and a fair wage.
If you want to hire me but want to see some real-world work first, give me an 8 hour contract and a fair wage.
So you have invested 2+ hours writing a cv and a cover letter, and another 4 to 5 getting ready for, going to and finally attending one or more interviews, and you balk at investing another 4 hours on a project, where your abilities can really shine over the competition?
I don't think the argument was about the misc. time it takes to do things. It takes time to commute, it takes time to shower, it even takes time to eat.
The issue is that you're working for a prospective company while not being compensated for it.
The issue is that you're working for a prospective company while not being compensated for it.
I read this post as being the same project for everyone, that was extracted from real work, not 'we're going to put this in production once you're done with it.'
That's correct. We already have an excellent implementation of what we're testing so there's no need to use what the interviewee made (this is simply an interviewing tool).
If you want to hire me but want to see some real-world work first, give me an 8 hour contract and a fair wage.
If only there was a website where one could upload and source code one had written for fun. Perhaps even with repo history. Like a hub of git activity or something.
Then employers could simply browse this website rather than demanding these annoying project interviews.
If only there was a website where one could upload and source code one had written for fun. Perhaps even with repo history. Like a hub of git activity or something.
Then employers could simply browse this website rather than demanding these annoying project interviews.
If you are working at a place that does OSS that puts you miles ahead of anyone working on closed source systems.
Additionally the legality of OSS commits is technically still kind of grey for a lot of people with generic "your code belongs to the company" in their contract with their employer.
Additionally the legality of OSS commits is technically still kind of grey for a lot of people with generic "your code belongs to the company" in their contract with their employer.
Yes, if an employer can easily determine you know what you are doing, you are miles ahead of people who are unknown quantities.
See also filters like college degrees and past experience, both of which also provide evidence you know what you are doing and put you miles ahead of people without them. Should we also not look at past experience and education?
See also filters like college degrees and past experience, both of which also provide evidence you know what you are doing and put you miles ahead of people without them. Should we also not look at past experience and education?
I am speaking from a requirement standpoint. By requiring OSS commits, I am heavily favoring people who worked for OSS companies. This may be a good thing or a bad thing.
I do not disagree on the value of looking at OSS, I just feel that requiring it is going to cause you to pre-emptively filter your candidate list.
Now if that is what you want there is nothing wrong with that, it just may not be obvious that is what you are doing.
Note that I am assuming you are looking for complex work, github is great for seeing if they can at least program, but I would trust a quick whiteboard problem more for that level of technical aptitude.
I do not disagree on the value of looking at OSS, I just feel that requiring it is going to cause you to pre-emptively filter your candidate list.
Now if that is what you want there is nothing wrong with that, it just may not be obvious that is what you are doing.
Note that I am assuming you are looking for complex work, github is great for seeing if they can at least program, but I would trust a quick whiteboard problem more for that level of technical aptitude.
The benefits don't stop on the hiring side. I would gladly do this for any position it looks like I'm qualified for, for two reasons. One is that it gives at least some idea what the work is like. The other is that I've discovered I turn into an idiot when asked to code in an interview setting while someone else is watching. :P
(Shameless plug: I am looking, if anyone is after a decent Python programmer. Email in profile.)
(Shameless plug: I am looking, if anyone is after a decent Python programmer. Email in profile.)
We're hiring Python programmers at Khan Academy too, not just JS! I can't promise that you'll get a project-based interview but I promise we'll be friendly.
It's interesting to see that employers are essentially requiring more effort from potential job candidates in order for them to be seriously considered, when the tech industry is currently a candidate's market.
I think tech is definitely a candidate's market but the corollary is that the tech market attracts a lot of candidates who can't perform. Real programming exercises are perhaps the only way to weed them out.
While I 100% agree you need some kind of code based part of your interview practice, there is a large range of exercises that are being spoken about.
It goes roughly from whiteboard problems to sitting at a computer with them over your shoulder to a couple hours before the interview to a full week of work. Everyone knows the lightest form is necessary the big question is how much you can get from increasing the complexity before people get annoyed.
It goes roughly from whiteboard problems to sitting at a computer with them over your shoulder to a couple hours before the interview to a full week of work. Everyone knows the lightest form is necessary the big question is how much you can get from increasing the complexity before people get annoyed.
We give prospective candidates a take-home programming project that should take 3-5 hours and is clearly not something we could use in production. We think that's a fair compromise.
It may be a seller's market, but it's worth noting that your chances of being hired without going through all of the inane interview processes frequently discussed here are still slim to none at tech giants and hot startups.
The large/hot companies have no problem finding enough candidates to jump through their hoops and, wisely or foolishly, many startups are content to sit on unfilled positions until they find someone they think meets their criteria. So only a subset of sellers have the leverage many think they do.
The large/hot companies have no problem finding enough candidates to jump through their hoops and, wisely or foolishly, many startups are content to sit on unfilled positions until they find someone they think meets their criteria. So only a subset of sellers have the leverage many think they do.
I agree that "hot" companies have their own localized demand, but I don't think that's reflective of the majority of companies in the tech field who are now being encouraged to imitate these top-tech players hiring practices.
For a long time, the whole 'Github profile' stuck in my craw, but recently my thinking has changed to more of a 'show us your code portfolio', which isn't much different than a lot of creative job interviews.
When hiring a designer, part of the interview process is showing things that you have done. It doesn't matter if they're real or not, only that you were the one who created them. You also talk about 'your process' when designing those things.
Much the same, I believe that hiring an engineer should be about showing your portfolio and talking to your 'process', whatever that may be. It's a far more interesting way to get to know a person, how they reason and what they're capable of.
When hiring a designer, part of the interview process is showing things that you have done. It doesn't matter if they're real or not, only that you were the one who created them. You also talk about 'your process' when designing those things.
Much the same, I believe that hiring an engineer should be about showing your portfolio and talking to your 'process', whatever that may be. It's a far more interesting way to get to know a person, how they reason and what they're capable of.
Hiring a designer is so much different than hiring a software engineer. And the process of designing should be the least of your worries.
Design is an artistically creative process. Takes talent, I wouldn't hire a designer if I don't perceive the work they've made as beautiful.
But hiring an engineer, there are certain things you can overlook in favour of their capacity to actually build something. I can overlook their code quality, but only because we have a boarding process in which one of the things we make sure a new engineer understands is our quality standard in writing code, the systems we use to enforce that standard from code reviews to pre-commit hooks and lint checking.
An engineer has the capacity to adapt to these changes of both quality and process.
IMHO, you can't teach a designer who isn't talented how to make beautiful things. Albeit beauty is subjective, but there's still a common ground upon which the majority will perceive this piece of art work as beautiful.
Design is an artistically creative process. Takes talent, I wouldn't hire a designer if I don't perceive the work they've made as beautiful.
But hiring an engineer, there are certain things you can overlook in favour of their capacity to actually build something. I can overlook their code quality, but only because we have a boarding process in which one of the things we make sure a new engineer understands is our quality standard in writing code, the systems we use to enforce that standard from code reviews to pre-commit hooks and lint checking.
An engineer has the capacity to adapt to these changes of both quality and process.
IMHO, you can't teach a designer who isn't talented how to make beautiful things. Albeit beauty is subjective, but there's still a common ground upon which the majority will perceive this piece of art work as beautiful.
I agree that they are different, however it sounds like I'm not communicating my idea correctly.
I'm very much NOT interested in code quality from an engineer. I very much AM interested in how an engineer thinks and approaches solving a problem.
One way to start that conversation is to have a code portfolio that they can talk to. Why did you do this? What were thinking about when you did that? Did you start writing some code then refactor? Did you think about the problem for 3 days before you started writing? Do you diagram on paper? Did you model the data structures before you modeled the application itself?
These are the sorts of questions I'm interested in when I hire someone; They're also the sorts of questions I want someone who is hiring me to be interested in.
Using your words:
To me, writing code is an artistically creative process as well, which takes talent. I wouldn't hire an engineer if I don't perceive their thinking as beautiful
The process for creativity doesn't change just because the expression changes. Compare the process of writing code with that of painting and you'll find a lot more similarities than differences.
PG even wrote a book about it.
I'm very much NOT interested in code quality from an engineer. I very much AM interested in how an engineer thinks and approaches solving a problem.
One way to start that conversation is to have a code portfolio that they can talk to. Why did you do this? What were thinking about when you did that? Did you start writing some code then refactor? Did you think about the problem for 3 days before you started writing? Do you diagram on paper? Did you model the data structures before you modeled the application itself?
These are the sorts of questions I'm interested in when I hire someone; They're also the sorts of questions I want someone who is hiring me to be interested in.
Using your words:
To me, writing code is an artistically creative process as well, which takes talent. I wouldn't hire an engineer if I don't perceive their thinking as beautiful
The process for creativity doesn't change just because the expression changes. Compare the process of writing code with that of painting and you'll find a lot more similarities than differences.
PG even wrote a book about it.
I'm inclined to agree with you, I'm not saying writing code isn't an artistically creative process.
If I'm are hiring an engineer, the last thing I'll consider is how well is their code because that can be enhanced (albeit to a certain level, as per pionar's comment which I also agree with)
If you are hiring a designer, then that's likely the main aspect you'll be looking at, how beautiful/creative their work/solutions are.
If I'm are hiring an engineer, the last thing I'll consider is how well is their code because that can be enhanced (albeit to a certain level, as per pionar's comment which I also agree with)
If you are hiring a designer, then that's likely the main aspect you'll be looking at, how beautiful/creative their work/solutions are.
> IMHO, you can't teach a designer who isn't talented how to make beautiful things.
Likewise, you can't teach a developer who doesn't "get it" how to make beautiful things in code.
Likewise, you can't teach a developer who doesn't "get it" how to make beautiful things in code.
Bravo.!
Let me tell you my story. I have been rejected just because I don't have enough activity on my github account. I am a young college senior. My most of "good" projects aren't on GitHub. They are not open source. So does that mean that I don't add up ? No. How can you expect undergrad college students to have 1000s of commits in popular repos ?
I tried project-based interview, the requirement was very simple build a tic-tac-toe iphone app (player vs device) sometimes device wins and sometimes player wins.
I made this iPhone app(https://github.com/jyothepro/tictactoe) but they rejected me. Reason for rejection was not very clear. I felt I wasted my time :(
I made this iPhone app(https://github.com/jyothepro/tictactoe) but they rejected me. Reason for rejection was not very clear. I felt I wasted my time :(
Maybe there were a swarm of applicants. You were obviously qualified but they just went with someone else for no reason that reflects on you.
Yes, that's the problem with similar open-ended tasks - the assessment is very subjective.
And I have a feeling that I'm gambling - in terms of allocating time vs what I'll get. That's why paid projects make more sense - with defined goals.
In the OP's case - tic tac toe game is completed? playable? that's it - he gets paid, no matter if they don't hire him for some other reasons.
And I have a feeling that I'm gambling - in terms of allocating time vs what I'll get. That's why paid projects make more sense - with defined goals.
In the OP's case - tic tac toe game is completed? playable? that's it - he gets paid, no matter if they don't hire him for some other reasons.
Paying someone for building software is tedious as they have to employ you as a contractor(many companies have policies that empolyees cannot work to any other employer), so I would suggest giving some swag - any tech gadget or amazon gift card or something on those lines.
Tediousness of hiring a contractor is exaggerated. I've had clients from many places, including USA. Signing the contract was as simple as putting my signature PNG file into PDF and sending back. Payment was done with invoices, through regular bank wire transfers.
Now that person has to file taxes for this money, he needs 1099 from the company to show that he earned this money. I am not a tax expert but this is how I interpret filing taxes for this kind of money
I believe it is needed for US contractors. Anyway, when the company asks you to spend substantial amount of time, they shouldn't hesitate to issue 1099 form as well.
> You were obviously qualified
Not so sure about that. I looked at parent's code submission. In particular, his game logic looks like this - https://github.com/jyothepro/tictactoe/blob/master/TicTacToe...
This is unduly verbose & has newbie programmer written all over it.
Here, I just wrote up the same tictactoe over lunch in the repl - https://gist.github.com/krishnanraman/7591435
Without much thought into optimiziation, refactoring etc., its about 20 lines of actual code. Random TicTacToe is a very simple game. Even if you take that Scala & translate line by line into Objective-C, you shouldn't see an exponential explosion from 20 lines to 400+ lines for game logic alone.
Not so sure about that. I looked at parent's code submission. In particular, his game logic looks like this - https://github.com/jyothepro/tictactoe/blob/master/TicTacToe...
This is unduly verbose & has newbie programmer written all over it.
Here, I just wrote up the same tictactoe over lunch in the repl - https://gist.github.com/krishnanraman/7591435
Without much thought into optimiziation, refactoring etc., its about 20 lines of actual code. Random TicTacToe is a very simple game. Even if you take that Scala & translate line by line into Objective-C, you shouldn't see an exponential explosion from 20 lines to 400+ lines for game logic alone.
When you say "unduly verbose" I am not sure what exactly the interviewer/you are looking here. I purposefully made it verbose so that anyone who calls himself a programmer should understand what I am doing.
Even I dint spend any time in refactoring or optimization as this is just a hack to get past a job interview.
When an interviewer gives you tasks like this do they expect optimized and refactored code?
Even I dint spend any time in refactoring or optimization as this is just a hack to get past a job interview.
When an interviewer gives you tasks like this do they expect optimized and refactored code?
[deleted]
I am very much in favor of a low time commitment project interview, something that takes maybe 3-4 hours to complete.
I've written a ton of code in my career, and 99% of it is wholly owned by the companies who I wrote it in the employ of, and I don't have the legal right to show it to anyone much less put it on public display. Some people would say the answer to that is doing OSS/other projects in my spare time... but I have hobbies, and other interests, and don't want to spend my whole life writing code.
Give me an opportunity at no cost to you and low (time) cost to me to show what I can do, and we will both be in a better position to evaluate each other.
I've written a ton of code in my career, and 99% of it is wholly owned by the companies who I wrote it in the employ of, and I don't have the legal right to show it to anyone much less put it on public display. Some people would say the answer to that is doing OSS/other projects in my spare time... but I have hobbies, and other interests, and don't want to spend my whole life writing code.
Give me an opportunity at no cost to you and low (time) cost to me to show what I can do, and we will both be in a better position to evaluate each other.
the intensity of the project should ramp with the seriousness of both sides; we start with a 30 minute phone screen then a very simple 30 minute "make these tests pass" and then bring people in for a day of coding/chatting. requiring a project as a first-pass filter is a bit much to ask of candidates and interviewers.
Yeah, that's essentially what I meant. A coding challenge that takes a handful of hours as a post phone screen/pre onsite screen.. not the first step in the process :)
If AT ALL possible I prefer to contract developers. We pay a candidate for their time and pair program with them for a week. This lets us evaluate not only their ability to code but also their desire to learn and their team culture fit. Since I've started doing this with a new team it's been a lot easier to manage the people that I work with. All the existing developers get to know someone and they all get veto power over whether its someone that they'd want to work with AFTER they're worked with them.
It's hard enough trying to find good people; as pleasant as that sounds, I can't imagine narrowing my criteria to "good people who can take a week off their current job to do contract work while we decide whether to make them a full offer."
That definitely seems tough. Maybe shortening the project time to a couple days (over a weekend?) or a few weeknights would help. From the hiring side at least, the time investment would seem worthwhile, considering you're evaluating a potential team member.
It doesn't work at all for people currently employed.
However, I've given that issue a lot of thought, and I've changed my mind in that there is something really really nice about it: it gives unemployed programmers a lot of breathing room.
You may think that it's really easy to get hired, and in the top-5 cities it might be, but in other markets even very qualified people can't get interviews lined up quickly.
If I were to get laid off tomorrow, I would love the fact that there are employers who, after an initial screen, would take me in for a week at contractor's rates to evaluate me.
1. I would get paid, and after being laid off that would be a tremendous relief.
2. It is much easier to get in the door. Employers always worry that the person they are bringing on won't cut it and will have to be let go. With this test, it is very easy for both sides to try it out without worrying about an expensive "divorce."
However, I've given that issue a lot of thought, and I've changed my mind in that there is something really really nice about it: it gives unemployed programmers a lot of breathing room.
You may think that it's really easy to get hired, and in the top-5 cities it might be, but in other markets even very qualified people can't get interviews lined up quickly.
If I were to get laid off tomorrow, I would love the fact that there are employers who, after an initial screen, would take me in for a week at contractor's rates to evaluate me.
1. I would get paid, and after being laid off that would be a tremendous relief.
2. It is much easier to get in the door. Employers always worry that the person they are bringing on won't cut it and will have to be let go. With this test, it is very easy for both sides to try it out without worrying about an expensive "divorce."
This. You have to make your offer pretty attractive, or have pretty desperate candidates for that to work.
As a complement to the project-based interviews that John mentions, we recently did this at Khan Academy with a candidate and are doing it with another this week. It's difficult because not everyone is free (or is interested) to take a week off what they're currently doing, but we're excited to keep trying this as it's a very valuable signal.
Come on, a place like Khan academy, who is creating virtual, anywhere classroom, and they don't have remote positions?
For all these types of coding interviews I think that the company should state up front how they evaluate it. Is that clicking keyboard in the background someone typing in the code to run it and see if it works? Are they looking for clear communications, do they want you to just solve the problem or is it more important to lay out that whiteboard code exactly the way you would in a source code file?
And what about unit tests? If they give you a problem to code on a whiteboard, do they want to see TDD red green refactor cycles?
And is this the only problem they want you to work on in the hour or is there another one coming when you finish?
And what about unit tests? If they give you a problem to code on a whiteboard, do they want to see TDD red green refactor cycles?
And is this the only problem they want you to work on in the hour or is there another one coming when you finish?
[deleted]
What if I have a Bitbucket profile? Or is that not hipster enough?
You should probably read the blog post. "And to clarify, some people seem to be getting caught up in the terminology – I see many people using ‘Github’ as a placeholder for ‘any OSS activity’, I hope no one is being that literal."
I know, I'm referring to the tweet you are responding to. But let's be honest, Bitbucket and hg do get some irrational hate in certain hacker circles.
They do have the flaw that they might not weed out candidates that are really slow (since they can generally take as much time as they want on a take-home project). They also don't weed out candidates who are willing to have someone else help them significantly with the project.
As such, I've found take-home projects useful in addition to an onsite interview. At a previous company, we would follow up a take-home project with an onsite interview involving a couple 1-hour "code this using any tools/resources you want with no one looking over your shoulder" questions to prove that they didn't cheat on the take-home and that they can get things done in a reasonable timeframe. I always felt it was a pretty useful way of evaluating programmer skill without stupid whiteboard coding.