Show HN: Recruit web developers by having them build a web app(cull.io)
cull.io
Show HN: Recruit web developers by having them build a web app
http://www.cull.io/index.html
24 comments
I agree with this 100%. Coding tests and filters like this make sense when we have a market with an oversupply of developers that need to be filtered out. Right now, we're in the opposite situation with more jobs than skilled devs.
I think a few companies (Google, Facebook, Twitter, etc) have more applicants than open job reqs, so for them these types of products have value.
I think a few companies (Google, Facebook, Twitter, etc) have more applicants than open job reqs, so for them these types of products have value.
Want to work here? Do work for us for free! Then maybe we'll hire you afterwards.
Great idea if you want to hire suckers.
Great idea if you want to hire suckers.
I agree with you. This trend in our field is really making me uncomfortable. Judge me on what I know and what I did before, no more, no less. Or, to make this even more futile, how about I ask the company to build a small app for me? Since I also want to know if I should work for them. If they think their main business is enough to judge them, then my past experience is equally enough. This is bordering ridiculing in my opinion.
"This trend in our field is really making me uncomfortable. Judge me on what I know and what I did before, no more, no less."
You are uncomfortable in demonstrating your knowledge in a practical way? I don't understand. You are looking for a role doing web development, which requires up-to-date knowledge not just of techniques, but of understanding. These code exercises make an excellent starting ground for a technical interview, and has the advantage of being built in your preferred choice of environment, entirely in your style. It's a time where you can show off your real skills. Which is what hiring companies want to know about.
There's very little room to hide in these coding challenges / exercise.
I've been on both sides of web developer tests / challenges, and the one I enjoyed the most was interviewing for Global Radio in London. I had a weekend to build a web app. So I built something I wanted to build anyway, gave it to them.
The next step was to go in with their web development team who then code reviewed it.
Still use that web app today.
You are uncomfortable in demonstrating your knowledge in a practical way? I don't understand. You are looking for a role doing web development, which requires up-to-date knowledge not just of techniques, but of understanding. These code exercises make an excellent starting ground for a technical interview, and has the advantage of being built in your preferred choice of environment, entirely in your style. It's a time where you can show off your real skills. Which is what hiring companies want to know about.
There's very little room to hide in these coding challenges / exercise.
I've been on both sides of web developer tests / challenges, and the one I enjoyed the most was interviewing for Global Radio in London. I had a weekend to build a web app. So I built something I wanted to build anyway, gave it to them.
The next step was to go in with their web development team who then code reviewed it.
Still use that web app today.
> You are uncomfortable in demonstrating your knowledge in a practical way?
The concern is potential employers trying to bilk free work out of developers. The small project strategy is hardly fool-proof either. If it becomes widespread, we'll just see phony developer put their assignments up on rent-a-coder and ask for a crib sheet to study so they can withstand a technical interview on what they've created.
Anecdotally, the 2 developers I know who were hired via the "do a small project for us first" method both ended up in companies with high dev turn-over, shitty tools (slow, single monitor computers), and were surrounded by lots of "green" co-workers who required significant hand-holding despite the fact that they too completed their small projects to get hired.
The concern is potential employers trying to bilk free work out of developers. The small project strategy is hardly fool-proof either. If it becomes widespread, we'll just see phony developer put their assignments up on rent-a-coder and ask for a crib sheet to study so they can withstand a technical interview on what they've created.
Anecdotally, the 2 developers I know who were hired via the "do a small project for us first" method both ended up in companies with high dev turn-over, shitty tools (slow, single monitor computers), and were surrounded by lots of "green" co-workers who required significant hand-holding despite the fact that they too completed their small projects to get hired.
surrounded by lots of "green" co-workers who required significant hand-holding despite the fact that they too completed their small projects to get hired
When I hear talk like this coming from devs in a given startup, it only confirms that their work culture is terrible. It doesn't make me think their devs are bad. It's more of the "rock star" and "ninja" wishlist stuff. No company-- not even Google, Apple, or Facebook-- is going to employ only developers who are all at the top of their field. If you're a startup, you must have a pathway to get your average programmers to greatness.
When I hear stuff like "hand-holding" it makes me think of companies that hire based on keyword matching. They don't want to put anything at all into their employees, they just want the best output right away. No wonder these firms are always complaining about not being able to hire talent. They could develop talent, but they don't-- it's either getting the creme of the crop engineers (which they're not going to be able to get anyway since they can't compete with the above-mentioned companies on salary and benefits) or nothing!
The really creative startups I'm seeing now are those that train. They know there's a war for talent, so they're willing to invest in somebody who isn't a rock star today, but has potential and very well may be tomorrow.
When I hear talk like this coming from devs in a given startup, it only confirms that their work culture is terrible. It doesn't make me think their devs are bad. It's more of the "rock star" and "ninja" wishlist stuff. No company-- not even Google, Apple, or Facebook-- is going to employ only developers who are all at the top of their field. If you're a startup, you must have a pathway to get your average programmers to greatness.
When I hear stuff like "hand-holding" it makes me think of companies that hire based on keyword matching. They don't want to put anything at all into their employees, they just want the best output right away. No wonder these firms are always complaining about not being able to hire talent. They could develop talent, but they don't-- it's either getting the creme of the crop engineers (which they're not going to be able to get anyway since they can't compete with the above-mentioned companies on salary and benefits) or nothing!
The really creative startups I'm seeing now are those that train. They know there's a war for talent, so they're willing to invest in somebody who isn't a rock star today, but has potential and very well may be tomorrow.
>The concern is potential employers trying to bilk free work out of developers.
This is why the "assignment" that I request takes less than two hours and is a greenfield project that is not going to be used by the company -- and I tell candidates this up-front. Think something like a blog, recipe manager or to-do list app. It is trivial, but you may be surprised at the range of outputs that you get.
What is interesting to me is how the resume isn't the best indicator for how well they will do on the coding project. Also, to avoid wasting people's time, I give the coding project as late in the interview process as possible, assuming the other filters have been passed.
This is why the "assignment" that I request takes less than two hours and is a greenfield project that is not going to be used by the company -- and I tell candidates this up-front. Think something like a blog, recipe manager or to-do list app. It is trivial, but you may be surprised at the range of outputs that you get.
What is interesting to me is how the resume isn't the best indicator for how well they will do on the coding project. Also, to avoid wasting people's time, I give the coding project as late in the interview process as possible, assuming the other filters have been passed.
"The concern is potential employers trying to bilk free work out of developers."
If the coding exercise looks like something that is of this sort, then I guess consider whether you want to work with that employer. From an employer perspective, this is the worst source of getting something built.
A candidate should be researching the companies submits his resume/CV too anyway.
"If it becomes widespread, we'll just see phony developer put their assignments up on rent-a-coder and ask for a crib sheet to study so they can withstand a technical interview on what they've created."
If that were feasible, I'd suggest the employer would much rather hire the rent-a-coder instead of the one paying him. That's a win for the employer there too, since they'll be able to offer the rent-a-coder guy far more than the scratchings he gets as a rent-a-coder. The ability to predict in advance a sufficient range of questions as to forearm a candidate will border on the mystic.
Coding exercises aren't a mouse in a maze puzzle, they are a starting point for gauging web development ability.
The companies that use coding challenges as a way of creating production ready material are going to be amply supplied by the rent-a-coder approach you outline. And the technical interview part of the test is probably geared with a formulaic approach that can be guessed. In that case, this candidate and that employer are well matched.
But not every candidate needs a rent-a-coder to pass a technical test, and not every company uses those tests to create production ready websites and applications.
A good candidate can sniff a bad employer just as well as vice versa. If a company seems to be asking for production ready material, then walk away. If the test seems on the up, you'd better have a very good reason for refusing.
If the coding exercise looks like something that is of this sort, then I guess consider whether you want to work with that employer. From an employer perspective, this is the worst source of getting something built.
A candidate should be researching the companies submits his resume/CV too anyway.
"If it becomes widespread, we'll just see phony developer put their assignments up on rent-a-coder and ask for a crib sheet to study so they can withstand a technical interview on what they've created."
If that were feasible, I'd suggest the employer would much rather hire the rent-a-coder instead of the one paying him. That's a win for the employer there too, since they'll be able to offer the rent-a-coder guy far more than the scratchings he gets as a rent-a-coder. The ability to predict in advance a sufficient range of questions as to forearm a candidate will border on the mystic.
Coding exercises aren't a mouse in a maze puzzle, they are a starting point for gauging web development ability.
The companies that use coding challenges as a way of creating production ready material are going to be amply supplied by the rent-a-coder approach you outline. And the technical interview part of the test is probably geared with a formulaic approach that can be guessed. In that case, this candidate and that employer are well matched.
But not every candidate needs a rent-a-coder to pass a technical test, and not every company uses those tests to create production ready websites and applications.
A good candidate can sniff a bad employer just as well as vice versa. If a company seems to be asking for production ready material, then walk away. If the test seems on the up, you'd better have a very good reason for refusing.
Seems like the bilking of devs for free work could be mitigated by making sure copyright of the test web app code remained with the developer being hired.
Like you said, a small project is a little bit of a red herring. No small test project done in a short amount of time (like a few days) is really going to be complex enough to test if a developer is actually proficient enough to work on a large project.
Like you said, a small project is a little bit of a red herring. No small test project done in a short amount of time (like a few days) is really going to be complex enough to test if a developer is actually proficient enough to work on a large project.
"You are uncomfortable in demonstrating your knowledge in a practical way?"
Isn't that what his past projects and employment demonstrate?
Isn't that what his past projects and employment demonstrate?
No. Past projects are typically not solo projects, so it's hard to establish who did what. We're hiring a developer for what he can do, not for what other people in his team did.
"Past projects are typically not solo projects, so it's hard to establish who did what."
That's why I give references, they can comment/confirm what I did. If they're bad (but professional) references, they'll just confirm that I worked with them.
Also, I wasn't clear, but I meant personal projects (with commit records) as a separate consideration from employment.
That's why I give references, they can comment/confirm what I did. If they're bad (but professional) references, they'll just confirm that I worked with them.
Also, I wasn't clear, but I meant personal projects (with commit records) as a separate consideration from employment.
Maybe hire me? I have applied at a few places without getting a response, let alone a phone interview. If I'm spending a couple of hours writing code for them, they might as well take out a couple of minutes to reply back to me.
True! After you spend a couple of hours, maybe they'll send you "We really enjoyed reviewing your code. At this time we do not feel this would be an ideal fit."
I don't think that's the idea. I've had multiple interviews where I've been asked to code something to show that I know how to do it (even something simple as modifying a website or showing a counter). This just eases that process.
10-15 minute code exercise and "build a webapp" are not the same.
More to the point, showing your skills after you've been granted an interview, and being forced to do spec work before you are allowed to apply (wtf?????) are not the same.
It could be an exercise to build a very simple webapp in 15 minutes. I mean, there are so many 'build a blog in 15 minutes' tutorials out there. It would test if you have enough rails/django/x-framework experience to build a simple crud app.
Besides, this is just a platform, it's up to the interviewer to set an appropriate task. The interviewer can provide an existing web app and ask the user to do something with it, instead of writing something completely from scratch.
Besides, this is just a platform, it's up to the interviewer to set an appropriate task. The interviewer can provide an existing web app and ask the user to do something with it, instead of writing something completely from scratch.
The examples given take way less than 10-15mins, though. OF course, they're not really "webapps", just very basic HTTP APIs.
You are right. I meant more of the kind "Hey, before your interview, do this". I misspoke when I said "interview problems".
Didn't YC fund a company doing something similar to this in the last cycle?
I think it had a Goldman Sachs co founder and another business development guy. Reached Tech Crunch some time in the last 6months.
That received some pretty harsh comments for precisely the reason you mentioned. This is a easy way to game developers and have a couple of hours free consulting.
I think it had a Goldman Sachs co founder and another business development guy. Reached Tech Crunch some time in the last 6months.
That received some pretty harsh comments for precisely the reason you mentioned. This is a easy way to game developers and have a couple of hours free consulting.
Did you look at the site? It's more like interview questions than free work. While I doubt this service will find the cream of the crop, it could be useful for broader-based programmer hiring.
OP here. Agree with this sentiment. Even I would not use cull.io to have candidates build a complex web app that would take days or even hours to build. I would use it as a tool to "Filter" candidates, not "Select" them. (It's amazing how the simplest of a coding challenge would filter all non-coding coders)
thanks for the review.
thanks for the review.
I've never been on the other(interviewer) side of the hiring process and I was on another side only once during the last decade, so I might not have enough insight, but I really don't understand non-coding coders filtering per se, be it your method or FizzBuzz...
When someone says about their years of experience in coding in their CV, is it a common case that they're lying?
When someone says about their years of experience in coding in their CV, is it a common case that they're lying?
I, somewhat cynically, divide job seekers into four categories:
1) the qualified (I know what I can do, and I can do what you need)
2) the unqualified (I know what I can do, but I can't do what you need)
3) the deluded (I honestly think I'm qualified, but I'm not)
4) the liars (I'm trying to get the job by any mean necessary).
I only want (1).
(2) are easy to deal with - you just have to know what you need.
(3) are common and mildly annoying - especially if you're a nice person and feel the need to explain where they are deluded :-)
(4) are not common - but are exceptionally annoying and often hard to sort out
Writing a really good job spec / advert will get rid of most of (2) for you, and help remove all but the most deluded from (3).
Filtering resultant CVs will usually get rid of the rest of (2), and a bit more of (3).
A quick chat on the phone will get rid of any remaining (2) and a big chunk of (3).
Simple tests similar to FizzBuzz will cull a surprising amount of (3) and (4).
Interviewing - especially good accomplishment based questions will tend to filter out all of (3) and a big chunk of (4).
Which, unfortunately, occasionally leaves some of (4) who seem to sometimes put more effort into deceiving me than it would take to actually learn the damn skills in the first place. Which is why I really like having some kind of solid work "with the team" session for at least a couple of hours. Pay 'em for it too if at all possible.
I, somewhat cynically, divide job seekers into four categories:
1) the qualified (I know what I can do, and I can do what you need)
2) the unqualified (I know what I can do, but I can't do what you need)
3) the deluded (I honestly think I'm qualified, but I'm not)
4) the liars (I'm trying to get the job by any mean necessary).
I only want (1).
(2) are easy to deal with - you just have to know what you need.
(3) are common and mildly annoying - especially if you're a nice person and feel the need to explain where they are deluded :-)
(4) are not common - but are exceptionally annoying and often hard to sort out
Writing a really good job spec / advert will get rid of most of (2) for you, and help remove all but the most deluded from (3).
Filtering resultant CVs will usually get rid of the rest of (2), and a bit more of (3).
A quick chat on the phone will get rid of any remaining (2) and a big chunk of (3).
Simple tests similar to FizzBuzz will cull a surprising amount of (3) and (4).
Interviewing - especially good accomplishment based questions will tend to filter out all of (3) and a big chunk of (4).
Which, unfortunately, occasionally leaves some of (4) who seem to sometimes put more effort into deceiving me than it would take to actually learn the damn skills in the first place. Which is why I really like having some kind of solid work "with the team" session for at least a couple of hours. Pay 'em for it too if at all possible.
Not lying. Even I am not sure why this happens though. I think two things are at play:
a) You could survive in big enough company without coding (You survive by tinkering xml files, db records, monitoring the servers.. stuff like that)
b) Most interviews don't require you to code. They rely on stuffs like puzzles, knowledge of complex algorithms and asking you explain things about your previous work. So a candidates keeps getting good at that since that's what is required to get the next job.
But i am not entirely sure. It surprises me too.
a) You could survive in big enough company without coding (You survive by tinkering xml files, db records, monitoring the servers.. stuff like that)
b) Most interviews don't require you to code. They rely on stuffs like puzzles, knowledge of complex algorithms and asking you explain things about your previous work. So a candidates keeps getting good at that since that's what is required to get the next job.
But i am not entirely sure. It surprises me too.
Maybe not lying per se, but embellishing the truth definitely occurs. It isn't unheard of people not being able to conceptually plow through the FizzBuzz even with a few years of experience.
Depends on the economy, I would guess. I haven't had to do a lot of hiring recently, but a couple years ago at my last company we found ourselves overwhelmed with applicants to PHP programming jobs. We created an application tool that asked applicants to solve a few PHP puzzles. These weren't super-hard MIT-level questions, just basic FizzBuzz. We found it to be a very effective screening process.
This probably works best for entry level/junior programmers responding to a help wanted ad. Senior people are being proactively recruited.
This probably works best for entry level/junior programmers responding to a help wanted ad. Senior people are being proactively recruited.
Can you share the app?
It used to be linked from this page but it looks like it's 404 ATM
http://www.surveygizmo.com/company/jobs/
http://www.surveygizmo.com/company/jobs/
OK as someone looking for employment, I feel ambivalent about this and probably would pull the plug on the process if you asked for non trivial work from me. I understand the need to weed out applicants that lack skills. This can and should be done by technical screening conducted by qualified individuals.
FWIW this opinion is based on experience. I've conducted more interviews than I have given. As an applicant I once went through a fairly lengthy interview process that first did a screening interviews including live coding exercises. They were conducted and critiqued in real time. I have no issue with these. You can either do what you say, or you can't and it becomes apparent fast.
The next stage is where I started to question the company's processes - They requested a small "project" followed by an interview. I _nearly_ pulled the plug on this, since it was a request for 10+ hours coding, but did it anyway.
The next interview involved a fairly in depth defense of my code and design choices, white board work that again put my skill set to the test real time. After several hours of this, I was requested to do another "small project" and I walked away.
TL;DR Ask me a million questions, put my skills to the test on the spot, but don't ask free work... I'll think you're unable to judge my skill/confused/a scammer or whatever.
FWIW this opinion is based on experience. I've conducted more interviews than I have given. As an applicant I once went through a fairly lengthy interview process that first did a screening interviews including live coding exercises. They were conducted and critiqued in real time. I have no issue with these. You can either do what you say, or you can't and it becomes apparent fast.
The next stage is where I started to question the company's processes - They requested a small "project" followed by an interview. I _nearly_ pulled the plug on this, since it was a request for 10+ hours coding, but did it anyway.
The next interview involved a fairly in depth defense of my code and design choices, white board work that again put my skill set to the test real time. After several hours of this, I was requested to do another "small project" and I walked away.
TL;DR Ask me a million questions, put my skills to the test on the spot, but don't ask free work... I'll think you're unable to judge my skill/confused/a scammer or whatever.
I've had experiences similar to yours and have also had the same reaction. Technical interviews and whiteboarding is one thing, but projects -- especially MULTIPLE projects! -- get very obnoxious very fast.
The actual project is a drain on my time and energy of course. In addition, by dragging out the interviewing process over weeks or months, you're killing my actual desire to work there. Two months of hoop-jumping is annoying, and I'm embarassed that I ever put up with it.
I make clear now to any company I'm interviewing with two things:
1. Let's get to "yes" or "no" quickly 2. I'll do one project as long as it can not only be completed in 2 hours, but that 2 hours' worth of work on the project will yield something that I can be proud to show you.
#1 is obvious, but #2 is very, very important. If your project is "create a blog app", yes I can complete it in two hours. But it will be gross and not something I'm proud of. I will definitely spend more time on it getting it just right so that when you see it you'll be blown away.
What does this mean for the interviewing company? Don't give me an open-ended project. Tell me specifically what kind of thing you want, what functionality you want, and don't put shit like "...anything else you think is cool for bonus points!" That, IMO, translates to hidden requirements and guessing what you actually want.
The actual project is a drain on my time and energy of course. In addition, by dragging out the interviewing process over weeks or months, you're killing my actual desire to work there. Two months of hoop-jumping is annoying, and I'm embarassed that I ever put up with it.
I make clear now to any company I'm interviewing with two things:
1. Let's get to "yes" or "no" quickly 2. I'll do one project as long as it can not only be completed in 2 hours, but that 2 hours' worth of work on the project will yield something that I can be proud to show you.
#1 is obvious, but #2 is very, very important. If your project is "create a blog app", yes I can complete it in two hours. But it will be gross and not something I'm proud of. I will definitely spend more time on it getting it just right so that when you see it you'll be blown away.
What does this mean for the interviewing company? Don't give me an open-ended project. Tell me specifically what kind of thing you want, what functionality you want, and don't put shit like "...anything else you think is cool for bonus points!" That, IMO, translates to hidden requirements and guessing what you actually want.
+1 and Agree. It's wrong to get free work.
However, just out of curiosity how would you have felt if the employer has actually paid you for those "small projects"? (To be clear: No guarantee that you will get the Job, but payment is guaranteed for the hours you put in)
Payment, or even a donation to charity for a token amount - say 1/3 the value of the work - would go far, but I still think it is the wrong approach.
Companies requiring this sort of project work should be aware it smells funny to the applicant and the only way to tell if the applicant did the work him/herself is a tear-down / code defence.
You have not saved any effort from the hire process - only tested if I am willing to jump through a hoop and made me question the company.
Companies requiring this sort of project work should be aware it smells funny to the applicant and the only way to tell if the applicant did the work him/herself is a tear-down / code defence.
You have not saved any effort from the hire process - only tested if I am willing to jump through a hoop and made me question the company.
I first posted a snarky comment saying how this is spec work and what not. I then looked closely and deleted it, because it's not a spec work.
Your description -
Your description -
Recruit web developers by having them build a web app
Recruit web developers by having them build a TEST web app
Or, hire a web developer by having them PROVE they have what it takes.
Or, hire a web developer with too little work experience to have any previous examples of what they can do, but still have them somewhat prove they have the appropriate technical knowledge at least handle the job ......... or something like that.
Anyway, if I had to run a developer through a test like this, I probably wouldn't be talking to him in the first place honestly. I tend to contract with people based on what I've already seen them do.
I understand that getting started is hard too, so hopefully some new developers will find themselves using this tool and it will help them get new jobs and build that portfolio.
Or, hire a web developer with too little work experience to have any previous examples of what they can do, but still have them somewhat prove they have the appropriate technical knowledge at least handle the job ......... or something like that.
Anyway, if I had to run a developer through a test like this, I probably wouldn't be talking to him in the first place honestly. I tend to contract with people based on what I've already seen them do.
I understand that getting started is hard too, so hopefully some new developers will find themselves using this tool and it will help them get new jobs and build that portfolio.
Hi HNers,
OP here. Cull.io is similar to top coder or interviewstreet but focussing just on recruiting web developers. It works by asking candidates to develop a web app on localhost instead of presenting a textarea where candidates types the code.
Advantages:
1) Candidates can virtually choose any language, framework, libraries or IDEs to solve the challenge. (Since they are developing on localhost with their own setup)
2) Because of 1) you need not limit your challenge to be overly algorithmic. You can for example ask questions like 'Pick the most popular article on hacker news today'
Disadvantages: Limited to candidates who can develop a web application
Advantages:
1) Candidates can virtually choose any language, framework, libraries or IDEs to solve the challenge. (Since they are developing on localhost with their own setup)
2) Because of 1) you need not limit your challenge to be overly algorithmic. You can for example ask questions like 'Pick the most popular article on hacker news today'
Disadvantages: Limited to candidates who can develop a web application
This filtering signal works both ways. While you do avoid some of the worst developers, you also reduce your chances of getting some of the better developers who will not likely waste time jumping through hoops unless you're an extremely desirable company to work at.
I think that there's a market for this type of product for larger companies who receive too many resumes to easily sort through and want to hire a lot of developers that meet some minimum standard. However, I think that this is much less useful to startups who just want the best possible small team initially.
Given the makeup of HN, I think that most people here would not look at your product too favorably. I think that you should be looking at finding ways to reach out to companies that have an actual HR department and increasing your price dramatically for them.
I think that there's a market for this type of product for larger companies who receive too many resumes to easily sort through and want to hire a lot of developers that meet some minimum standard. However, I think that this is much less useful to startups who just want the best possible small team initially.
Given the makeup of HN, I think that most people here would not look at your product too favorably. I think that you should be looking at finding ways to reach out to companies that have an actual HR department and increasing your price dramatically for them.
At the risk of playing devil's advocate, I would argue that giving candidates the ability to use virtually any language or framework and develop on localhost is not necessarily an advantage.
When I'm looking to hire a Rails dev then I want to know as soon as possible in the interview process that they actually know some of the basics of Rails and MVC - not just if they can build a web app. Furthermore, by hiding the candidate's choice of language/framework (and code quality) behind localhost the benefits of assigning an arbitrary coding challenge are lost. I fear that the only solution would be for the candidate to submit the full source code.
However, as a very quick "can this candidate actually write code" filter, I think you're well on track.
When I'm looking to hire a Rails dev then I want to know as soon as possible in the interview process that they actually know some of the basics of Rails and MVC - not just if they can build a web app. Furthermore, by hiding the candidate's choice of language/framework (and code quality) behind localhost the benefits of assigning an arbitrary coding challenge are lost. I fear that the only solution would be for the candidate to submit the full source code.
However, as a very quick "can this candidate actually write code" filter, I think you're well on track.
Yes that's a valid point. We do ask the candidate to submit the source code once the they have solved the challenge. So employers who need only say "Rails" developers can mention it in the instructions and check the source later.
"However, as a very quick "can this candidate actually write code" filter, I think you're well on track."
Yes, that's exactly what I wanted this to be. Quickly filter out the 80% and move to the next stage where you can do much better justice reviewing the candidate's past work, working with the candidate on a project etc.
"However, as a very quick "can this candidate actually write code" filter, I think you're well on track."
Yes, that's exactly what I wanted this to be. Quickly filter out the 80% and move to the next stage where you can do much better justice reviewing the candidate's past work, working with the candidate on a project etc.
Leaving aside whether or not this is a valid way to attract top people, my suggestion is to rebrand the name - "cull" has too negative a connotation ("cull the herd").
So #1, I'm part of "the herd", and #2 this tool is being used to possibly cull me. Great. No thanks, I'll spend my time applying to a place that doesn't have a tool dedicated to weeding me out. It doesn't portray the hiring org or your tool very well.
I'd be afraid that you're turning off devs right away, before they even get to the site (which has the potential to turn them off even more). My $0.02
So #1, I'm part of "the herd", and #2 this tool is being used to possibly cull me. Great. No thanks, I'll spend my time applying to a place that doesn't have a tool dedicated to weeding me out. It doesn't portray the hiring org or your tool very well.
I'd be afraid that you're turning off devs right away, before they even get to the site (which has the potential to turn them off even more). My $0.02
Thanks, I never thought of it that way. Will check.
Cull is perfect from HR's point of view and speaks to the heart of the problem they are having trying to find viable candidates in a sea of resumes.
Yes, it speaks to HR's problem - but in a way that jeopardizes their objective (i.e. hiring good people).
Quality devs are in short supply already... explicitly saying that you're treating them like cattle is not going to result in better outcomes. If good devs don't participate = this is worse than a regular process.
In my experience, devs want to feel wanted / valued / pursued, not like commodities that need to be filtered.
/your experience may vary
Quality devs are in short supply already... explicitly saying that you're treating them like cattle is not going to result in better outcomes. If good devs don't participate = this is worse than a regular process.
In my experience, devs want to feel wanted / valued / pursued, not like commodities that need to be filtered.
/your experience may vary
I think you'll have traction problems on the developer side.
Doesn't matter what kind of job it is, you can be pretty much 100% guaranteed that the job will be listed on the company website without any of the extra hoops to jump through.
Requiring someone to build a web app just to apply is a huge requirement. It's fine if you gave someone an interview, and this is the final step before you hire...but requiring someone to code just to submit an application is abuse.
Doesn't matter what kind of job it is, you can be pretty much 100% guaranteed that the job will be listed on the company website without any of the extra hoops to jump through.
Requiring someone to build a web app just to apply is a huge requirement. It's fine if you gave someone an interview, and this is the final step before you hire...but requiring someone to code just to submit an application is abuse.
It is true that having it as the only option to apply for the job or even having it as a totally separate option wouldn't be the best idea.
I can see this as a useful tool for pre-phone screen or post-phone screen evaluation. At the startup where i work we do send the candidate a small code problem after the phone screen and before the on-site interview. I know lot of companies do that. Gives you a little more insight about the candidate and also ensures that you are not wasting your teammates' time as well as the candidate's time.
Also, submitting the resume/github link along with a solution to a programming challenge increases the probability of getting to the phone interview stage. Highly applicable for famous companies like FB or twitter where they receive hundreds of similar applicants every day. Of course it is a stupid method if you are trying to hire a team lead or senior engineer but for fresh graduates or junior developers it works well. I don't think they will mind spending an hour solving a problem.
I can see this as a useful tool for pre-phone screen or post-phone screen evaluation. At the startup where i work we do send the candidate a small code problem after the phone screen and before the on-site interview. I know lot of companies do that. Gives you a little more insight about the candidate and also ensures that you are not wasting your teammates' time as well as the candidate's time.
Also, submitting the resume/github link along with a solution to a programming challenge increases the probability of getting to the phone interview stage. Highly applicable for famous companies like FB or twitter where they receive hundreds of similar applicants every day. Of course it is a stupid method if you are trying to hire a team lead or senior engineer but for fresh graduates or junior developers it works well. I don't think they will mind spending an hour solving a problem.
This idea/service needs to sold to HR departments who are seeking developers.
As someone who hires I want to know what they can do vs what they say they can do. When I hire I ask job applicant to cut up and code a PSD. Some refuse while the eager ones whom are really serious and are excited about the job happily do so.
Overall it's good practice and we give each applicant feedback on how to improve. The flip side allows us to see who cut up and coded the PSD the best (validates) and quickest. That is the person we go with; majority of the time.
Jobs aren't easy to come by these days and in non metropolitan cities the competition is fierce.
As someone who hires I want to know what they can do vs what they say they can do. When I hire I ask job applicant to cut up and code a PSD. Some refuse while the eager ones whom are really serious and are excited about the job happily do so.
Overall it's good practice and we give each applicant feedback on how to improve. The flip side allows us to see who cut up and coded the PSD the best (validates) and quickest. That is the person we go with; majority of the time.
Jobs aren't easy to come by these days and in non metropolitan cities the competition is fierce.
> I ask job applicant to cut up and code a PSD
You judge job applicants by their willingness to jump through boring scut work hoops in the hope of getting a job? I don't know what you're optimising for there, but it certainly isn't programming talent.
I've known a few C-class bumblers who would probably do that for a job interview. The number of grade-A talented engineers, the people I would want to hire, who would put themselves through some pointless, arbitrary slog just to impress you with their "eagerness"? Precisely zero.
You judge job applicants by their willingness to jump through boring scut work hoops in the hope of getting a job? I don't know what you're optimising for there, but it certainly isn't programming talent.
I've known a few C-class bumblers who would probably do that for a job interview. The number of grade-A talented engineers, the people I would want to hire, who would put themselves through some pointless, arbitrary slog just to impress you with their "eagerness"? Precisely zero.
These tests are for a web design and branding agency.
There are tons and tons of resumes and the shop is not located in a huge metropolitan area like San Fran, NYC, Chicago, etc.
There are tons and tons of resumes and the shop is not located in a huge metropolitan area like San Fran, NYC, Chicago, etc.
It dosent have to be a complex web app. It's can be the same problems you would solve on topcoder if you need. but it can be more than that too.
Even those tests they give developers to do during the interview process are off-putting, especially given there's a talent shortage in many parts of the world. That said, good luck. I think this sort of thing will help most with the recruitment of graduate developers.
Thanks ticks. Out of curiosity I'd love to know what do you think should be the best way to filter developers when you make a Job post. Puzzles, Hr questions(tell me something bad about yourself) and other questions like ('how many golf balls can you put in a mini van) dosen't seems to be the best way either.
A lot of people are commenting that they don't want to have to solve this type of problem before applying, but I don't see an issue. I could whip up a simple program that listens for HTTP requests and e.g. reverses a string in minutes, using any high-level language. I spend more time than that writing a nice cover letter.
Given the number of "non-coding coders," I think this is a decent service.
Given the number of "non-coding coders," I think this is a decent service.
Thanks!
Seeing things like this is why I love working in open source, a quick look at github + the projects I am involved with and someone can see what my interests are, what my abilities are and how I work in a team.
Not every work place produces open source code and not everyone wants to spend their limited spare time 'working', but I cant think of better advice for eager job seekers / students / graduates to look for open source projects to contribute to.
(of course it isnt the only reason, making software you love, with awesome people, and making the world a better place arent to be ignored)
Not every work place produces open source code and not everyone wants to spend their limited spare time 'working', but I cant think of better advice for eager job seekers / students / graduates to look for open source projects to contribute to.
(of course it isnt the only reason, making software you love, with awesome people, and making the world a better place arent to be ignored)
I think it is important for some cases at least, that the libraries used by a candidate be limited. For some algorithms (lets say a search algorithm), it would be convenient to use a searching library. What library a person chooses is informative, but not as much as how a person implements it himself/herself. I do understand that this is essentially a screening test, but if lets say, DuckDuckGo was hiring, then how a person implements searching, even if trivial goes a long way in deciding how applicable that person would be for the job. This will be extremely difficult to implement though, I'm just saying one can require them to submit source code and then manual checking can be done for candidates who pass the screen. The reason is that there are a lot of drag-and-drop style wizards that can just generate some skeletal web service that could be used to break this screening test.
In the end, this is an amazing service, but sometimes, these tasks are essentially trivial, like, for example, exposing a function to a web service. I believe (and this is just my opinion), that the function is more essentially than how one can wire it to different interfaces.
But, what do I know, I'm too drunk right now.
In the end, this is an amazing service, but sometimes, these tasks are essentially trivial, like, for example, exposing a function to a web service. I believe (and this is just my opinion), that the function is more essentially than how one can wire it to different interfaces.
But, what do I know, I'm too drunk right now.
There's definitely some merit in this approach over something like Fizzbuzz and whiteboard problems. At my previous place of employment [1] , after the initial interview, you were given 48 hours to produce a simple web addressbook app. You could use anything that you wanted to produce the app - at the end of the 48 hour period, you had to supply thme with the code and instructions on how to install it on a Ubuntu instance. It was actually quite fun (and then after getting the job and testing other people's entries, you begin to see how it separated people with impressive CVs who couldn't program for toffee, and others who had much less experience, but managed to come up with something that worked!)
[1] That would be Open Source Integrators, a great Durham-based outfit who I believe are still looking for people to staff their Chicago offices: http://osintegrators.com/
[1] That would be Open Source Integrators, a great Durham-based outfit who I believe are still looking for people to staff their Chicago offices: http://osintegrators.com/
Great idea... but did you let them know ahead of time? I'd hate to do a phone interview and then realize I'm building a web app over my weekend trip to Chicago.
The nice thing was that after you passed the initial interview, the company would let you start whenever (within reason, obviously). You'd send them an email that you were ready, they'd send details, and the clock would start ticking from then. Much more relaxed.
[deleted]
@ksat - I think it's easy to get the wrong idea of what your site is for from the heading - "...having them develop a real web application". Of course building an entire web app just to apply for a job is overkill. But it looks like that's not what you meant? I read further down the page and the examples given are more like short coding challenges. That's acceptable, and I think you need to focus your site more on filling that need. Position it as some sort of "Automated Fizzbuzz Test".
Got it. Thanks. I'll see how I can word it better.
What stops me from asking a friend to help me? Or worse still, if this is used as a campus recruitment tool, someone might mail working code to the entire college!
Well yes, but thats applicable any remote interviews. Or even telephonic interviews.
As a solution you could have the candidate solve a challenge on premise. or having him explain the code or make small modification on premise(what i do, if I have a doubt)
[deleted]
I have not seen the service itself for the developer, but as for the screen I would recommend a way of customizing a bit more what site it should check. Right now it seems you can only change the port, but plenty of times I just create a new directory under root for the webapp (something like http://localhost:8080/mywebapp) because it's easier in some web servers.
Got it. Din't think of that. thanks. It'll be a nice to perform all requests relative to a webapp directory. Will send you a mail once it's done.
Oh, now programmers are graphic designers too? Don't want to play the submit three logo comps and maybe we'll hire you game?
Geez, even a hired gun has to shoot someone for free once in a while just to show potential employers he's still got it! No one really knows what you did before (unless you have code groupies). Employers certainly do not give a shit in today's job market, they want to know you will fit in with the team and that you work as advertised. No one in their right mind believes advertising or resumes these days. So what's the alternative? You want to slip them source someone else paid you to make! I sure as shit would not hire you if you share like that. Competitions surely suck for losers who do not learn from their loss(es).
Suck on this - if you are bored then YOU are boring. If you won't build a small (not going to make anyone else money) solution to show a potential employer you can, you probably can't.
Geez, even a hired gun has to shoot someone for free once in a while just to show potential employers he's still got it! No one really knows what you did before (unless you have code groupies). Employers certainly do not give a shit in today's job market, they want to know you will fit in with the team and that you work as advertised. No one in their right mind believes advertising or resumes these days. So what's the alternative? You want to slip them source someone else paid you to make! I sure as shit would not hire you if you share like that. Competitions surely suck for losers who do not learn from their loss(es).
Suck on this - if you are bored then YOU are boring. If you won't build a small (not going to make anyone else money) solution to show a potential employer you can, you probably can't.
Small nitpick: it's typical to have your logo link to the homepage.
It's a pretty nice idea. That music is beyond horrible, however.
Payed $47 to musicbakery.com for that :(
Agree with antihero. The music is not great.
it better be a very simple web app.
otherwise the applicant can only apply to a couple jobs a week/month.
makes me think: ask a car manufacturer to design & build a new car before you choose them to design & build your company's custom car.
makes me think: ask a car manufacturer to design & build a new car before you choose them to design & build your company's custom car.
Yes! I think the complexity should be reasonable too. I've listed why I built it here: http://www.cull.io/why.html
To start with, if you are recruiting a web developer it can be an simple alternative which lets candidates choose any framework, language or library.
P.s: It is another reason why cull.io never tracks the time a candidates take to complete the challenge. Honestly I think services like this will merely be a way to filter out candidates not select them. To select candidates you still need to work with them on a project and see their previous projects
To start with, if you are recruiting a web developer it can be an simple alternative which lets candidates choose any framework, language or library.
P.s: It is another reason why cull.io never tracks the time a candidates take to complete the challenge. Honestly I think services like this will merely be a way to filter out candidates not select them. To select candidates you still need to work with them on a project and see their previous projects
Interesting idea, but what stops me from just writing the content instead of programming it?
A Few things:
1) For some requests, the correct response can be hidden and just reply 'pass' or 'fail'.
2) You can use javascript to randomize the requests. For example create a random expression and ask it to be evaled
3) Candidates submit the code after completing the challenge.
Thanks for checking out!
1) For some requests, the correct response can be hidden and just reply 'pass' or 'fail'.
2) You can use javascript to randomize the requests. For example create a random expression and ask it to be evaled
3) Candidates submit the code after completing the challenge.
Thanks for checking out!
The intro video would benefit by demonstrating this. I had the same question.
love it , on the side note , it might help to train fresh developer, example create series of request and get the fresh grad developer to go through ?
"Print the tweet given _its_ Id"
:). Will fix. Thanks
http://www.cull.io/images/home/answersannot.png - Has the text "Resume and other intersting stuff", there's a typo on interesting.
Nice idea by the way!
Nice idea by the way!
Thanks! Will fix.
If you look at filtering mechanisms themselves, there are companies that try to have puzzles that they ask you to solve before joining (eg: ITA software). But they are really challenging puzzles that I would solve just for fun anyway, (which is what I did for months till I one day decided to apply for a job there). The fact that these puzzles are also made by the company (and not a third party website) makes a difference. It tells me about the people that work there and potentially about their culture. On the other hand, building a random web app to prove basic skills is not something I would imagine most talented web developers would like to do anyway.
If you look at it from the perspective of a company that is hiring, tools like this solve merely one part of the puzzle. It takes the subset of developers who are already in the market for a new job and filters them. As a startup though, what I want to do is lure the great engineers who are content at their regular jobs and not looking at all because of inertia. It sounds like there is more of a "search for developers" problem that will expand the breadth of all the developers available to me, instead of focusing on the ones who are already out there (and are most likely not the cream of the crop by definition).
In short, I don't mean to be negative, and am not saying that there might not be a market for a tool like this. But I am pretty certain that if you are a great developer, you don't have an incentive to make a web app on this.