Software Raises Bar for Hiring(webcache.googleusercontent.com)
webcache.googleusercontent.com
Software Raises Bar for Hiring
http://webcache.googleusercontent.com/search?q=cache:pghpLbngybIJ:online.wsj.com/article/SB10001424052702304821304577436172660988042.html
14 comments
I've been on the hiring manager's side and the employee's side, more the former than the latter.
I don't get your beef.
The phone interviews are there to screen the 80% of the applicants who don't merit the time of my team to interview them. The puzzles are there to watch people think on their feet. The detailed questions are to see if you're looking at someone superficial or someone with deep knowledge of the subject matter. The programming challenges are to see if the applicant wants the job badly enough to spend a couple of hours working on a programming problem that can then be used as a showcase for how they write some software.
Earlier in my career, I didn't have all those interview techniques in place. When I started, hiring people was a complete crap shoot and I ended up with some total lemons. Like the warning labels on products you buy, each interview technique I added to the process was the result of a lame hire that in hindsight, I could have avoided had I used that technique.
The last job I had to interview for used all the techniques above. I found the interview process to be thorough and challenging. I appreciated the rigor of the interview process since it meant that the people hiring really gave a shit about the quality of the people joining their small team. I got the job. In point of fact, if they hadn't had a thorough interview process, I probably would have passed on the job offer since that would have been an indication to me that they were naive and didn't have their crap together.
Do you want to send me your $100 via Paypal?
I don't get your beef.
The phone interviews are there to screen the 80% of the applicants who don't merit the time of my team to interview them. The puzzles are there to watch people think on their feet. The detailed questions are to see if you're looking at someone superficial or someone with deep knowledge of the subject matter. The programming challenges are to see if the applicant wants the job badly enough to spend a couple of hours working on a programming problem that can then be used as a showcase for how they write some software.
Earlier in my career, I didn't have all those interview techniques in place. When I started, hiring people was a complete crap shoot and I ended up with some total lemons. Like the warning labels on products you buy, each interview technique I added to the process was the result of a lame hire that in hindsight, I could have avoided had I used that technique.
The last job I had to interview for used all the techniques above. I found the interview process to be thorough and challenging. I appreciated the rigor of the interview process since it meant that the people hiring really gave a shit about the quality of the people joining their small team. I got the job. In point of fact, if they hadn't had a thorough interview process, I probably would have passed on the job offer since that would have been an indication to me that they were naive and didn't have their crap together.
Do you want to send me your $100 via Paypal?
I don't have a beef with it. I also think that of you apply to the jobs posted on HN, you will get my point.
Sadly, I don't use Paypal. You can make a donation to your local hackerspace with the $100 that you will soon lose.
Good luck.
PS. Have you ever asked the people you have interviewed what they think about the process?
Sadly, I don't use Paypal. You can make a donation to your local hackerspace with the $100 that you will soon lose.
Good luck.
PS. Have you ever asked the people you have interviewed what they think about the process?
"Have you ever asked the people you have interviewed what they think about the process?", especially, perhaps, the people you didn't hire?
Almost without exception, people who don't get hired never hear back from a company as to why. Yes, I know all about "everyone's afraid to get sued for saying something wrong". So... develop a neutral feedback form to candidates as to why they were passed over, skills the interviewer(s) thought were lacking, etc. This will mean that people can get better, perhaps get another job (yes, maybe at a competitor, but also maybe somewhere else entirely), and continue to earn income, pay taxes, and contribute to society in a productive way. Some people can contribute without a job, sure, but right now most people need jobs.
Telling someone "we're sorry - we had 8 candidates apply for this position, and we ended up taking on someone with more experience in X, Y and Z compared to your experience level. We wish you the best of luck in your job search". This would be courteous, professional and helpful all at the same time. Between my own experience and that of several colleagues, fewer than 5% of employers ever provide something even remotely useful in terms of feedback.
How do we expect the job seeking population to get better without providing feedback mechanisms for them to learn from?
Almost without exception, people who don't get hired never hear back from a company as to why. Yes, I know all about "everyone's afraid to get sued for saying something wrong". So... develop a neutral feedback form to candidates as to why they were passed over, skills the interviewer(s) thought were lacking, etc. This will mean that people can get better, perhaps get another job (yes, maybe at a competitor, but also maybe somewhere else entirely), and continue to earn income, pay taxes, and contribute to society in a productive way. Some people can contribute without a job, sure, but right now most people need jobs.
Telling someone "we're sorry - we had 8 candidates apply for this position, and we ended up taking on someone with more experience in X, Y and Z compared to your experience level. We wish you the best of luck in your job search". This would be courteous, professional and helpful all at the same time. Between my own experience and that of several colleagues, fewer than 5% of employers ever provide something even remotely useful in terms of feedback.
How do we expect the job seeking population to get better without providing feedback mechanisms for them to learn from?
Telling someone "we're sorry - we had 8 candidates apply for this position, and we ended up taking on someone with more experience in X, Y and Z compared to your experience level.
And what about the cases where your reasons are not "more experience in X, Y, Z"? Cultural fit is a biggie - one might turn down a candidate who is more qualified because his banking attitudes would go over badly at a startup, for example.
And what about the cases where your reasons are not "more experience in X, Y, Z"? Cultural fit is a biggie - one might turn down a candidate who is more qualified because his banking attitudes would go over badly at a startup, for example.
That's true, and in those cases, a more generic "we had other qualified candidates" letter/email would be fine. but all too often corporate america just ignores people altogether - won't return emails/phone calls, and basically leaves the person out to dry. Working via a recruiter, you at least have a person at the agency who might be sympathetic.
Companies don't seem to realize that treating applicants bad is just as detrimental in many cases as poor customer service. I've had bad experiences with job application processes, and I've told many people chapter and verse about the companies involved. If they can't even treat people well who want to work there, how will they treat customers after the money is received?
Companies don't seem to realize that treating applicants bad is just as detrimental in many cases as poor customer service. I've had bad experiences with job application processes, and I've told many people chapter and verse about the companies involved. If they can't even treat people well who want to work there, how will they treat customers after the money is received?
That's true, and in those cases, a more generic "we had other qualified candidates" letter/email would be fine.
If unskilled candidate X gets "other candidates had more specific skillz" while uncultured candidate Y gets "sorry, you suck for unspecified reasons", it sounds like a lawsuit risk.
If unskilled candidate X gets "other candidates had more specific skillz" while uncultured candidate Y gets "sorry, you suck for unspecified reasons", it sounds like a lawsuit risk.
Telling the candidate anything specifically negative invites argument and misinterpretation. In practice, the better companies always use the "other qualified candidates" letter. The worse ones tell you nothing.
> PS. Have you ever asked the people you have interviewed what they think about the process?
Specifically asked? I've received lots of feedback from people who made it through the process. They've added to my bag of interview techniques here and there. In general, they appreciated a hiring manager who spent a lot of time and effort fielding good people. Those kinds of comments tend to be a bit self-selecting, though.
I've had a couple of candidates who didn't make it get frustrated with the process. One in particular lost his cool just trying to pseudocode a routine to determine prime factors of the input or something like that. If you lose your composure doing something like that, I don't want you in the trenches with me late at night when we have a demo to show to customers the next day and something isn't working.
When people have asked me for feedback on why they weren't made an offer, I've always tried to be as candid as possible; especially when it seemed that they were sincerely trying to improve. Screw HR and their obsessive need to CYA.
Specifically asked? I've received lots of feedback from people who made it through the process. They've added to my bag of interview techniques here and there. In general, they appreciated a hiring manager who spent a lot of time and effort fielding good people. Those kinds of comments tend to be a bit self-selecting, though.
I've had a couple of candidates who didn't make it get frustrated with the process. One in particular lost his cool just trying to pseudocode a routine to determine prime factors of the input or something like that. If you lose your composure doing something like that, I don't want you in the trenches with me late at night when we have a demo to show to customers the next day and something isn't working.
When people have asked me for feedback on why they weren't made an offer, I've always tried to be as candid as possible; especially when it seemed that they were sincerely trying to improve. Screw HR and their obsessive need to CYA.
>> The puzzles are there to watch people think on their feet
Genuinely curious: why do you consider spontaneity an essential trait in a programmer?
Genuinely curious: why do you consider spontaneity an essential trait in a programmer?
I agree with everything you've said except for the bit about puzzles -- I've never found any correlation between the results from burn-the-ropes/odd-shaped manhole cover-type questions and strong programming skills. I've had at least one candidate who did superbly on the puzzlers (had clearly not seen them before)- best performance we'd seen-and could not program his way out of a wet paper bag. We stopped the puzzlers shortly thereafter. Brain teasers are a waste of time.
Yea I have to agree with other posters. This is easy money for most software engineers because they have exactly the skills that will be posted on the ads, criteria that are checked, etc. I think even your challenge misses the point of the article which is when we post wanted ads we put down such specific skill sets that if we rigidly stick those skill sets, just like our software does, we'll miss the people that can grow into those roles.
The irony is that software engineers are so much rarer than everyone else in the market we can easily be hand screened thus bypassing the very software we created. Finding software engineers is really hard because there are so few so either you don't get a lot of people applying, which makes it easy to screen by hand, or you set the bar very low to get people in the door. Then you go through 6 months to find the right person.
If you aren't a software person its much much harder to find a job because there are so many other people looking for jobs. You have to compete much harder, downward wage pressure, etc. Hence the volume increasingly demands automated solutions that make it even harder to get past the screeners.
I happen to agree with this article that we do a poor job of specifying the skills that are absolutely mandatory vs. what's nice to have. If we back off on rigid skill set we can find great people than can grow into a job.
The irony is that software engineers are so much rarer than everyone else in the market we can easily be hand screened thus bypassing the very software we created. Finding software engineers is really hard because there are so few so either you don't get a lot of people applying, which makes it easy to screen by hand, or you set the bar very low to get people in the door. Then you go through 6 months to find the right person.
If you aren't a software person its much much harder to find a job because there are so many other people looking for jobs. You have to compete much harder, downward wage pressure, etc. Hence the volume increasingly demands automated solutions that make it even harder to get past the screeners.
I happen to agree with this article that we do a poor job of specifying the skills that are absolutely mandatory vs. what's nice to have. If we back off on rigid skill set we can find great people than can grow into a job.
That makes no sense to me. I know exactly what skills my company needs (or rather, what the hiring people look for on the CV). I know exactly what the interview is looking for. I know exactly what programming language to brush up on, and I know the complete set of coding problems set in the interview.
I know it so well that the last person I recommended (for the recruitment bounty) breezed through after I tipped her off on the coding questions. I could get through the whole process easily. I doubt I'm the only person.
What kind of people set questions in interviews without knowing what answer they're hoping for? How do they know if the candidate is any good if they don't know the answer themselves? That sounds.... well, it sounds insane.
I know it so well that the last person I recommended (for the recruitment bounty) breezed through after I tipped her off on the coding questions. I could get through the whole process easily. I doubt I'm the only person.
What kind of people set questions in interviews without knowing what answer they're hoping for? How do they know if the candidate is any good if they don't know the answer themselves? That sounds.... well, it sounds insane.
I think you are missing the point. The challenge is to design a set of hurdles that are equivalent to those faced by potential hires, not the exact ones. Of course you ought to know what your company asks for. Furthermore, I don't think its all about the questions. Part of the challenge is to design the exact same circumstances, i.e. puzzles, phone interviews, travelling, amount of time to review what the company requires, length of interviews, and so on.
breezed through after I tipped her off on the coding questions
I probably botched an interview where they expected me to come up with an RB tree fast enough (it was a company that has posted on HN).
Tipping people off to questions is cheating the test, so you have some people trying to temporarily bulk memorize algorithms that they'll forget in another month before an interview. Does anyone else not find this stupid?
I probably botched an interview where they expected me to come up with an RB tree fast enough (it was a company that has posted on HN).
Tipping people off to questions is cheating the test, so you have some people trying to temporarily bulk memorize algorithms that they'll forget in another month before an interview. Does anyone else not find this stupid?
> What kind of people set questions in interviews without knowing what answer they're hoping for?
The kind of people who want to "see how you think."
The kind of people who want to "see how you think."
I'll add another $100. I think the fundamental problem is that employers are unwilling to take on someone unless they are absolutely certain that the candidate is the right person because firing is extremely difficult and complicated (with potential for law suits) if the new hire does not prove to be a good fit.
It's not just lawsuits. Getting a new hire equipped and up to speed takes resources, and drains productivity away from existing employees.
Also, once you've done a little training the sunk costs fallacy can kick in, even if you wouldn't hire them in hindsight.
Also, once you've done a little training the sunk costs fallacy can kick in, even if you wouldn't hire them in hindsight.
I thought this was what probationary periods were for; evaluating a new employee with the opportunity to not continue their employment at the end of it.
What in America the land of "at will" employment I find that very hard to believe.
You might find out something frightening. Getting the job mostly just came down to luck.
People in the real world realize that outside factors have a lot to do with hiring.
But when these conversations turn into arguments it all comes down to some applicants being more "qualified."
There are lots of reasons people get hired to do jobs they've never done before and lot of those reasons are subjective.
But when these conversations turn into arguments it all comes down to some applicants being more "qualified."
There are lots of reasons people get hired to do jobs they've never done before and lot of those reasons are subjective.
Exactly.
Maybe the CEO got a not so gentle nudge from the investors to hurry up and hire somebody and you were the one they were interviewing next. Stuff like that happens a lot. The process is more about outside factors than the actual interviewing process itself.
Maybe the CEO got a not so gentle nudge from the investors to hurry up and hire somebody and you were the one they were interviewing next. Stuff like that happens a lot. The process is more about outside factors than the actual interviewing process itself.
I've never received an offer to work as a software engineer other than by going through the whole interview process. My impression was that the vast majority of other people who became software engineers at the companies I applied to had done the same.
I'll take your bet, if you are still inclined to give away money.
I'll take your bet, if you are still inclined to give away money.
In most cases that's how the engineers got hired on in the first place, so I don't understand your beef. Plus, what you really want is to hire people better than the ones you already have.
Well if I am working at a place that would tend to imply I did get through all that. Why would my chances be less a second time around? That said, I do agree that the hiring process is rather broken, just not sure that this would be a good test to show that.
Most of them had to go through the same thing to be hired.
This is spot on. I've been on both sides of the fence, and in multiple disciplines. The strange thing I found is that finding a job as a developer has been just as difficult, if not more so, than finding a non-technical job.
On the non-technical side, the basic requirements are exactly that - basic. Have some experience, a certification or two, and a specialized degree. That doesn't make it easy, but it makes the path somewhat concrete.
And on the technical side, there are very few companies willing to hire entry level developers. They all seem to want code ninjas, which is fine, but here's the thing...if I'm a code ninja, why would I work for someone else when I could build something of my own without the requirement of commuting to an office and clocking in/out.
On the employer side, I've found far less people applying for technical jobs than non-technical jobs. It has taken much longer to fill mid-level technical positions than entry-mid level non-technical positions.
Building a team is difficult. Joining a team is difficult.
It's a very strange time indeed.
On the non-technical side, the basic requirements are exactly that - basic. Have some experience, a certification or two, and a specialized degree. That doesn't make it easy, but it makes the path somewhat concrete.
And on the technical side, there are very few companies willing to hire entry level developers. They all seem to want code ninjas, which is fine, but here's the thing...if I'm a code ninja, why would I work for someone else when I could build something of my own without the requirement of commuting to an office and clocking in/out.
On the employer side, I've found far less people applying for technical jobs than non-technical jobs. It has taken much longer to fill mid-level technical positions than entry-mid level non-technical positions.
Building a team is difficult. Joining a team is difficult.
It's a very strange time indeed.
> The strange thing I found is that finding a job as a developer has been just as difficult, if not more so, than finding a non-technical job.
It's easier to bullshit when interviewing for non-technical jobs. Technical jobs have mostly-objective tests you can give to candidates where they have to program stuff, solve equations, etc. Non-technical jobs have squishy mostly-subjective tests that aren't very reliable at gauging anything.
It's easier to bullshit when interviewing for non-technical jobs. Technical jobs have mostly-objective tests you can give to candidates where they have to program stuff, solve equations, etc. Non-technical jobs have squishy mostly-subjective tests that aren't very reliable at gauging anything.
The US should find ways to make employers less risk-averse. There are lots of ways worth exploring. For example: maybe if you fire someone in the first 6 months you are immune from wrongful termination lawsuits and don't have to pay higher unemployment insurance. A lot of resistance to firing is also caused by moral concerns -- we could fix that by making sure these fired workers are still eligible for unemployment.
This would be extra beneficial for young people, who really just need to get a foot in the door to prove themselves -- they need someone to take a risk. Well, maybe we should reduce the risk.
This would be extra beneficial for young people, who really just need to get a foot in the door to prove themselves -- they need someone to take a risk. Well, maybe we should reduce the risk.
Most US states are at-will states, meaning they're already immune from wrongful termination, except for discriminatory reasons. No matter how long the employee has been there.
While "At-Will" in theory means that they're immune from wrongful termination, there's some major exceptions in most of those states.
See "Covenant of good faith and fair dealing" (which in some states mean you have to be fired for 'just cause') and "Implied contract exceptions".
I say these not to indicate you're wrong, mostly to make sure the entrepreneurs around here understand that even in a lot of "At-Will" states they might not be able fire completely with impunity.
w/r/t the GP as long as the rules around a trial period are set via an employee handbook or in the contract, generally in the US we're fairly free to try people out and let them go, the US is actually more liberal in this then a lot of European countries.
http://en.wikipedia.org/wiki/At-will_employment
See "Covenant of good faith and fair dealing" (which in some states mean you have to be fired for 'just cause') and "Implied contract exceptions".
I say these not to indicate you're wrong, mostly to make sure the entrepreneurs around here understand that even in a lot of "At-Will" states they might not be able fire completely with impunity.
w/r/t the GP as long as the rules around a trial period are set via an employee handbook or in the contract, generally in the US we're fairly free to try people out and let them go, the US is actually more liberal in this then a lot of European countries.
http://en.wikipedia.org/wiki/At-will_employment
Your proposal sounds a lot like the risk reduction schemes implemented in France a few years ago under Sarkozy. The net result of the legislation was the creation of a micro-job economy like we see in Germany today (and a lot of street riots). Basically, employees are "hired" with an unspoken understanding that the job is essentially a temporary position, and they will be fired without severance near the end of their 6 month term. If you look into how consumer buying power among young micro-employees in Germany has decreased, the idea may seem a bit less appealing.
You'd end up seeing a lot of 5 month and 29 day employment terms for people. It'd be no effective difference than 'contract and perhaps hire later', except so many people are averse to 'contract' work for reasons (sometimes valid ones) that have little to do with the work and more to do with the culture of 'benefits'.
I don't quite know how we as a culture do it, but we need to wean ourselves off of 'employer-provided benefits'. Getting back to a more consumer-driven market vs large employer group-driven market will make our job/work situations easier for generations to come.
I don't quite know how we as a culture do it, but we need to wean ourselves off of 'employer-provided benefits'. Getting back to a more consumer-driven market vs large employer group-driven market will make our job/work situations easier for generations to come.
A simple solution would be to exchange employer-provided benefits for government-provided benefits :-/
There's a failure of logic in this article that no one is pointing out.
If employers are so picky that they can't find anyone to hire, then they will have to lower their standards if they want to hire anyone.
There might be a delay--like the guy that waits for the supermodel to come along, but eventually settling for the pretty girl next door. Ultimately the problem is self-correcting.
If employers are so picky that they can't find anyone to hire, then they will have to lower their standards if they want to hire anyone.
There might be a delay--like the guy that waits for the supermodel to come along, but eventually settling for the pretty girl next door. Ultimately the problem is self-correcting.
And employers are not that picky either. Most of the things they ask, at least in Europe, are pretty basic. For a finance is usually a bachelor, English skills, some prove you are not a lazy ass. Of course, requirements go up with better positions, but this article deceives into thinking that employers are looking for stars even in entry level jobs, it doesn't look like that to me.
I am still convinced the problem is on the employee side...
I am still convinced the problem is on the employee side...
He makes good points about the software filtering out qualified applicants, but I don't think that is the underlying problem.
The underlying problem is that there is a lot of unemployment and therefore too many people applying for too few jobs.
We just don't need that many people to order around anymore now that we have so much hardware and software automation. The public education system is designed to create wage slaves, and is effectively doing that, but we don't need that many anymore.
Another part of it of course is just that we are in an 'economic' down period and therefore have to be extremely selective when hiring since no one can afford any dead weight or even to do much training.
The entire 'economic' model has been invalidated by technology.
The underlying problem is that there is a lot of unemployment and therefore too many people applying for too few jobs.
We just don't need that many people to order around anymore now that we have so much hardware and software automation. The public education system is designed to create wage slaves, and is effectively doing that, but we don't need that many anymore.
Another part of it of course is just that we are in an 'economic' down period and therefore have to be extremely selective when hiring since no one can afford any dead weight or even to do much training.
The entire 'economic' model has been invalidated by technology.
The public education system is designed to create wage slaves, and is effectively doing that, but we don't need that many anymore.
The other half of the problem is that our economy creates very few sources of income for the supposed "creative class", the newest Class of the New. If we say that our economy has evolved beyond the need for factory workers and we now need artists, writers, and hackers, we need to actually work on creating jobs for artists, writers and hackers (as distinct from mediocre programmers).
The other half of the problem is that our economy creates very few sources of income for the supposed "creative class", the newest Class of the New. If we say that our economy has evolved beyond the need for factory workers and we now need artists, writers, and hackers, we need to actually work on creating jobs for artists, writers and hackers (as distinct from mediocre programmers).
Well.. I actually think that anything that qualifies as a job is more or less going in the category of wage slave.
I just think that the extreme hierarchies and subservience are unnecessary and detrimental. I think we need to examine our basic assumptions and have a realistic perspective about the actual level of suffering that is currently going on in the world.
We need to see the extreme inequality as being unacceptable and move away from the Social Darwinian rationalizations.
I just think that the extreme hierarchies and subservience are unnecessary and detrimental. I think we need to examine our basic assumptions and have a realistic perspective about the actual level of suffering that is currently going on in the world.
We need to see the extreme inequality as being unacceptable and move away from the Social Darwinian rationalizations.
I agree with you, but then we have to overthrow capitalism completely. Unfortunately, socialism is considered impolite on HN.
Well, just to clarify further, whenever I talk like this, people think I mean to completely do away with money and/or move towards a traditional socialist or techno-communist model.
I don't want to do that. I think that almost all anti-capitalists are missing a very important lesson that we can take as a major success of capitalism: diversity and distribution is very important for robustness and ease of system evolution. Technocrats, techno-communists, social democrats, almost all seem to be aiming for centralization which has been proven to be extremely inadequate.
So I think the way forward is to create a system that while encouraging a great diversity of distributed, localized solutions to problems, at the same time the local solutions are developed and evaluated holistically over a comprehensive common global scientific information schema and database.
I don't want to do that. I think that almost all anti-capitalists are missing a very important lesson that we can take as a major success of capitalism: diversity and distribution is very important for robustness and ease of system evolution. Technocrats, techno-communists, social democrats, almost all seem to be aiming for centralization which has been proven to be extremely inadequate.
So I think the way forward is to create a system that while encouraging a great diversity of distributed, localized solutions to problems, at the same time the local solutions are developed and evaluated holistically over a comprehensive common global scientific information schema and database.
You'd be surprised at the variability and diversity of the approaches that fall under the term "socialism".
"His son, now 25 years old, graduated in 2010 with a degree in classics from St. John's College and couldn't find a job."
This is the problem right here. Gee I wonder why you couldn't find a job in "Classics"? Because there aren't any, at least with an undergraduate degree. People pay you to work because you can perform a given task with a level of quality and consistency. We tell kids that they should study whatever makes them happy in college, but then when they graduate and can't find a job it's not their fault. Fine you study your BS degree, but don't expect to be handed a job when you graduate. Expect to work some menial job, or do what this kid did, go back to school and learn a trade. You'll have to prove to someone that you have some employable skill if you want to earn money. Having a job isn't a right.
This is the problem right here. Gee I wonder why you couldn't find a job in "Classics"? Because there aren't any, at least with an undergraduate degree. People pay you to work because you can perform a given task with a level of quality and consistency. We tell kids that they should study whatever makes them happy in college, but then when they graduate and can't find a job it's not their fault. Fine you study your BS degree, but don't expect to be handed a job when you graduate. Expect to work some menial job, or do what this kid did, go back to school and learn a trade. You'll have to prove to someone that you have some employable skill if you want to earn money. Having a job isn't a right.
The underlying problem is that most companies rely on hiring procedures that are empirically demonstrated to be suboptimal. Companies rely on industry tradition or the gut feelings of the boss rather than on research to decide hiring practices. The review article "The Validity and Utility of Selection Models in Personnel Psychology: Practical and Theoretical Implications of 85 Years of Research Findings"
http://mavweb.mnsu.edu/howard/Schmidt%20and%20Hunter%201998%...
sums up, current to 1998, much of the HUGE peer-reviewed professional literature on the industrial and organizational psychology devoted to business hiring practices. There are many kinds of hiring screens, such as resume reviews for job experience, telephone interviews, in-person interviews, checks for academic credentials, and so on. Most of the most commonly used means for screening job applicants, such as those mentioned in the submitted (like matching job titles in previous positions) are basically useless. There is much published study research on how job applicants perform after they are hired in a wide variety of occupations.
The overall summary of the industrial psychology research in reliable secondary sources is that two kinds of job screening procedures work reasonably well (but only about at the 0.5 level, standing alone). One is a general cognitive ability test (an IQ-like test, such as the Wonderlic personnel screening test). Another is a work-sample test, where the applicant does an actual task or group of tasks like what the applicant will do on the job if hired. Each of these kinds of tests has about the same validity in screening applicants for jobs, with the general cognitive ability test better predicting success for applicants who will be trained into a new job. Neither is perfect (both miss some good performers on the job, and select some bad performers on the job), but both are better than anything else that has been tested in rigorous research, across a wide variety of occupations. So if you are hiring for your company, it's a good idea to think about how to build a work-sample test into all of your hiring processes.
For legal reasons in the United States (the same consideration does not apply in other countries), it is difficult to give job applicants a straight-up IQ test (as was commonplace in my parents' generation) as a routine part of a hiring process. The Griggs v. Duke Power, 401 U.S. 424 (1971) case in the United States Supreme Court
http://scholar.google.com/scholar_case?case=8655598674229196...
held that cognitive ability tests used in hiring that could have a "disparate impact" on applicants of some protected classes must "bear a demonstrable relationship to successful performance of the jobs for which it was used." In other words, a company that wants to use a test like the Wonderlic, or like the SAT, or like the current WAIS or Stanford-Binet IQ tests, in a hiring process had best conduct a specific validation study of the test related to performance on the job in question. Some companies do the validation study, and use IQ-like tests in hiring. Other companies use IQ-like tests in hiring and hope that no one sues (which is not what I would advise any company). Note that a brain-teaser-type test used in a hiring process might be illegal if it can be shown to have disparate impact on some job applicants and is not supported by a validation study demonstrating that the test is related to successful performance on the job. Companies outside the United States are regulated by different laws. So don't set up stupid HR procedures like those described in the submitted article, even if those stupid procedures can be automated. Instead, figure out how to give applicants a work sample test, or figure out how to validate a general cognitive ability test for the applicants you desire to hire. Automating USEFUL processes can be helpful. There is no reason to automate a process that is not worth doing.
http://mavweb.mnsu.edu/howard/Schmidt%20and%20Hunter%201998%...
sums up, current to 1998, much of the HUGE peer-reviewed professional literature on the industrial and organizational psychology devoted to business hiring practices. There are many kinds of hiring screens, such as resume reviews for job experience, telephone interviews, in-person interviews, checks for academic credentials, and so on. Most of the most commonly used means for screening job applicants, such as those mentioned in the submitted (like matching job titles in previous positions) are basically useless. There is much published study research on how job applicants perform after they are hired in a wide variety of occupations.
The overall summary of the industrial psychology research in reliable secondary sources is that two kinds of job screening procedures work reasonably well (but only about at the 0.5 level, standing alone). One is a general cognitive ability test (an IQ-like test, such as the Wonderlic personnel screening test). Another is a work-sample test, where the applicant does an actual task or group of tasks like what the applicant will do on the job if hired. Each of these kinds of tests has about the same validity in screening applicants for jobs, with the general cognitive ability test better predicting success for applicants who will be trained into a new job. Neither is perfect (both miss some good performers on the job, and select some bad performers on the job), but both are better than anything else that has been tested in rigorous research, across a wide variety of occupations. So if you are hiring for your company, it's a good idea to think about how to build a work-sample test into all of your hiring processes.
For legal reasons in the United States (the same consideration does not apply in other countries), it is difficult to give job applicants a straight-up IQ test (as was commonplace in my parents' generation) as a routine part of a hiring process. The Griggs v. Duke Power, 401 U.S. 424 (1971) case in the United States Supreme Court
http://scholar.google.com/scholar_case?case=8655598674229196...
held that cognitive ability tests used in hiring that could have a "disparate impact" on applicants of some protected classes must "bear a demonstrable relationship to successful performance of the jobs for which it was used." In other words, a company that wants to use a test like the Wonderlic, or like the SAT, or like the current WAIS or Stanford-Binet IQ tests, in a hiring process had best conduct a specific validation study of the test related to performance on the job in question. Some companies do the validation study, and use IQ-like tests in hiring. Other companies use IQ-like tests in hiring and hope that no one sues (which is not what I would advise any company). Note that a brain-teaser-type test used in a hiring process might be illegal if it can be shown to have disparate impact on some job applicants and is not supported by a validation study demonstrating that the test is related to successful performance on the job. Companies outside the United States are regulated by different laws. So don't set up stupid HR procedures like those described in the submitted article, even if those stupid procedures can be automated. Instead, figure out how to give applicants a work sample test, or figure out how to validate a general cognitive ability test for the applicants you desire to hire. Automating USEFUL processes can be helpful. There is no reason to automate a process that is not worth doing.
> Other companies use IQ-like tests in hiring and hope that no one sues (which is not what I would advise any company).
It's weird, as the actual policy was "requiring a high school education or passing of a standardized general intelligence test".
Surely there's a lot of African Americans with a high IQ, but no degree. Wouldn't this case mean that requiring a certain level of education is worse than requiring a high IQ?
It's weird, as the actual policy was "requiring a high school education or passing of a standardized general intelligence test".
Surely there's a lot of African Americans with a high IQ, but no degree. Wouldn't this case mean that requiring a certain level of education is worse than requiring a high IQ?
In the UK the govt wants to implement a report commissioned from a hedge fund friend of the PMs - one of the key "political" recommendations was to introduce no fault firing - making firing staff ridiculously easy because that is obviously want is wrong with UK PLC. The elephant in the room is that if you have hired large numbers of crap staff and you can't get rid of them because it will cost too much or they will all sue, then clearly your past hiring practises were wrong.
It's interesting how we still run to the government to solve our own past mistakes.
It's interesting how we still run to the government to solve our own past mistakes.
>like the SAT
I would love it if companies hired people like colleges select students.
I'm very good at interviews, but I still hate doing them. If would love it if I could write an essay and send them my SAT scores without going through that awful process.
It would probably also weed out people who are just inordinately good at interviews.
I don't overtly try to exaggerate, but a few times I've been offered a job where I thought in the back of the mind that I might not really be qualified. To wit, I thought my interview skills, not my coding skills got me the offer.
I would love it if companies hired people like colleges select students.
I'm very good at interviews, but I still hate doing them. If would love it if I could write an essay and send them my SAT scores without going through that awful process.
It would probably also weed out people who are just inordinately good at interviews.
I don't overtly try to exaggerate, but a few times I've been offered a job where I thought in the back of the mind that I might not really be qualified. To wit, I thought my interview skills, not my coding skills got me the offer.
I get what you mean, but colleges are probably the worst analogue for successful hiring practices given selection based on legacy status, wealth, minority status, athletic ability, etc
A note about the above. The article actually states that the best indicators of performance wasn't actually just a GMA (General Mental Aptitude) test or just a trial but in fact the combination of a GMA test and a integrity test or a GMA test and a structured interview. You're not off the hook by just testing well.
Of the combinations of predictors examined, two stand out as being both practical to use for most hiring and as having high composite validity: the combination of a GMA test and an integrity test (composite validity of .65); and the combination of a G M A test and a structured interview .(com posite validity of .63).
Of the combinations of predictors examined, two stand out as being both practical to use for most hiring and as having high composite validity: the combination of a GMA test and an integrity test (composite validity of .65); and the combination of a G M A test and a structured interview .(com posite validity of .63).
Resumes can be VERY misleading.
We just finished interviews last week and a few people who looked perfect on paper were not even considered after the interviews.
I was one of two interviewers and I only threw softballs.
We just finished interviews last week and a few people who looked perfect on paper were not even considered after the interviews.
I was one of two interviewers and I only threw softballs.
But it wouldn't weed out the people who can do well on tests but can't apply their skills to real problems. Part of the interview process is getting a better look at the person behind the resume, and mailing in test scores defeats that.
Also, all the SAT does is see how well you can take a 4-hour test.
Also, all the SAT does is see how well you can take a 4-hour test.
Some people are also really good at essays. It's also like talking to a wall, and it would filter out many effective immigrant workers.
Wait, I'm confused... Are you saying that it can be illegal to have hiring practices that are suboptimal? For example, you mention that brain-teaser tests might have disparate impact on some applicants--and they're also not that effective anyway.
Diaparate impact means disparate racial impact (or disparate against another protected class, but usually racial).
So if you hire a smaller fraction of any racial category of applicants than there were in the pool of applicants or in the workforce of your region, then your hiring method has disparate impact. If you use an objective test that contributes to that disparate impact, you have to be able to prove by thorough studies that it was the best method to pick staff for the specific job you are hiring for. Such studies start around seven figures and won't work for positions with multiple responsibilities or difficult to measure outputs.
So if you hire a smaller fraction of any racial category of applicants than there were in the pool of applicants or in the workforce of your region, then your hiring method has disparate impact. If you use an objective test that contributes to that disparate impact, you have to be able to prove by thorough studies that it was the best method to pick staff for the specific job you are hiring for. Such studies start around seven figures and won't work for positions with multiple responsibilities or difficult to measure outputs.
>So if you hire a smaller fraction of any racial category of applicants than there were in the pool of applicants or in the workforce of your region, then your hiring method has disparate impact.
Is it necessarily because of the hiring method, though? Couldn't it be possible that there is something about the company as a whole that causes people in a protected class to avoid applying to the company? (For example, if the company has racist ads.)
Is it necessarily because of the hiring method, though? Couldn't it be possible that there is something about the company as a whole that causes people in a protected class to avoid applying to the company? (For example, if the company has racist ads.)
I believe you're parsing it wrong. Read "Note that a brain-teaser-type test used in a hiring process might be illegal if it can be shown to have disparate impact on some job applicants and is not supported by a validation study demonstrating that the test is related to successful performance on the job" as
"Note that a brain-teaser-type test used in a hiring process (might be illegal if [both] it can be shown to have disparate impact on some job applicants and [if it] is not supported by a validation study demonstrating that the test is related to successful performance on the job)",
not "Note that a brain-teaser-type test used in a hiring process (might be illegal if it can be shown to have disparate impact on some job applicants) and ([furthermore] is not supported by a validation study demonstrating that the test is related to successful performance on the job.)"
As I read the sentence, a general brain-teaser is supported (by all the research showing that tests of general mental ability are predictive), but it is not legal unless you've done a study specifically for that brain-teaser.
"Note that a brain-teaser-type test used in a hiring process (might be illegal if [both] it can be shown to have disparate impact on some job applicants and [if it] is not supported by a validation study demonstrating that the test is related to successful performance on the job)",
not "Note that a brain-teaser-type test used in a hiring process (might be illegal if it can be shown to have disparate impact on some job applicants) and ([furthermore] is not supported by a validation study demonstrating that the test is related to successful performance on the job.)"
As I read the sentence, a general brain-teaser is supported (by all the research showing that tests of general mental ability are predictive), but it is not legal unless you've done a study specifically for that brain-teaser.
To explain the issue further, tersely, at the risk of being downvoted, in America, the average score among blacks is a standard deviation lower than that of whites, on IQ tests, that is, 15 points. Given that IQ is normally distributed for the most part, using IQ as a factor in hiring will reduce the proportion of blacks who pass the 'bar.'
This gap may diminish if you're looking at a pool of people who passed some previous threshold, eg they have a degree in math. But even then, some of the gap remains, due to the shape of normal distributions. Think of the heights of men and women, and how even if you looked only at people six feet and taller, the proportion of women in that pool who just barely meet the threshold of 6' would be much higher than the similar group of men; despite the cutoff, the average height of the women would still be less than the average height of the men, within that group. Draw overlapping bell curves to see this visually, and analyze the areas under the graph of each distribution, to the right of a given cutoff point.
We do have a law against invidious racial discrimination. The current implementation of the law states that in practice, if some selection mechanism has a passing rate for one minority that is less than 80% of that whites, that is evidence of discrimination in the absence of validation of that mechanism. This is known as the four fifths rule. A test in which 10% of whites and 7% of blacks passed would fall afoul of this law. Say you set a 115 IQ threshold. That is +1 S.D. from the white mean roughly, and +2 S.D. from the black mean, meaning 16% of whites and 2.3% of blacks would pass, if drawing from the general population. That wouldn't even pass a 'One fifth rule,' let alone a four fifths one.
And validation is no sure thing either - police and fire departments have had these tests validated and still have the results thrown out regularly, due to disparate racial impact, i.e. relatively few blacks and Hispanics (average IQ: ~90) passing the test.
On a related note, explicit racial and gender quotas are more efficient than the race- sex- and IQ-blind, approach in use today. A quota would mandate in effect different thresholds for different groups, but it would allow one at least to use this selection mechanism. Within given groups, one would obtain more efficient outcomes. The problem with quotas is that they make racial preferences explicit, while abandoning such selection methods entirely is only an implicit, de facto racial preference, a rather subtle and hidden one at that.
This gap may diminish if you're looking at a pool of people who passed some previous threshold, eg they have a degree in math. But even then, some of the gap remains, due to the shape of normal distributions. Think of the heights of men and women, and how even if you looked only at people six feet and taller, the proportion of women in that pool who just barely meet the threshold of 6' would be much higher than the similar group of men; despite the cutoff, the average height of the women would still be less than the average height of the men, within that group. Draw overlapping bell curves to see this visually, and analyze the areas under the graph of each distribution, to the right of a given cutoff point.
We do have a law against invidious racial discrimination. The current implementation of the law states that in practice, if some selection mechanism has a passing rate for one minority that is less than 80% of that whites, that is evidence of discrimination in the absence of validation of that mechanism. This is known as the four fifths rule. A test in which 10% of whites and 7% of blacks passed would fall afoul of this law. Say you set a 115 IQ threshold. That is +1 S.D. from the white mean roughly, and +2 S.D. from the black mean, meaning 16% of whites and 2.3% of blacks would pass, if drawing from the general population. That wouldn't even pass a 'One fifth rule,' let alone a four fifths one.
And validation is no sure thing either - police and fire departments have had these tests validated and still have the results thrown out regularly, due to disparate racial impact, i.e. relatively few blacks and Hispanics (average IQ: ~90) passing the test.
On a related note, explicit racial and gender quotas are more efficient than the race- sex- and IQ-blind, approach in use today. A quota would mandate in effect different thresholds for different groups, but it would allow one at least to use this selection mechanism. Within given groups, one would obtain more efficient outcomes. The problem with quotas is that they make racial preferences explicit, while abandoning such selection methods entirely is only an implicit, de facto racial preference, a rather subtle and hidden one at that.
There is updated research on the issue of differing distributions of scores between the groups of people characterized as "black" and "white" in the United States. The score gap is narrowing, according to an important review article published this year by the American Psychological Association,
http://www.apa.org/pubs/journals/releases/amp-67-2-130.pdf
and investigating how the gap arose in the first place has been very illuminating about the effect of early childhood environment and formal schooling on the development of IQ. But, yes, the reason the Supreme Court ruled as it did in the 1971 Duke Power case is that there has been a difference in the distribution of IQ scores among "white" and "black" people in the United States, and also an odious legacy of efforts to deny employment opportunity to black people, so if a company has a particular hiring process, it had better make sure that the hiring process will "bear a demonstrable relationship to successful performance of the jobs for which it was used."
See the writings of James R. Flynn, particularly
http://www.apa.org/pubs/journals/releases/rev1082346.pdf
http://www.iapsych.com/iqmr/fe/LinkedDocuments/dickens2006a....
http://www.iapsych.com/iqmr/fe/LinkedDocuments/dickens2006b....
http://www.iapsych.com/iqmr/fe/LinkedDocuments/flynn2007c.pd...
and
http://www.iapsych.com/iqmr/fe/LinkedDocuments/flynn2010a.pd...
for background on the narrowing gap between "race" groups in the United States.
http://www.apa.org/pubs/journals/releases/amp-67-2-130.pdf
and investigating how the gap arose in the first place has been very illuminating about the effect of early childhood environment and formal schooling on the development of IQ. But, yes, the reason the Supreme Court ruled as it did in the 1971 Duke Power case is that there has been a difference in the distribution of IQ scores among "white" and "black" people in the United States, and also an odious legacy of efforts to deny employment opportunity to black people, so if a company has a particular hiring process, it had better make sure that the hiring process will "bear a demonstrable relationship to successful performance of the jobs for which it was used."
See the writings of James R. Flynn, particularly
http://www.apa.org/pubs/journals/releases/rev1082346.pdf
http://www.iapsych.com/iqmr/fe/LinkedDocuments/dickens2006a....
http://www.iapsych.com/iqmr/fe/LinkedDocuments/dickens2006b....
http://www.iapsych.com/iqmr/fe/LinkedDocuments/flynn2007c.pd...
and
http://www.iapsych.com/iqmr/fe/LinkedDocuments/flynn2010a.pd...
for background on the narrowing gap between "race" groups in the United States.
Great explanation! Just one small nitpick: IQ is not "normally distributed for the most part", it's fitted to the normal curve by definition.
How is this a black vs. white issue? There are plenty of smart people of all races who are unemployed.
Like the article stated, it's due to lack of experience in the job they are applying for, coupled with the skills gap. Employees are asking for more experience than applicants have.
Intelligence is not a factor if they are only looking for experience.
Like the article stated, it's due to lack of experience in the job they are applying for, coupled with the skills gap. Employees are asking for more experience than applicants have.
Intelligence is not a factor if they are only looking for experience.
Most companies will put a litany of skill sets on the requirements list, however what they are really looking for is what crusso pointed out, someone with critical thinking skills.
Be it a customer service rep, or a software engineer, when the employer says "Do X" if you don't know how to do it, figure it out. So many people will just sit there and stare into their screen or at a wall. That is the type of person these tests are designed to weed out.
As others pointed out, if you are not willing to put in the effort and jump through the hoops, why bother, either you feel your too good for the job and won't apply yourself, or you lack the basic understanding of problem solving and troubleshooting.
Be it a customer service rep, or a software engineer, when the employer says "Do X" if you don't know how to do it, figure it out. So many people will just sit there and stare into their screen or at a wall. That is the type of person these tests are designed to weed out.
As others pointed out, if you are not willing to put in the effort and jump through the hoops, why bother, either you feel your too good for the job and won't apply yourself, or you lack the basic understanding of problem solving and troubleshooting.
Maybe in theory, but I read a couple of posts on HN in just the last few days of people complaining that potential hires they were interviewing couldn't even rhyme off things like basic data structures.
If critical thinking was the skill being sought, rote memorization seems like it shouldn't have any emphasis at all. You can always look up data structures, or whatever, but that is often not good enough.
If critical thinking was the skill being sought, rote memorization seems like it shouldn't have any emphasis at all. You can always look up data structures, or whatever, but that is often not good enough.
This just smacks of big dumb companies trying to "automate away" HR and not really paying attention to the consequences. And, candidates not figuring out HR systems.
What I fear, however, is that this isn't really a problem for the executives for a public company. Their real compensation these days is all tied to stock prices; I don't see the incentive to really care about HR. How does adding people change the stock price? (Usually I only hear about it from the other way; cutting jobs to cut costs.)
Have you ever heard of investor pressure to hire candidates?
What I fear, however, is that this isn't really a problem for the executives for a public company. Their real compensation these days is all tied to stock prices; I don't see the incentive to really care about HR. How does adding people change the stock price? (Usually I only hear about it from the other way; cutting jobs to cut costs.)
Have you ever heard of investor pressure to hire candidates?
It becomes an issue when there are too many vacancies and revenues begin to fall. Most c-level executives don't have their comp tied to stock prices, but to revenue growth (which is as bad as having it tied stock prices.) That being said, this definitely a heavy case of big & dumb.
Every C-level exec with share-based (RSU or option) compensation quite clearly has their comp tied to share prices. I doubt very many C-level execs at public companies have no share-based comp.
While I agree that this indeed sounds like a problem, and a genuine one at that, for the HN community this should also sound like an opportunity, to create a resume sorting systems that is much much better than the ones out there.
Are there startups already looking at this problem? If not, why not? And, who's up for a challenge?
Are there startups already looking at this problem? If not, why not? And, who's up for a challenge?
The problem is not the software. The problem are the employers. Everyone wants the perfect candidate. But no candidate is perfect. Everyone has good skills and bad baggage to carry from job to job. The aim should be to find people who will add to the team rather than building a perfect team. You can't do that with software (yet).
Well you're agreeing with me when you say "yet", and hence my comment about opportunity.
The problem is the software and the software is not going to go away. The opportunity is to make the software more intelligent so that the output is more in line with what would work.
The problem is the software and the software is not going to go away. The opportunity is to make the software more intelligent so that the output is more in line with what would work.
Oh, I'm not disagreeing on the opportunity available. You are, to quote Hermes from Futurama, technically correct (the best kind).
I do think that this problem is up there with world peace.
I do think that this problem is up there with world peace.
Technology, as wonderful as it is, is not going to change employers, or rather, certain ideas often found in employers.
Why haven't DICE or Indeed.com or whoever seem to have done any work on that front? I'm thinking the ROI isn't there.
Considering that it costs 20% for a match by a recruitment company, there is enourmous ROI for a program like that. It would almost be as good as the ability to print money.
Not to mention the cost you could get for the unemployed to sign up ("It costs you on average 2k/month not to use our service").
Not to mention the cost you could get for the unemployed to sign up ("It costs you on average 2k/month not to use our service").
I think this could probably be handled using crowd sourcing approach within a company. The problem that I see with typical job postings is the tendency for people to over-tailor resumés (as is mentioned in the article).
I mentioned it in another comment already, but this problem is what I'm tackling with They Meet You (http://theymeetyou.com). At a minimum, I'll be using it where I work, but check it out if you want.
How it works is really simple: You can send a message to anyone that proves they work for a company, if you interest them then they can share their contact information with you.
I mentioned it in another comment already, but this problem is what I'm tackling with They Meet You (http://theymeetyou.com). At a minimum, I'll be using it where I work, but check it out if you want.
How it works is really simple: You can send a message to anyone that proves they work for a company, if you interest them then they can share their contact information with you.
Doomed to fail. If the company doesn't hire the person, that person considers me responsible. That opens me to potential legal liability. Why expose your employees to that risk?
The biggest problem I see in hiring people is that good resumes get filtered out by HR departments because the resumes don't have the precise keywords. Fix that by hiring smarter HR people.
The biggest problem I see in hiring people is that good resumes get filtered out by HR departments because the resumes don't have the precise keywords. Fix that by hiring smarter HR people.
I think the challenge is the opposite. When resumes are read exclusively by machines, they become protocol. We need a compiler that takes your skills and the job listing as input and produces output that satisfies the filter.
mojoLive - count everything
http://mojolive.com
The thing is, it's not a resume-sorting problem at all. Most people aren't even qualified to be who they are.
In general, people aren't qualified to do their jobs. That doesn't mean they shouldn't be doing them, or be hired to do them. It means the qualification system is broken.
In general, people aren't qualified to do their jobs. That doesn't mean they shouldn't be doing them, or be hired to do them. It means the qualification system is broken.
I think it is strange to rely on HR and recruiters as a filter. I would prefer someone to introduce themselves to the entire company: tell me what you can do and what you want to do, send me a reference to any relevant portfolio, and I'll take it from there. Hopefully this approach will discourage overly tailored cover letters, who knows. The article talks about this specifically, "the trick is parroting all the words in the job description but not just copying and pasting the text"
This is pretty much why I started building They Meet You (theymeetyou.com) during my spare time. The way the site works right now is pretty simple. You can write a message that is meant for anyone at a company (@customink.com), anyone that proves they have an email address there can read it and exchange their contact info.
This is pretty much why I started building They Meet You (theymeetyou.com) during my spare time. The way the site works right now is pretty simple. You can write a message that is meant for anyone at a company (@customink.com), anyone that proves they have an email address there can read it and exchange their contact info.
While I think the problems discussed in the article need some resolution there is another issue that they do not discuss and which you allude to. Too many people are leading with their resumes and, more or less, blindly applying to jobs. Getting to know people in the industry you want to work in and the companies you would like to work at is a significantly better approach.
One thing I hope to be able to do in my career is, as I come into points where I am hiring, I can look at everything in my resume queue without having some software or some unqualified individual tell me who is qualified and who isn't.
I hope to be able to spread a wide net to find the best... not a fine net to find a unicorn.
I hope to be able to spread a wide net to find the best... not a fine net to find a unicorn.
Software that discards applicants based on simple keyword mismatch doesn't "raise the bar."
[deleted]
I have $100 here that says most won't (if any).
* If we can't hire people due to a broken system, how do we expect to succeed?