Why Software Maker Fog Creek Is Helping Its Competitors Hire Women(fastcompany.com)
fastcompany.com
Why Software Maker Fog Creek Is Helping Its Competitors Hire Women
http://www.fastcompany.com/3043082/most-creative-people/why-software-maker-fog-creek-is-helping-its-competitors-hire-women
15 コメント
"And it may be that it intimidates more women than men."
I've been looking around for a developer recently, and in looking at the profiles of women vs. men at places like AngelList I noticed a few key differences in how they described themselves. We're talking about people with very similar academic backgrounds here with similar levels of experience.
Women would say things like "I worked on a team that did X," while men -- who obviously did the same kind of work -- would simply say "I did X." Women would highlight the cooperative nature of their work, while men would give the impression that they did it all themselves. In my experience the former is typically more honest, especially if you actually were working on a team or on a team-oriented college project.
Women also tend to highlight team-oriented and non-technical skills more than men in general, which to techie-minded hacker types gives the impression that they are not strong on the technical end.
I also noted my reaction to these differences. Introspecting, I noticed that I tended to assume that "being on a team" meant riding the coattails of the other team members. If I noted this bias in myself, it's probably pretty common.
I decided to try to de-program myself of this bias by thinking of all the counter-examples I've seen.
I do believe in the "10X programmer" -- for the simple reason that there are 10X everything elses. There are 10X athletes, 10X businesspeople/salespeople, 10X artists, 10X anythings. When we look for employees or contractors we are looking for that 10X standout, or at least someone who's going to grow into one. But being really good at something is not mutually exclusive with working on a team or highlighting non-technical skills, and the inverse is also not correlated. I've met plenty of monomaniacal "closet programmers" who really aren't any good. They sit alone stewing in their huge egos and churn out crap. (But then again I've met some who are. There does not seem to be a strong correlation.)
There's probably quite a few really talented female programmers out there who are being passed over because they don't conform to the "whip it out and let me measure it" customs of technical recruitment. I also suspect that this does drive women from the field.
I've been looking around for a developer recently, and in looking at the profiles of women vs. men at places like AngelList I noticed a few key differences in how they described themselves. We're talking about people with very similar academic backgrounds here with similar levels of experience.
Women would say things like "I worked on a team that did X," while men -- who obviously did the same kind of work -- would simply say "I did X." Women would highlight the cooperative nature of their work, while men would give the impression that they did it all themselves. In my experience the former is typically more honest, especially if you actually were working on a team or on a team-oriented college project.
Women also tend to highlight team-oriented and non-technical skills more than men in general, which to techie-minded hacker types gives the impression that they are not strong on the technical end.
I also noted my reaction to these differences. Introspecting, I noticed that I tended to assume that "being on a team" meant riding the coattails of the other team members. If I noted this bias in myself, it's probably pretty common.
I decided to try to de-program myself of this bias by thinking of all the counter-examples I've seen.
I do believe in the "10X programmer" -- for the simple reason that there are 10X everything elses. There are 10X athletes, 10X businesspeople/salespeople, 10X artists, 10X anythings. When we look for employees or contractors we are looking for that 10X standout, or at least someone who's going to grow into one. But being really good at something is not mutually exclusive with working on a team or highlighting non-technical skills, and the inverse is also not correlated. I've met plenty of monomaniacal "closet programmers" who really aren't any good. They sit alone stewing in their huge egos and churn out crap. (But then again I've met some who are. There does not seem to be a strong correlation.)
There's probably quite a few really talented female programmers out there who are being passed over because they don't conform to the "whip it out and let me measure it" customs of technical recruitment. I also suspect that this does drive women from the field.
> " Women would highlight the cooperative nature of their work, while men would highlight that they did it all themselves. In my experience the latter is typically more honest, especially if you actually were working on a team or on a team-oriented college project.
Did you mean to say the former was more honest, i.e. that saying “we did X” is more honest than “I did X?"
Did you mean to say the former was more honest, i.e. that saying “we did X” is more honest than “I did X?"
Yes. Edited.
It depends on the context of course. If it was your project alone, then you did it. If others contributed non-trivial amounts, it was a team effort. I noted that the males tended to bury that, while the females tended to highlight it.
It depends on the context of course. If it was your project alone, then you did it. If others contributed non-trivial amounts, it was a team effort. I noted that the males tended to bury that, while the females tended to highlight it.
I really want to hear what the individual contributed, though, and actually couldn't care less what their team did as a whole. I've heard way too many "we"s that were actually "they"s, while the subject was present, but not actually contributing that much. It's really easy for a bad developer to hide in a large team.
EDIT: and it's also easy for a good developer to get quagmired in a bad team! Just because the project as a whole didn't do much doesn't mean the individual is bad. So I still really only want to hear about the individual efforts.
EDIT: and it's also easy for a good developer to get quagmired in a bad team! Just because the project as a whole didn't do much doesn't mean the individual is bad. So I still really only want to hear about the individual efforts.
You're missing one final problem: it's easy for a bad developer to talk themselves up.
This would be why I spend almost no effort on the interview process. I put people to work right away to force them to prove whether or not they can do the job.
Well, I suppose they could be subcontracting it out, but at that point, I don't care. The work is getting done and at a price I can manage.
Well, I suppose they could be subcontracting it out, but at that point, I don't care. The work is getting done and at a price I can manage.
Well I think the more honest part is I myself worked on part X and we the team created product Y made out these multiple parts. How big of a part you had does matter.
For example, I made the iOS UI part of this text chat app, and my team members created data models and networking code that I used to make the UI part. Another team member made the android part using the cross platform code and a bunch of others made the server part and so on. Together we made a text chat app pair.
For example, I made the iOS UI part of this text chat app, and my team members created data models and networking code that I used to make the UI part. Another team member made the android part using the cross platform code and a bunch of others made the server part and so on. Together we made a text chat app pair.
> Women would say things like "I worked on a team that did X," while men -- who obviously did the same kind of work -- would simply say "I did X.
I find this a very interesting fact, but I see a number of other possible explanations for "I do X": - It is simply shorter. Men being used to more compact communication? - It is also more boastful - perhaps men being used to market themselves more aggressively than women? Or, put in another words, having and/or presenting bigger egos in their CVs? - Women being insecure about their personal contribution in X? Maybe the coattail-riding assumption is not always wrong.
In any case, I would withhold judgement before an interview.
I find this a very interesting fact, but I see a number of other possible explanations for "I do X": - It is simply shorter. Men being used to more compact communication? - It is also more boastful - perhaps men being used to market themselves more aggressively than women? Or, put in another words, having and/or presenting bigger egos in their CVs? - Women being insecure about their personal contribution in X? Maybe the coattail-riding assumption is not always wrong.
In any case, I would withhold judgement before an interview.
There's lots of possibilities. It's really hard to tell whether any of these things are cultural or come from actual gender dimorphism (to use the technical biological term). Ultimately it doesn't matter if they are irrelevant to ability.
What I'm saying is that when we are evaluating peoples' abilities, we should take care to evaluate actual ability rather than unreliable proxies for ability. The latter are prejudices, basically, and they blind us to a lot of talent that might not fit those lazy filters.
I'm neither a "PC liberal" nor a conservative/reactionary on this issue. Liberals pretend differences in ability don't exist, while conservatives like to defend the accuracy of our "gut" prejudices and argue there's nothing wrong. IMHO equality of opportunity comes when people make an effort to look at actual ability as objectively as possible.
It's in your best interest as an employer to do this. Good people are very hard to find, and if you're running some kind of dumb ape-brain filter that excludes ~50% of the population from consideration you are handicapping your recruitment efforts pretty badly. Think like a human being.
What I'm saying is that when we are evaluating peoples' abilities, we should take care to evaluate actual ability rather than unreliable proxies for ability. The latter are prejudices, basically, and they blind us to a lot of talent that might not fit those lazy filters.
I'm neither a "PC liberal" nor a conservative/reactionary on this issue. Liberals pretend differences in ability don't exist, while conservatives like to defend the accuracy of our "gut" prejudices and argue there's nothing wrong. IMHO equality of opportunity comes when people make an effort to look at actual ability as objectively as possible.
It's in your best interest as an employer to do this. Good people are very hard to find, and if you're running some kind of dumb ape-brain filter that excludes ~50% of the population from consideration you are handicapping your recruitment efforts pretty badly. Think like a human being.
I (male, for the record, but gender-specific behavior isn't my point here) used to write/say that I was "part of a team" that did X or "worked on a team" that did Y, but I stopped doing it for a couple reasons, the most prominent being that "worked on a team that..." is usually obvious given the context of the conversation and the scale of the work discussed. In those cases, explicitly mentioning it feels like you're going out of the way to distance yourself from the work and describe yourself as a bystander - someone who just happened to be on the team that did this great work.
Personally, I hate phrases like "I worked on a team that did X" because I can't tell what you did. Where you the lead, a junior, or did you get called in when everything exploded and saved the day?
I'd rather see "I did the Y on the team that did X" as it's more descriptive and leaves less to my imagination.
I'd rather see "I did the Y on the team that did X" as it's more descriptive and leaves less to my imagination.
I worked with a 10x developer once, he was awful and I couldn't stand to work with him. Although he could produce an insane amount of code and functionality his presence was like throwing a grenade into the middle of the dev team everyday. He left to take a job at google and the overall team productivity went up significantly. He basically did 10X individually,but took 11x away from the team. So team work is pretty darn important.
I think that wasn't a 10x programmer, then.
I've worked with 10x programmers, and while certainly a lot of their value is simply their ability to write code, they also have a lot of value because they help other people excel. The 10x programmers I've worked with are people who push for code reviews, better tools, proper testing, realistic timelines, good communication with clients, etc.
And in the long run, those programmers made the programmers around them better. A 10x programmer creates a lot of 2x-5x programmers. I'm maybe a 3x programmer and it's because a lot of 10x programmers have taken the time to help me get better.
I've worked with 10x programmers, and while certainly a lot of their value is simply their ability to write code, they also have a lot of value because they help other people excel. The 10x programmers I've worked with are people who push for code reviews, better tools, proper testing, realistic timelines, good communication with clients, etc.
And in the long run, those programmers made the programmers around them better. A 10x programmer creates a lot of 2x-5x programmers. I'm maybe a 3x programmer and it's because a lot of 10x programmers have taken the time to help me get better.
Here's my favorite quote by @bmdhacks that reflects my experience dealing with such people:
How to be a 10x engineer: Incur technical debt fast enough to appear 10x as productive as the ten engineers tasked with cleaning it up.
https://twitter.com/bmdhacks/status/560949130999365633
How to be a 10x engineer: Incur technical debt fast enough to appear 10x as productive as the ten engineers tasked with cleaning it up.
https://twitter.com/bmdhacks/status/560949130999365633
Spot on!
> How to be a 10x engineer: Incur technical debt fast enough to appear 10x as productive as the ten engineers tasked with cleaning it up.
We have a guy in our onsite team that keeps making our user facing application suck with his changes one day and then fix those every other day. So lots of git commits but limited productivity in the team, and a very tough maintainence task for every developer in the team.
> How to be a 10x engineer: Incur technical debt fast enough to appear 10x as productive as the ten engineers tasked with cleaning it up.
We have a guy in our onsite team that keeps making our user facing application suck with his changes one day and then fix those every other day. So lots of git commits but limited productivity in the team, and a very tough maintainence task for every developer in the team.
I think you're misunderstanding what "10x developer" is supposed to mean. The impression I've gotten is that they are considered to be 10 times as productive because of the work they don't do. They write code that has less bugs, is more scalable, requires less refactoring, is easier for other developers to understand, etc. You'd never see it in LOC or sprint points completed, but there'd be a noticeable difference of pace between a project of 10x devs and a project with 1x devs.
I wanted to reply to all the comments so I just decided to reply to my own comment. For those telling me I'm misinterpreting what a 10x dev is, I don't think a solid definition of the 10x developer actually exists it's mostly in the eye of the beholder.
I think it's mostly in what other people think of them especially that guy that hands out raises. So it's incredibly subjective.
I say a 10x developer is someone that can create features 10x faster than everyone else, regardless of their other skills.
To be clear my dream rockstart software engineer is not a 10x developer. They are 5x developer, a 5x Team player, 5x architect and 10x Mentor. So maybe we need a dictionary of terms around what people by various nicknames and concepts given to the various kinds of software developers.
I think it's mostly in what other people think of them especially that guy that hands out raises. So it's incredibly subjective.
I say a 10x developer is someone that can create features 10x faster than everyone else, regardless of their other skills.
To be clear my dream rockstart software engineer is not a 10x developer. They are 5x developer, a 5x Team player, 5x architect and 10x Mentor. So maybe we need a dictionary of terms around what people by various nicknames and concepts given to the various kinds of software developers.
I agree everyone has their own definition, but the one you are giving is a pointy-haired definition, where the only metric is LoC and a successful demo of some feature.
Even though it's hard to measure a lot of things like technical debt, architectural benefits, bugs, edge case handling, UX polish, teamwork, etc, it doesn't mean that you can reasonably get away with discounting those effects when you talk about 10x. The 10x is referring to productivity, which is implicitly a net measure (gross productivity is just activity, and no one willingly pays for that).
So yeah, you can be 10x in terms of cranking out working code a lot faster (rare), but if that is blowing other people up then it's not meaningful to say they are more productive, in fact it's extremely harmful to risk putting them on a pedestal like this in front of clueless management. For me, true 10x is more intangible than cranking out functionality, it's more about the ability to reconcile the details with the big picture, and zeroing in on solutions that solve the most problems and create the fewest. How this actually looks in practice varies from project to project, but I think it rarely correlates with sustaining a 200wpm typing speed for 8 hours a day.
Even though it's hard to measure a lot of things like technical debt, architectural benefits, bugs, edge case handling, UX polish, teamwork, etc, it doesn't mean that you can reasonably get away with discounting those effects when you talk about 10x. The 10x is referring to productivity, which is implicitly a net measure (gross productivity is just activity, and no one willingly pays for that).
So yeah, you can be 10x in terms of cranking out working code a lot faster (rare), but if that is blowing other people up then it's not meaningful to say they are more productive, in fact it's extremely harmful to risk putting them on a pedestal like this in front of clueless management. For me, true 10x is more intangible than cranking out functionality, it's more about the ability to reconcile the details with the big picture, and zeroing in on solutions that solve the most problems and create the fewest. How this actually looks in practice varies from project to project, but I think it rarely correlates with sustaining a 200wpm typing speed for 8 hours a day.
I'm going to quote this on my blog, I love everythin about this comment.
Wow thanks, let me know where your blog is.
impressmyself.co , It probably won't be up until next week sometime though. I have to figure how I want to write the blog post, and then actually do it.
Those exist too. It's possible to be really good at something but to be so awful at everything else that the net effect is zero to negative. But not all "10X developers" are assholes.
I read a book about the development of writing in mesopotamia (http://www.amazon.com/gp/product/0674049683 , but I can't really recommend it as a general-interest book); it mentioned that a particular king had seen his city flourish under his rule and boasted accordingly, in the records we have, of how irrigation canals were dug and roads were built and generally a lot of great stuff happened on his watch.
In a parenthetical, it also noted that, as is typical for the records of the time, they actually attributed all these works to the king personally ("I dug canals in XX regions").
In a parenthetical, it also noted that, as is typical for the records of the time, they actually attributed all these works to the king personally ("I dug canals in XX regions").
That may be a men/women thing, but I actually screen for that when I'm interviewing and significantly prefer to hire people that say "we did X" rather than "I did X". It's my job as an interviewer to figure out which person on that team you were, but if you describe your work as I I I I I and never We We We We, it is pretty telling.
It also goes for managing. If my team fucked something up, I say "I fucked up Y", even if it was a report of mine. If my team did something great, it's a "We did Z" even if I did it alone.
When you put it out that plainly, it sounds hokey and stupid. But trust me, over the long run those tiny things are the difference between a good company culture and a bad company culture.
It also goes for managing. If my team fucked something up, I say "I fucked up Y", even if it was a report of mine. If my team did something great, it's a "We did Z" even if I did it alone.
When you put it out that plainly, it sounds hokey and stupid. But trust me, over the long run those tiny things are the difference between a good company culture and a bad company culture.
It's interesting that you say your bias is toward perceiving someone self-reporting as working on a team as "everyone on the team did the work but me".
Does this result from primarily working alone, or have you just been granted license to be more autonomous than most coders I know? A lot of shops both employ pair programming and have a series of pairs working on one project, or have a t the very least a bunch of people working on different aspects of the project.
Of course this could be spun as "I did y for my project, x" but I'm curious as to why "As part of the team working on x project, I did y" sounds less like accomplishing something technical to you.
Say I had a project out of which I'm submitting 3-4 conference papers to various conferences for publication, working with my advisor and another student. Would it be better to say I did it all, or to be honest and say the other student did 1/3, I did 2/3, and my advisor helped shape our vision?
Does this result from primarily working alone, or have you just been granted license to be more autonomous than most coders I know? A lot of shops both employ pair programming and have a series of pairs working on one project, or have a t the very least a bunch of people working on different aspects of the project.
Of course this could be spun as "I did y for my project, x" but I'm curious as to why "As part of the team working on x project, I did y" sounds less like accomplishing something technical to you.
Say I had a project out of which I'm submitting 3-4 conference papers to various conferences for publication, working with my advisor and another student. Would it be better to say I did it all, or to be honest and say the other student did 1/3, I did 2/3, and my advisor helped shape our vision?
I felt the same as 'api' when he explained that and to me, it came from my bad experiences working on teams, in high school, where 1 or 2 people did all the work and the rest rode the coattails to an easy A.
Of course my experiences in high school shouldn't factor into my judgement of reading judging peoples capabilities (because chances are everyone worked on a team that did X). Something like that might just be a hidden cultural bias - I'm sure everyone has an experience where they forced to worked with others who didn't feel like pulling their weight - and its those negative experiences that shape our notions of what it means to work on a team, rather than the multitude of positive experiences we may have had.
Of course my experiences in high school shouldn't factor into my judgement of reading judging peoples capabilities (because chances are everyone worked on a team that did X). Something like that might just be a hidden cultural bias - I'm sure everyone has an experience where they forced to worked with others who didn't feel like pulling their weight - and its those negative experiences that shape our notions of what it means to work on a team, rather than the multitude of positive experiences we may have had.
I have often worked alone, but not always. I've never worked in a pair programming shop, and I would probably find it distracting for anything but debugging or syncing up. The trouble is that complex programming tasks usually have the nature where communicating the ideas takes longer than simply coding them. You'd spend all your time syncing state between you and the other person.
But I don't think this explains the bias. I'm not really sure where it comes from but I found it interesting.
But I don't think this explains the bias. I'm not really sure where it comes from but I found it interesting.
> Women would say things like "I worked on a team that did X," while men -- who obviously did the same kind of work -- would simply say "I did X."
Depending on what X is, and on context, it can be obvious that "worked on X" simply means that they worked on X, but that doesn't mean that others did not also work on it, with them. Especially if "X" is too big and complex for one person to do on ones own. In particular, researchers in CS (maybe coincidentally majority of men) tend to say that they "work on" or "are interested in" X, Y, Z - but that doesn't really mean that they don't collaborate or work with other people on those things. It's not exclusionary at all.
But this interpretation doesn't fit with the usual men-are-egotistical-narcissists-compared-to-women. So pay this comment no mind.
Depending on what X is, and on context, it can be obvious that "worked on X" simply means that they worked on X, but that doesn't mean that others did not also work on it, with them. Especially if "X" is too big and complex for one person to do on ones own. In particular, researchers in CS (maybe coincidentally majority of men) tend to say that they "work on" or "are interested in" X, Y, Z - but that doesn't really mean that they don't collaborate or work with other people on those things. It's not exclusionary at all.
But this interpretation doesn't fit with the usual men-are-egotistical-narcissists-compared-to-women. So pay this comment no mind.
"But this interpretation doesn't fit with the usual men-are-egotistical-narcissists-compared-to-women. So pay this comment no mind."
I'm just reporting the patterns I saw. That's all.
I wouldn't say "egotistical narcissist," just that the culture probably holds out different behavioral expectations for men and women and that gives rise to differences in how people present themselves.
I'm just reporting the patterns I saw. That's all.
I wouldn't say "egotistical narcissist," just that the culture probably holds out different behavioral expectations for men and women and that gives rise to differences in how people present themselves.
> I'm just reporting the patterns I saw. That's all.
Well let's not kid ourselves. :) If a pattern you happened to observe reflected poorly on women - like if you observed that many more of them were incompetent than men[1] - you wouldn't report on it here. Or else you would just get downvoted or ignored. Can you honestly object to that?
Casually reporting on something which shows women as a group (or subgroup, like in tech) is viewed at best as being in poor taste, or at worst as sexism or misogyny. And there are good arguments for calling it that. But do the same for men and it's just innocuous "observations".
[1] For the sake of the argument: totally hypothetical.
Well let's not kid ourselves. :) If a pattern you happened to observe reflected poorly on women - like if you observed that many more of them were incompetent than men[1] - you wouldn't report on it here. Or else you would just get downvoted or ignored. Can you honestly object to that?
Casually reporting on something which shows women as a group (or subgroup, like in tech) is viewed at best as being in poor taste, or at worst as sexism or misogyny. And there are good arguments for calling it that. But do the same for men and it's just innocuous "observations".
[1] For the sake of the argument: totally hypothetical.
I'm not an athlete, but I'll run a mile against anyone who claims to be 10X faster than me.
"Women would say things like "I worked on a team that did X," while men -- who obviously did the same kind of work -- would simply say "I did X." Women would highlight the cooperative nature of their work, while men would give the impression that they did it all themselves."
Cannot emphasise this quote enough. It is the difference between, "We" and "I", the basis of any community where you put yourself second. It is also the essence of good leadership.
If you want a lone wolf programmer as a new-hire in a small startup, maybe this type isn't for you. As your company grows, the type of programmer described by @api above, the one who works in teams where your livelyhood and codebase is on the line, is the person I'd want to work with and for.
Cannot emphasise this quote enough. It is the difference between, "We" and "I", the basis of any community where you put yourself second. It is also the essence of good leadership.
If you want a lone wolf programmer as a new-hire in a small startup, maybe this type isn't for you. As your company grows, the type of programmer described by @api above, the one who works in teams where your livelyhood and codebase is on the line, is the person I'd want to work with and for.
> Women would say things like "I worked on a team that did X," while men -- who obviously did the same kind of work -- would simply say "I did X."
While this anecdote seems to match traditional gender narratives, I'm interested in how you knew the men "obviously did the same kind of work."
You observed that women were more likely than men to emphasize teamwork and not their individual contributions. It seems that there are two hypotheses which result from that:
1. Women are more likely to emphasize team aspects and de-emphasize their contribution (less narcissistic)
2. Women are less likely to individual contribute and more likely to rely on teams for projects (less independent)
How did you differentiate between those two?
While this anecdote seems to match traditional gender narratives, I'm interested in how you knew the men "obviously did the same kind of work."
You observed that women were more likely than men to emphasize teamwork and not their individual contributions. It seems that there are two hypotheses which result from that:
1. Women are more likely to emphasize team aspects and de-emphasize their contribution (less narcissistic)
2. Women are less likely to individual contribute and more likely to rely on teams for projects (less independent)
How did you differentiate between those two?
I think programming challenges are the only recruitment tool we have that can cut through this and many other similar questions.
We (www.gapjumpers.me) are trying to solve the gender diversity issue in tech by using blind auditions as a first filter. Our data shows that blind auditions result in greater diversity numbers.
By using a technical challenge as the first step and fast forwarding people to an advanced interview stage, we're eliminating candidate reluctance and increasing transparency (the candidate interacts directly with the hiring manager through our platform).
In our experience, telling the candidate "Show us your skills, impress us, and you'll get an interview" is better than "Tune up your resume to pass every possible filter". Showing your skills early on also removes a lot of stress from the candidate and gives you something solid to talk about during the interview.
Edit:Our homepage shows some numbers and case studies are available on request: http://www.gapjumpers.me/#genderResults
By using a technical challenge as the first step and fast forwarding people to an advanced interview stage, we're eliminating candidate reluctance and increasing transparency (the candidate interacts directly with the hiring manager through our platform).
In our experience, telling the candidate "Show us your skills, impress us, and you'll get an interview" is better than "Tune up your resume to pass every possible filter". Showing your skills early on also removes a lot of stress from the candidate and gives you something solid to talk about during the interview.
Edit:Our homepage shows some numbers and case studies are available on request: http://www.gapjumpers.me/#genderResults
Do you have any numbers you can share? Do your results look significantly different from the background population?
Hi Kalium,
for numbers, please have a look here at what our data is showing to date:
www.gapjumpers.me/#genderResults
for numbers, please have a look here at what our data is showing to date:
www.gapjumpers.me/#genderResults
Uhm. I was hoping for something much more comprehensible and comprehensive than a series of blurbs and unlabeled pie charts. A statistical comparison of your cohort compared to the background population and relative performance is what interests me.
I want to know how your program does compared to a hypothetical ideal gender-bias-free hiring system. If what you present there is truly all that your data shows, I have some grave concerns about your company.
I want to know how your program does compared to a hypothetical ideal gender-bias-free hiring system. If what you present there is truly all that your data shows, I have some grave concerns about your company.
I wonder if you could also make the matching part of the process blind as well? That is, the article mentions that women might be less likely to apply to places like Fog Creek - but if you matched applicants to employers blindly both ways (blindly to them, of course, you the matcher would not be blind), so the applicants didn't know who they were interviewing for during the interview? Then developers with lower self-esteem but great skills might end up interviewing for high-end places, and not even be stressed during the interview, because they don't know who it's with.
I'd find this very interesting myself, although I'd have concerns about potential employers in some ways. As long as I was able to provide a list of employers I DON'T want to work for up front and have those auto-filtered from my blind interviews I'd be totally open to trying out this process through a recruitment firm. I personally tend to pick my employers broadly on three questions: 1) Are they doing something that's actually innovative? 2) Do they behave ethically in the marketplace towards their employers, vendors, and customers? 3) Is the position located somewhere that I'd be okay with living?
If the answer to any of those is "no", I won't even bother applying/interviewing and I automatically turn down recruiters from that company. #2 in particular is a big one, there are way too many businesses in the world from startups to big corps that behave unethically, and I refuse to be a part of it.
Being able to work with a recruiting platform where I can detail these qualifications up-front and trust that they'll be respected would give me the piece of mind to put myself in a situation where I'd be matched to an employer based on their needs and my qualifications without concern that I'd be getting screwed somehow.
If the answer to any of those is "no", I won't even bother applying/interviewing and I automatically turn down recruiters from that company. #2 in particular is a big one, there are way too many businesses in the world from startups to big corps that behave unethically, and I refuse to be a part of it.
Being able to work with a recruiting platform where I can detail these qualifications up-front and trust that they'll be respected would give me the piece of mind to put myself in a situation where I'd be matched to an employer based on their needs and my qualifications without concern that I'd be getting screwed somehow.
This is an interesting approach. We will probably try it in the future but at the moment feedback says that candidates almost always want more information (more about the company, more about the people they'd be working with, more about the kind of work they'd be doing). But I definitely agree on how it might take the stress out of the process.
The only issue is that hiring and getting hired requires significant commitment on both ends and long term anonymity might not work really well under those conditions.
The only issue is that hiring and getting hired requires significant commitment on both ends and long term anonymity might not work really well under those conditions.
As long as I'd be able to submit a list of companies that I did not want to work with ahead of time.
Interesting that you mention gender diversity but there's no mention about racial diversity.
Our profile capture does not require applicants to include their demographic information, hence the only way we've been able to easily split by gender is by identifying their profile images and names.
Of course, outcomes have shown that racial diversity is also helped along. Our hypothesis is that if you work to remove bias through process there should be a measurable (even if it's small) change in diversity metrics.
Of course, outcomes have shown that racial diversity is also helped along. Our hypothesis is that if you work to remove bias through process there should be a measurable (even if it's small) change in diversity metrics.
It doesn't draw the connection directly, but part of the training program deals with technical interviews and getting the fellows more practice, and hopefully more comfort, with them. My guess is that once people have been through some mock interviews, the intimidation factor goes down quite a bit.
I'm glad the "rigorous testing process" was mentioned. The facade of "we only hire superninja-kickass-hyperexperts and we will verify that you are such a programmer before we hire you" attracts a certain type of person, and intimidates another.
Yup. And I've always suspect that this cant was ultimately something of a foil, and that the actual messages it's intended to send are of a rather different sort. To wit:
Internal: Hoo-ray for you! You made it through the gauntlet! Aren't you just swell and awesome!
External: You know you have to invest in us / buy our products because we only hire the top 0.001 percent! And we have a rigorously testing process (as verified by the magic of survivorship bias) to prove it!
Yup. And I've always suspect that this cant was ultimately something of a foil, and that the actual messages it's intended to send are of a rather different sort. To wit:
Internal: Hoo-ray for you! You made it through the gauntlet! Aren't you just swell and awesome!
External: You know you have to invest in us / buy our products because we only hire the top 0.001 percent! And we have a rigorously testing process (as verified by the magic of survivorship bias) to prove it!
[deleted]
>attracts a certain type of person, and intimidates another. And it may be that it intimidates more women than men.
This is classic horseshoe theory, where extremists on both ends of a spectrum actually wrap around and start to sound the same. In this case they both believe that women are delicate flowers who can't handle the pressure, stress, and competition of the working world like men can.
This is classic horseshoe theory, where extremists on both ends of a spectrum actually wrap around and start to sound the same. In this case they both believe that women are delicate flowers who can't handle the pressure, stress, and competition of the working world like men can.
There's a difference between "lower self-confidence" and "can't handle pressure, stress, and competition". It's also possible that even women who have high self confidence in their abilities are less confident that they will be judged positively by others.
There's plenty of research to the effect that men will apply to a job if they meet 40% of the listed requirements, whereas women won't until they meet 90%. And when you don't meet someone's default expectations of what kind of person a "software developer" (or a "nurse" or a "CEO") is, you often have to work harder to prove that you are one.
But also, why does the working world have to be competitive rather than collaborative?
There's plenty of research to the effect that men will apply to a job if they meet 40% of the listed requirements, whereas women won't until they meet 90%. And when you don't meet someone's default expectations of what kind of person a "software developer" (or a "nurse" or a "CEO") is, you often have to work harder to prove that you are one.
But also, why does the working world have to be competitive rather than collaborative?
I'm not saying that working world has to be competitive rather than collaborative.
The OP is talking situations that are presented as being competitive.
The OP is talking situations that are presented as being competitive.
To me at least, what's intimidating about applying for a job described as Fog Creek's are isn't that they're competitive -- they don't say "you'll be in a knock-down drag-out competition to beat 100 other candidates". It's that the criteria for acceptance are high -- only those who have an extremely high opinion of their own abilities will consider themselves likely not only to meet those criteria but to be judged by others as meeting those criteria. And many people who are very capable are unlikely to judge themselves so favorably. Women in particular are socialized to underrate our own abilities and conditioned to expect our abilities to be underrated by others, especially in STEM fields.
[deleted]
Several people have already mentioned it here, but I'm also very pleased to read this line:
"Hall asked a simple question: Why didn't Chipps, a brilliant engineer, ever apply to work at Fog Creek? Chipps said she didn't think she would make it through their rigorous testing process."
I do think that the grueling and intimidating software interview process is a very serious problem in our field. This one sentence hints at the extent of the harm.
"Hall asked a simple question: Why didn't Chipps, a brilliant engineer, ever apply to work at Fog Creek? Chipps said she didn't think she would make it through their rigorous testing process."
I do think that the grueling and intimidating software interview process is a very serious problem in our field. This one sentence hints at the extent of the harm.
"I do think that the grueling and intimidating software interview process is a very serious problem in our field."
I'd have to agree. The interview process is irreducibly stressful, but that doesn't mean we have to go out of our way to make it worse. It isn't even good for hiring most men either, after all! It's just a bad idea.
I've personally had fairly good success hiring people who happen to be women, and I suspect a good part of that is that my interview attitude is that I'm there to find out what you can do, not to find out what you can't, which seems to be the unexamined default of a lot of interviewers. You don't start at "perfect" and get marks off for failure... you start at essentially zero and work your way up. "Starting at zero" may sound superficially harsh when I put it into English, but in practice this produces a much friendlier interview environment, I think, because we're discovering positive aspects instead of negative ones.
(Another reason to prefer this is that in practice this also helps avoid the problem of the senior candidate being thrown softballs, or the junior candidate being asked to design a cloud infrastructure. It really doesn't take that long for me to key in on where your most likely "knowledge frontier" is and probe for details there. As opposed to just going through a rote set of questions looking for demerits. It is a harder style of interviewing, though, precisely because you can't just go through a rote set of questions... even the standard question set I use has a lot of ways of approaching them and several "levels of detail" I can get into depending on experience and skill level.)
The end result is still, frankly, stressful. It's a big deal, and the interviewee would have to be outright irrational to forget that. But it helps if you as an interviewer remember that, too, and have some compassion.
I'd have to agree. The interview process is irreducibly stressful, but that doesn't mean we have to go out of our way to make it worse. It isn't even good for hiring most men either, after all! It's just a bad idea.
I've personally had fairly good success hiring people who happen to be women, and I suspect a good part of that is that my interview attitude is that I'm there to find out what you can do, not to find out what you can't, which seems to be the unexamined default of a lot of interviewers. You don't start at "perfect" and get marks off for failure... you start at essentially zero and work your way up. "Starting at zero" may sound superficially harsh when I put it into English, but in practice this produces a much friendlier interview environment, I think, because we're discovering positive aspects instead of negative ones.
(Another reason to prefer this is that in practice this also helps avoid the problem of the senior candidate being thrown softballs, or the junior candidate being asked to design a cloud infrastructure. It really doesn't take that long for me to key in on where your most likely "knowledge frontier" is and probe for details there. As opposed to just going through a rote set of questions looking for demerits. It is a harder style of interviewing, though, precisely because you can't just go through a rote set of questions... even the standard question set I use has a lot of ways of approaching them and several "levels of detail" I can get into depending on experience and skill level.)
The end result is still, frankly, stressful. It's a big deal, and the interviewee would have to be outright irrational to forget that. But it helps if you as an interviewer remember that, too, and have some compassion.
> my interview attitude is that I'm there to find out what you can do, not to find out what you can't,
Have you ever considered trying to find out and confirm what the candidate has done in the past, instead of the almost impossible task of finding out what they "can do" in a few short hours? Barring some major new life issue, a developer's past performance is highly predictive of their future performance.
Edit: Removed longish rant.
Have you ever considered trying to find out and confirm what the candidate has done in the past, instead of the almost impossible task of finding out what they "can do" in a few short hours? Barring some major new life issue, a developer's past performance is highly predictive of their future performance.
Edit: Removed longish rant.
That's what interviews for hardware engineers are like. I'm an embedded systems engineer, and as such about half my work is programming. I've only worked for companies that are primarily electronics manufacturers. I've hesitated to apply to companies that are more software focused largely because of my perception of their interview process.
Interviews for electronic engineers consist mostly of talking about projects the candidate has worked on (for young people these might be college and hobby projects). The interviewer's questions are things like "what did you consider the most difficult part of that project?" or "What tools did you use for that?" or "why did you do it that way and not this other way?" No one has ever asked me to design something on a whiteboard, nor would they dream of asking me to rattle off the pinout of some random microprocessor.
The software company interviews I hear about sound more like someone being tested to prove they aren't lying about their qualifications than like adults discussing a business deal.
Interviews for electronic engineers consist mostly of talking about projects the candidate has worked on (for young people these might be college and hobby projects). The interviewer's questions are things like "what did you consider the most difficult part of that project?" or "What tools did you use for that?" or "why did you do it that way and not this other way?" No one has ever asked me to design something on a whiteboard, nor would they dream of asking me to rattle off the pinout of some random microprocessor.
The software company interviews I hear about sound more like someone being tested to prove they aren't lying about their qualifications than like adults discussing a business deal.
If I can, yes. But many interviewees are not prepared for that, and I've had to adapt to that.
If you are interviewing, please by all means bring samples of your work and be ready to discuss them, if you possibly can. But some people come from places where they can't talk about what they did (like, literally, Federal law in some cases), or for some other good reason can't really substantiate what they did.
You also have to account for the fact that a lot of the ways of finding out what they did in the past are somewhat unreliable. Many past employers have their own reasons for not being forthcoming or entirely truthful. The interviewee is incentivized to exaggerate their claims. In this context it's hard to get very much solid information of the type you're asking for.
My preference is to see past accomplishments, then use the interview to substantiate that they are indeed your accomplishments and you were not just generally in the area of accomplishments that happened to be occurring. But alas, that's not always an option.
If you are interviewing, please by all means bring samples of your work and be ready to discuss them, if you possibly can. But some people come from places where they can't talk about what they did (like, literally, Federal law in some cases), or for some other good reason can't really substantiate what they did.
You also have to account for the fact that a lot of the ways of finding out what they did in the past are somewhat unreliable. Many past employers have their own reasons for not being forthcoming or entirely truthful. The interviewee is incentivized to exaggerate their claims. In this context it's hard to get very much solid information of the type you're asking for.
My preference is to see past accomplishments, then use the interview to substantiate that they are indeed your accomplishments and you were not just generally in the area of accomplishments that happened to be occurring. But alas, that's not always an option.
What about developers with little experience, that come right out of school, or are applying for an internship position? Are you going to cut them out of the process? If so, are you okay with the fact that you'll often be missing out on getting an early lead on good junior developers? Good developers rarely kick around on the job market, and so it's often much more difficult to hire them when they've become more senior. Joel Spolsky's take:
http://www.joelonsoftware.com/articles/FindingGreatDeveloper...
http://www.joelonsoftware.com/articles/FindingGreatDeveloper...
I'm not suggesting to do this exclusively. By all means, hire promising devs out of college using the method the GP prefers.
I'm just suggesting that when an applicant like this exists before you, it makes more sense (assuming you're happy with their accomplishments) to confirm that these are indeed their accomplishments than to try to cram in some ad hoc heuristic for determining their range of skills in an hour or two.
And if my peers are any indication, the reason it's become so hard to hire senior devs is that no one wants to go through the headache of the interview process anymore once you've reached a moderately satisfactory job situation.
I'm just suggesting that when an applicant like this exists before you, it makes more sense (assuming you're happy with their accomplishments) to confirm that these are indeed their accomplishments than to try to cram in some ad hoc heuristic for determining their range of skills in an hour or two.
And if my peers are any indication, the reason it's become so hard to hire senior devs is that no one wants to go through the headache of the interview process anymore once you've reached a moderately satisfactory job situation.
When I used to hire people, I focused more on their intelligence, their focus, and their motivation. Any idiot can google snippets of code or memorize answers to questions. What I wanted was someone that could handle learning what they don't know, motivate themselves to get the job done, and most of all, get the job done.
And you can't figure those things out just by looking at a resume or by quizzing them for hours until they break. You have to actually talk with them. Engage them in conversation. Learn who they are and how they fit into this field. Learn about their past experiences and their future goals. Then the question as to whether or not to hire them is easy.
Resumes are a starting point. Previous knowledge is a time-saver. Problem-solving and motivation is where its at.
*As a side note, most of my interviews were 30 minutes or less. I only regretted one hire, and I got rid of him pretty quickly.
And you can't figure those things out just by looking at a resume or by quizzing them for hours until they break. You have to actually talk with them. Engage them in conversation. Learn who they are and how they fit into this field. Learn about their past experiences and their future goals. Then the question as to whether or not to hire them is easy.
Resumes are a starting point. Previous knowledge is a time-saver. Problem-solving and motivation is where its at.
*As a side note, most of my interviews were 30 minutes or less. I only regretted one hire, and I got rid of him pretty quickly.
Well if you want to be an innovator in that space, then you can offer an interview process that is more friendly to them.
That means, no onsite interview processes that require taking vacation days to complete mostly. Or off hours interview processes.
That means, no onsite interview processes that require taking vacation days to complete mostly. Or off hours interview processes.
Talk to them about school projects or classes. I recall one candidate was in a compiler class, and told me about the compiler s/he had working. I asked 'what's a recursive descent parser'. They didn't recognize the term. I asked what grammar the language had. They didn't know. Other's show excitement, talk about having had to study things on their own because it wasn't covered in class but they really wanted to learn it, and so on. You quickly get a sense of who has a fire burning and who is going through the motions.
It's tough though, because when you are a small company that expects developers to jump in and be productive, you do need some kind of process. That process can be referral based, in which case testing isn't as necessary because you can rely on past shared experience, but that obviously limits your talent pool.
When you are interacting with an unknown (or a less known) person, you have to have some kind of vetting process. And with a small company culture, typically false negatives are far less harmful than false positives.
When you are interacting with an unknown (or a less known) person, you have to have some kind of vetting process. And with a small company culture, typically false negatives are far less harmful than false positives.
Only if you have a policy of not having positions for Junior programmers.
Fog Creek has the same hypocrisy as any other company that demands X years of experience from applicants but consider it somebody else's problem how they get that.
Fog Creek has the same hypocrisy as any other company that demands X years of experience from applicants but consider it somebody else's problem how they get that.
As a business owner who is a developer and hires developers: getting my experience was my responsibility. It's now yours. And you won't do it on my dime.
I'm not in the business of using money to make developers. I'm in the business of using developers to make money. The jobs I offer are not because I want to give someone a job, they're because I want someone to come do a job for me.
I'm not in the business of using money to make developers. I'm in the business of using developers to make money. The jobs I offer are not because I want to give someone a job, they're because I want someone to come do a job for me.
Until very recently, the vast majority of Fog Creek's devs came through the internship program. Which is restricted to people who are going back to school afterwards (and, with a few exceptions, is only for undergrads).
Worse: they demand that you can code, and don't consider it their problem to teach you how to do that.
Anyway I thought it was well established that years of experience is a bullshit measurement anyway and you should just apply no matter what - worst thing happens, you end up getting hired.
Anyway I thought it was well established that years of experience is a bullshit measurement anyway and you should just apply no matter what - worst thing happens, you end up getting hired.
You're making fair points. I think I understand the equation you're setting up here (harm of false positive vs harm of false negative, along with probabilities for each). I'm just starting to believe that the damage of so many false negatives to avoid the occasional false positive may be harming the industry more than we realize.
Let's not forget, we're an industry where employers very consistently claim a serious and economically harmful shortage of talented developers. At the same time, we're an industry that accepts an interview process they open acknowledge will result in the rejection of good candidates because they feel that the damage of a false positive is so high.
Is there a chance that there is a strong "negativity bias" at work here?
http://en.wikipedia.org/wiki/Negativity_bias
In other words, is it possible that the bad memories of a false positive is masking the thousand paper cuts that are inflicted on the high tech workforce though grueling interviews? What overall harm does it do to an industry when a programmer who would have been successful gets rejected from a promising job due to a poor performance on a one hour data structures problem at a white board? I think there's the harm you see, and then the harm you don't.
This one line suggests the harm may be serious and extensive. I'm not making any particular prescription here, I'm just saying that when developers experience repeated rejections due to repeated false negatives, this may be vastly more harmful in a global sense than people currently realize.
EDIT: false positive/negative switch (vonmoltke mentioned below).
Let's not forget, we're an industry where employers very consistently claim a serious and economically harmful shortage of talented developers. At the same time, we're an industry that accepts an interview process they open acknowledge will result in the rejection of good candidates because they feel that the damage of a false positive is so high.
Is there a chance that there is a strong "negativity bias" at work here?
http://en.wikipedia.org/wiki/Negativity_bias
In other words, is it possible that the bad memories of a false positive is masking the thousand paper cuts that are inflicted on the high tech workforce though grueling interviews? What overall harm does it do to an industry when a programmer who would have been successful gets rejected from a promising job due to a poor performance on a one hour data structures problem at a white board? I think there's the harm you see, and then the harm you don't.
This one line suggests the harm may be serious and extensive. I'm not making any particular prescription here, I'm just saying that when developers experience repeated rejections due to repeated false negatives, this may be vastly more harmful in a global sense than people currently realize.
EDIT: false positive/negative switch (vonmoltke mentioned below).
> I'm just starting to believe that the damage of so many false positives to avoid the occasional false negative may be harming the industry more than we realize.
I think you have "false positive" and "false negative" backwards, but I agree with your point. I have long thought that the industry systematically understates the damage of false negatives and overstates the damage of false positives.
If you are hiring, it fits one of two broad categories: catch as catch can, or specific need. In a catch as catch can situation, the "always hiring" mode, the company can afford to place more emphasis on minimizing the false positive rate at the expense of the false negative rate because the desire is to skim the cream off the top of the applicant pool as it comes to you. The company doesn't need new people per se; it finds slots for these highly-capable people somewhere in the company after the hire is made.
In the specific need situation, the company is hiring to fill a hole. Either work is not getting done, or other people are overburdened with the responsibilities this new hire will take on. In this case, every day that passes costs the company in some way. The cost of a false negative is related to the quality and size of the applicant pipeline. A false negative could add anywhere from a few days, if the company has a strong pipeline, to a few months, if the company has a weak pipeline, to the timeline for filling this position. The "false negatives are cheap" position comes from the former, but I would say that very few companies actually have an applicant pipeline that strong. Furthermore, if you can go months without filling a position and not suffer any ill effects, you need to ask yourself why you are looking for someone in the first place.
I think you have "false positive" and "false negative" backwards, but I agree with your point. I have long thought that the industry systematically understates the damage of false negatives and overstates the damage of false positives.
If you are hiring, it fits one of two broad categories: catch as catch can, or specific need. In a catch as catch can situation, the "always hiring" mode, the company can afford to place more emphasis on minimizing the false positive rate at the expense of the false negative rate because the desire is to skim the cream off the top of the applicant pool as it comes to you. The company doesn't need new people per se; it finds slots for these highly-capable people somewhere in the company after the hire is made.
In the specific need situation, the company is hiring to fill a hole. Either work is not getting done, or other people are overburdened with the responsibilities this new hire will take on. In this case, every day that passes costs the company in some way. The cost of a false negative is related to the quality and size of the applicant pipeline. A false negative could add anywhere from a few days, if the company has a strong pipeline, to a few months, if the company has a weak pipeline, to the timeline for filling this position. The "false negatives are cheap" position comes from the former, but I would say that very few companies actually have an applicant pipeline that strong. Furthermore, if you can go months without filling a position and not suffer any ill effects, you need to ask yourself why you are looking for someone in the first place.
You're on to something here. There's the specific, need-based interview, and there's the "sure, we're always looking for talent" interview.
The latter can lead to an unintended problem. I once went on an interview where I was passed from room to room, on the hour, and taken through my technical paces each time. Each interview was intensely technical. If my interview had been my only exposure to the company (i.e., I hadn't gone to the site, read press releases and articles, and so forth), I would have no idea what the company does, other than that it seems to involve operations-research style math (optimization and stochastic processes), data structures and algorithms, and complex sql. I certainly would have no idea what they planned on having me do from a business point of view. I didn't get an offer (was busy, hadn't adequately refreshed my knowledge of tree traversal, markov chains, and linear optimization to immediate post-exam undergrad/grad levels), but I wasn't hot on the company anyway, because it left me with the impression that they wouldn't value my business input in any way.
So, why would they do this? Maybe they're just sort of looking generally for people. They figure, eh, if we find candidates who can rock an exam[1], sure, let's go ahead and make an offer, I'm sure we'll find something for them to do. Because their need isn't immediate, their standards (no false positives) will be set unusually high.
The unintended consequence is that yet another developer in this space is now reluctant to interview, because who wants to prepare for and re-take their undergraduate/graduate math and CS exams?
[1] At this point, I don't even think we should call this an interview, it was an exam that lasted longer than my MS graduate exams, with far less information about what would be tested.
The latter can lead to an unintended problem. I once went on an interview where I was passed from room to room, on the hour, and taken through my technical paces each time. Each interview was intensely technical. If my interview had been my only exposure to the company (i.e., I hadn't gone to the site, read press releases and articles, and so forth), I would have no idea what the company does, other than that it seems to involve operations-research style math (optimization and stochastic processes), data structures and algorithms, and complex sql. I certainly would have no idea what they planned on having me do from a business point of view. I didn't get an offer (was busy, hadn't adequately refreshed my knowledge of tree traversal, markov chains, and linear optimization to immediate post-exam undergrad/grad levels), but I wasn't hot on the company anyway, because it left me with the impression that they wouldn't value my business input in any way.
So, why would they do this? Maybe they're just sort of looking generally for people. They figure, eh, if we find candidates who can rock an exam[1], sure, let's go ahead and make an offer, I'm sure we'll find something for them to do. Because their need isn't immediate, their standards (no false positives) will be set unusually high.
The unintended consequence is that yet another developer in this space is now reluctant to interview, because who wants to prepare for and re-take their undergraduate/graduate math and CS exams?
[1] At this point, I don't even think we should call this an interview, it was an exam that lasted longer than my MS graduate exams, with far less information about what would be tested.
I'm a big fan of looking at developers' github contributions. It's a good way to see that they can actually write code and it doesn't involve a high-pressure whiteboard test.
On the other hand, it can be a negative for somebody working for a company that discourages or forbids open sourcing their work. Or people who don't have free time to spend on FOSS projects.
On the other hand, it can be a negative for somebody working for a company that discourages or forbids open sourcing their work. Or people who don't have free time to spend on FOSS projects.
...or who work on open source projects that aren't hosted on github?
Don't confuse activity for productivity. Most of the activities people engage in when searching for a new employee are complete wastes of time, at best.
I personally find it is net-better to stop wasting that time, just try people at essentially random, and get back to my own work.
I personally find it is net-better to stop wasting that time, just try people at essentially random, and get back to my own work.
I agree.
Personally, I can handle all sorts of adversarial situations, but I've never done well with "rigorous" interviews. I think I was raised to be humble and that's built into my DNA, but the business world embraces overpromise/underdeliver.
At the same time, I've successfully managed large teams, budgets, interfaced with super-high level people and negotiated hard-nosed deals. I gotten myself into these roles by other means -- 90% of interviews that I have done have been formalities.
IMO, that's a symptom of a bigger problem, as what you do has more to do with where you are and who you know than any other factor.
Personally, I can handle all sorts of adversarial situations, but I've never done well with "rigorous" interviews. I think I was raised to be humble and that's built into my DNA, but the business world embraces overpromise/underdeliver.
At the same time, I've successfully managed large teams, budgets, interfaced with super-high level people and negotiated hard-nosed deals. I gotten myself into these roles by other means -- 90% of interviews that I have done have been formalities.
IMO, that's a symptom of a bigger problem, as what you do has more to do with where you are and who you know than any other factor.
It's why I switched to a PM role. I miss programming every day, but I hate the thought of doing another stupid technical interview.
As I often say, interviews based on the final year algorithms and datastructures exam are just a way to sneak ageism under the HR radar.
What final year algorithms and data structures exam are you referring to?
Not all companies require them if you have decent experience. My friend interviewed with several companies and flat out told them "I don't do code tests anymore, my experience should speak for itself." Obviously your milage will vary but he ended up in a good position at a large well known company.
Edit: Also if you have a connection into the company that can vouch for you, that will work as well. The best coding test that can be administered is "yeah I've worked with this guy before and he's a good developer."
Edit: Also if you have a connection into the company that can vouch for you, that will work as well. The best coding test that can be administered is "yeah I've worked with this guy before and he's a good developer."
i can only imagine people who work at Google are some sort of interview ninjas.
I disagree. How many people actually do the bare minimum and work their way through Cracking the Code Interview, and make sure they know all of the material covered by those questions? I think having that would do a long way towards proving oneself in the tech interviews.
I applied for a job at Fog Creek Software in 2012 where I was accepted for an internship, and I would have to say the application process was difficult, but fair. I was told that there were hundreds of applicants, so being one of five interns selected seemed rather prestigious at the time, but it was still just an opportunity to work on my skills as a developer.
I don't think there is anything that would preclude women from getting technical jobs at a company like Fog Creek except maybe, as was said in the article, a self-selecting attitude. These companies are looking for well-rounded individuals and people who are competent with their skills. It's nothing beyond impossibility to get a job there, it just takes determination.
I don't think there is anything that would preclude women from getting technical jobs at a company like Fog Creek except maybe, as was said in the article, a self-selecting attitude. These companies are looking for well-rounded individuals and people who are competent with their skills. It's nothing beyond impossibility to get a job there, it just takes determination.
> I applied for a job at Fog Creek Software in 2012 where I was accepted for an internship
Does this mean you applied for a position but you were offered an internship instead?
Does this mean you applied for a position but you were offered an internship instead?
No; they don't do that. He means he applied for an internship and got an internship.
I have a very small consulting company. Both of my employees right now were non-traditional students when they finished their CS degrees. Actually, I'm not even sure the one guy has finished his degree yet. Regardless, neither of them had their degrees when they started working for me.
And honestly, I didn't really even know. I never asked them for a resume and they never provided one.
I don't really test the people I hire. I get to know them on a personal level. I meet them at parties, at meetups, conferences, etc. I then ask if they'd like to make a little side money with a freelance gig. We talk about the work, and if the person says they can do it, I give them a task. A real task, it goes right into the project I'm working on. It usually takes about a week or two.
And if they do a good job, I ask them to do more. If they don't, I pay them for the work they did, thank them, and don't mention they were ever interviewing for a more long-term relationship.
This has been so incredibly much better of an experience than the hiring processes I was involved with as a working stiff. Resumes that don't say shit. Stupidly simplistic coding tests. Interviewing anxiety. All of it, a nightmare.
And it didn't seem to make any difference. We would put more and more effort into hiring, to the point I was spending 10 to 15 hours interviewing and reviewing candidates a week, for weeks on end. It was having a detrimental impact on my performance, without any correlated impact on the performance of the people we hired.
So when I went independent, I decided that, if the results are really, actually random, then that means the inputs have no impact. So I threw away the interviewing process. I just ask people, "can you do this job"?
And really, I don't need to know anything else. That's what it all boils down to. They can lie on their resume just as much as they can lie to the question. They can cram on interview questions the night before. They can be really good at regurgitating academic theory but complete crap at thinking creatively. So I just ask, "can you do this job?"
And then I let them prove it.
And honestly, I didn't really even know. I never asked them for a resume and they never provided one.
I don't really test the people I hire. I get to know them on a personal level. I meet them at parties, at meetups, conferences, etc. I then ask if they'd like to make a little side money with a freelance gig. We talk about the work, and if the person says they can do it, I give them a task. A real task, it goes right into the project I'm working on. It usually takes about a week or two.
And if they do a good job, I ask them to do more. If they don't, I pay them for the work they did, thank them, and don't mention they were ever interviewing for a more long-term relationship.
This has been so incredibly much better of an experience than the hiring processes I was involved with as a working stiff. Resumes that don't say shit. Stupidly simplistic coding tests. Interviewing anxiety. All of it, a nightmare.
And it didn't seem to make any difference. We would put more and more effort into hiring, to the point I was spending 10 to 15 hours interviewing and reviewing candidates a week, for weeks on end. It was having a detrimental impact on my performance, without any correlated impact on the performance of the people we hired.
So when I went independent, I decided that, if the results are really, actually random, then that means the inputs have no impact. So I threw away the interviewing process. I just ask people, "can you do this job"?
And really, I don't need to know anything else. That's what it all boils down to. They can lie on their resume just as much as they can lie to the question. They can cram on interview questions the night before. They can be really good at regurgitating academic theory but complete crap at thinking creatively. So I just ask, "can you do this job?"
And then I let them prove it.
I think this method doesn't work in all situations. For example in embedded programming. First there are not many of those who actually work in this field so unlikely to just meet up with them. Most are stabily employed. Secondly, there is lots of proprietary hardware and knowledge. Third, hard to work on a trial job when you are already working. Lastly there are a good number of people who are just not skilled enough and will waste your time during the trial period.
Actually, one of them does do some embedded programming for me.
Granted, my sample size is small, but traditional hiring practices are completely ineffective, so you shouldn't waste your time thinking you have any influence over the process.
Granted, my sample size is small, but traditional hiring practices are completely ineffective, so you shouldn't waste your time thinking you have any influence over the process.
There is a war for Women Dev's in silicon valley. Google's HR head mentioned in a podcast that they are simply poaching female devs' from smaller players to get their diversity numbers up. This puts smaller players at a major disadvantage as VC's have started tying their funding with diversity numbers. There is no way a startup can compete directly with Goog for the female dev's who are in major short supply.
This diversity madness is putting SV at a major disadvantage.
This diversity madness is putting SV at a major disadvantage.
> This diversity madness is putting SV at a major disadvantage.
On the contrary, evidence suggest Silicon Valley is putting itself at a disadvantage by failing to attract talented women. Other non-SV technology markets do far better: http://www.inc.com/kimberly-weisul/women-tech-problem-silico...
On the contrary, evidence suggest Silicon Valley is putting itself at a disadvantage by failing to attract talented women. Other non-SV technology markets do far better: http://www.inc.com/kimberly-weisul/women-tech-problem-silico...
If it's not too much trouble, could I ask for sources on 1)
>Google's HR head mentioned in a podcast that they are simply poaching female devs' from smaller players to get their diversity numbers up.
and 2)
> VC's have started tying their funding with diversity numbers.
>Google's HR head mentioned in a podcast that they are simply poaching female devs' from smaller players to get their diversity numbers up.
and 2)
> VC's have started tying their funding with diversity numbers.
If VC do anything but focus on getting investment in the best companies they are not fulfilling their fiduciary duty to their investors - even if that anything is saving starving children in Africa.
Maybe Joel needs to add another point to his "test"
13. Do you foster Junior/Mentor relationships?
13. Do you foster Junior/Mentor relationships?
What company is not going to check that box?
I've worked for a company that claimed they did and it was always talked about internally in their "look how great our culture is" corporate-speak. Some developers even had the title of "mentor," but there wasn't any actual time allocated for it and no one ever took it seriously. In my entire time there I never met with my "mentor."
I've worked for a company that claimed they did and it was always talked about internally in their "look how great our culture is" corporate-speak. Some developers even had the title of "mentor," but there wasn't any actual time allocated for it and no one ever took it seriously. In my entire time there I never met with my "mentor."
The same can be said for just about every item on the Joel test
I genuinely think you should put that to him. Not enough organisations recognise the value of junior developers imo.
And get rid of hallway usability tests.
Not sure why you think that. I have found getting feedback from team members before user testing finds a lot of low-hanging issues.
I'd be okay if it just said "usability testing". In my company that's just a part of the testing phase. But no way I'm just going to grab someone walking down the hall. I assume people are busy enough.
Is this meant to be taken literally? Fog Creek companies work remotely. How would you even physically grab someone in the hallway? As for how we do internal testing, we simply ask around on Slack and get a solid group of people to test for us before moving forward. Works out great.
> But no way I'm just going to grab someone walking down the hall. I assume people are busy enough.
That seems like a kind of joyless approach to life.
That seems like a kind of joyless approach to life.
I agree that lack of gender diversity in tech is an obvious problem, and I laud Fog Creek for trying to address this problem.
However, I'd like to strongly emphasis the fact that one of the main problems Fog Creek identified as hindering the most talented female developers from applying to Fog Creek, is also a huge problem that prevents many of the most talented male developers from applying to Fog Creek. Ironically, these companies, with their legendarily difficult and arduous application processes, are selecting for mediocre developers:
1) We all know that the Dunning-Kruger effect is real: there's an abundance of quantified and repeatable psychological research that demonstrates that the least skilled individuals are on average the most likely to estimate their skill to be above average. Likewise, the most skilled individuals are the most cautious in estimating their skill level.
So who applies for more of these types of jobs? Well, based on what I've heard, Google, Fog Creek, and many other top employers receive a huge number of grossly unqualified applicants. And while I'm sure they'd like to think they're getting the majority of their desired candidate pool to apply, research on Dunning-Kruger, as well as anecdotes like the one on the article, all suggest that's probably not the case.
This isn't the first time I've read about a top employer asking a highly-desired employee why they'd never applied in the past, and receiving as an answer, "oh, I didn't think I'd make it through your interview process". (btw, I love watching these kinds of interactions)
2) The most qualified developers can make a nice consultant salary without ever having to go through the headache of a demeaning 5-day interview process that often treats developers like disposable commodities. Why would you want to subject yourself to that kind of process if you're making $100,000-200,000 a year as a contract software dev (which is the range my colleagues working contract tend to make, depending on their hours and expertise)? So whenever I get an email from employers like Facebook, I say thanks but no thanks.
This is not to say that these employers never get highly-skilled employees. That's obviously not the case. But they get a very specific type of highly-skilled applicant: from top CS departments, coming right out of college. They're much less likely to get a highly-skilled dev to apply who has taken an alternate route to the top of his or her profession.
So what happens on average? "Top" employers like Google, Facebook, and smaller "desirable" employers like Fog Creek, effectively scare off a large number of the devs they want the most, both male and female. And they simultaneously attract a large number of unqualified applicants.
Yet again I ask: maybe the software developer interview process is completely broken? Are employers in enough pain yet that they're willing to revisit their hiring process? Or does it need to get worse?
However, I'd like to strongly emphasis the fact that one of the main problems Fog Creek identified as hindering the most talented female developers from applying to Fog Creek, is also a huge problem that prevents many of the most talented male developers from applying to Fog Creek. Ironically, these companies, with their legendarily difficult and arduous application processes, are selecting for mediocre developers:
1) We all know that the Dunning-Kruger effect is real: there's an abundance of quantified and repeatable psychological research that demonstrates that the least skilled individuals are on average the most likely to estimate their skill to be above average. Likewise, the most skilled individuals are the most cautious in estimating their skill level.
So who applies for more of these types of jobs? Well, based on what I've heard, Google, Fog Creek, and many other top employers receive a huge number of grossly unqualified applicants. And while I'm sure they'd like to think they're getting the majority of their desired candidate pool to apply, research on Dunning-Kruger, as well as anecdotes like the one on the article, all suggest that's probably not the case.
This isn't the first time I've read about a top employer asking a highly-desired employee why they'd never applied in the past, and receiving as an answer, "oh, I didn't think I'd make it through your interview process". (btw, I love watching these kinds of interactions)
2) The most qualified developers can make a nice consultant salary without ever having to go through the headache of a demeaning 5-day interview process that often treats developers like disposable commodities. Why would you want to subject yourself to that kind of process if you're making $100,000-200,000 a year as a contract software dev (which is the range my colleagues working contract tend to make, depending on their hours and expertise)? So whenever I get an email from employers like Facebook, I say thanks but no thanks.
This is not to say that these employers never get highly-skilled employees. That's obviously not the case. But they get a very specific type of highly-skilled applicant: from top CS departments, coming right out of college. They're much less likely to get a highly-skilled dev to apply who has taken an alternate route to the top of his or her profession.
So what happens on average? "Top" employers like Google, Facebook, and smaller "desirable" employers like Fog Creek, effectively scare off a large number of the devs they want the most, both male and female. And they simultaneously attract a large number of unqualified applicants.
Yet again I ask: maybe the software developer interview process is completely broken? Are employers in enough pain yet that they're willing to revisit their hiring process? Or does it need to get worse?
> "oh, I didn't think I'd make it through your interview process".
I can attest to that. I was a contractor for Google for several years. I built all of their election results and voter information maps for the 2008-2012 US elections. I also did results maps for a number of international elections, both on my own and working with local teams, along with various special mapping and visualization projects.
The maps were very popular and people in multiple teams at Google really liked my work.
Of course it occurred to me several times that I ought to apply for a full-time job there, and every time I thought, "Naw, I'd never make it through a Google interview process."
This, after years of doing successful, highly visible work for them!
I can attest to that. I was a contractor for Google for several years. I built all of their election results and voter information maps for the 2008-2012 US elections. I also did results maps for a number of international elections, both on my own and working with local teams, along with various special mapping and visualization projects.
The maps were very popular and people in multiple teams at Google really liked my work.
Of course it occurred to me several times that I ought to apply for a full-time job there, and every time I thought, "Naw, I'd never make it through a Google interview process."
This, after years of doing successful, highly visible work for them!
This isn't the first time I've read about a top employer asking a highly-desired employee why they'd never applied in the past, and receiving as an answer, "oh, I didn't think I'd make it through your interview process"
Exactly. And to someone who is already gainfully employed elsewhere, the opportunity cost of going through an interview gauntlet--which often consists of one or more full days of intense testing, and many more days of practice to prepare for that testing--is very real and daunting. So they will naturally tend to attract fresh college graduates who have nothing to lose and can afford to make such a huge time investment in the interview process for a particular prospective employer.
Exactly. And to someone who is already gainfully employed elsewhere, the opportunity cost of going through an interview gauntlet--which often consists of one or more full days of intense testing, and many more days of practice to prepare for that testing--is very real and daunting. So they will naturally tend to attract fresh college graduates who have nothing to lose and can afford to make such a huge time investment in the interview process for a particular prospective employer.
You are wrong about the Dunning-Kruger effect - there is not a negative correlation between perceived and actual ability, just a weaker positive correlation than you might expect. There's plenty of debate about the causes of the effect in the original Dunning-Kruger paper. Moreover, if you look at the actual paper (rather than exaggerated retellings of it), you will see that perceived ability is actually positively correlated with actual ability: most people tend to think that they're above-average - the bottom quartile tends to think that they're in to 50th to 60th percentiles, while more able people tend to think they're in the 70th to 80th percentiles.
See Figure 1 at: http://psycnet.apa.org/journals/psp/77/6/1121.html
Also, $100,000-200,000 a year is the range you'd make at those companies and have more job security and income consistency, so I'm not sure what your point is.
See Figure 1 at: http://psycnet.apa.org/journals/psp/77/6/1121.html
Also, $100,000-200,000 a year is the range you'd make at those companies and have more job security and income consistency, so I'm not sure what your point is.
I regret my initial phrasing about Dunning-Kruger, which was incorrect as you have pointed out (and as I conceded below some 3 hours before you wrote your rebuttal).
Also, my point is that if I'm making $100-$200,000 and am happy with my situation, particularly when I live in a lower cost area than the Bay Area, then I am much less likely to put myself through the pain of one of these interview processes. Unlike lots of workers, software devs actually have a choice.
Also, my point is that if I'm making $100-$200,000 and am happy with my situation, particularly when I live in a lower cost area than the Bay Area, then I am much less likely to put myself through the pain of one of these interview processes. Unlike lots of workers, software devs actually have a choice.
It's not a 5 day process. This is the standard bay area interview process:
1. 15 minute recruiter call
2. 1 hour remote screener interview
3. 4-6 45m-1hr onsite interviews. They can be done all on one day or possibly split up if your local.
4. Wait 1 day to a couple of weeks to get an offer or a rejection.
The real pain in the ass part is the onsite one. But that is 1 day, not a week. And because they go for high false negative rates, and you need multiple offers for higher salaries, you need to do onsites at multiple places.
1. 15 minute recruiter call
2. 1 hour remote screener interview
3. 4-6 45m-1hr onsite interviews. They can be done all on one day or possibly split up if your local.
4. Wait 1 day to a couple of weeks to get an offer or a rejection.
The real pain in the ass part is the onsite one. But that is 1 day, not a week. And because they go for high false negative rates, and you need multiple offers for higher salaries, you need to do onsites at multiple places.
It's a provocative thesis that Fog Creek hires mediocre developers. The evidence of their products suggests otherwise.
Of course as Atwood's friend Scott says, "We're all amateurs," so perhaps Fog Creek just hires the less mediocre on an absolute rather than relative scale.
Of course as Atwood's friend Scott says, "We're all amateurs," so perhaps Fog Creek just hires the less mediocre on an absolute rather than relative scale.
From the context of the statement, I think the thesis is that Fog Creek's process increases the probability that mediocre developers will apply.
This is exactly the thesis. And by the same thesis, if Fog Creek's ability to correctly identity high-quality applicants remains at a constant rate, and the share of qualified devs grows smaller, then Fog Creek will hire more and more mediocre devs.
Does it? I don't know a single person who uses any of their products. And I don't think it is plausible that the best devs in the world are flocking to work on a website for project managers.
Do you know anyone who uses StackOverflow? Not the same fictitious person as Fog Creek, but the many of the same people.
https://www.fogcreek.com/about/
https://www.fogcreek.com/about/
To add to what brudgers said, Trello also came out of Fog Creek (and somewhat more recently).
I don't know what that is, or anyone who uses it.
Oh. Well, in my defense that was pretty unlikely. Hacker news search shows that it's been mentioned in the title of 475 stories, at least 25 of which hit the front page.
Well, we've still got Stack Exchange. Surely you've heard of that?
Well, we've still got Stack Exchange. Surely you've heard of that?
I think the Dunning-Kruger thing is a nice idea but it's immediately ruined once you become aware of it, like a Heisenberg uncertainty sort of thing.
I used to think I was crap, but Dunning-Kruger, so I must be great.
I used to think I was great, but Dunning-Kruger, so I must be crap.
It just seems a bit trite to base any real world conclusions or behaviour off of.
I agree that the interview process is broken. The reason I wouldn't apply to Google/Facebook/Fog Creek is personal and nothing to do with their interview process.
I used to think I was crap, but Dunning-Kruger, so I must be great.
I used to think I was great, but Dunning-Kruger, so I must be crap.
It just seems a bit trite to base any real world conclusions or behaviour off of.
I agree that the interview process is broken. The reason I wouldn't apply to Google/Facebook/Fog Creek is personal and nothing to do with their interview process.
> I think the Dunning-Kruger thing is a nice idea but it's immediately ruined once you become aware of it, like a Heisenberg uncertainty sort of thing.
Not at all! You're being too binary with it. Dunning-Kruger is about the relationship between discernment and capability. In particular, it's saying discernment drives capability. A great way of applying it is this Ira Glass quote:
http://www.goodreads.com/quotes/309485-nobody-tells-this-to-...
Basically, the way I apply it is to say that if I can tell that my work isn't as good as I want it to be, that's ok: I have to use that as fuel to improve. And if I'm dealing with something where my work looks perfect to me, it's time to up my discernment.
In hiring, the way I apply it is to avoid listening much to self-assessment and related measures of competence (e.g., confidence). Instead, I go for demonstrations of competence, like short essay answers instead of cover letters, or pair programming interviews [1] over "tell me about a time when" discussions.
[1] https://www.quora.com/What-are-the-best-programming-intervie...
Not at all! You're being too binary with it. Dunning-Kruger is about the relationship between discernment and capability. In particular, it's saying discernment drives capability. A great way of applying it is this Ira Glass quote:
http://www.goodreads.com/quotes/309485-nobody-tells-this-to-...
Basically, the way I apply it is to say that if I can tell that my work isn't as good as I want it to be, that's ok: I have to use that as fuel to improve. And if I'm dealing with something where my work looks perfect to me, it's time to up my discernment.
In hiring, the way I apply it is to avoid listening much to self-assessment and related measures of competence (e.g., confidence). Instead, I go for demonstrations of competence, like short essay answers instead of cover letters, or pair programming interviews [1] over "tell me about a time when" discussions.
[1] https://www.quora.com/What-are-the-best-programming-intervie...
I like that Ira Glass quote, maybe some day I'll get there.
I think you are just positively improving your work, I don't think what you are saying is an application of Dunning-Kruger. Why in the first instance do you give your negative assessment credence, and have doubts about your positive assessment in the second?
Dunning-Kruger says both assessments are likely to be wrong, and in both cases you are striving to improve - so Dunning-Kruger has little to do with it.
In fact you could go further in a "worse is better" sense and say your striving to improve one thing is letting down something else you could be paying attention to.
Dunning did an AMA on reddit a few months back:
http://www.reddit.com/r/science/comments/2m6d68
Your hiring approach does sound very common sense and I am sure people did as you do long before Dunning Kruger was a thing.
I think you are just positively improving your work, I don't think what you are saying is an application of Dunning-Kruger. Why in the first instance do you give your negative assessment credence, and have doubts about your positive assessment in the second?
Dunning-Kruger says both assessments are likely to be wrong, and in both cases you are striving to improve - so Dunning-Kruger has little to do with it.
In fact you could go further in a "worse is better" sense and say your striving to improve one thing is letting down something else you could be paying attention to.
Dunning did an AMA on reddit a few months back:
http://www.reddit.com/r/science/comments/2m6d68
Your hiring approach does sound very common sense and I am sure people did as you do long before Dunning Kruger was a thing.
It's about how one strives to improve. Have you read the original paper?
http://psych.colorado.edu/~vanboven/teaching/p7536_heurbias/...
Incompetent people think they're doing fine because they can't tell the difference between good and bad work. Highly competent people, readily seeing their own flaws, think they don't do as well as they are doing. (At least until you have them grade a bunch of other work, at which point they are better calibrated.) Competent people are competent because their metacognitive ability, their ability to tell good work from bad, drives them forward.
So basically, if I already can see the problems in my work, then my metacognitive ability is sufficient to drive improvement. When I improve enough that my work looks fine to me, further improvement is prevented because I literally can't tell whether changes I make are better or worse. There, the path to improvement is to improve metacognition.
As to the latter, my approach may be common sense, but it is not in fact common. Last I was interviewing developers, most people had never done a pair programming interview before. The default is talk-talk interviews. I'm sure some people did it the other way, of course: science frequently trails practice. But the Dunning-Kruger paper in specific is what convinced me that self-assessments were at best highly unreliable and at worst thoroughly misleading.
http://psych.colorado.edu/~vanboven/teaching/p7536_heurbias/...
Incompetent people think they're doing fine because they can't tell the difference between good and bad work. Highly competent people, readily seeing their own flaws, think they don't do as well as they are doing. (At least until you have them grade a bunch of other work, at which point they are better calibrated.) Competent people are competent because their metacognitive ability, their ability to tell good work from bad, drives them forward.
So basically, if I already can see the problems in my work, then my metacognitive ability is sufficient to drive improvement. When I improve enough that my work looks fine to me, further improvement is prevented because I literally can't tell whether changes I make are better or worse. There, the path to improvement is to improve metacognition.
As to the latter, my approach may be common sense, but it is not in fact common. Last I was interviewing developers, most people had never done a pair programming interview before. The default is talk-talk interviews. I'm sure some people did it the other way, of course: science frequently trails practice. But the Dunning-Kruger paper in specific is what convinced me that self-assessments were at best highly unreliable and at worst thoroughly misleading.
haven't read it thanks. are you saying the paper is about how one strives to improve? - because it's not titled that. the title is pretty clear. so is the abstract.
i think we agree that self assessments aren't worth much.
i think our disagreement stems from me applying it from a neutral ability perspective and you applying it to your own self assessment, where you don't question your basic competency.
i think we agree that self assessments aren't worth much.
i think our disagreement stems from me applying it from a neutral ability perspective and you applying it to your own self assessment, where you don't question your basic competency.
> haven't read it thanks. are you saying the paper is about how one strives to improve? - because it's not titled that. the title is pretty clear. so is the abstract.
I am not. You didn't see how one could usefully apply their work. I'm explaining how I have applied it. You asked "Why in the first instance do you give your negative assessment credence, and have doubts about your positive assessment in the second?" When I say, "It's about how one strives to improve," I am answering your question.
> i think our disagreement stems from me applying it from a neutral ability perspective and you applying it to your own self assessment, where you don't question your basic competency.
Not at all. If for X I am a person of low ability, then I am likely to perceive myself as doing pretty well at X because I cannot see the flaws in my work. So if I'm at a state where my work looks good to me, the thing to do is, as I said, to improve my discernment until I can see flaws in my work. Then I can work to improve, increasing my competency.
This strategy works no matter what you relative competence is. And note that relative competence is distinct from absolute competence. Even people in the top quartile of a given sample can keep improving, and I believe Glass's quote helps explain why: their taste is still ahead of their skill, which if they keep practicing yields increased absolute competence.
I am not. You didn't see how one could usefully apply their work. I'm explaining how I have applied it. You asked "Why in the first instance do you give your negative assessment credence, and have doubts about your positive assessment in the second?" When I say, "It's about how one strives to improve," I am answering your question.
> i think our disagreement stems from me applying it from a neutral ability perspective and you applying it to your own self assessment, where you don't question your basic competency.
Not at all. If for X I am a person of low ability, then I am likely to perceive myself as doing pretty well at X because I cannot see the flaws in my work. So if I'm at a state where my work looks good to me, the thing to do is, as I said, to improve my discernment until I can see flaws in my work. Then I can work to improve, increasing my competency.
This strategy works no matter what you relative competence is. And note that relative competence is distinct from absolute competence. Even people in the top quartile of a given sample can keep improving, and I believe Glass's quote helps explain why: their taste is still ahead of their skill, which if they keep practicing yields increased absolute competence.
I initially stated that you are just doing self improvement and dunning kruger isn't informing it.
you haven't said anything that disagrees with what i said, neither does the paper. at best you are applying half of it, the positive side. cherry picking?
if you're competent and you assess your work and it appears to be bad, then dunning-kruger says that it is not as bad as you think.
if you're incompetent and you assess your work and it appears to be good, then dunning-kruger says you don't have the competence to make that assessment and it's probably not good work.
if you had somebody competent in to assess your work instead of you, then that would probably be a more accurate application of it.
sure self improvement works, but self assessment doesn't really and that's the main take away.
you haven't said anything that disagrees with what i said, neither does the paper. at best you are applying half of it, the positive side. cherry picking?
if you're competent and you assess your work and it appears to be bad, then dunning-kruger says that it is not as bad as you think.
if you're incompetent and you assess your work and it appears to be good, then dunning-kruger says you don't have the competence to make that assessment and it's probably not good work.
if you had somebody competent in to assess your work instead of you, then that would probably be a more accurate application of it.
sure self improvement works, but self assessment doesn't really and that's the main take away.
I'll try one more time to get my point across. I agree with this:
> if you're incompetent and you assess your work and it appears to be good, then dunning-kruger says you don't have the competence to make that assessment and it's probably not good work.
Definitely. Ergo, if my work looks good to me, I should not assume that it is. Instead, I should look to improve my ability to evaluate the work. (What D+K call metacognition.) One way to do that, of course, is to have other people evaluate my work and tell me what they see. But there are plenty of others.
> if you're competent and you assess your work and it appears to be bad, then dunning-kruger says that it is not as bad as you think.
Exactly. Therefore, I should not be discouraged, but instead focus on upping my game to the level of my taste.
> if you're incompetent and you assess your work and it appears to be good, then dunning-kruger says you don't have the competence to make that assessment and it's probably not good work.
Definitely. Ergo, if my work looks good to me, I should not assume that it is. Instead, I should look to improve my ability to evaluate the work. (What D+K call metacognition.) One way to do that, of course, is to have other people evaluate my work and tell me what they see. But there are plenty of others.
> if you're competent and you assess your work and it appears to be bad, then dunning-kruger says that it is not as bad as you think.
Exactly. Therefore, I should not be discouraged, but instead focus on upping my game to the level of my taste.
your overly positive outlook means you are either deliberately ignoring or unable to see the other side of it, and so there's no point in our continuing this conversation.
you've stated your position over and over, i've reworded mine, no progress will be made.
> Exactly. Therefore, I should not be discouraged, but instead focus on upping my game to the level of my taste.
it doesn't mean you should do anything. there's no need for this self improvement. the self improvement tactics are not informed by dunning kruger. they are not an application of dunning kruger.
in the absence of dunning kruger if you run with a self improvement mentality the input, action and outcome are the exact same. your behaviour is not informed by it. if you weren't self improving before dunning kruger, then you might be on to something - but i'm certain that wasn't the case.
in fact you are in danger of spending too much time on the improvement, whereas dunning kruger could be telling you, as you are competent "it's good enough".
you've stated your position over and over, i've reworded mine, no progress will be made.
> Exactly. Therefore, I should not be discouraged, but instead focus on upping my game to the level of my taste.
it doesn't mean you should do anything. there's no need for this self improvement. the self improvement tactics are not informed by dunning kruger. they are not an application of dunning kruger.
in the absence of dunning kruger if you run with a self improvement mentality the input, action and outcome are the exact same. your behaviour is not informed by it. if you weren't self improving before dunning kruger, then you might be on to something - but i'm certain that wasn't the case.
in fact you are in danger of spending too much time on the improvement, whereas dunning kruger could be telling you, as you are competent "it's good enough".
I suggest going with either "there is no point in continuing" or "I will repeat again my possibly mistaken notion about why I think you're wrong." Because doing both makes you look like somebody who cares more about "winning" than having an actual discussion where people learn things. That's certainly the feeling I'm leaving with.
I believe your interpretation of the Dunning-Kruger effect is slightly wrong. The effect is not that the least skilled estimate their skill to be above average. The effect is that the least skilled tend to overestimate their skill level greater than those with higher skill levels. Also, those at the highest tend to overestimate the least (and indeed often times underestimate).
You're right, your phrasing is the correct way to put it. I was writing quickly and even now am wasting time when I should be working.
I don't know about D-K and mediocre developers, but I do know that those coming out on top of any test only proves one thing: that you are good at the test.
Let's say for argument's sake that the test is accurate (it flawlessly measures what its creators intended to measure.) This still doesn't mean that those who score highest are the "best." It only indicates congruence with the test designers.
The fact that Fog Creek has 0% gender diversity is evidence of this and indicates a underlying lack of diversity in many areas. So the question remains "Is a company that is an echo chamber full of mini-mes superior to one with a variety of opinions, habits, thinking styles, and working practices?"
I don't know the answer, but I think it's a good question.
Let's say for argument's sake that the test is accurate (it flawlessly measures what its creators intended to measure.) This still doesn't mean that those who score highest are the "best." It only indicates congruence with the test designers.
The fact that Fog Creek has 0% gender diversity is evidence of this and indicates a underlying lack of diversity in many areas. So the question remains "Is a company that is an echo chamber full of mini-mes superior to one with a variety of opinions, habits, thinking styles, and working practices?"
I don't know the answer, but I think it's a good question.
If I understand what you are saying, we can look at the results of an interview test and say, “this test has few or no false-positives,” meaning that everyone who passes the test is competent.
But the way that the industry is set up, we have very little knowledge of whether the test produces false negatives, meaning we don’t know how many of the people who failed the test turn out to be successful or even spectacular somewhere else.
But the way that the industry is set up, we have very little knowledge of whether the test produces false negatives, meaning we don’t know how many of the people who failed the test turn out to be successful or even spectacular somewhere else.
> The fact that Fog Creek has 0% gender diversity is evidence of this
No it's not. Do the math - at the rate of women in the application pool, and the total number of devs Fog Creek has ever hired, it would actually be fairly surprising if they'd ever hired a woman. The internship program, on the other hand (larger sample size) runs at more or less the expected rate, if memory serves.
This seems to make it pretty clear that the applicant pool is the problem, no?
No it's not. Do the math - at the rate of women in the application pool, and the total number of devs Fog Creek has ever hired, it would actually be fairly surprising if they'd ever hired a woman. The internship program, on the other hand (larger sample size) runs at more or less the expected rate, if memory serves.
This seems to make it pretty clear that the applicant pool is the problem, no?
When I applied at Fog Creek, I asked about diversity in the workplace and got what I considered to be a hasty, defensive, almost-canned response.
Prior to their jarring, incongruous answer, I was worried I was not good enough at software development to get the job. Afterward, I suspected it was just another "cultural fit" thing. They did use videoconferencing in the interview process, if I recall correctly. I thought at the time that maybe age discrimination could have been a factor, or regionalism.
Of course, Fog Creek, just like everyone else I have ever interviewed with, declined to give any sort of meaningful feedback afterward. "No, you were great, really; we just didn't pick you. No, we won't say why."
Not knowing any of the real reasons why I didn't make the cut, I am free to imagine any stupid, bogus reason that I can rationalize to myself. The only reason I would really accept with finality, in a way that wouldn't diminish my image of the company, is "we don't think your software skills are good enough to work here." But they very carefully implied that was not the reason. So my image of the company is diminished.
And a lengthy and arduous screening process only makes it more frustrating when the company won't provide any useful feedback.
I guess that sword swings both ways. They also don't know why they don't get more female applicants. I must admit feeling a little schadenfreude over that. It's mean and petty, of course, but I still feel it.
Prior to their jarring, incongruous answer, I was worried I was not good enough at software development to get the job. Afterward, I suspected it was just another "cultural fit" thing. They did use videoconferencing in the interview process, if I recall correctly. I thought at the time that maybe age discrimination could have been a factor, or regionalism.
Of course, Fog Creek, just like everyone else I have ever interviewed with, declined to give any sort of meaningful feedback afterward. "No, you were great, really; we just didn't pick you. No, we won't say why."
Not knowing any of the real reasons why I didn't make the cut, I am free to imagine any stupid, bogus reason that I can rationalize to myself. The only reason I would really accept with finality, in a way that wouldn't diminish my image of the company, is "we don't think your software skills are good enough to work here." But they very carefully implied that was not the reason. So my image of the company is diminished.
And a lengthy and arduous screening process only makes it more frustrating when the company won't provide any useful feedback.
I guess that sword swings both ways. They also don't know why they don't get more female applicants. I must admit feeling a little schadenfreude over that. It's mean and petty, of course, but I still feel it.
How about "Your software skills were great, but so-and-so's were better"? Of course, not knowing you and not working at Fog Creek, there's no way to know if that was the actual reason or not - but I would think it would be an acceptable reason.
Something I've realized since ending up on the hiring side of things is that there's a bit of a mismatch between how candidates and companies see the hiring process. As a candidate applying for jobs, I used to think I was applying to "the company" as an entity, and I was either good enough or I wasn't. The company, however, sees its candidates as applying for a specific position - like college admissions, there may be more qualified applicants than spots the budget allows for - especially at a place like Fog Creek.
(And, anecdotally, I am female and many years ago applied to Fog Creek for an internship. I didn't get it, but I enjoyed the interview process - it was one of the more interesting and challenging technical interviews I've done.)
Something I've realized since ending up on the hiring side of things is that there's a bit of a mismatch between how candidates and companies see the hiring process. As a candidate applying for jobs, I used to think I was applying to "the company" as an entity, and I was either good enough or I wasn't. The company, however, sees its candidates as applying for a specific position - like college admissions, there may be more qualified applicants than spots the budget allows for - especially at a place like Fog Creek.
(And, anecdotally, I am female and many years ago applied to Fog Creek for an internship. I didn't get it, but I enjoyed the interview process - it was one of the more interesting and challenging technical interviews I've done.)
I have heard that before, actually. It doesn't help if they won't tell you who they hired, and why they were better. The fact that they picked someone "better" is obvious if they picked someone at all, and it wasn't you.
Also, when I hear that particular line, I follow up with "I'm glad you were able to find someone you like. Will you keep me in mind the next time you're hiring?" If I get a response at all, it is something noncommittal, like "We keep all applications for a minimum of six months."
Perhaps companies should not focus so narrowly on the current position. I have applied multiple times to the same companies for different positions, provided my experience with them remains positive.
But if, for instance, a company like SAIC schedules an interview with me on a military base, sends an escort to the gate to bring me to the work site, and only then tells me that no one is available to interview me that day, because the supervisors are all halfway across the country, that company lands itself on my blacklist. Later experiences with employees of that company moved them to my permanent, public blacklist. Not only will I not apply for any of their positions, but if they appeared on my doorstep with an offer letter in hand, I'd order them off the property. And I would recommend that everyone else do the same, for both SAIC and for spinoff spawn Leidos. They are the sole reason why I ask "Who's the prime contractor?" whenever I interview with a contracting company.
When you create an energy potential by posting an opening, candidate-anticandidate pairs don't pop into existence to interview with you and then self-annihilate after you reject them. Actual, persistent people stick around, remember how you treated them, and talk to each other about their experiences. They stay in the industry, and may one day be customers or competitors.
As it happens, Fog Creek is not on my blacklist, either temporarily or permanently. But I do feel like their interview process would be a waste of my time if I had to endure it again from the beginning, and I don't expect them to ever pick me up again from where we left off. So I ignore their job postings now. It won't hurt them any, because they get plenty of candidates anyway.
But it doesn't surprise me at all that anyone who doesn't believe that they are good enough to top the 99 other people all applying for the same position wouldn't bother investing all that energy to compete, especially when that won't help them in their search if they fail. With no feedback, no referrals, and no helpful suggestions, it really is a lot of effort for a high probability of no reward whatsoever.
There may be something embedded in our tribal caveman brains that makes high-risk behaviors more attractive to men than to women. If so, Fog Creek might be able to attract more female applicants just by guaranteeing something of value at the end of the interview process, regardless of whether the candidate is selected for the next round.
Also, when I hear that particular line, I follow up with "I'm glad you were able to find someone you like. Will you keep me in mind the next time you're hiring?" If I get a response at all, it is something noncommittal, like "We keep all applications for a minimum of six months."
Perhaps companies should not focus so narrowly on the current position. I have applied multiple times to the same companies for different positions, provided my experience with them remains positive.
But if, for instance, a company like SAIC schedules an interview with me on a military base, sends an escort to the gate to bring me to the work site, and only then tells me that no one is available to interview me that day, because the supervisors are all halfway across the country, that company lands itself on my blacklist. Later experiences with employees of that company moved them to my permanent, public blacklist. Not only will I not apply for any of their positions, but if they appeared on my doorstep with an offer letter in hand, I'd order them off the property. And I would recommend that everyone else do the same, for both SAIC and for spinoff spawn Leidos. They are the sole reason why I ask "Who's the prime contractor?" whenever I interview with a contracting company.
When you create an energy potential by posting an opening, candidate-anticandidate pairs don't pop into existence to interview with you and then self-annihilate after you reject them. Actual, persistent people stick around, remember how you treated them, and talk to each other about their experiences. They stay in the industry, and may one day be customers or competitors.
As it happens, Fog Creek is not on my blacklist, either temporarily or permanently. But I do feel like their interview process would be a waste of my time if I had to endure it again from the beginning, and I don't expect them to ever pick me up again from where we left off. So I ignore their job postings now. It won't hurt them any, because they get plenty of candidates anyway.
But it doesn't surprise me at all that anyone who doesn't believe that they are good enough to top the 99 other people all applying for the same position wouldn't bother investing all that energy to compete, especially when that won't help them in their search if they fail. With no feedback, no referrals, and no helpful suggestions, it really is a lot of effort for a high probability of no reward whatsoever.
There may be something embedded in our tribal caveman brains that makes high-risk behaviors more attractive to men than to women. If so, Fog Creek might be able to attract more female applicants just by guaranteeing something of value at the end of the interview process, regardless of whether the candidate is selected for the next round.
You might be right on point 1, but it doesn't offer an alternative. If you state that you take mediocre developers, better developers won't want to work there either.
Point 2 is also sort of moot; if people can make it on their own in the contract world, then it doesn't really matter what hyperbolized interview process (5-days? What?) stands between them and employment. I'm in the same boat, I work contract now and don't want to go back. The interview process has nothing to do with it.
I think you're far from concluding that the process is broken, though. Some companies may want to hire top CS grads straight out of college. In fact, a small company could probably fill their ranks through that alone.
Point 2 is also sort of moot; if people can make it on their own in the contract world, then it doesn't really matter what hyperbolized interview process (5-days? What?) stands between them and employment. I'm in the same boat, I work contract now and don't want to go back. The interview process has nothing to do with it.
I think you're far from concluding that the process is broken, though. Some companies may want to hire top CS grads straight out of college. In fact, a small company could probably fill their ranks through that alone.
Well, maybe not 'mediocre', but they have certainly self-selected their applicant pool.
Any softare company that has their developers working on a component called DonutCounter has my vote as an awesome palce to work! :-)
As a younger male about to finish his software engineering degree and start applying I agree with these points. I am intimidated at the rigorous testing some companies use. I don't have the confidence that others might have in there ability.
I am ultimately looking for a mentoring workplace ( just to start) where I can build my confidence quickly and start producing for the company.
I am ultimately looking for a mentoring workplace ( just to start) where I can build my confidence quickly and start producing for the company.
>>just to start
One place I worked at had a senior dev slash architect who really enjoyed mentoring green developers. This company did tend to hire people straight out of college, and this guy was really good at bringing them up to speed. It was really quiet nice to behold.
The problem was, the hires that were 'good' tended to leave after a year or two, leaving the less talented and motivated to muddle along. I think this is why you rarely see this nowadays.
One place I worked at had a senior dev slash architect who really enjoyed mentoring green developers. This company did tend to hire people straight out of college, and this guy was really good at bringing them up to speed. It was really quiet nice to behold.
The problem was, the hires that were 'good' tended to leave after a year or two, leaving the less talented and motivated to muddle along. I think this is why you rarely see this nowadays.
I don't think that has anything to do with mentorship. I think it's just that most companies don't give most young programmers big enough raises to reflect their increased value on the open market.
Lots of companies don't want to hire new grads, so there's a significant price premium for hiring someone with a couple years of experience. The companies that are willing to hire new grads aren't always willing to give annual raises that reflect that price premium. So lots of programmers with a couple of years of experience see that they can get a 20% raise by switching jobs. So they switch jobs.
(This happens with programmers later in their career too, but the difference between 8 and 10 years of experience is much less dramatic than the difference between 0 and 2. If a company gives the same annual raise to all its programmers in the same performance band regardless of experience, that raise may be too small for the newest ones, even if it's just right for the middle ones.)
Lots of companies don't want to hire new grads, so there's a significant price premium for hiring someone with a couple years of experience. The companies that are willing to hire new grads aren't always willing to give annual raises that reflect that price premium. So lots of programmers with a couple of years of experience see that they can get a 20% raise by switching jobs. So they switch jobs.
(This happens with programmers later in their career too, but the difference between 8 and 10 years of experience is much less dramatic than the difference between 0 and 2. If a company gives the same annual raise to all its programmers in the same performance band regardless of experience, that raise may be too small for the newest ones, even if it's just right for the middle ones.)
> Fog Creek doesn't get any immediate benefits
I'd say articles like this one are pretty significant benefits. I doubt they overlooked the PR benefits of this program. (Which of course doesn't mean it isn't still a great thing. Just that "no benefit" is a bit of a stretch.)
I'd say articles like this one are pretty significant benefits. I doubt they overlooked the PR benefits of this program. (Which of course doesn't mean it isn't still a great thing. Just that "no benefit" is a bit of a stretch.)
Headlines?
There are more women than men in employment in the US, where is the daily articles on that? And the effort to tackle that? And the constant shaming of companies that employ more women than men?
There are 1.4x the amount of women graduating college than men. Where are the daily articles on that massive issue? Where is the effort to do something about that? Where is the constant headlines on how many more women are graduating than men, put in a very negative light as the lack of quote underlined "diversity" in the tech industry is?
This absolute, unrelenting and incredibly transparent (tech = money, status currently) narrative being sharted at everyone in the tech industry on a daily basis is driving me toward getting out of it altogether.
But hey, lets ignore some massive issues on the table in favour of constantly debating why women who never showed an interest in the industry can't pick and choose the finest and most well paid positions in that industry and see where that gets society in 10 years' time.
There are 1.4x the amount of women graduating college than men. Where are the daily articles on that massive issue? Where is the effort to do something about that? Where is the constant headlines on how many more women are graduating than men, put in a very negative light as the lack of quote underlined "diversity" in the tech industry is?
This absolute, unrelenting and incredibly transparent (tech = money, status currently) narrative being sharted at everyone in the tech industry on a daily basis is driving me toward getting out of it altogether.
But hey, lets ignore some massive issues on the table in favour of constantly debating why women who never showed an interest in the industry can't pick and choose the finest and most well paid positions in that industry and see where that gets society in 10 years' time.
There have been a vast slew of articles on the problem of male unemployment, and particularly the problem of the unemployment rate in minority males, who got hit particularly hard by the recession and have not seen those jobs come back. Around the recession this was all over the news.
Higher education has also been tackling the issue of getting men into college, and keeping them there until graduation. As has primary and secondary education, asking questions like what it takes to get boys to read more, and the effects of having mostly female teachers.
That you have to ask where they are suggests to me that you haven't looked very hard.
And the assumption that you cannot tackle multiple issues at the same time, worrying about both diversity in tech and unemployment in men, is simply false.
Higher education has also been tackling the issue of getting men into college, and keeping them there until graduation. As has primary and secondary education, asking questions like what it takes to get boys to read more, and the effects of having mostly female teachers.
That you have to ask where they are suggests to me that you haven't looked very hard.
And the assumption that you cannot tackle multiple issues at the same time, worrying about both diversity in tech and unemployment in men, is simply false.
This absolute, unrelenting and incredibly transparent (tech = money, status currently) narrative being sharted at everyone in the tech industry on a daily basis is driving me toward getting out of it altogether.
It's not being thrown at the tech industry. It's people within the industry calling out the problem that we have. It's very telling that you think of the industry as a community under siege.
It's not being thrown at the tech industry. It's people within the industry calling out the problem that we have. It's very telling that you think of the industry as a community under siege.
>It's not being thrown at the tech industry. It's people within the industry calling out the problem that we have. It's very telling that you think of the industry as a community under siege.
Actually, the vast majority of it is being carried out by people on the fringe of the industry in roles as "journalists" and similar who've never spent a day working within it.
And yes, I see the tech industry as being under siege.
Actually, the vast majority of it is being carried out by people on the fringe of the industry in roles as "journalists" and similar who've never spent a day working within it.
And yes, I see the tech industry as being under siege.
[deleted]
But the article seems to glide past that and presents the idea that five years of experience might change someone's willingness to apply. And it probably will help, but that amount of experience is unlikely to change someone's basic perception of their ability to succeed in a high-stakes one-chance prove-yourself situation.