Programmers and “productivity”(clipperhouse.com)
clipperhouse.com
Programmers and “productivity”
http://clipperhouse.com/blog/post/Programmers-and-productivity.aspx
13 comments
Ah,
2.6 "Best to median" sounds like a much more plausible figure. The x10 figure is usually quoted in such a way that it seems like it is "best to median".
2.6 "Best to median" sounds like a much more plausible figure. The x10 figure is usually quoted in such a way that it seems like it is "best to median".
Numbers I cannot disclose to you from a major Japanese multinational which actually tracks programmer productivity (grumble grumble why is this rare grumble grumble) cause me to not have major plausibility problems with the x10 best to median.
Can you talk about the metric(s) by which they measure productivity? I would appreciate it.
We're essentially a large consultancy who bills by the kousuu -- "unit of engineering labor", somewhat arbitrarily defined as "what we think one engineer should be able to do in a day". Projects are successful if we bill more than N * K * the average labor cost of producing one kousuu, where K is the number of kousuu and N is a constant representing overhead, profit, and the cost of sale.
Projects are sized based on a negotiation between us and the customer. It is then broken into parts, which are given their own kousuu value, via guesstimation by a senior engineer on the team. (You'll notice this process involves an awful lot of guesstimation. This tends to converge on reasonable numbers because guesstimation is happening among multiple actors who don't have an incentive to collude -- kinda-sorta like a market of players with imperfect knowledge can converge staggeringly quickly on a good guess of the intrinsic value of something.)
We then measure how many kousuu of work you complete for a particular project per day you report you are working on the project. That is currently our primary productivity measure, although we also track customer satisfaction, "claims" (a backchannel by which customers here get free work done to fix issues in the deliverable which are determined to be due to our mistakes), subjective quality assessments from the engineers, follow-on business from the same customer, and any value we can extract from the project for the company. (For example, in a 10 kousuu project if we manage to extract 2 kousuu worth of functionality that the customer paid for into our internal framework, then deploy that to seven customers and charge them each for it, we make out like bandits.)
Anyhow, we keep obsessive records of who does what and how long it took, so figuring out productivity is "just" a matter of dividing. (We only figured out the dividing trick recently. Hard to move a large organization from the way it typically does things, you know?)
Projects are sized based on a negotiation between us and the customer. It is then broken into parts, which are given their own kousuu value, via guesstimation by a senior engineer on the team. (You'll notice this process involves an awful lot of guesstimation. This tends to converge on reasonable numbers because guesstimation is happening among multiple actors who don't have an incentive to collude -- kinda-sorta like a market of players with imperfect knowledge can converge staggeringly quickly on a good guess of the intrinsic value of something.)
We then measure how many kousuu of work you complete for a particular project per day you report you are working on the project. That is currently our primary productivity measure, although we also track customer satisfaction, "claims" (a backchannel by which customers here get free work done to fix issues in the deliverable which are determined to be due to our mistakes), subjective quality assessments from the engineers, follow-on business from the same customer, and any value we can extract from the project for the company. (For example, in a 10 kousuu project if we manage to extract 2 kousuu worth of functionality that the customer paid for into our internal framework, then deploy that to seven customers and charge them each for it, we make out like bandits.)
Anyhow, we keep obsessive records of who does what and how long it took, so figuring out productivity is "just" a matter of dividing. (We only figured out the dividing trick recently. Hard to move a large organization from the way it typically does things, you know?)
The expectation of market clearing for programmer wages is completely implausible here.
Going independent requires additional skills beyond the programming productivity that we're supposedly trying to measure, is perceived as riskier (and it's generally understood that most people are irrationally risk-averse), and may involve additional practical complications, such as if our hypothetical programmer relies on salaried employment to provide health insurance for a family. So "would this programmer make more independently" is pretty much an irrelevant question.
As for moving between "regular" jobs, there's little evidence that most hiring processes can distinguish programmers by productivity, nor that most organizations measure the productivity of employed programmers in a useful way. Switching jobs also has practical obstacles, such as the disruption of relocating, and avoiding the perception of "job hopping". So as far as we can tell, salaries have little relationship with productivity, which is kind of the whole point.
Fundamentally, the article makes the mistake of taking principles that are simplifying assumptions in economics and then arguing as if they're universally correct. This is roughly akin to an argument like "Things tend to fall down, so it's a fallacy to ask why helium balloons float."
Of course, you can simply define someone's productivity as the market value of their time and thus "prove" that order of magnitude differences in productivity between programmers don't generally exist, but proof-by-redefining-the-terms doesn't really make for a persuasive argument.
Going independent requires additional skills beyond the programming productivity that we're supposedly trying to measure, is perceived as riskier (and it's generally understood that most people are irrationally risk-averse), and may involve additional practical complications, such as if our hypothetical programmer relies on salaried employment to provide health insurance for a family. So "would this programmer make more independently" is pretty much an irrelevant question.
As for moving between "regular" jobs, there's little evidence that most hiring processes can distinguish programmers by productivity, nor that most organizations measure the productivity of employed programmers in a useful way. Switching jobs also has practical obstacles, such as the disruption of relocating, and avoiding the perception of "job hopping". So as far as we can tell, salaries have little relationship with productivity, which is kind of the whole point.
Fundamentally, the article makes the mistake of taking principles that are simplifying assumptions in economics and then arguing as if they're universally correct. This is roughly akin to an argument like "Things tend to fall down, so it's a fallacy to ask why helium balloons float."
Of course, you can simply define someone's productivity as the market value of their time and thus "prove" that order of magnitude differences in productivity between programmers don't generally exist, but proof-by-redefining-the-terms doesn't really make for a persuasive argument.
"If our newly-independent programmer does not make 10x the money, we must admit that she is not the 10x programmer we thought she was." Not a good metric. The signal is at least polluted by social skills and geography - in some areas of the midwest, the social construct of the traditional company is much more entrenched than in some places in CA. Also, I'm 29 and I have no family - I could work totally different places and locations than a family man with roots, which opens up my possibilities much more. imho, conflating the concept of profitability and productivity is not a good idea.
Yes,
I'm actually a skeptic on the generic x10 programmer-productive claim but the claim "Markets tend to clear" when applied anything and everything is more of a justifying fiction than a useful measure. Alan Greenspan and friends used it repeated to justify the housing bubble. And yet this kind of reasoning is still here.
Sigh...
I'm actually a skeptic on the generic x10 programmer-productive claim but the claim "Markets tend to clear" when applied anything and everything is more of a justifying fiction than a useful measure. Alan Greenspan and friends used it repeated to justify the housing bubble. And yet this kind of reasoning is still here.
Sigh...
I'm actually a skeptic on the generic x10 programmer-productive claim
Why? There have been many studies of this and their findings are consistent. IIRC the numbers range from 5-to-1 to 100-to-1 with the median in the order-of-magnitude range.
Pages 14-15 of Robert Glass' Facts and Fallacies of Software Engineering cover this topic.
http://books.google.com/books?id=3Ntz-UJzZN0C&printsec=f...
Why? There have been many studies of this and their findings are consistent. IIRC the numbers range from 5-to-1 to 100-to-1 with the median in the order-of-magnitude range.
Pages 14-15 of Robert Glass' Facts and Fallacies of Software Engineering cover this topic.
http://books.google.com/books?id=3Ntz-UJzZN0C&printsec=f...
I agree with the other commenters: the author rather trivially assumes his conclusion by taking for granted that the market is rational. It isn't. One can argue about why. I tend to believe it's because we're still early in the history of the software: the closest analogy we have is the Industrial Revolution, and Ford and Taylor don't appear until well over a century into that.
The other side of the coin, of course, is that an irrational market represents major opportunity for anyone who figures out how to exploit it. So we all should stop complaining and start exploiting. Hopefully the hacker renaissance and startup wave of the last few years are the start of such a process, but it's too early to know for sure.
The other side of the coin, of course, is that an irrational market represents major opportunity for anyone who figures out how to exploit it. So we all should stop complaining and start exploiting. Hopefully the hacker renaissance and startup wave of the last few years are the start of such a process, but it's too early to know for sure.
Article appears to be deleted. Cannot find anything with similar title in recent archives either.
It's working fine for me - maybe my ISP is caching it or something??
I uploaded a copy (temporarily) to my site, feel free to grab it (PDF format): http://mqudsi.com/junk/Programmers_and_Productivity.pdf
I uploaded a copy (temporarily) to my site, feel free to grab it (PDF format): http://mqudsi.com/junk/Programmers_and_Productivity.pdf
You are a nice guy.
1. My source for the productivity figure is Peopleware. They cite a set of "coding war games" that found a repeatedly measured 10x variation between the best and the worst programmers in their work environment. The productivity difference showed up under a variety of different measurements. There is no vagueness about the measurement techniques or doubt about the figure.
2. The 10x figure was from best to worst, from best to median was (IIRC) 2.6 times.
3. A significant portion of the figure they measured was correlated with environment, and the strongest predictor of performance was the performance of your coworkers. That said their study technique couldn't tell the difference between whether environment attracted the best people, or whether environment caused people to become more productive. (From the point of view of a smart employer there isn't too much difference between those hypotheses.)
4. People accept compensation in many ways, and not all of them involve money. I believe that the better a programmer is, the more apt they are to be willing to trade salary for better work environment, more interesting co-workers, more interesting problems, etc. As a concrete example I have heard anecdotes from many people claiming that people are willing to accept sometimes substantial pay cuts for the privilege of working at Google.
5. Whatever the cause, in the Peopleware study the difference in pay between the top and bottom half of programmers was less than 10%. Furthermore productivity was basically uncorrelated with various factors people look at in the hiring process such as years of experience. That is direct evidence that it is a bad idea to measure productivity by salary.
When you put all of these factors together, you simply can't point to market forces then salaries to discount real performance differences between working programmers.