Software Estimation Is a Losing Game (2014)(rclayton.silvrback.com)
rclayton.silvrback.com
Software Estimation Is a Losing Game (2014)
https://rclayton.silvrback.com/software-estimation-is-a-losing-game
12 comments
> That uncertainty is never built into the mathematical model used by project managers
It depends on the project manager.
The PERT estimation technique was based on giving a 3-part range estimate for broken-down tasks. These are then rolled up with a simple formula.
In the PERT method independence is handled by creating explicit links between tasks, which leads to the concept of the critical path.
Nobody does these. Because it's very tedious. There are a lot of tasks for which "just try it and see" is less effort than performing a full-blown estimation raindance.
It depends on the project manager.
The PERT estimation technique was based on giving a 3-part range estimate for broken-down tasks. These are then rolled up with a simple formula.
In the PERT method independence is handled by creating explicit links between tasks, which leads to the concept of the critical path.
Nobody does these. Because it's very tedious. There are a lot of tasks for which "just try it and see" is less effort than performing a full-blown estimation raindance.
Thanks. Yeah, there's a fairly large number of formal management techniques, but like you said, they're very rarely used.
PERT: https://en.wikipedia.org/wiki/Program_evaluation_and_review_...
Barry Boehm's recent work:
http://www.amazon.com/Incremental-Commitment-Spiral-Model-Pr...
PERT: https://en.wikipedia.org/wiki/Program_evaluation_and_review_...
Barry Boehm's recent work:
http://www.amazon.com/Incremental-Commitment-Spiral-Model-Pr...
I worked on an estimation tool[1] inspired by PERT 3-point estimates for a while, simply because there weren't any when I went looking one time.
It was a lot of fun reading up on the estimation literature in various disciplines. Personally I think that the real magic of PERT 3-point estimation is the "unpacking effect" -- making people think about the subtasks of a headline issue.
When I'm in situations where I point a story, I like to ensure that we engineers say aloud what assumptions we're pointing on ("Bootstrap will make responsive mobile almost free") and then a quick low-detail breakdown of expected tasks ("new page, new RESTful endpoint, new model code").
I find that the latter, in particular, makes a big difference in how pointing goes.
[1] http://confidest.com
It was a lot of fun reading up on the estimation literature in various disciplines. Personally I think that the real magic of PERT 3-point estimation is the "unpacking effect" -- making people think about the subtasks of a headline issue.
When I'm in situations where I point a story, I like to ensure that we engineers say aloud what assumptions we're pointing on ("Bootstrap will make responsive mobile almost free") and then a quick low-detail breakdown of expected tasks ("new page, new RESTful endpoint, new model code").
I find that the latter, in particular, makes a big difference in how pointing goes.
[1] http://confidest.com
The other part of 3 part estimates that I like is that it forces everyone to consider the worse case. What can go wrong? What risks are there to this task? And only then do we talk about how long we think it will probably take.
The problem I have always had afterwards is communicating the results. Most people I've worked with cannot intuitively grasp confidence levels (90% vs 95% vs ...). They also invariably want a break down of how long each task or group of tasks will take so that they can negotiate down the estimate, not understanding that the total estimate is not a simple sum of each task. I'd love to hear better ways of explaining/using these sorts of estimates.
The first time was my fault: I left the columns that had intermediate values visible. One of those was the variance in each estimate, which is just the Std Dev squared. The numbers were big. No one could stop talking about how big the numbers were. No one knew what a variance was, nor appreciated that it was just the number in the previous column, multiplied by itself. Learned the very important lesson that you hide all your work, no one gets it.
The problem I have always had afterwards is communicating the results. Most people I've worked with cannot intuitively grasp confidence levels (90% vs 95% vs ...). They also invariably want a break down of how long each task or group of tasks will take so that they can negotiate down the estimate, not understanding that the total estimate is not a simple sum of each task. I'd love to hear better ways of explaining/using these sorts of estimates.
The first time was my fault: I left the columns that had intermediate values visible. One of those was the variance in each estimate, which is just the Std Dev squared. The numbers were big. No one could stop talking about how big the numbers were. No one knew what a variance was, nor appreciated that it was just the number in the previous column, multiplied by itself. Learned the very important lesson that you hide all your work, no one gets it.
Culture, really. Trust.
Where I work, Engineering owns the estimates. No ifs, buts or maybes. We own estimates. Product managers have unlimited ability to reprioritise, but we and we alone get to say how hard it's going to be.
That said, another secret is: use story points. They are dimensionless units that are only meaningful within a single project. As soon as you give dollars or days, everybody has an opinion or an agenda.
But if you use points and project based on velocity, it's hard to negotiate. Because you're not negotiating. Those are the actual numbers. The actual historical data, staring you in the face.
(Edited for clarity)
Where I work, Engineering owns the estimates. No ifs, buts or maybes. We own estimates. Product managers have unlimited ability to reprioritise, but we and we alone get to say how hard it's going to be.
That said, another secret is: use story points. They are dimensionless units that are only meaningful within a single project. As soon as you give dollars or days, everybody has an opinion or an agenda.
But if you use points and project based on velocity, it's hard to negotiate. Because you're not negotiating. Those are the actual numbers. The actual historical data, staring you in the face.
(Edited for clarity)
> One of those was the variance in each estimate, which is just the Std Dev squared. The numbers were big. No one could stop talking about how big the numbers were. No one knew what a variance was, nor appreciated that it was just the number in the previous column, multiplied by itself.
Just out of curiosity, were any of them interested in learning the answers to these mysteries?
Just out of curiosity, were any of them interested in learning the answers to these mysteries?
I spent a while trying to explain the purpose behind the numbers when it came up, but no, it really just kept coming back to the variance numbers being big. Not "why is the estimate for this so uncertain?", but more "Why is this number 100000? That's big".
This was actually not a time estimate. We were estimating the footprint for our production deployment for a very complicated distributed system. Most of the pieces were only in early development and the load model itself was uncertain by orders of magnitude.
It's a situation where statistics is easily applied, if you know statistics. If you don't, you'll never even realize it would have helped and keep saying that math is useless outside of school.
This was actually not a time estimate. We were estimating the footprint for our production deployment for a very complicated distributed system. Most of the pieces were only in early development and the load model itself was uncertain by orders of magnitude.
It's a situation where statistics is easily applied, if you know statistics. If you don't, you'll never even realize it would have helped and keep saying that math is useless outside of school.
So, small tasks you CAN estimate, large ones not so much. Perhaps the technique to use is right in front of you: decomposition.
On a small task, everything is small. The percent error can be totally gross (not precise at all) and not have a meaningful impact overall. A large error on small is just a multiplier of small. It's all small.
When you decompose a larger task into smaller ones, that's all still true for each of the smaller ones, right?
But you must remember to add the errors! And that is precisely why overall confidence may not likely improve with decomposition vs. an aggregate, or coarse, high level estimate of the whole thing.
And this is true for more than software. Why?
New tasks can be broken down into two coarse buckets: done before and novel.
Novel things really are novel! It's important to recognize this case and qualify it. Progress might look like:
week 1 - 10 percent done, week 2 - 10 percent done, week 3 - 15 percent done, week 4 - 50 percent done! Yes!, week 5 - 5 percent done... everybody take a drink, week 6 - 40 percent done,
etc...
Because it's novel, there are new dynamics in play, and even the best planning may not account for, and actually usually does not account for, things that just come up and that must be dealt with.
For the done before case, it's better as a lot is known, but there are still many factors out of the team control. Updates, changes, regressions, unfound bugs, all contribute to variance. The more the organization completely owns, the less this may be true, but it's always somewhat true.
The pragmatic person might classify everything as having some percentage of novel, and break it down into known novel and unknown, "it just hit us!" type novel and go from there and likely be better off when having to manage expectations.
Another way to think of this, depending on it being software or something else, is production vs "R&D" and the research and development part just isn't cog / production like. Time and materials estimates, if they are even given, must be considered accordingly.
If they are not, expectations do not match realities, and now there is a management and potentially a funding problem in addition to the primary problem needing a solution.
When you decompose a larger task into smaller ones, that's all still true for each of the smaller ones, right?
But you must remember to add the errors! And that is precisely why overall confidence may not likely improve with decomposition vs. an aggregate, or coarse, high level estimate of the whole thing.
And this is true for more than software. Why?
New tasks can be broken down into two coarse buckets: done before and novel.
Novel things really are novel! It's important to recognize this case and qualify it. Progress might look like:
week 1 - 10 percent done, week 2 - 10 percent done, week 3 - 15 percent done, week 4 - 50 percent done! Yes!, week 5 - 5 percent done... everybody take a drink, week 6 - 40 percent done,
etc...
Because it's novel, there are new dynamics in play, and even the best planning may not account for, and actually usually does not account for, things that just come up and that must be dealt with.
For the done before case, it's better as a lot is known, but there are still many factors out of the team control. Updates, changes, regressions, unfound bugs, all contribute to variance. The more the organization completely owns, the less this may be true, but it's always somewhat true.
The pragmatic person might classify everything as having some percentage of novel, and break it down into known novel and unknown, "it just hit us!" type novel and go from there and likely be better off when having to manage expectations.
Another way to think of this, depending on it being software or something else, is production vs "R&D" and the research and development part just isn't cog / production like. Time and materials estimates, if they are even given, must be considered accordingly.
If they are not, expectations do not match realities, and now there is a management and potentially a funding problem in addition to the primary problem needing a solution.
I can try to decompose a ~2-month task into one-day tasks, and still not improve my overall confidence level.
One of the comments on the original article nails it. The real problem is conflating "estimates" with "deadlines" or "commitments". You see it with the casual replacement of "story estimating" with "story sizing" in our language. "Sizing" implies a degree of certainty that may not be there.
Estimating is most definitely useful. But you have to fight every step of the way to keep an estimate from becoming a "promise". Promises that can't be kept are not good for any of the parties involved and lead to things like developer burnout, poor quality, technical debt, surreptitiously reduced scope, lack of trust...it goes on and on.
Estimating is most definitely useful. But you have to fight every step of the way to keep an estimate from becoming a "promise". Promises that can't be kept are not good for any of the parties involved and lead to things like developer burnout, poor quality, technical debt, surreptitiously reduced scope, lack of trust...it goes on and on.
Pardon my French, but it's a load of bollocks. Sure, I hate estimating and I'm bad at it, but I've worked with many others who were better than me at it and did consistently deliver what promised, when they promised it. And yes I'd much rather just sit and contemplate and re-re-rework 'the design' and be all-important about 'it's done when it's done' - but I fully well understand that any customer will just fire me and take someone who just does what he says he will.
Building construction is also fraught with unexpected events and project management issues etc., but I'd laugh any building who would propose that I just keep paying them until it's done, or who will start building and go home when whatever budget I give him is used, out of the room. Why would my customers be any different?
(articles like this poison the minds of young impressionable developers who think it's actually reasonable to have ideas like this, btw, which is why we shouldn't just shrug them off)
Building construction is also fraught with unexpected events and project management issues etc., but I'd laugh any building who would propose that I just keep paying them until it's done, or who will start building and go home when whatever budget I give him is used, out of the room. Why would my customers be any different?
(articles like this poison the minds of young impressionable developers who think it's actually reasonable to have ideas like this, btw, which is why we shouldn't just shrug them off)
>Building construction is also fraught with unexpected events and project management issues etc., but I'd laugh any building who would propose that I just keep paying them until it's done,
In practice, though, this is exactly how building construction and civil engineering works. After all, if the bridge is half-built, what's the customer going to do? Walk away? When was the last time you heard about any large civil engineering project coming in on time or on budget? It's to the point where we're relieved when the project ends up within an order of magnitude of its projected cost.
In practice, though, this is exactly how building construction and civil engineering works. After all, if the bridge is half-built, what's the customer going to do? Walk away? When was the last time you heard about any large civil engineering project coming in on time or on budget? It's to the point where we're relieved when the project ends up within an order of magnitude of its projected cost.
>After all, if the bridge is half-built, what's the customer going to do?
Maybe sue you for liquidated damages whilst pursuing all other possible avenues in the contract for claiming against a breach. Maybe file an insurance claim, and weather the mobilization costs for another contractor to take over your scope.
But before any of that you first have to estimate the potential legal costs and determine what's going to work more in your favor. :)
Maybe sue you for liquidated damages whilst pursuing all other possible avenues in the contract for claiming against a breach. Maybe file an insurance claim, and weather the mobilization costs for another contractor to take over your scope.
But before any of that you first have to estimate the potential legal costs and determine what's going to work more in your favor. :)
Actually, that's an excellent point -- how do buildings/bridges/etc. actually get built? I mean, laugh if you will, but can you not think of any examples of construction projects that straight up ran out of money and had to halt? Or that became enormous money and time sinks, inching their way but never-quite-making it to completion?
As I understand it (still pretty new to the industry), we're supposed to give realistic estimates for a product's completion, considering that 1. requirements are always incomplete and 2. unexpected complications will always arise. 'Done' is a moving target, and part of its estimation is ensuring the product is deliverable at every stage (with varying degrees of completeness), regardless of whether how many times everything goes pear shaped.
As I understand it (still pretty new to the industry), we're supposed to give realistic estimates for a product's completion, considering that 1. requirements are always incomplete and 2. unexpected complications will always arise. 'Done' is a moving target, and part of its estimation is ensuring the product is deliverable at every stage (with varying degrees of completeness), regardless of whether how many times everything goes pear shaped.
Sure. Developments get abandoned all the time, either because they run out of time and money or because economic conditions change. One I was reading about recent is the Fontainbleu Resort in Las Vegas[1]—the second-tallest building on the Strip and unfinished.
And even more developments end up hilariously past their budget on both time and money. "Megaprojects" like the Big Dig[2] are common examples of this.
These are the failures big enough to be notable. I'm sure there are far more smaller, less interesting but more indicative failed projects as well.
[1]: https://en.wikipedia.org/wiki/Fontainebleau_Resort_Las_Vegas
[2]: https://en.wikipedia.org/wiki/Big_Dig
And even more developments end up hilariously past their budget on both time and money. "Megaprojects" like the Big Dig[2] are common examples of this.
These are the failures big enough to be notable. I'm sure there are far more smaller, less interesting but more indicative failed projects as well.
[1]: https://en.wikipedia.org/wiki/Fontainebleau_Resort_Las_Vegas
[2]: https://en.wikipedia.org/wiki/Big_Dig
It was more of a rhetorical question, with the remaining literalness directed at the OP :) but yeah, those are fantastic examples! The Berlin Brandenburg [1] was what came to mind immediately, but you're absolutely right that history is like to be littered with other dead developments.
[1]: http://www.bloomberg.com/news/features/2015-07-23/how-berlin...
[1]: http://www.bloomberg.com/news/features/2015-07-23/how-berlin...
https://en.wikipedia.org/wiki/Bertha_(tunnel_boring_machine)
Seattle's current pear-shaped project. Check out the missed estimates!
Seattle's current pear-shaped project. Check out the missed estimates!
There is a huge difference between building a bridge and building a piece of software.
The complexity is just many many times bigger when it comes to software projects. With bridges it's actually fairly straight forward and the budgets are often of a completely different scale.
The complexity is just many many times bigger when it comes to software projects. With bridges it's actually fairly straight forward and the budgets are often of a completely different scale.
Software time/cost estimates are essential. If I run a business, I need to have some understanding if an investment will have a return. If I have no way to gauge that at all, then there are many more predictable endeavours where I can spend the money and realize a return. When you invest, do you simply say the future is unknowable and bet on raw emotion? I should hope not. That's not to say you may err in your judgment of available facts, you may still lose, but at least you understand some of the parameters about what constitutes success.
Software time/cost estimates are exactly that: estimates not guarantees. We tend to think of such things in terms of project management and contract enforcement. If anything, that is where we go wrong. What we should do is not try to understand the whole project in terms of an unalterable roadmap, but rather as a guideline for managing expectation and for informing decisions that are made during the course of a project. I use my estimates to sell the project, yes, but I also tell the client that as we go along we will be making many decisions that will move us away from our initial plan as we better understand both the issues and opportunities a project affords. Sometimes spending more time than planned will make sense, sometimes it won't and we cut the feature. Sometimes we find that we have failed to identify important features and sometimes we identified a feature early on that isn't as important as we thought. The overall project plan and estimate does set a benchmark to ensure the economics are still correct, after all if we lose that we lose the point of the project entirely, but we don't turn that plan into dogma. We don't lose sight of the reason that caused the project to exist.
So, yes, we need estimates no matter how hard they are to get accurate. What we need to stop doing is equating that estimate as an oath to be upheld regardless of what new knowledge comes along.
Software time/cost estimates are exactly that: estimates not guarantees. We tend to think of such things in terms of project management and contract enforcement. If anything, that is where we go wrong. What we should do is not try to understand the whole project in terms of an unalterable roadmap, but rather as a guideline for managing expectation and for informing decisions that are made during the course of a project. I use my estimates to sell the project, yes, but I also tell the client that as we go along we will be making many decisions that will move us away from our initial plan as we better understand both the issues and opportunities a project affords. Sometimes spending more time than planned will make sense, sometimes it won't and we cut the feature. Sometimes we find that we have failed to identify important features and sometimes we identified a feature early on that isn't as important as we thought. The overall project plan and estimate does set a benchmark to ensure the economics are still correct, after all if we lose that we lose the point of the project entirely, but we don't turn that plan into dogma. We don't lose sight of the reason that caused the project to exist.
So, yes, we need estimates no matter how hard they are to get accurate. What we need to stop doing is equating that estimate as an oath to be upheld regardless of what new knowledge comes along.
If I'm ever asked for an estimate, it's after the project schedule has already been set in stone.
On my current project, I am the sole full-time developer. The requirements were gathered from the client without my involvement. The schedule was laid out without my input. Then the whole thing was dropped on me.
On my current project, I am the sole full-time developer. The requirements were gathered from the client without my involvement. The schedule was laid out without my input. Then the whole thing was dropped on me.
I really wish more articles on software engineering practices would at least acknowledge that this is the reality for a huge number of developers in the real world. I just can't take them seriously without it.
If your new-fangled-to-you development ideology starts by assuming a fundamental level of congenial cooperation between different departments of your organization, then you've designed your system for a fairy-tale world. I don't suspect most people are able to change organizations easily. I suspect most people are stuck dealing with the hand they've been dealt. The lies they were told during their interview. The other employees that lied during their interviews. It's a really messy world out there.
If your new-fangled-to-you development ideology starts by assuming a fundamental level of congenial cooperation between different departments of your organization, then you've designed your system for a fairy-tale world. I don't suspect most people are able to change organizations easily. I suspect most people are stuck dealing with the hand they've been dealt. The lies they were told during their interview. The other employees that lied during their interviews. It's a really messy world out there.
I think the problem is that software process engineering does not really include the skills necessary for transforming organizations. To be honest, I don't know of any really good software process person who thinks that a process will do anything on its own. The processes and practices are only half of the battle.
It gets even more complicated because, at least in my experience, software processes are really subtle and it is extremely easy to appear to be "doing the right thing", while at the same time failing miserably at it. Throw in the power politics, the bad actors who don't really care if something is successful or not (as long as they can extract money from it), and naive developers who are absolutely sure they know what to do but really don't, and it makes it very difficult.
It is difficult to be the one instigating process change. Such a role is fraught with danger. You will be attacked by the power politics people. You will be attacked by the money extracting people. You will be attacked by the naive developers who think they know better than you. You almost certainly will not be thanked by anyone (except very far down the road if you somehow manage to succeed... and even then probably nobody will remember it was you who started the ball rolling). And finally, because it is incredibly hard and the necessary skills are very difficult to acquire, you will almost certainly fail many, many times before you ever succeed.
Even worse, it's not something you can write down and say "Do X,Y, and Z and everyone will be a happy family working hard to ensure success". It is something that requires dedication to understanding every single individual who is involved in the development. You have to sit and listen to them, understand their goals and help them meet those goals (even if they aren't "Make a successful software product" and even if you personally think the person is unbearably awful). You have to devote yourself to finding common ground and bringing everyone together.
Don't get me wrong. The processes and practices are crucial to a successful organizational change. These practices, if well chosen, will support you as you influence, cajole, flatter, criticize and motivate the people around you. It is next to an impossible job and without these tools, you will almost certainly fail.
It is unfortunate that people dramatically underestimate the difficulty of cultural change in an organization. They write about the easy bits (the processes that support the change) and sweep the hard bits under the carpet. To be fair, I'm not sure what you would say that wouldn't sound like "business BS" to most engineers, though.
Anyway, most organizations do not have many people conscientious (possibly read "stupid") enough to attempt real, lasting cultural change. The vast majority that do try are often incredibly naive and lack all of the basic skills necessary to be successful. The end result is unfortunately all too predictable.
To anyone who is really serious about building an amazing team, I encourage you to suffer the slings and arrows of outrageous management and to work on the people skills that will eventually bring you what you want. It sucks for a very long time, but as you get a better understanding, it can get better.
It gets even more complicated because, at least in my experience, software processes are really subtle and it is extremely easy to appear to be "doing the right thing", while at the same time failing miserably at it. Throw in the power politics, the bad actors who don't really care if something is successful or not (as long as they can extract money from it), and naive developers who are absolutely sure they know what to do but really don't, and it makes it very difficult.
It is difficult to be the one instigating process change. Such a role is fraught with danger. You will be attacked by the power politics people. You will be attacked by the money extracting people. You will be attacked by the naive developers who think they know better than you. You almost certainly will not be thanked by anyone (except very far down the road if you somehow manage to succeed... and even then probably nobody will remember it was you who started the ball rolling). And finally, because it is incredibly hard and the necessary skills are very difficult to acquire, you will almost certainly fail many, many times before you ever succeed.
Even worse, it's not something you can write down and say "Do X,Y, and Z and everyone will be a happy family working hard to ensure success". It is something that requires dedication to understanding every single individual who is involved in the development. You have to sit and listen to them, understand their goals and help them meet those goals (even if they aren't "Make a successful software product" and even if you personally think the person is unbearably awful). You have to devote yourself to finding common ground and bringing everyone together.
Don't get me wrong. The processes and practices are crucial to a successful organizational change. These practices, if well chosen, will support you as you influence, cajole, flatter, criticize and motivate the people around you. It is next to an impossible job and without these tools, you will almost certainly fail.
It is unfortunate that people dramatically underestimate the difficulty of cultural change in an organization. They write about the easy bits (the processes that support the change) and sweep the hard bits under the carpet. To be fair, I'm not sure what you would say that wouldn't sound like "business BS" to most engineers, though.
Anyway, most organizations do not have many people conscientious (possibly read "stupid") enough to attempt real, lasting cultural change. The vast majority that do try are often incredibly naive and lack all of the basic skills necessary to be successful. The end result is unfortunately all too predictable.
To anyone who is really serious about building an amazing team, I encourage you to suffer the slings and arrows of outrageous management and to work on the people skills that will eventually bring you what you want. It sucks for a very long time, but as you get a better understanding, it can get better.
I guess there's really no estimation to be done at all then, right? Are they expecting any?
They always ask. They're basically just asking you to draw the Gantt chart for them, so they can then ride your shoulder and bug you for each piece as it gets done.
A more appropriate question is not, does software estimation help improve project management, but is project management a suitable model for software?
I am a proponent of the idea software is a new form of literacy. We generally do not project manage the next paragrpH or the next chapter. writers are often faced with deadlines, but rarely have to justify themselves on a daily basis.
Improving estimation will just result in slightly better inputs to a process that at best hinders us and sucks creativity and joy.
I am a proponent of the idea software is a new form of literacy. We generally do not project manage the next paragrpH or the next chapter. writers are often faced with deadlines, but rarely have to justify themselves on a daily basis.
Improving estimation will just result in slightly better inputs to a process that at best hinders us and sucks creativity and joy.
Creativity and joy are fantastic qualities that are difficult to quantify on an accountant's spreadsheet. Clients have to be able to budget sufficiently for project work, and the only way they can do that is with an estimate. This is how business is done. "Uh..we'll send you a bill when we're done for...like...whatever it costs. I guess?" Isn't going to go over well when responding to a potential clients RFP.
Of course, given the historical fact that traditional project management doesn't work and the estimates are a farce, why should we happily continue doing this cargo cult uselessness?
They're not useless. Except sometimes they are.
I have to give estimates all the time. They're estimates, but they're in the ballpark.
But recently, I had to give an estimate for porting some software to a new environment. I knew it was going to present me with a bunch of obstacles, and I didn't know what the obstacles were, or how long each one would take. I didn't even know how many obstacles there would be.
I didn't want to give an estimate for that. They made me give them one anyway. I did, but I knew that no reliance whatsoever could be placed on the estimate. Whereas other tasks, estimates were close to "bet the company on it" solid.
I have to give estimates all the time. They're estimates, but they're in the ballpark.
But recently, I had to give an estimate for porting some software to a new environment. I knew it was going to present me with a bunch of obstacles, and I didn't know what the obstacles were, or how long each one would take. I didn't even know how many obstacles there would be.
I didn't want to give an estimate for that. They made me give them one anyway. I did, but I knew that no reliance whatsoever could be placed on the estimate. Whereas other tasks, estimates were close to "bet the company on it" solid.
When we run into stuff like that at work we budget a set number of hours to breaking the problem up into chunks, researching and prototyping the sticky parts, and then using that information to put together a realistic estimate. We call it a "planning project".
That's sensible - and is a way to lose bad clients (not itself a problem).
I am not saying project mgmt does not work, I am saying the plan the task, do the task approach is not appropriate for software development, at a given level of granularity.
If you ever watch Grand Designs, they start with a design, a plan, a schedule, and it all goes out the window ten minutes later. This is not that being on top of the process will not give you control, but that relying on estimates and plans as a means of control is worthless - and that's for something people have been doing for hundreds of years
I am not saying project mgmt does not work, I am saying the plan the task, do the task approach is not appropriate for software development, at a given level of granularity.
If you ever watch Grand Designs, they start with a design, a plan, a schedule, and it all goes out the window ten minutes later. This is not that being on top of the process will not give you control, but that relying on estimates and plans as a means of control is worthless - and that's for something people have been doing for hundreds of years
Because "Uh..we'll send you a bill when we're done for...like...whatever it costs. I guess?" Isn't going to go over well when responding to a potential clients RFP. It's that or give them an estimate, unless you have a 3rd way?
Also, how did you come to the conclusion that "project management doesn't work" is a historical fact?
Also also, if your teams' estimates are a farce you might consider starting to track time spent on projects and build up a base of real world numbers you can use as guidelines for future estimates. It worked wonders for us.
Also, how did you come to the conclusion that "project management doesn't work" is a historical fact?
Also also, if your teams' estimates are a farce you might consider starting to track time spent on projects and build up a base of real world numbers you can use as guidelines for future estimates. It worked wonders for us.
I have to laugh when I see these posts. So, software development is a special activity that should not be constrained by budgets or deadlines (or estimates)? Where do I sign up? There are so many strawmen here I can't possibly address them all. If you can't provide a credible estimate for a piece of work, you either a) aren't very good, b) don't properly understand the work to be done, or c) perceive some aspect of the job that is unknowable to you (through lack of experience or other novelty). Fixing each of these is up to you. 'a' is obvious (you either work hard to get better or find something else to do). 'b' means you need to ask more questions that allow you to sufficiently complete your understanding of the problem so an estimate is possible. 'c' means you need to isolate the novelty and explain that this part of the problem is new to you and you are not qualified to provide an estimate. Perhaps someone else can provide that insight or you can define an initial activity designed to help you better understand the novel problem until you can estimate the remaining work. NO area of engineering is so cut and dried that estimation is easy.
What IS common is a combination of poorly defined projects and a profound lack of self-awareness. By all means, push back against poorly defined projects ("I can't estimate the work until you tell me what it is."). Lack of self-awareness is squarely in YOUR lap ("I don't understand why it always takes longer than I thought it would..." Well, duh, maybe you should consider that reality for what it is and account for it next time).
Estimating work and committing to those estimates is what a professional does. Deal with it. If estimates are imposed on you that you do not agree with, that has nothing to do with the virtue of estimation. That's a social problem, not a technical one.
What IS common is a combination of poorly defined projects and a profound lack of self-awareness. By all means, push back against poorly defined projects ("I can't estimate the work until you tell me what it is."). Lack of self-awareness is squarely in YOUR lap ("I don't understand why it always takes longer than I thought it would..." Well, duh, maybe you should consider that reality for what it is and account for it next time).
Estimating work and committing to those estimates is what a professional does. Deal with it. If estimates are imposed on you that you do not agree with, that has nothing to do with the virtue of estimation. That's a social problem, not a technical one.
"So, software development is a special activity that should not be constrained by budgets or deadlines (or estimates)?"
Of course it is constrained, but how can you estimate it? Because "software development" is the continuous process of converting a real world idea into instructions that are so tiny and specific that a computer can understand them. Obviously you can't know in advance what those instructions are, or you've already written the program. When do you stop "pushing back against poorly defined projects"? You can literally continue asking for more specifics until the project is complete. This is different from other areas of engineering, because almost NOTHING demands preciseness like a computer does.
"Estimating work and committing to those estimates is what a professional does. Deal with it."
The reality is, we have been trying to deal with, for decades. And failing badly. The author is suggesting (probably rhetorically) that perhaps we need to abandon the idea entirely, because it is not working (for very large projects).
Of course it is constrained, but how can you estimate it? Because "software development" is the continuous process of converting a real world idea into instructions that are so tiny and specific that a computer can understand them. Obviously you can't know in advance what those instructions are, or you've already written the program. When do you stop "pushing back against poorly defined projects"? You can literally continue asking for more specifics until the project is complete. This is different from other areas of engineering, because almost NOTHING demands preciseness like a computer does.
"Estimating work and committing to those estimates is what a professional does. Deal with it."
The reality is, we have been trying to deal with, for decades. And failing badly. The author is suggesting (probably rhetorically) that perhaps we need to abandon the idea entirely, because it is not working (for very large projects).
We can estimate better but it's often not worth it.
It makes sense when there is a large cost to not meeting a deadline. Or when it's difficult to change scope.
It makes sense when there is a large cost to not meeting a deadline. Or when it's difficult to change scope.
Most of the problems are organisational rather than with estimating per se.
If I'm doing prod support for three different products and build work for two products, then I'm naturally going to be juggling priorities. There are a lot of different and changing variables in the mix and it makes committing to deadlines very difficult.
If an organisation wants to isolate developers and have them work on only build work for a single product estimation becomes much more accurate. Of course most orgs would rather less accurate estimation instead.
If I'm doing prod support for three different products and build work for two products, then I'm naturally going to be juggling priorities. There are a lot of different and changing variables in the mix and it makes committing to deadlines very difficult.
If an organisation wants to isolate developers and have them work on only build work for a single product estimation becomes much more accurate. Of course most orgs would rather less accurate estimation instead.
"Estimating work and committing to those estimates"
I agree with you that estimation has value, especially at the large scale when you are trying to budget for a multi-year project. (and at that level- the detail you think should be there simply isn't) You are also right that at some level it is a social problem, estimates and commitments are two very different things- and you don't just commit to an estimate. Estimates are predictions, statements of probability about the unknown future, hopefully based on solid data from past performance and some understanding of the work to be done. Commitments are promises to make things happen that are only partially within one's control. In many cases, an estimate is needed prior to getting the level of information that one would need to make a truly accurate prediction, so we respond with a range of dates to reflect that uncertainty, but this can be too early to make any kind of commitment.
If you estimate at the point in time where you are 50% likely to get the job done, and commit to that, half of the time you will be late, even though it is "average time" it would take to do that work. You can't take the easy way out and commit at 100% likely time, because you could be sick in the hospital tomorrow (and with a big enough project, someone eventually is). If you commit to an 90% confidence estimate, you get stuck with Parkinson's law type problems. Not to mention that those predictions with percentages are hard to base on real data...but if you make enough of them, you can actually get feedback on your accuracy.
The problem is that most audiences for estimates aren't able to deal with the uncertainty, and many givers of estimates are easily pushed into overcommitting. This is why some of the ideas popularized by the Agile movement seem to work- if you fix the time increment, it is much easier to flex on the scope, especially when it is not possible for it to be well-defined in advance. For example, there are many cases where an high confidence estimate can't be made- as is often the case in more research oriented projects, where the path to a working solution, or whether one will even be found, is unknown, and it is part of the job of the project to discover that. You are stuck with estimating how long it will take you to try approaches x, y, and z.
McConnell's classic "Software Estimation" probably says it better. See ch 1 of McConnell's book here- page one is on the commitment/target/estimation problem: http://www.stevemcconnell.com/estimation-01.pdf
I agree with you that estimation has value, especially at the large scale when you are trying to budget for a multi-year project. (and at that level- the detail you think should be there simply isn't) You are also right that at some level it is a social problem, estimates and commitments are two very different things- and you don't just commit to an estimate. Estimates are predictions, statements of probability about the unknown future, hopefully based on solid data from past performance and some understanding of the work to be done. Commitments are promises to make things happen that are only partially within one's control. In many cases, an estimate is needed prior to getting the level of information that one would need to make a truly accurate prediction, so we respond with a range of dates to reflect that uncertainty, but this can be too early to make any kind of commitment.
If you estimate at the point in time where you are 50% likely to get the job done, and commit to that, half of the time you will be late, even though it is "average time" it would take to do that work. You can't take the easy way out and commit at 100% likely time, because you could be sick in the hospital tomorrow (and with a big enough project, someone eventually is). If you commit to an 90% confidence estimate, you get stuck with Parkinson's law type problems. Not to mention that those predictions with percentages are hard to base on real data...but if you make enough of them, you can actually get feedback on your accuracy.
The problem is that most audiences for estimates aren't able to deal with the uncertainty, and many givers of estimates are easily pushed into overcommitting. This is why some of the ideas popularized by the Agile movement seem to work- if you fix the time increment, it is much easier to flex on the scope, especially when it is not possible for it to be well-defined in advance. For example, there are many cases where an high confidence estimate can't be made- as is often the case in more research oriented projects, where the path to a working solution, or whether one will even be found, is unknown, and it is part of the job of the project to discover that. You are stuck with estimating how long it will take you to try approaches x, y, and z.
McConnell's classic "Software Estimation" probably says it better. See ch 1 of McConnell's book here- page one is on the commitment/target/estimation problem: http://www.stevemcconnell.com/estimation-01.pdf
The problem is that we still don't have the tools or language to specify software projects with the same degree of precision that other engineering disciplines do. Give me a spec as complete as precise and as a blueprint for a building or a bridge and I can give you a pretty good estimate.
What I get instead is a stack of vaguely worded "user stories" and, if I'm lucky, some wireframes that will invariable wind up looking nothing like the final design.
What I get instead is a stack of vaguely worded "user stories" and, if I'm lucky, some wireframes that will invariable wind up looking nothing like the final design.
> Give me a spec as complete as precise and as a blueprint for a building or a bridge and I can give you a pretty good estimate.
I would argue that that would in fact be the source code for the program!
I know that sounds flippant, but - civil engineers and architects will correct me I'm sure - a blueprint to me is a set of pretty clear and hopefully unambiguous instructions, just not necessarily in a chronologically ordered list format. To torture the metaphor even further: the act of building the building from the blueprints is the same as running the program.
The key difference is that in the case of building a computer program the hard part is writing the instructions and in the case of building a building the hard part is execution of the instructions. Building designers have the advantage of centuries (millennia, even) of knowledge and some pretty hard limitations set by physics. Money and time aside, computer program designers are more or less limited by hardware and imagination.
I really do hope someday programmers will look back at conversations like this and think not that their counterparts of the past just needed to deal with it and be more professional but that we just didn't know any better way, yet.
I would argue that that would in fact be the source code for the program!
I know that sounds flippant, but - civil engineers and architects will correct me I'm sure - a blueprint to me is a set of pretty clear and hopefully unambiguous instructions, just not necessarily in a chronologically ordered list format. To torture the metaphor even further: the act of building the building from the blueprints is the same as running the program.
The key difference is that in the case of building a computer program the hard part is writing the instructions and in the case of building a building the hard part is execution of the instructions. Building designers have the advantage of centuries (millennia, even) of knowledge and some pretty hard limitations set by physics. Money and time aside, computer program designers are more or less limited by hardware and imagination.
I really do hope someday programmers will look back at conversations like this and think not that their counterparts of the past just needed to deal with it and be more professional but that we just didn't know any better way, yet.
There are Mathematical Limits to Software Estimation: http://scribblethink.org/Work/kcsest.pdf, http://scribblethink.org/Work/Softestim/softestim.html
In general, unless you've done something very similar before, estimation can't be done accurately. Your manager should know and accept this.
The best you can do is probably this: Repeatedly split the project into tasks, estimate the time it will take for each, until the next task takes several hours, and then do the next task. Keep track of estimates and actual times spent. Your manager should ask you to do this and keep it updated. Joel Spolsky wrote something similar here: http://www.joelonsoftware.com/articles/fog0000000245.html (I'm sure I read similar advice elsewhere, but can no longer find it.)
It should go without saying that you should include unit testing, code review, debugging, etc. in your estimates. And, of course, making and updating the time estimates.
The project, when "complete", will still usually require maintenance, so work on the project only stops when the software is no longer used.
You should never be given deadlines, as you won't know in advance what problems you might encounter, and the quality of your work (and probably your health) will suffer. This will result in a build up of technical debt, increasing maintenance costs.
It's well worth reading The Mythical Man-Month, especially if you're managing a project, or working with others on the same project.
In general, unless you've done something very similar before, estimation can't be done accurately. Your manager should know and accept this.
The best you can do is probably this: Repeatedly split the project into tasks, estimate the time it will take for each, until the next task takes several hours, and then do the next task. Keep track of estimates and actual times spent. Your manager should ask you to do this and keep it updated. Joel Spolsky wrote something similar here: http://www.joelonsoftware.com/articles/fog0000000245.html (I'm sure I read similar advice elsewhere, but can no longer find it.)
It should go without saying that you should include unit testing, code review, debugging, etc. in your estimates. And, of course, making and updating the time estimates.
The project, when "complete", will still usually require maintenance, so work on the project only stops when the software is no longer used.
You should never be given deadlines, as you won't know in advance what problems you might encounter, and the quality of your work (and probably your health) will suffer. This will result in a build up of technical debt, increasing maintenance costs.
It's well worth reading The Mythical Man-Month, especially if you're managing a project, or working with others on the same project.
The Mythical Man-Month is REQUIRED reading for anyone who might ever come close to estimating a software project in any setting. +++++1
I've found that using past data is very helpful. Look at your past projects, compare your initial estimates to the actual outcome when the project was finished. After a dozen or so projects, you should be able to formulate a percentage buffer to help you get closer on future estimates. Continually make this assessment and eventually you will find that you are getting much closer to accurate budgets/timelines.
If I have any past projects that are good indications of my current work, it means I'm stuck doing the same things over and over again.
I don't think building on your past successes and mistakes and using the raw data to improve your future endeavors should be considered stuck doing the same thing over and over again. Surely there will be some overlap of knowledge with each project you take on... Or do you only go for projects with completely foreign problems to anything you've ever worked on before?
I've never worked on two projects that did the same things other than "let people login", which I usually have settled in the project before anyone even starts talking about estimates. I've never worked on two projects with even the same reporting frameworks. Hell, only a few of them have even been in the same application framework.
I've never worked on two projects with the same team. There's as much learning about how to work with new, particular individuals as there is to learn about what the client needs.
Insomuch that anything has remained constant, it was pretty much just the underlying programming language, but even that has changed dramatically over the years, as well as opinions on the best way in which one should write it. Certainly my understanding of it has drastically improved.
And I'm not exactly sure what you're suggesting by "raw data". What sort of data? Collected and stored how? Reported on in what ways?
I've never worked on two projects with the same team. There's as much learning about how to work with new, particular individuals as there is to learn about what the client needs.
Insomuch that anything has remained constant, it was pretty much just the underlying programming language, but even that has changed dramatically over the years, as well as opinions on the best way in which one should write it. Certainly my understanding of it has drastically improved.
And I'm not exactly sure what you're suggesting by "raw data". What sort of data? Collected and stored how? Reported on in what ways?
Open article
Ctrl+f "Risk Management" "Zero results"
Ctrl+f "Risk" "Zero results"
Now, I don't work in software engineering, I work in traditional engineering, but I find the lack of any formal Risk Assessment, Management or Lessons Learned in any Project Management methodology/framework a little scary.
Now, I don't work in software engineering, I work in traditional engineering, but I find the lack of any formal Risk Assessment, Management or Lessons Learned in any Project Management methodology/framework a little scary.
I use my best judgement to estimate the time the task will take. I then double that number and report that as my estimate. For times great than one month - multiple by a time and a half instead.
One month? No, six weeks. Two hours? No, four hours.
The reality is typically somewhere between the two numbers. So the estimate remains fairly accurate and most things are finished ahead of schedule.
There are too many variables that cannot be accounted for when giving an estimate. How often am I going to get interrupted? Is there going to be legacy code that causes a bug to surface and has to be refactored? Etc.
One month? No, six weeks. Two hours? No, four hours.
The reality is typically somewhere between the two numbers. So the estimate remains fairly accurate and most things are finished ahead of schedule.
There are too many variables that cannot be accounted for when giving an estimate. How often am I going to get interrupted? Is there going to be legacy code that causes a bug to surface and has to be refactored? Etc.
Aside from starting with some nirvana fallacy at the beginning, the conclusion is basically an accidental advertisement for XP.
Other tasks I can accurately say I don't know how long they'll take. This new feature is... a couple of weeks? This new application is... a few months? I can roughly guess (estimate!) but there's a wide certainty margin.
That uncertainty is never built into the mathematical model used by project managers, which is invariably a simple addition of scalar estimates. The only common way of dealing with uncertainty is to pad the estimates (multiply by two, etc), but that's still not a statistical boundary -- it's just a scalar.
Furthermore, when adding uncertain values, they wind up not being independent, for a number of reasons -- the lessons from one feature are carried on to the next, or the next task is easier because of new existing code, or because as you approach the original deadline you invariably throw out feature requirements. So you can't necessarily just add all the uncertainty values.
So a better approach to estimation would be to formulate a (simple) model which encodes each estimate as {guess, uncertainty, independence} and then provides a function for compounding the guesses and uncertainty and independence factors in a realistic way.