(note: I wrote this article 28 years ago and while something have changes, these human factors in software development have mostly remained the same. The proliferation of tools and deployments may have made it worse. The promise of Vibe Coding in my opinion is not going to change this, the durations might be shorter but the problems with estimation will be the same.)
Why are estimation always so far off?
Estimating is about prediction. At the outset it is impossible to arrive at an accurate estimate.
Failure is the perception of unrealized expectation even when those expectations were unrealistic.
An estimate is successful if it is accurate. This can only be determined at the end of a project. The estimating process and the development process are inseparable. The success or failure of one can be attributed to the other.
Confusion between project sizing versus project estimating causes many estimates to be misrepresented. Unfortunately project sizing is what is often used to sell or approve a project. Managers and users remember the initial project sizing and continue to judge the project success on it. Project size is expressed in desired overall effort, cost, time to deliver and therefore usually have a high margin of error. Detail estimates are based on a work breakdown structure and never meet the initial sizing expectations. Project sizing needs to be very conservative and pessimistic to make sure that the project estimate does not exceed the initial project sizing.
Estimating unknown phases will guarantee failure due to omissions. Because the estimates will need to be increased.
The average project is underestimated by a factor of at least 2.85 (Center for Project Management).
Manual estimates are wrong more than 75% of the time, and the errors were almost always on the side of excessive optimism — that is, they significantly underestimated schedules, costs, application complexity, paper work, quality control, learning curves, and exaggerate staff experience.
Most professionals have not been trained to estimate projects, nor have they any experience with forming a project plan. Even if they know how to form a project plan one would need experience managing to a plan to make the estimates mimic reality. This is most likely the largest contributing factor to poor estimates.
The lack of formal methods of estimating make it a black art.
How to improve our estimates!
Assign the initial estimating task to the final developers, testers and implementers. This builds commitment to timely project completion. Estimators (Sales, Marketing, Consulting, Account Manager etc.) may tend to underestimate assuming that lower estimates are more likely to get the project approved and they personally need not meet the schedule. Estimators may not have any direct experience with the technology being used and may underestimate out of ignorance. User, of course, put severe downward pressure on estimates since they know that it will be used as the basis for costing the project.
Revising an estimate in the middle does not increase its accuracy nor does revising it in response to user changes. The project needs to be managed to stay within the initial estimates.
Rely on documented facts, standards, and simple formulas rather than guessing and intuition. Facts increase the accuracy of the estimates.
Delay finalizing the initial estimate until the end of a thorough study. An estimate needs to be as accurate as possible otherwise it will be subject to several revisions.
Estimate only known work and apply very conservative factors to remaining unknown work to arrive at a pessimistic estimate.
Estimates are never exact therefore state them as a range of values and always document the assumptions, tradeoffs and risk.
Anticipate and control user changes. User changes and the users lack of understanding of their own requirements are the most severe cause of inaccurate estimates. Both of these issues occur after the estimate is finalized and the project starts. Changes should be moved to subsequent projects or if absolutely necessary the users are charged extra for the changes. Estimators should thoroughly understand the user requirements for the proposed system before estimating.
Monitor the progress of the plan. Frequent check points are needed to compare progress to the plan. Monitoring indicates a commitment to stay within the estimates and acts as an incentive to the estimators and developer to consider the estimates seriously.
Evaluate proposed project progress by using independent auditors. More accurate estimates occur when independent auditors are involved. Three practices are commonly used to hide inaccurate estimates; omitting portions of the system, inaccurate reporting of hours, working disproportionately harder towards the end of the project.
Use the estimates to do performance evaluations of project personnel. Cost estimate are primarily used to staff projects, control implementation, select projects, schedule them, and quoting charges to the users. Holding estimators and developer directly accountable improves the accuracy of the estimates.
Computing management should carefully study and approve the cost estimate. A cost-benefit is required to justify the project. User should be given final approval of the estimates. Only computing management involvement will increase the accuracy of the estimates.
Don’t rely on cost estimating software for an accurate estimate. It only makes the estimating process more productive not more accurate.
Factors that produce more realistic estimates:
– Knowledge of the business application is required to estimate magnitude and complexity.
– Project scope, preventing changing requirements.
– Constraints on staff, budget, hardware, quality, legal and schedule.
– Comprehensive task list; omissions are common.
– Task dependencies.
– Labor cost
– Fixed cost
– Resource skills
– Historic data
Guesstimating effort is inaccurate:
– About 15 – 35% of a day is wasted in meetings, breaks & administrative tasks.
– Typical learning curve requires 32% of work time until mastered then its about 5% of the day.
– No more than half a day of real work can be expected on average.
– The range of work done per day is from 2.5hr untrained to 6.5hr expert level of real work.
– If the estimates are guesses it is better to use a duration driven instead of a resource driven project plan.
Use an Estimating method:
– Guesstimates are used (42%)
– Dictated deadlines (34%)
– Effort distribution over each phase and task. This is based on historical data. It determines % of total effort used at each phase and extrapolated for subsequent phases.
– Formula based using lines of code and applying weighting factors for cost drivers. Most well known is Constructive Cost Model (Cocomo).
– Function Point Analysis may be used after the requirements are document and is more accurate than Cocomo.
– Task based uses a detailed work breakdown structure to estimate effort or duration for each task and adjust for cost factors.
Reference: Zawrotny, Stan. Demystifying The Black Art of Project Estimating. Application Development Trend, July 1995.
Reference: Jayesh Prasad, Nine Management Guidelines for Better Cost Estimating, Communications of the ACM, Feb. 1992/Vol.35 No.2
Rule based Estimating
Use simple rules cautiously because they are not accurate and have a high margin of error. Simple rules are better than simply guessing but are not as good as thoroughly analyzing the project for detailed task.
Rules should not be used for contracts or bids.
Rules can be used as a “sanity check” or in place of wild guesses.
The application of rules assumes the use of Function Point 3.0.
1. Source Code
One function point = 100 logical source code statements for a typical 3gl language. This rule needs to be adjusted for OO languages, application generators or 4gl languages.
2. Paper Deliverable
Page count for documentation = raise the number of function points to the 1.15 power. One page of documentation = approximately 400 English words.
3. Creeping Requirements
Creeping user requirements will grow at an average rate of 1% per month over the entire development schedule. The use of cost per function point is becoming prevalent with a sliding scale that becomes more expensive further into the project.
4. Test Cases
Number of test cases created = raise function point to the 1.2 power. Assume that each test case will be executed about four to six times during software development.
5. Defect Potentials
Number of defects = raise function points to the 1.25 power for new software and 1.27 for enhanced software.
6. Defect Removal Efficiency
Each software review, inspection, or test step will find and remove 30% of the bugs present. Six to 12 consecutive defect removal cycles are needed to achieve high quality levels.
7. Software Schedule
Approximate development schedule in calendar month = raise function points to the 0.4 power. This calc interval between start of requirements until first delivery to a client.
8. Staffing
Number of development personnel = function points divided by 150.
Number of maintenance personnel = function point divided by 500.
Application life expectancy = raise function point to the 0.25 power
Staff month = 22 working days per month with 6 productive hours each day or 132 work hours per month.
10. Development effort
Staff month of efforts = development months times the number of personnel.
Estimating factors:
- Function point
- duration
- resource skill level
- Team dynamics
- learning curve
- Spec, design, code, test, review,
- Size
- Tools
- complexity iterations, acceptance, release, system setup config, training, documentation
- Confidence interval,
- Risk
- external interface,
- Political factor,
- Environment productivity factor
- Hardware constraints
- User’s understanding of IT
- Volatility of requirements
- Productivity declines as size increases.
- reference: Computer, Software Challenges, Software estimating rules of thumb, Capers Jones, Software Productivity Research Inc.
Ray Garcia, original doc written in Oct 1997 while CTO at Clearcross.