Agile Milestones

I’ve been engaged in a lively debate at work about the role of milestones in an agile environment.  How and when should they be used?  Is the definition of a milestone in an agile environment any different than in a traditional project?  From the outset, I want to go on record as saying milestones are important, but doing them incorrectly can lead us down a slippery slope away from agility.

We take the concept of a milestone from the numbered markers placed along roads, which provided reference points to travelers, and date back to the Roman Empire.  It is a convenient metaphor to apply this concept to a project management, but when applied to a software development effort, the metaphor sometimes tends to break down.  In the Project Management Book of Knowledge (PMBOK), a milestone is defined as “a significant point or event in the project” (p. 136).  This definition doesn’t really help us in the discussion, though, as further interpretation of the definition often leads to disagreement and confusion.  The PMBOK gives us a little more information, though, when it further describes one attribute of a milestone: whether it is mandatory (i.e. contractual) or optional.  I think we can use this key differentiator to help guide our discussion.

We must first realize that a traditional, waterfall view of projects, which have an “on time/ in budget/manage to the plan” approach, requires deterministic activities to be successful.  A great deal of modern project management still successfully uses this approach to these types of projects; construction is a good example of this.  We can see this in the approach to risk management.  Risk management in construction is fairly well understood – for example, actuarial models exist for most types of construction projects.  Plans for deterministic projects are generally static, the focus is on controlling cost, and the amount of time it will take to complete the project is, for the most part, predictable.

However, software development is a non-deterministic activity: it is a process of change, invention and design, not one of manufacturing.  It has “unknown unknowns” at the start of a project, some of which can never be fully understood until project completion.  Edison filled more than 40,000 pages of notes and tested over 1600 materials before he had a light bulb that would last just 40 hours.  Tesla had been trying to solve the puzzle of a rotating magnetic field for years.  One day while walking through a park in Budapest, reciting Faust, the solution presented itself to him in a blinding flash of insight.  Imagine asking Edison if his project to invent the light bulb were “on time”, or asking Tesla when he would complete the invention of the AC induction motor.  They would have probably scoffed at you.

Fortunately, software development is not completely non-deterministic as it is not purely invention.  We don’t have to re-invent databases, recursion or Unix; we stand on the shoulders of giants such as Jim Gray, Edsger Dijkstra and Dennis Ritchie for these and countless other advancements in computer science and engineering.  We can leverage this knowledge and re-use tools and design patterns when developing new software, and so not have to completely re-invent the wheel every time we develop a new system.  Good developers know a broad array of these design patterns, know how to apply the right tools and technologies to solve business problems, and more broadly align IT with business needs – one of the primary functions of the CIO (or in some organizations, the CTO).

However, software development projects remain at least partially non-deterministic, and they cannot be fully managed with traditional project management methods.  Success must often be defined as delivering as much value as possible in a reasonable timeframe.  Plans for non-deterministic projects are volatile, the focus must be on making tradeoffs between value and cost, and the amount of time it will take to complete the project is either unknown or only known within some degree of uncertainty.

Are milestones in software projects then useless?  I don’t think so.  To understand whether a milestone has value or not, as my friend Kenny Rubin said, we need to look at the economics.  Ask: what is the economic value of meeting the milestone?  What is the cost of being early or late?  It might be helpful to look at two different examples.

Example 1: Milestones with time-based economic impact

In the first example, imagine that a team needs to complete features in a software project for a demonstration at an upcoming annual convention.  If the software is late, the company will lose an opportunity to demonstrate new product features and corporate technology progress to a large audience.  Conversely, if the product is completed too early, then there is also value lost in terms of opportunity cost.  Figure 1 illustrates a simple value curve based on time.  One can of course argue the shape and slope of the curves based on a number of factors: a contract might specify that if a deliverable is late, penalties may be incurred; as well, having the software completed late may still add value (just not as much as if it were delivered on time), but the point is that maximum ROI is achieved by (just) meeting the milestone.



Figure 1: milestone based on economic value

Example 2: Milestones with no time-based economic impact

In the second example, let’s assume an operations team would like to have a tool developed to automate some part of the IT operational process.  Not having the tool is costing the company operational expenses in terms of manual processes that need to be run by hand every month.  In this example, the company is continuously paying the operational cost of the manual process, which for the sake of argument we can assume does not change over the short term.  Figure 2 illustrates this cost curve.  Given that there are probably many other projects queued up on which the team could work, and the team executes them based on the ROI of each project, as long as the development team is reliably producing value at a good, sustainable rate, what is the purpose of having milestones at all for this type of work?

Figure 2:  Milestone ROI with no time-based variability

There are many old-school arguments against this, but the most typical goes like this:  “How do we know if a team is meeting its commitments?”  This sounds good on the surface, but if we analyze it deeply we find an insidious belief: that meeting constraints is more important than delivering value.  The undercurrent of this belief still pervades a large part of the software industry, and agilists must constantly be on their guard against it, and work to educate others about it.  This is not easy; I have struggled to do this for over a decade.  I have not always been successful.

There is a simple reason this thinking is insidious: incentives work.  If the primary incentive for a software development team to meet delivery dates, then they will do their very best to make sure those dates are met.  But the upshot is that, in their quest to lower risk and do everything possible to meet dates, several things happen.  First, estimates become inflated.  What would have been estimated at 2 days now might take two weeks.  Let’s be clear: developers aren’t lying or being underhanded when they do this - they are simply reflecting the non-determinism of their craft back into their estimates in an attempt to meet the success criteria given to them.  Second, Parkinson’s Law starts to operate: “work expands so as to fill the time available for its completion.”  Focusing on constraints instead of value will cause developers who complete tasks early to continue to add features, bells and whistles until time runs out.  We call this gold plating.  Again, developers aren’t being underhanded when this happens.  Who wouldn’t want to say “not only did I get the work done on time, but I went the extra mile and added X!”

When we incent value instead of constraints, this does not happen.  Developers give the best estimates they can and strive to deliver as much value as possible in a given time frame.  The difficulty, of course, is measuring value.  Admittedly, it is much easier to look at a calendar and ask whether a team met a date than it is to do the really hard work of measuring how much value was delivered.  Personally, I think it’s worth it.  Business members of teams need to participate in the process, roll up their sleeves and do the hard work of calculating business ROI.