Problem: DoD established use of EVM as a requirement for periodically measuring linear programs with firm baselines established prior to starting development. EVM is not well suited as a measurement tool in an Agile environment, which is dynamic by design…
Given the dynamic nature of Agile, implementing a batch-oriented EVM system has limited value in an Agile environment. By its nature, Agile provides dynamic and ongoing feedback to stakeholders participating on development teams. The opposite is true for EVM techniques, which take a static measurement at a point in time. Legacy accounting and program management tracking systems can only accrue performance data once or twice a month. Today, timekeeping systems are linked to a cost system that updates weekly.
Another substantial shortcoming of EVM is that it does not measure product quality. A program could perform ahead of schedule and under cost according to EVM metrics, but deliver a capability that is unusable by the customer.
That was from a very on point Recommendation 19 of the Report of the Advisory Panel on Streamlining and Codifying Acquisition Regulations, Jan. 2018. Here’s my take on how EVMS is incompatible with Agile.
The backlash to EVMS seems to be largely based on its association with waterfall — yet even in valid waterfall environments, EVMS is an unhelpful encumbrance. EVMS had at its root that the mistaken belief that the expenditure of money represents the cost. The real cost is the next-best thing that could have been tried, but wasn’t. It is through the EVMS rituals of planning, tracking, and assigning money outlays that managers lose sight of the path not taken — the opportunity cost of their actions.
The sentence below references PMI guide to EVM, perhaps not the most unbaised source:
EVM, an important management tool for DoD for the last several decades, is also used in commercial industry and advocated for by organizations such as the Project Management Institute (PMI).
I’d like to know who runs a certifiable EVMS in the purely commercial world.
One last note. It’s interesting that the backlash to waterfall has slipped over to EVMS and not the program budget (PPBE process). However, it is no coincidence that both were instituted around the same time (EVMS was rolled out in 1962, PPBS in 1961 — both had their roots from years earlier however). At their core is a static baseline created up front from which all progress is measured. All knowledge is assumed inherent in the plan.
I’d go even further, Eric — EVM is poorly suited to software development in general. EVM requires that you always have a good idea how much work you have accomplished and how much is left to do. This is never true of any but the smallest software projects. (It is a long-standing joke to say that a software project is “90% complete, and has been for four years.”) Add in the fact that development never ends for any software system that is still being used, and you can see the problem.
Agile just makes it impossible to pretend any more that you know how many labor hours of effort it will take to be “finished”. On the other hand, I suspect that it is both possible and useful to apply EVM specifically to the phase of the project that goes from initiation to delivery of the Minimum Viable Product. The MVP should (if you do it right) have fixed and concrete requirements that can be expressed as function points or feature points and performance specifications, and standard effort estimating techniques can be applied to those. Estimating remaining work as you go is tricky, but can be done through a combination of completed function points, code growth models, and integration effort estimation (e.g. using something like COSYSMO).
Is the resulting information worth that much effort? For very large projects, such as a new GPS ground control system, I think it would be.
Good points, David. I find it interesting that, after all these years (remember, agile software development isn’t new), the community is still struggling with how much of the traditional project management rigor (say as espoused in the PMBoK) applies in an agile environment and how it applies.
Eric, I don’t know that I agree that EVMS is an “unhelpful encumbrance”. If the work packages that are the root of EVM analysis are created correctly – i.e. they represent a subset of the project work that has a definitive scope that can be evaluated and measured with appropriate quality standards – then, EVM provides a legitimate analysis of the work that has been completed compared to the plan. EVM is an indicator that the project manager should integrate with all the other data available to get a sight-picture of the status of the project in order to make decisions to positively affect the future.
I would be interested in any responses you get to the question about people using EVMS in the commercial world. My understanding is that it used pretty extensively in construction projects.
Bottom line: I don’t think EVM is the useless tool it is sometimes characterized as; I think it has been haphazardly implemented – or only implemented as a box-checker.
The problem with agile is it gives teams an excuse for not providing performance or ECDs. Nobody knows if/when the program will meet the customer’s major milestones until it is too late. Then, the program gets further and further behind each week/month. EVMS done properly, ensures that you have predictability in meeting program deliverables which allows for adjustments well before a risk is realized. In my experience, agile teams hide behind a curtain of unpredictability and find excuses not to be accountable for meeting deliveries. That does NOT mean that agile is not effective for iterative type of scope, it certainly is for SW. I would say that iterative work could be represented as an enhanced version of LOE type activity. What bothers me is when we go overboard and implement agile on effort that it is clearly not compatible; such as production work.