The reason that far too little… flexibility is purchased—aside from a development philosophy which results in large technical as well as financial commitments very early in the game—are, as I have suggested, that its costs are typically overestimated (in time as well as in dollars), and that its benefits are typically underestimated. While lower echelon organizations sometimes underestimate the cost of program changes, my observations indicate that upper echelons almost invariable overestimate them. Often the costs of making any changes in a particular configuration are made to seem astronomical, even before a single piece of metal has been bent.
That was Burton Klein in “Policy Issues Involved in the Conduct of Military Development Programs,” Mansfield, Edwin (Ed.). (1968). Defense, Science, and Public Policy. New York, NY, W.W. Norton & Company.
Of course, this still rings true today. The idea that the you have to lock down specifications first is largely a result of industrial age realities. Most of the cost wasn’t in design, but in the raw materials and physical processes like bending metal. This is the rationale for lifecycle cost estimates — it’s too expensive to start anything until you plan everything.
If Burton Klein understood that it’s more expensive NOT to make early changes that correct design errors, then that logic is even more powerful today with digital processes. Yet the acquisition system still treats all programs as if they are major hardware systems from the 1960s.
The other reason that too little flexibility is purchased is that the current budgeting system imposes an enormous de facto discount rate on spending. A dollar today is worth an infinite number of dollars outside the FYDP. Spending today on things like modular architectures, data rights, open systems, etc. might very well reduce both eventual total cost and eventual development leadtimes, but it requires precious current funds.