Here’s Secretary of the Air Force Frank Kendall quoted in a recent Space News article, speaking about problems in current acquisition:
“Overall, we start more programs than we can afford, and we don’t prioritize the most promising ones early so that we can ensure they cross the valley of death to production and fielding,” Kendall said. “We still have too much bureaucracy.” Over the past 30 years, he said, “the U.S. has not stood still, but we have not moved fast enough.”
There’s a tension in that statement. On the one hand, DoD hasn’t moved fast enough on transitioning new weapons and there’s too much bureaucracy. On the other hand, DoD isn’t doing enough analysis to prioritize the right programs and make sure that they get into fielded to the troops.
Certainly, more up-front analysis to get the perfect program slows things down and is a major source of bureaucracy. It takes 1-3 years to get a requirement, another 2 years and 49 major documents to get through a milestone, and 2-5 years to get a program funded. All of this lead-time is analysis and decision. It’s hard to say DoD doesn’t analyze enough.
Role of Analysis
Here’s more on that from Kendall in a segment where he spoke with Vago Muradian on the Defense & Aerospace daily podcast.
I’m equally focused on speed as I am on getting decisions right. Then there’s an element of tenacity and commitment to the things you decide to do so even if it doesn’t go as well as you’d hope you stick with it and get it fielded. It’s always a balancing act to do these things. The thing I have not seen enough of, quite frankly, in the past few years is the effort to make sure the decisions are sound.
I would argue that spending a years trying to define the exact right requirement, materiel solution, and acquisition plan has been an enduring source of failure since the mantra of weapon systems analysis has taken over the DoD in the 1960s. This was called Planning-Programming-Budgeting (PPB) and still exists today under the PPBE process. As scholar Allen Schick noted in 1971:
Expressing a widely held view, one of PPB’s leading exponents has termed systematic analysis the “heart and soul of PPB”…. During the early 1960’s, McNamara’s men used their analytic technologies to override military judgment and to make crucial decisions, often on the basis of paper studies of weapons systems and defense requirements. Confidence in the analysts and their analyses has been impaired by TFX, cost overruns, Vietnam, and other disappointments. Under Laird, analysis has been stripped of its pivotal position, even though PPB (in revised form) has been retained.
It is painfully clear that analysis fails for two reasons: it packages prediction and optimization. First, analysis cannot solve uncertainty. Second, it cannot resolve to “criteria” problem — or selection of the units of cost-effectiveness. Even if we had a crystal ball of the future, are we maximizing cost-per-pound of airframe, or cost-per-payload of JDAMs, or cost-per-sortie, or something else? Selection of the criteria is overwhelmingly important, but modern systems could have dozens or hundreds of measures of effectiveness. They cannot be compared for the optimization.
Moreover, once the bureaucracy has made a decision, to put your head down and stick with it despite enormous errors coming to light is the way DoD got into the F-35 and numerous other “acquisition malpractices.” These over-analyzed programs inevitably become too brittle and expensive to operate. Kendall’s strategy here is to sink good money after bad. And it ignores best practices emerging from commercial industry that focus on modularity, iteration, speed, and competition — not just in software and data, but even in deep tech hardware. Adaptation is superior to executing the optimal plan.
Centralization of Decisions
Back to Kendall:
When I came back to government in 2010, I noticed we were making requirements decisions essentially by intuition. In my Cold War experience, we need analysis to support decisions — to pay attention to that analysis. It should help guide us. Professional judgment is of value, but it shouldn’t be the only thing we’re focused on. We’re dealing with a lot of things difficult to understand intuitively without a lot of help. A lot of the work on the imperatives I’ve started is about doing that analysis to make sure we’re making good decisions.
If something is difficult to deal with intuitively, it doesn’t mean the problem in amenable to linear programming and other forms of analysis. Indeed, most real-world problems are highly complex and nonlinear in nature, meaning we cannot adequately model them in advance to provide the right answer. Hypersonics is an obvious area that requires empirical tests. Related to that, here’s Kendall on hypersonics:
“I rethink all of our programs all the time,” [SecAF Frank Kendall] said when asked whether he’s considering altering the Air Force’s approach. But hypersonic projects would likely continue “in one form or another,” he added.
There is something of a great-man theory to the way DoD acquisition is set up. Basically, the secretary of an armed service and the top military officer are the only point at which decisions are integrated. They are expected to weigh judgment on hundreds or thousands of complex programs. It reminds me of the “active” management philosophy of Robert McNamara: “McNamara’s strong centralization only works if the secretary is willing to personally make a huge number of budget decisions, which have been estimated to total about 700 per year during his tenure.”
Meanwhile, the context for even a single budget decisions (because it is linked to technical programs) can take several years to get up to speed on and really appreciate. Simplified analysis associated with requirements, an analysis of alternative, and so forth, are superficial at best and hide the real risks/opportunities.
Back to Kendall’s conversation with Vago, you can get an idea of how complex decisions are centralized in the PPBE funding process:
There are a few areas such as the uncrewed combat platforms where the technology is making it pretty clear we have an opportunity to do something if we can sort out exactly what that looks like and move forward on it. Other areas, it’s not as obvious what the right thing is. So we’re going to make sure to do those tradeoffs. Do that effectiveness analysis — operations research using an older term — and head down the right path.
One of the things General Brown and I in particular — and General Raymond and I will do this as well — is to take a look at the things we started and go through them to understand the things that are high priority, we should definitely move forward with. Other things that, even if they’re successful, we probably aren’t going to buy them, so we can shift those resources to something else. Then things in between, where we need to see progress and assess them later on. We need to make better, earlier decisions about the things we’re interested in and focus on the ones that will make the greatest difference.
One issue is that DoD gets leadership change every 2-5 or so years. These changes in priorities chosen by a leader take time to filter through, and by the time they are in execution there’s a new leader who has their own priorities. The result is that no body is in the place to plan.
The opposite of central planning is not no planning. It is delegating planning to individuals in the right places, and having a strong evaluation/filter mechanism so that the right plans get scaled.
Indeed, Kendall is actually benefitting from the optionality provided to him by the experimentation fostered under his predecessors. That doesn’t mean select the best and kill off the rest. That would close off optionality to future leaders. But it also doesn’t mean letting everything survive. Experimentation, evaluation, and evolution are key to progress, and DoD doesn’t have the filtering mechanism of private markets. But there should be incentives at lower levels to stop doing things that don’t work. The first step is to let them pivot to something else.
Problem of Contingencies
Another problem with this line of thinking is that selecting the defense programs that will “make the greatest difference” is literally a statement about judgment, not analysis. Alternative actions will have extensively overlapping outcome distributions due to uncertainty. A decision-maker usually cannot “maximize,” but only choose between outcome distributions based on preference.
There will be multiple contingencies in strategic threats (e.g., land war in Europe, fight against global terrorism, stopping a crossing of Taiwan straits), technologies (e.g., success of quantum, autonomy, hypersonics), and CONOPS (e.g., human-machine teaming, JADC2, Agile Combat Employment, Distributed Maritime Ops).
If you selected the “most likely” contingency that makes “the biggest difference” then you open yourself to massive risks that you’re unprepared if a different contingency becomes the reality. If anything, starting more programs than DoD ends up fielding is the exact right process. Here’s a small part from famed economist Armen Alchian who ripped apart DoD’s reliance on analysis in 1952 and 1954:
For some problems, great gains will come from unique binding choices resulting from systems analyses; for others the gain will come from diversity of actions… In what situations is the latter principle of diversity preferable? And in what situation is the former appropriate? Do systems analyses help us to answer these questions? Does it help us select the diverse or unique actions?
… We, therefore, must recommend the development of a menu of several alternative weapons—guaranteeing that ignorant or malevolent critics will be able to show that a large majority of them were ‘useless’ and ‘wasted’ millions of dollars—but assuring ourselves flexibility in order to have safety and economy with optimal weapons in actual use.
Platform-centric decisions
Here’s one last quote from Kendall’s discussion with Vago:
Cyber is one of those things — and I would add electronic warfare, munitions, and base resilience might be another one — that tend to get neglected in our budgetary deliberations relative to buying platforms. Those are all critical things for successful warfighting. We are reviewing all of those areas.
I think these cross-cutting, or enabling, technologies get short shrift because they are not so easily amendable to analysis. They get baked into a larger “weapon system platform” which undergoes a full-up analysis, and then a suite of brand new subsystems are designed to match that platform requirement rather than selecting mature subsystems independently developed. It’s easier to put everything into one program, run that one analysis and decision chain, rather than doing it dozens of times and integrating later. (Base resilience is a little bit different — I’d link it closer to the acquisition system’s bias against infrastructure and enterprise tooling.)
Conclusion
To conclude, I’d say Frank Kendall is as knowledgeable a Secretary of the Air Force as is possible to find. His judgment might even be superior to anyone else on matters of weapon systems. However, even the greatest of individuals has but the smallest fraction of the knowledge required to make decisions in one of the most complex organizations in the world.
The challenge is creating a system where individuals, each of whom is unable to assess the global situation, can still work together in a coordinated way. That is the whole enterprise of this blog and PPBE reform. I think it is nicely represented in this post which quotes Michael Polanyi:
It is clear that only fellow scientists working in closely related fields are competent to exercise direct authority over each other; but their personal fields will form chains of overlapping neighborhoods extending over the entire range of science…
All that I have said here about the workings of mutual adjustment and mutual authority… suggests a way by which resources can be rationally distributed between any rival purposes that cannot be valued in terms of money. All cases of public expenditure service collective interests are of this kind. This is, I believe, how the claims of a thousand government departments can be fairly rationally adjudicated, although no single person can know closely more than a tiny fraction of them.
Leave a Reply