The DoD does not have an effective process for doing a thorough exploration of alternative technologies and concepts. Exploration is usually done only as a part of the build process, because exploration is expensive and adequate funds are not made available unless a decision to build has been made. The build process, however, tends to shut off exploration, partly to save money and partly to make sure that no new idea will arise to interfere with decisions already made.
Much of the difficulty now being experienced with acquisition stems from setting detailed requirements before adequate exploration has taken place. Lacking the discipline that real knowledge brings to what is doable and how best to do it, these requirements are usually overstated, leading to the delays, overruns, and performance shortfalls that are so common.
Perhaps even more serious, the build process fails to take advantage of new ideas and possibilities, both technical and operational. Serious consideration should be given to revising this procedure. We should explore first and then ask whether a buildable system is worth the cost rather than determining what is required first and then struggling to build it, whatever the cost.
This dichotomy is evident in the SDI [strategic defense initiative] program. Although the SDI is supposed to be a research and development program, the build model has been applied and has led to fixing the system design too early before adequate exploration of alternative technologies was completed. The system has been divided into components, component descriptions have been set in concrete (or at least in molasses), and innovation has been thwarted despite efforts to encourage.
Serious consideration should be given to applying the exploratory design approach (of which Brilliant Pebbles is an example) across the SDI, to both the system and the elements. The same approach should be considered for other DOD programs as well. The exploratory approach involves the design by a capable organization with technical depth and experiment resources, operating under a minimum of procedural restraints, and with system specifications not yet fixed.
That was from a Dec. 1989 Defense Sciences Board report, “Report of the Defense Science Board on SDIO Brilliant Pebbles,” quoted in Donald Baucom’s excellent chapter in Providing the Means of War called “The Strategic Defense Initiative and Acquisition Reform,” pp. 236-37.
This is (in my opinion) the biggest problem that the Section 809 Panel missed entirely. The AoA process is broken; instead of doing an honest resource-informed assessment of the possible courses of action — including gaining more information by prototyping — the Services invariably leap into a full-fledged SED program to invent a new magic bullet. They then spend billions per year trying to mature the immature technologies, narrowing the scope and shedding requirements until they have an exquisite point solution that isn’t what they want and is a dead end for future capabilities. They do this in part because starting major programs is (politically) hard, so you do it whenever you can, regardless of whether that’s the right time to be starting that program.