DoD is failing on the three E’s: experimentation, evaluation, and evolution

Here is the Director of Defense Research & Engineering Harold Brown in 1962, discussing how he has responsibility for all R&E activities in DoD since the Defense Reorg. Act of 1958 empowered DDR&E:

My authority here is to approve, modify, or disapprove research and development projects and programs of the military departments and other Defense agencies, to eliminate unpromising or unneccesarily duplicative programs, and to initiate or support promising ones.

The idea that one individual (and his staff) can make tens of thousands of very detailed project decisions is the height of hubris. It is this belief in the coordinating function of a top administrator that drove so much of defense management reform in the 1950s and 60s. Unfortunately, it is rubbish on the face of it if for no other reason than the information requirements. As Friedrich Hayek wrote:

…. while it is perhaps conceivable that all theoretical knowledge might be combined in the heads of a few experts and thus made available to a single central authority, it is this knowledge of the particular, of the fleeting circumstances of the moment and of local conditions, which will never exist otherwise than dispersed among many people.

That knowledge of local conditions is not just important to businessmen responding to local prices, surpluses, and shortages, but also particulars about where inventory is, what types of design features are likely to be successful, what tastes and experiences customers may prefer. Here’s Hayek again:

…the sort of knowledge with which I have been concerned is knowledge of the kind which by its nature cannot enter into statistics and therefore cannot be conveyed to any central authority in statistical form. The statistics which such a central authority would have to use would have to be arrived at precisely by abstracting from minor differences between the things, by lumping together, as resources of one kind, items which differ as regards location, quality, and other particulars, in a way which may be very significant for the specific decision.

Making the point a bit differently is Arnold Kling:

Once we take seriously the fact that most new ideas are wrong, we can compare the market and and government as processes for sifting new ideas. From the standpoint of the three e’s, this comparison strongly favors the market.

 

When it comes to experimentation, a decentralized market is able to generate many more experiments than a centralized government. How should we address the harms that come from Facebook and Twitter? Government could turn major social media companies into regulated utilities, which would perpetuate their dominance and force all would-be competitors to fit a single regulatory mold. Or the market can continue to experiment, with entities like Clubhouse and Substack and innovations yet to be discovered, until platforms emerge that provide more satisfying experiences.

 

When it comes to evaluation, the market measures costs and benefits using the unforgiving criteria of profits and losses. If an enterprise provides less value than the resources that it uses, it will suffer losses. Only those enterprises that improve the use of resources will enjoy profits and expand.

 

In theory, government can evaluate programs according to costs and benefits. But these evaluations are rarely undertaken, and they are never tested for accuracy. There is no check against government officials who overstate benefits and understate costs, or who ignore the benefit-cost criteria altogether.

 

Finally, with government programs evolution is stymied. In the market, most new firms fail. They go out of business, their ideas having been coldly discarded by the profit-and-loss system. But government programs almost never get discarded. Whether they are helpful or harmful, with benefit-cost ratios that are high or low, they persist in perpetuity.

There’s a lot of truth there. A central DoD authority over R&D efforts would (1) limit experimentation, and indeed that was the whole point — to stamp out competition and duplication; (2) would use overly simplified metrics for evaluation, and because of their non-expertise with the particulars will make systematically wrong choices; and (3) will fail adopt more cost-effective alternatives. All three of these errors of non-market decision making are clearly apparent in DoD. There’s little doubt.

Of course, most people would say: DoD is a non-market environment! There’s just one buyer (DoD) and few sellers (defense industrial base) and there’s no way that private individuals in the economy can coordinate to supply military outputs (it is a “public good”).

That’s all true, but it doesn’t mean that a government led process cannot have the basic principle behind markets: the process of mutual adjustment. This isn’t just how markets work, but how science, fashion, and the natural world work. Individuals act according to local knowledge only, and yet through their complex interactions come to complex global solutions that could never be devised by any one person with the “most information.”

And so DoD must look to its history, when there were multiple buyers in the services competing and cooperating with one another. They had something that looked like property rights in running their various bureaus and technical services. Sellers chased profits in production and sustainment by investing private capital into weapon systems. It all looked so very disorganized, inefficient. But it was creative, fast paced, and effective.