Perhaps the best way to explain some variations of a practical fly-before-you buy policy is to take several examples. The AX program is one which is truly fly-before-you-buy, for in this program we have two competing firms. Each will produce development models of the AX, and these development models will be flown and compared before a production contractor is selected…
A second approach, using the fly-before-you-buy principle, is the B-1 program. In this case it was too expensive to develop two new bombers, and test them against each other. The contractor will build three prototypes and we will thoroughly test those before a production decision is made.
… the B-1 project manager has done an excellent job in cutting out unnecessary paperwork. This is a case where the project manager has adequate authority, good communication with the top decision-making people in the Air Force, and the right kind of a contract. His management of this program has reduced the cost of this development by several hundred million dollars.
That was former DepSecDef David Packard discussing his fly-before-you-buy principle before the Appropriations Committee on March 18, 1971. The AX program, of course, flew the YA-9 vs. the YA-10 and led to the beloved A-10 Warthog we have today.
The B-1 prototype began in June 1970, and reached first flight in December 1974. Another major development finished in less than 5 years. It got underway right at the end of the swing-wing design era, and perhaps led to it’s high price tag. The B-1 program was ultimately cancelled during the Carter administration, only to make a comeback as the B-1B Lancer during Reagan. The bombers are inching closer into retirement.
Packard criticized the F-14 for not having been a competitive development, like the one he was backing for the Air Force in the Lightweight Fighter Program, the YF-16 vs. the YF-17.
Of course, it’s also important that the prototype be close enough to the operational system that you actually learn enough in the prototype competition to burn down risk. The F-35 is notoriously dissimilar to its prototype; the competition generated enough information to inform the decision about which design family to prefer, but could in no way be considered “fly before you buy”.
Of course, as Packard originally proposed fly-before-you-buy, he meant that the thing you buy should already be operationally capable as-is. That’s vanishingly rare these days.
Operational realism seems to be a problem, same issue for the F-22 competition where the only real aspect prototyped was super-cruise. While it’s probably best to prototype only one or two new features at first, we should probably expect multiple prototyping rounds rather than one prototype and one full-scale development.