The thing that I will cry from the rooftops is that we just need more uncorrelated experiments. If you spend all the money in the same way that you have been spending the money before you literally just go down the list of projects that would have been funded and instead of funding, the top five, you fund the top 10.
And so you’re just funding five more worst projects of the same type. Instead of saying this is how it should be spent, I would. Would have wanted more uncorrelated experiments.
… Now everybody’s all excited about SpaceX, but everybody thought it was a terrible idea. The NSF would never have funded it.
And let’s look at the vaccines that so many of us have. I’m blanking on her name [Katalin Karikó], but the woman who was really instrumental in creating mRNA vaccines could not get funding for years.
That was Ben Reinhardt speaking with Jordan Schneider on the China Talk podcast, DARPA and how to do R&D right. Ben also has a podcast and has written a great paper on DARPA.
The idea of uncorrelated experiments, or a diversity of paths to funding, should be one of the most important concepts in defense acquisition reform. The DoD already has a long unfunded priorities list, program requirements that were generated in the exact same process and filter through the exact same decision makers as existing programs leaders are unsatisfied with.
Adding more funding to the budget will not transform defense technology until project selection mechanism is opened up. Middle Tier acquisition efforts had a period of irregularity there, but they fundamentally were the same programs already being lined up.
The crux of the uncorrelated experiments issue is then: Who decides on which projects get funded? How does oversight keep track of them and measure performance? I think there’s a lot of wisdom to the two things that make DARPA work well. Here’s Ben:
I think that it’s a combination of empowered program managers who can basically — they don’t have to go through a bureaucracy. And at the same time it has to do with coordination that would not happen otherwise. So you actually have someone who really gets it and is technically trained with a strong hypothesis of the steps that technology needs to go through to actually become a useful thing, as opposed to just like a proof of concept.
I think there’s something to the idea that you cannot design a good solution by committee. There needs to be this responsible and empowered individual or team that can say ‘no’ to the demands of dozens of functionals. They must deliver evidence of their progress or risk losing their funding.
My general feeling is that project choice is intimate and personal. It requires anticipations of the future that cannot be perfectly articulated to third-parties. People’s anticipations are naturally divergent and create these uncorrelated experiments.
You then need a process of reputation (who has a track record) and filtering (who should lose funding). Ben mentioned that Boeing’s SLS rocket did one bench test-fire in the time SpaceX test flew four Starship prototypes. Whatever you thought of SpaceX when they started, they wasted no opportunity to quickly test and learn. Of course, it took a while for SpaceX, but his reputation (and wealth) from PayPal perhaps gave him a little extra runway before getting filtered out. And thus it should be.
Leave a Reply