Here are some slices from Annie Duke, world series of poker legend, discussing her new book on the a16z podcast, “How to Decide.”
… we want to really think about what types of decisions merit taking time, and what types of decisions merit going fast. And it just turns out that most of the decisions that you’re going to make on a daily basis are ones that you should be going fast on — much faster than you actually do. And in some ways, I think that people, sort of, have it reversed.
She asks how long you think the average person spends deciding on what food to eat, what shows to watch, and what to wear. These inconsequential decisions, regardless of the outcome, will not have any impact on you overall happiness that a day or two later. She says the average person spends between six and seven work weeks a year on just those three decisions! So people spend too much time agonizing over small decisions they should go fast on.
When you draw out decision time to perform analysis, calculate expected values and the like, you can generally increase your accuracy. But when the outcomes don’t have systemic impacts, then you should speed up decision time. There are a couple reasons for this. First, if the single outcome doesn’t have a large impact, then you’ll get a lot of attempts at it. And when you get a lot of attempts, you can explore the tradespace and gather information for the big decisions:
… when we can actually cycle these decisions really quickly — and I’m not really too worried about, like, making sure I’m making the best possible decision in terms of accuracy. What I’d rather do is get a lot of cracks — get a lot of at-bats — so that the world can start giving me information back more quickly, and I can start cycling that feedback a lot faster. Then I’m going to build much better models of the world. And what my own preferences are, and what my own goal goals are, and what my own values are, and what works and doesn’t work. Such that when I do actually make a decision that really matters, my models of the world are going to be more accurate — by having just, sort of, like, done a whole bunch of stuff really fast and not really cared whether I won or lost…
And that becomes much more important in a situation where we are at a paucity of information. And then it starts to actually close feedback loops more quickly for you, which also increases the quality of your models and information.
I think this is incredibly important for defense acquisition. So many decisions for new programs are made under high uncertainty. Decision time is lengthened to 3 or 5 or 10 years to make sure the baseline is “right” on the no-fail mission because the system will stay around for decades with a massive logistical tail. But not every acquisition decision needs to be like this. And in fact, when there is little information about success (e.g., autonomy) the best course of action is diverse action with quick feedback loops to collect the data that will be important for when that really big, impactful decision is made.
For example, Middle Tier of Acquisition restricts DoD efforts to a 5-year prototyping cycle along with cost thresholds. But what really matters is: Will this prototype be fielded? How long will it stay? What are the logistics? Is this a big consequential decision that needs time, or is it a smaller decision that we can cycle on?
A Major MTA like Mobile Protected Firepower, a big ground vehicle platform, probably needs more time but it also has tons of relevant historical data. A Non-Major MTA like Integrated Visual Augmentation System (IVAS) is smaller, depends on software, and has relatively low commitments in sustainment, so it should move faster. This despite the fact both programs in the Army are MTAs that cost about $850 million for the rapid prototyping phase.
This was recognized in the MTA policy, DoDI 5000.80. Non-Major systems like IVAS do not require an approved requirement, acquisition strategy, or cost estimate to initiate the program. A Major system is one that is greater than $115M in 1990 dollars of RDT&E or more than $540M in Procurement. Interestingly, both MPF and IVAS break that threshold.
But the real point is: when you start a prototype, should you really know the distant future of the outcomes? Do you have enough information? And finally, will oversight start adding more of the Major System requirements to Non-Major in an effort to see more certification around a decision?
[Update: note that even Non-Major MTAs still have to create a plan for when they create those requirements, cost estimates, and acquisition strategies within a couple years — they just don’t require them ahead of time.]
Here’s a little more on using certification processes to reach decisions, which is more like a warm blanket than something that leads to good decisions:
When we start to use something that feels objective, like a pros and cons list, we get that feeling of like — well, now I can have confidence that it’s a really good decision. So, one of the things that I’m very wary of — is that I think that there’s certain things that can come into a decision process that feel like it’s certifying the process. So, we end up with this combo of a decision that isn’t really better, but that we feel is much more certified.
Leave a Reply