Victor Deal talks with me on the role of contracting, how the price of a screw can be worth the tank it goes on, the conditions for failing smart and failing fast, how a brick can be considered innovative, issues around the technological “valley of death,” whether anyone has end-to-end accountability for a weapons program, and much more.
That is all from the first Acquisition Talk podcast interview. Download it on iTunes, Google, Stitcher, or any of your favorite podcast apps! You can also find it on SoundCloud or right here!
The next episodes will be with Rick Whittle on the V-22 tiltrotor project development history, and Mark Mandeles on technological change and organizational design. Please reach out if you have interview recommendations.
In the first episode, Victor teaches us about defense initiatives to stimulate collaborative innovation with commercial firms. As an Acquisition Innovation Advocate, he played a central part in rolling out contracting approaches for the DOD, including Commercial Solutions Openings (CSOs) and Other Transactions Authorities (OTAs). While OTAs allow the Government to by-pass contracting regulations when it tries to buy innovation, CSOs provide the Government a better way to do market research and proposal evaluation using merit-based selection. Here is Victor:
It’s only in the world of research and development is there this idea of a merit-based selection. If you imagine, previously, the paradigm was, I will either buy a technology based on the lowest-priced technically acceptable, which seems anathema to this idea of cutting-edge technology. The lowest priced, yet, the most cutting-edge technology.
On the other end, I’ll do a best-value trade-off… but the problem is it’s research and development, and no one knows what will pan out to be the better technology.
Merit-based selection is different because you do not need to evaluate cost-realism. Usually contract source selections need to follow many processes. Source selection boards must choose a proposal based on the articulated information connecting cost, schedule, and technical attributes. They require each item to have a basis of estimate, usually from historical data.
But in R&D, we find that such articulated information is lacking, and instead, decisions depend on inarticulate knowledge about the promise of the future. Here is Victor again:
Much like venture capital firms who are looking at new technologies, they value companies not based upon their current revenue streams of today but what they hope will be the revenue stream in the future. Now, for the Department of Defense, they’re not evaluating revenue streams, they’re evaluating capability, lethality, efficiency, something that’s better tomorrow than it is today.
To try to pin a dollar value on paper for a technology that has not yet been delivered makes it really difficult to pin a cost realism number on it. Hence why the merit based approach says, I’m not even going to pretend to know the cost. I’m going to base it on the best merit. And when the merit changes, because we live in a dynamic world with new technologies coming on-board every day, I’m going to pass the merit baton to the next idea.
Unlike in buying commodities, R&D is unique because neither the customer nor the supplier really knows at the outset what the requirements and technical solutions might be. R&D is a search of the unknown.
But the traditional contract process assumes that the Government can fully state its requirements upfront at the Request for Information [RFI] stage. After getting some feedback, they solicit proposals, and must select a single contractor’s proposal without modification. Here is Phillip Cooper on the traditional contract process:
One of the peculiarities of the government contract is that it is a contract of adhesion. The contracting officer writes the document and the contractor is left with a take-it-or-leave-it situation.
But CSOs are different. They provide a “nonlinear” option. As Victor tells us:
This approach obviates the use of RFIs or solicitations because instead of both sides being inefficient in describing what they want to sell or what they want to buy, the two can marry up and work on what they want to build together…
From what I gather, in a CSO, you basically have this long open period of a very broad scope, allowing contractors to submit a 1-2 page white paper whenever they have a good solution. Then they can get the pitch and potentially some work from the Government, but that happens as more of a back-and-forth, almost negotiated, but within a competitive, open, CSO environment.
Rather than, “here’s my requirements, everyone give me your proposals and I’ll pick the best one. Done.” It’s more of a “there’s lots of options and technologies going on, everyone, give me what you got when you feel ready, and we will select not just one, but potentially many promising solutions, and provide iterative guidance or feedback.”
But commercial solutions openings are not designed to solve all problems of weapon systems R&D:
Victor: When you compare it to weapon systems, you really come down to a very small number of companies that can manage a billion dollars, and CSOs will not likely be an appropriate solicitation method for something so large. But a CSO would be very appropriate for things that hang off that technology…
Eric: This sounds to me a little like the platform methodology going on in Silicon Valley. When you said CSOs are not a correct vehicle for a major weapon system, I’m thinking, OK, so CSOs might not be the best vehicle for a major platform like the ship system or the aircraft/airframe system itself, but then CSOs might do very well to provide applications or components that rapidly upgrade that platform and bring in some non-traditional vendors. Do you care to comment on the “platform,” and how it is seen in the Department of Defense versus Silicon Valley?
Victor: That completely gels with what I had envisioned for how CSOs would be used…
One of the things about platform design is that the platform owner not try to take too much of the value, and open up its architecture to applications from new vendors. So on a ship system, you can use CSOs to upgrade the fire-control system, the radar, the ordnance, and so forth.
I compared this to Amazon Web Services, which doesn’t go around soliciting requirements for new applications on its platform. The platform is made more valuable by allowing others to come in and compete on its apps. I suggested that an equivalent might be unsolicited proposals to the Government for a new component on the weapon platform. It turns out that CSOs in effect give the Government this capability but avoids some of the legal problems surrounding unsolicited proposals. Victor wisely responds that:
Unsolicited proposals are very rare. Partly it’s because when they submit a proposal that has not been solicited, they take a major risk. One, when they approach a customer with an idea, they don’t know what the customer will do with it.
He later told me that if a contractor gives an unsolicited proposal, the Government might then take that innovative idea and solicit proposals from everyone else based on it. An unsolicited proposal might be an invitation to have your idea shared with industry so that the Government can see what will happen to the price.
For example, Lockheed gave the Government an unsolicited proposal for a lightweight fighter, which DepSecDef David Packard said, Alright, let’s have a lightweight fighter competition and select at least two proposals for a prototype fly-off. The winners were General Dynamics (YF-16) and Northrop (YF-17, later F-18). Lockheed didn’t even get any work out of the competition their unsolicited proposal made possible in the first place. It isn’t clear that John Boyd and the fighter mafia would have got the lightweight program off the ground themselves.
CSOs, on the other hand, mitigate that risk of spillover.
Here is Victor on some of the intent of CSOs:
If the acquisition model mimicked what was going on in the engineering world of modularity, and the technologies or the concepts exploration process was chunked into smaller bites, then maybe we would have a better chance in reaching out to industry to bring them on board.
In many was, the proliferation of other transactions authority and commercial solutions openings is an acknowledgement of a capitulation where the federal government admits it is no longer the major decision maker in the direction of research and development. And instead, in industry — since the 1960s, 70s, and 80s — has started to invest more of their own private funds into the direction of technology, and the federal government… is coming to the table ready to do business more like industry does.
We come back to the idea of a platform and component applications, of a modular architecture. Now, this concept seemed to have largely failed in the case of the Littoral Combat Ship, but there are many promising features. For example, this idea of chunking programs into “smaller bites.” The primes already chunk a program into many smaller bites in the technical space through a huge network of subcontracting. But the problem is that the Government takes it as a package, and it hasn’t been chunked it into smaller bites over time (except IDIQs).
Usually, a major development contract simultaneously outlines major technical components and backs up plans for design, build, integration, and test. That allows it to be put under one contract with a lead-systems integrator. But if you admit that first, you don’t know what the proper specifications of the technical components should be, and second, that you don’t know how the technologies will mature, then you should partition the task into smaller bites.
This was exactly the point of Noble laureate Oliver Williamson on his excellent paper on defense contracting. Rather than add more regulations and oversight, or devise incentive schemes, a better way to reducing uncertainty is to partition contracts into smaller, better defined, pieces of work. That also sharpens the contract requirements, minimizes discretion, and reduces transaction costs.
One issue is that chunking a program into smaller pieces means that you are increasing the engineering coordination led by the Government. The DOD program offices do not have the technical knowledge they once had in the days of in-house technical services and bureaus, when the Government performed major development work itself. It brings us back to the problem of the proper role of the program manager, who in a partitioned task environment, would require a high-level of technical knowledge.
Victor tells us that CSOs actually take R&D decisions out of the hands of the Government. To me, this means maximizing the discretion of both the customer and supplier, rather than Oliver Williamson’s goal of minimizing discretion. To me, it implies more of a relational form of contracting as opposed to what Williamson was getting at, a more tightly-coupled contract that minimizes ambiguity.
Which brings us to another interesting part of the conversation:
Eric: What’s the role of trust in contracting, and culture in acquisition more generally?
Victor: The ideal form of contracts is one based only on trust, where there is no written document… The Federal Acquisition Regulation [FAR] is such a massive book because of all the lessons learned for when certain aspects of an agreement are not articulated, and so, over time it adds all these barnacles that really are lessons learned. If you have no trust, you have the FAR end of the spectrum, which is massive regulation. If there’s a high degree of trust, then you have a hand-shake…
And culture, culture is the hardest aspect of any organization to change…
Victor then tells us that the acquisition system is a “complicated business.” He says that
Having healthy dose of what came before, and understanding the pitfalls, ensures that the new generation does not make the same mistakes. Which is why when I saw your website and read through all the articles you’ve posted looking back over the last 50 years of acquisition initiatives, it was very heartening to see your respect for the institution. Not that there are any answers that anyone can take and formulate a “silver bullet,” but it starts by understanding what came before and taking the best out of that and making the adaptations that need to occur in order to improve.
It was gratifying to hear that from Victor. I strongly agree that problems of defense acquisition are hard ones of culture, institutions, and history in addition to quantitative problems such as found in systems analyses.
Here are some other interesting points from Victor. First on regulations:
I would never advocate the demolition of all the lessons learned, all the case law, that created today’s regulations. There’s definitely a time and place for that. At the same time, the use of OTAs is an acknowledgement that, in order to grasp new technologies, business as usual will likely give us the same results.
And second on the role of paper justifications:
It comes down to, what is the relative worth of that paper. Is it success 100% of the time, which that paper seems to have achieved, versus success 90% or less. And that becomes a question of what you value. Paperwork itself is not the enemy. Its you’re tolerance for the risks you’re willing to take based upon what you’re willing to pay.
I pushed back a little. Finally, here he is on the two problems of competition in the DOD:
The first problem is maybe the requirements are too big. The second problem is, when we think of competition, often times it’s look at as, this is the statement of work, this is the statement of objectives, and everyone who submits a proposal to that is trying to optimize against a single approach, rather than wild approaches that look at completely different ways to solve a problem.
He gives us some great examples of thinking outside the box. A lot more to say on that later.
I want to thank Victor Deal for speaking with me on Acquisition Talk. I highly recommend the podcast. Victor tells us so much more than the snippets I pulled out here.
The purpose of the podcast, over the course of time, will be to give the listener a good feeling for the context of defense acquisition. It is an enormously complex business, and Victor is right, there are no “silver bullets.” Each function is nuanced, from contracting to budgeting to system engineering, management, requirements, operations research, and many others besides. Hopefully the podcast, and this blog, will help provide a “lay-of-the-land” so that professionals can connect acquisition practice to theory, and ultimately improve the acquisition system itself.
Leave a Reply