I was pleased to have Bruce Cameron on the Acquisition Talk podcast. He is the director of the System Architecture Lab at Massachusetts Institute of Technology. Bruce also co-founded a consulting firm — Technology Strategy Partners — where he has worked with many of the leading firms in tech, aerospace, logistics, and consumer goods.
In the episode, Bruce tells us about open architectures, how the term modular is used so broadly as to be mean almost nothing, what leads to product lock-in effects, the three lens model of organizations, whether cybersecurity has fundamentally changed anything about architecture, and much more.
During the discussion, he tells us about the results of the MIT commonality study. It gives us a more positive framing of the F-35 program. Decreases in commonality from 80 to 90 percent down to 20 to 30 percent is common in joint defense programs and industry platforms. He provides four criteria for judging whether platforms of common parts will succeed.
Bruce also describes two of the “big levers” that we can use to improve product developments: quality of people and risk posture of the firm/agency. Without pulling these levers, agile processes cannot be used to a great effect. He also explains whether agile can be used for new systems architecture, or whether it is limited to the development of applications.
Podcast annotations.
Here is part of Bruce’s explanation of the lock in problem:
[It’s] similar to the idea of a monopoly, but usually refers to some past technology that we acquired and we’d like to make changes to, or some past contract that we’d like to grow in scope…
What we got out of this [study] is that when you’re looking specifically at computer code, there’s actually a pretty good way to tell whether you’re locked in or not. It was surprisingly descriptive in terms of the structure of the code tells you a lot about your relative ability to carve a portion of the code out, and recompete it.
We didn’t study hardware programs in this context, but what we found was that programs that essentially had relatively modular architectures tended to avoid lock in, versus programs which had – for a lack of a better word – hairball architectures, code that was intimately connected to all other pieces of the code, tended to display more lock in behavior.
I like how Bruce separated the lock in problem from monopoly. For example, HP doesn’t have a monopoly on printers, but once you’ve invested in an HP printer, you are locked into their ink cartridges unless you’re willing to invest in a new printer.
Now imagine that the printer is a weapon system platform, and the ink cartridges are change orders, upgrades, spares and repairs, and so forth. Indeed, it has been a long standing strategy for contractors to “buy in” on the development contract, and once the program is underway, recoup costs on the later stages.
This is particularly easy to do for contractors because the weapon system process that has been in vogue since the 1950s created a selection preference for hairball architectures. The hairball is a great metaphor, makes you think about the mess of entanglement between system components. Weapon systems often push the frontier on all components simultaneously, and have end-state integration in mind from the beginning.
Weapon systems have tended not to be technically separable. It seems that the result is less a fact of life than the weapon systems design philosophy process pushes for hairball architectures. When the DOD attempts to make a system modular, like for the LCS, usually that modularity is preordained for the platform rather than a logical consequence of an ecosystem of independent developments. With all components reliant on the contractor’s unique platform, rather than standard interfaces, modularity wouldn’t seem to mitigate the lock in problem.
Technically separable is also one of Bruce’s conditions for the success of agile development processes. Unfortunately, in Bruce’s experience with aerospace and defense, many products do not lend themselves to technical separation, parallel development, and independent testing.
The real challenge of running agile in a mature, particularly a complex, development area is our ability to carve off a set of parallel features that can be independently tested. Where agile has really succeeded in the software world, we have tremendous tools to essentially parallelize the work…
By contrast, when we get to complex programs, one of the challenges is that we get these emergent properties that come up as an interrelationship between a whole bunch of components that destroys our ability to parallelize it.
So unfortunately, most of my experience with the word agile in the aerospace and defense word is as a byword for “let’s just run development programs faster.” That does not imply anything truly different about the way the development program is run, or about the risk posture of the firm running the development.
There are many statements in the DOD about the importance of speed and agile processes. Yet when it comes down to it, programs seem to be run in the same way. “Rapid acquisition” means throwing off parts of the requirements and contract processes. But that doesn’t mean that the people doing the work have changed, or that there is a tolerance for failure and adapting the direction of the program.
Instead, the regular course of program development is asked to be compressed into a 2-5 year window. That often means ramping up more people faster, particularly if problems are encountered. DOD developments are known for their “boom and bust” cycles. But as Bruce explains, bringing on a host of new people to a project lowers average productivity. On highly technical projects, it is easy to suffer from the “too many cooks in the kitchen” effect.
Bruce points out that DARPA has had a good record of finding high quality people and accepts a great deal of risk. However, he wonders how scalable that model is:
I think the challenge that a lot of organizations in aerospace and defense face is getting a lot of high quality, motivated, people.
From my perspective, the DARPA model is not scalable in the current institutional setting. Talent is scarce. DARPA has some intangible qualities that attract top-level talent from academia and industry. But for the whole defense acquisition system to swell with those people would require a bidding war for higher compensation.
Even if the government were willing to pay top dollar for top talent, that would not necessarily lead to a total increase in motivated people. I think that the only scalable option for the DOD is to cultivate the talent from within. Admiral Rickover did this with his nuclear program, but even then, his talent was coming at the expense of the surface Navy to some degree.
Here’s a good part on commonality:
Bruce: The Joint Strike Fighter program is a fascinating case study. It was known to be a troubled program when we began the MIT commonality study, and yet a lot of the output of the study was in some sense a positive framing of the Joint Strike Fighter program. Among other things we found is that the divergence that the JSF program has seen, i.e., the movement from the target of 80-90 percent parts sharing to achievement levels in the 20 and 30 percent range, is not a unique failure of that program, but a commonly displayed behavior in many joint programs, and also in industry, at many attempts at commonality we find this same behavior…
Eric: Do you think part of that was because they prematurely tried to pursue commonality on the F-35? They were developing those components in conjunction with the platform, and with the growth of knowledge they realized there needed to be [changes]… What do you think about incrementally stepping toward commonality from the bottom-up versus planning it from the top-down ahead of time and trying to reach that goal?
Bruce: Well it turns out that there are failure modes on both ends of that spectrum…
I’ll leave it there. Be sure to listen to the podcast for a lot more insights from Bruce, including his advice to anyone working on the Space Development Agency’s RFI for a new space architecture.
I’d like to thank Bruce Cameron for speaking with me. I recommend checking out his website, which has dozens of interesting papers on system architecture and more about his program at MIT. He also has some YouTube lectures, such as one on Product Development for Complex Systems and Product Platforms. If you’re interested in learning more, he will be teaching a two day course, “Managing Product Platforms,” which is open for applications.
Leave a Reply