You mentioned intuition and I think that can be developed too believe it or not. I’ve helped many product managers develop their product sense. How can you tell, with pretty good accuracy before you have numbers, whether a product will work or not, or reach its goals? It turns out you can develop that by just trying many different kinds of products and seeing what works and what doesn’t, and having this massive library in your head with patterns of how products might work, and of how successful products work, and being able to refer to that library and apply it to new situations. That’s what Jeff was able to do — that’s what great product managers do — recall anecdotes and things from his time Yahoo! and say, we tried something similar to this and here’s what happened, then apply it to this situation.
That was Elliot Shmukler on the Venture Stories podcast, Principles of Product Management and Growth Marketing from Legendary PM who helped build LinkedIn, Wealthfront, and Instacart. I don’t think we quite know what goes on in people’s head to draw on correlation and anticipate winning decisions. Certainly, people can over-index on the past and completely miss why the next time is different. But the point I want to make is that in DoD, acquisition folks can spend an entire career on a single program. More than likely, folks in the workforce will jump between programs, rarely seeing one from all the way through. That kind of system makes it hard for an individual to draw lessons from experience and apply it to new situations. Their career becomes a series of disconnected events; no feedback loops are closed.
The fact that so much is lumped into a single program at the same time makes it basically impossible to learn concrete lessons about what did or did not work. Too many things are changing over too long a period of time. There is no control. Partitioning major programs into multiple overlapping projects where only one “miracle” is attempted at a time isn’t just a proposition about creating better programs, or better data for statistical analysis — it is a proposition for getting people the experience they need to make the right decisions.
This doesn’t mean that a good product manager uses intuition alone. Shmukler says product managers must be data driven, but that there is no perfect set of metrics. Data must be collected, experimented with, and the metrics emerge from analysis and a deep understanding of the product and the user. In my mind, that means data driven product management is all about context. Unfortunately, in DoD, the metrics are universal and intended to be used by cost-conscious managers from afar. That’s a massive mistake in the structure of defense oversight.
Leave a Reply