The Measurement Trap: Why New PM Productivity Metrics Might Be Solving the Wrong Problem
The scramble to quantify what product managers actually do has spawned a fresh generation of metrics. Most of them are measuring the PM, not the product work itself.
Something odd is happening in performance review conversations across the tech industry. After years of hand-wringing about whether you can even measure PM performance, organisations are now experimenting with metrics that sound suspiciously like they were invented by someone who has never actually shipped a product.
I first noticed this last quarter when a friend showed me her company’s new PM scorecard. It included something called “decision latency.” The basic premise is straightforward: measure the gap between when a decision is requested and when it gets made. Faster equals better. Lower numbers mean you’re agile. Higher numbers suggest bottlenecks, indecision, or what one exec apparently called “governance bloat.”
On paper, this makes sense. And there’s data to back it up. Research suggests that if decision latency of a team is reduced from 5 hours to 1 hour, their time to market increases by 30%. Medium That’s the kind of number that gets put on slides. It travels well in leadership meetings.
But here’s what nobody seems to be asking: what counts as a decision?
Because in my experience, the best PMs are often the ones who delay decisions deliberately. They hold off until they’ve gathered one more data point. They wait to see how the competitor’s launch lands. They let the engineer finish the technical spike before committing to an approach. Measuring decision speed without measuring decision quality is like celebrating a restaurant for how quickly they burn your steak.
The Hypothesis Cycle Obsession
Then there’s “hypothesis cycle time.” The premise here is that you should track how quickly your team moves from “we believe X” to “we now know whether X is true.” Speed of learning. Velocity of validated bets.
I’ve watched this play out twice now at companies that got very excited about experimentation culture. What actually happens is that teams start running smaller, safer experiments because those complete faster. The big bets, the ones that might actually reveal something meaningful about customer behaviour, get quietly deprioritised because they take longer and drag down the cycle time number.
A PM at a payments company told me recently:
“We optimised ourselves into running forty button colour tests and zero pricing experiments. Our cycle time looked incredible. We learned almost nothing that mattered.”
This is the classic trap of optimising for what’s measurable instead of what’s meaningful. Hypothesis-driven development is genuinely valuable. But measuring the time between hypothesis and conclusion, without weighting for the significance of what you’re testing, creates perverse incentives that anyone who has worked in quarterly targets could see coming.
Roadmap Accuracy and the Certainty Theatre
Perhaps the most concerning metric I’ve encountered is “roadmap accuracy.” The idea is simple: look at what you said you’d ship six months ago, then look at what you actually shipped. High correlation means you’re reliable. Low correlation means you’re bad at planning or easily swayed.
This one genuinely baffles me.
If your roadmap accuracy is consistently above 90%, you’re not doing product management. You’re doing project management with better marketing. The entire point of continuous discovery is that you’re supposed to learn things that change your direction. You’re supposed to talk to customers who reveal that the feature you planned is solving a problem they don’t have. You’re supposed to encounter technical constraints that make your original approach unworkable.
Spotify famously abandoned their squad model because teams were spending more time following the prescribed structure than building products. But for years, every company copied it anyway because it looked organised on a diagram. Roadmap accuracy feels similar. It’s certainty theatre. It rewards teams for ignoring signals that should change their plans.
A former Atlassian PM described their approach to early-stage bets:
“Prove that your product gives value to a small number of users. Then, and only then, prove that your product gives value to a large number of users.”
That’s the opposite of roadmap accuracy. That’s roadmap flexibility as a feature, not a bug.
Customer Bet Success Rate: The One That Almost Works
The metric I’m most curious about is “customer bet success rate.” The concept tracks the percentage of product bets, major initiatives framed as customer-focused hypotheses, that achieve their stated outcomes within a defined timeframe.
This is closer to measuring what matters. It focuses on outcomes rather than activities. It acknowledges that PMs make bets and some of those bets fail, which is actually how good product development works.
But even this one has problems.
First, who defines success? A feature that hits its adoption target but cannibalises revenue from another product line might look successful in isolation. Second, what’s the time horizon? Some bets take eighteen months to reveal their true impact. Third, and most importantly, if your bet success rate is too high, you’re probably not taking enough risks. A 100% success rate suggests you’re only pursuing the obvious.
I’ve yet to meet a company that has figured out how to calibrate this properly. The ones who’ve tried usually end up either defining success so narrowly that everything passes, or so broadly that the metric becomes meaningless.
What We’re Actually Measuring
Here’s the uncomfortable truth beneath all of this. These new metrics exist because organisations still don’t trust their PMs, and they’re trying to find numbers that will do the trusting for them.
But product management resists quantification precisely because it’s about judgement, context, relationships, and the ability to hold contradictory information in your head while still making forward progress. You can measure the outputs of those things. You can sometimes measure the outcomes. But measuring the quality of the thinking itself?
Rich Mironov puts it well: he’s “hesitant to assign numeric goals to individual product managers” ProductPlan because “as soon as there is a numerical, measurable target to be reached, it’s human nature that employees will alter their behavior to hit this designated goal instead of taking a big-picture approach to the job.” ProductPlan
I don’t think decision latency, hypothesis cycle time, roadmap accuracy, or customer bet success rate are useless. Some of them might even be helpful if used carefully, with context, and as one signal among many.
But I worry that we’re building measurement systems for the PMs we wish we had, in the organisations we wish we worked in, rather than acknowledging the actual complexity of product work. The best PMs I know are simultaneously fast and slow, certain and uncertain, planful and reactive.
Try putting that on a dashboard.