How can I keep track of all these product metrics?

Hi Jock,

Do you have any advice on productivity tools for tracking product metrics? I’m seeking guidance on streamlining feedback and metrics management. Juggling continuous discovery insights, team feedback, and metric tracking has become increasingly overwhelming.

How do other teams effectively prioritize and process this information? I’m particularly interested in recommendations for tools, routines, and methodologies for focusing on the most critical data points. Any insights or best practices would be greatly appreciated.

Thanks,

H

Hi H,

It certainly feels like we’re being asked to track more and more metrics these days. The idea underpinning this is that we can only work to change something we’re actively measuring. But we we’re usually not trying to change that many things at the same time. So one of the ways we can reduce our workload is to focus on tracking only the most important things we’re trying to change right now (rather than trying to measure everything, all the time).

One meaningful metric at a time #

Some writers such as Alistair Croll and Ben Yoskovitz (Lean Analytics) talk about ‘the one metric that matters’. In their North Star playbook, John Cutler and Jason Scherschligt also talk about a single, meaningful metric, and then a handful of contributing inputs that tend to move the North Star metric in the right direction.

Operational metrics #

But what about operational metrics, tracking the day-to-day stuff to catch when it’s not working as it should? This could be anything ranging from SaaS product uptime and performance, all the way through to whether the team has had sufficient user research interactions this month. Unless we’re actively trying to improve something in this area, we ideally want to track this passively (and automatically if possible), and only do something if an operating metric drops out of its desired normal operating range.

So if the website goes down or is responding too slowly, we get an alert. If the team isn’t getting out and talking to users often enough, this (hopefully) gets noticed in daily/weekly catch-ups and then we temporarily apply a bit more focus on improving that particular metric again. And once things are back to the desired state, we don’t need to give them as much attention again.

When unsure, start with a line in the sand #

The thing is that for many of the things (at least to begin with), we simply don’t have enough detail to be able to quantify it, except in general terms.

So in discovery, right at the beginning when we’re trying to find an opportunity to investigate further (but we don’t know what it’s going to be yet), we might set ourselves the metric of interviewing at least 5 users in a particular segment each week. The problem we’re trying to solve is lack of insights, so we measure how frequently we’re speaking to people with the hope of uncovering insights.

Why 5 interviews a week? It’s simply a realistic starting point we’ve guessed. Not too few, because that would mean we’re not likely to uncover insights very quickly; not too many because we have other work we need to be doing at the same time. Perhaps when we’ve become more experienced at running discovery interviews, we may decide that we can reasonably manage 8 interviews a week. All we’ve done is to use our practical experience to refine our initial guess. But now at least we have some actual data to inform the change.

So the point is that sometimes we just need to start with a reasonable guess, gather some evidence, then revisit our guess to make it more accurate in the light of the actual experience.

Automate and streamline metric gathering where possible #

In terms of practical ways to reduce the effort, try to automate measurement where possible. If it’s a manual measurement, keep the process of recording the metric as lightweight as possible to ensure people actually do it.

I think you mentioned you’re already using ProdPad. There are probably ways to automate collection of relevant metrics using their API, particularly given they integrate with Zapier, which in turn connects with many other apps. Many of the teams I have worked with in the past have also used Grafana to track and alert on the operational metrics I mentioned earlier.

Hopefully that gives you some ideas, let me know if you need more.

Cheers,

Jock

Further reading #

PRODUCTHEAD: A primer on analytics for beginners – I Manage Products

PRODUCTHEAD: The joy of metrics – I Manage Products

PRODUCTHEAD: How product teams can measure discovery – I Manage Products