PRODUCTHEAD: Everything you wanted to know about ROI but were afraid to ask

pyramid product #

every PRODUCTHEAD edition is online for you to refer back to

---

tl;dr

Why is it so hard to forecast return on investment (ROI) for a product?

ROI is not the only measure of product value

You can take account of uncertainty in your estimates in a systematic way

---

hello

Whether or not you believe we’re in an AI bubble that continues to defy investment logic, eventually someone is going to turn around and ask you awkward questions such as when we’re going see a return on the investment we’ve made, and how big a return. Wild market valuations are nice and all, but nothing beats cold, hard cash in the bank — at least when your CFO (Chief Financial Officer) has any say in the matter.

As I’ve covered before, a product manager is continually trying to find the right balance between solving valuable problems for users and addressing the needs of the business. Funding operations is one of the those fundamental needs. Efficiently exchanging the value your products bring to users into revenue for your organisation is what allows you to continuing operating. Commercial and non-profit-making organisations alike need income from somewhere.

At some point in their careers, every product manager will have to think about how all their efforts to create and bring to market a successful product will translate into a meaningful return on investment (ROI). Maximising ROI — or shortening the time to break even — may not be the primary goal of the product strategy, but in most cases* it’s a necessary consideration.

* More complex product portfolio strategies can involve loss-leader products to entice customers to buy the more profitable products later, or use of profits from mature cash cow products to subsidise new and less profitable products. (But that’s not important right now.)

Calculating ROI after the fact #

With the benefit of hindsight and when assessed in aggregate, it becomes relatively easy to determine ROI for your product. You can usually figure out how much you’ve invested in activities directly linked to the product, and how much that product has earned over time.

In simple cases, ROI is your net return divided by your original investment amount, often expressed as a percentage. So if you spent $5 million in total bringing a product to market and it then generated $6 million in revenue, your ROI would be ( $6 million − $5 million ) / $5 million × 100% = 20%.

Forecasting ROI #

However, forecasting ROI for a speculative product or feature is an entirely different challenge, even at a high level. There are three main considerations we need to account for: uncertainty, investment lag and granularity.

1. Uncertainty #

When you forecast the ROI for your product you are trying to predict the future, essentially by making a collection of informed guesses, each of which have a degree of uncertainty. To counter this, you could anchor some of your inputs, for example by assuming a fixed budget for research and development or for go-to-market activities. And through experimentation and sampling you could perhaps get a reasonable sense of who your target users are, how many of them there are, and what proportion of them would commit to purchase your product for a given price.

Nevertheless, there’s no guarantee that real life will mirror your assumptions. Aside from all the less easily quantified factors affecting product adoption (competitive pressures, alignment with customer purchasing cycles, and so on), you’re unlikely to create a product with perfect product-market fit, nor can even the most skilled go-to-market teams execute perfectly.

All of this introduces uncertainty and makes any ROI forecasts imprecise. So your first consideration is to take account of uncertainty in your assumptions. You can apply rough and ready confidence ratings to each assumption you make, where 100 percent is a stone-clad guarantee that something will happen and zero means it won’t. (Your data will likely be too imprecise to use confidence in the strictly statistical sense.)

You can apply the confidence rating in a few ways. You can simply multiply whatever numerical predictions you make by the confidence rating. So if you’re estimating the addressable market value to be $5 million and you’re 50% confident about that, then you’d state it as $2.5 million instead (= $5m × 50%).

Alternatively, you can use your confidence rating to create a +/− margin of error. Using the same example, your addressable market value would end up as a range between $2.5 million and $7.5 million. You might use this approach if you’re just as likely to be underestimating something as you are overestimating.

A third way only makes sense when there’s a hard upper limit to what you’re estimating. If you were guessing people would use your product 5 days a week and you were again only 50% confident, your range is going to be between 2.5 and 7 days (not 7.5) because nobody can use your product more than 7 days a week.

When you roll up all of these assumptions and confidence ratings into your ROI value, depending on the method you’ve chosen, you’ll either have a single ROI figure, or a worst case / best case range. It’s entirely possible that you end up with a worst case figure that is negative. As you test out more of your assumptions and turn guesses into observed evidence, your confidence ratings will increase, and your resulting ROI range will narrow.

Precision is the last aspect to think about. Use lower resolution numbers to avoid lending a false sense of accuracy to numbers where precision doesn’t exist. Round to the nearest significant figure.

2. Investment lag #

Another reason why ROI can be hard to forecast is the gap in time between starting to invest time and money in research, development and go-to-market, and the point at which the resulting product is actually available for customers to purchase. This is particularly the case when you don’t yet know how long it will take to develop the product in the first place.

Time to generate ROI can also matter. A product that can generate a 10% return on investment within 6 months may be more desirable than a product which takes several years to achieve the same 10% return. You can visualise this more easily with a diagram similar to the one below:

Key:

TM — time to market

BET — break-even time (elapsed time from the beginning of R&D)

MR — market release (the point at which the product goes on sale)

RF — return factor at 1 year, 2 years (= ROI)

BEAR — break-even after release (elapsed time from market release)

The modern practice of iterative releases muddies the waters, but only slightly. In the diagram above, investment plateaus around the time of release. This only makes sense for a one-and-done product such as a hardware device. With software, you’d expect investment to be constantly increasing, or perhaps increasing in waves corresponding with subsequent development cycles. This will have the effect of pushing back the break-even point further into the future.

3. Granularity #

The third consideration when thinking about ROI is how granular you can be. Imagine you have a software product already on the market. Subsequent releases will probably add some new features, fix some bugs, and maybe refactor some aspects to make life easier for your developers later on. You may be able to forecast incremental ROI for that release as a whole, but things become more tricky if you’re asked to predict ROI for each line item on the changelog.

Not everything directly generates revenue. Some features are table stakes, and are not something you would typically charge extra for. For example, social sign-on for a web app is expected by users and unlocks the ability to generate revenue from other features, but it’s unlikely anyone would pay to use social sign-on in isolation. Similarly, bug fixes and quality-of-life improvements are more about reducing churn than generating new revenue. Refactoring is about improving developer efficiency in the future, so it reduces the future cost of development slightly.

Final thoughts #

All of this is to say that ROI is not the only way of measuring value. It’s not unreasonable for the Powers That Be to ask what the business is getting back from its investment in ongoing software development. However, making ROI the single measure of value runs the risk of forgetting that revenue is really just a side-effect of finding that sweet spot where value to users, value to the business, and an effective business model (to facilitate the exchange of value) intersect.

For you this week #

Matt LeMay chats with Janna Bastow about practical ways to quantify the ROI of your work; Itamar Gilad introduces his Confidence Meter as a way to systematically determine the Confidence score used in prioritisation based on supporting evidence, keeping us honest about our assumptions; and Rich Mironov argues that ROI is fundamentally ill-suited for the uncertainty and complexity of development trade-offs and proposes approaches to square the circle.

Speak to you soon,

Jock

---

---

what to think about this week

Because you’re worth it: How to show the ROI of your product work with Matt LeMay

Do you find it challenging to quantify the return on investment (ROI) of your product work? Are you tired of investing time and resources into product development without being able to demonstrate its true value? If so, you’re not alone, especially in these uncertain times. Many product professionals face the same dilemma.

[VIDEO] Avoid ‘the low-impact death spiral’

[Matt LeMay & Janna Bastow / ProdPad]

“My CEO is a Finance Guy Stuck on ROI…”

Head of Product: My CEO is a finance guy, and is pushing the product team hard to prioritize all engineering work solely on ROI. As VP Product, I’m having trouble selling him on investing in user experience, software quality, “meet the competition” features, and other work that doesn’t immediately convert to revenue. We’re both frustrated. How do I get my point across?

Let’s try to recast product strategy in financial terms

[Rich Mironov / Product Bytes]

---

---

Idea Prioritization With ICE and The Confidence Meter

Prioritization questions are at the heart of product management. The penalty for choosing the wrong product idea can be quite high — cost of development + cost of deployment + cost of maintenance + opportunity cost + other residual costs. We are often tempted to make a decision based on weak signals: majority votes, highest-paid person opinion (Hippo), industry trends, etc, but those have been shown time and again to be bad heuristics that are not any better than putting chips on a roulette table (hence the term “Big Bet”).

How to assign confidence ratings to y0ur bets

[Itamar Gilad]

ROI: Seductive but of Limited Value in Product Planning

I’m part of many discussions where tech company execs try to apply Return on Investment (ROI) tools to make hard choices about what to build, or where to invest costly-and-scarce development resources. It rarely turns out to be as useful as we hoped. Where ROI is absolutely fundamental to pure financial decisions, it’s typically a very poor fit for the uncertainty and complexity of product/engineering planning and trade-offs. And often makes the product team appear incompetent. (Or makes Engineering appear lazy and disorganized.) Let’s unpack some ROI challenges and propose alternatives.

ROI is wildly imprecise for forward-looking product/technical trade-offs

[Rich Mironov / Product Bytes]

recent posts

Are developers vibe coding themselves out of a job?

And is the increasing reliance by junior developers on AI coding assistants storing up a generational skills shortage for the future – ‘professional debt’, if you will?

So simple, anyone could do it. Wait – don’t fire me

[I Manage Products]

Cloud computing for non-technical product managers

To understand how cloud computing works, we’re going to start with the basic building blocks and work our way up.

And why is it a cloud anyway? (All is revealed)

[I Manage Products]

Navigating your product management career

Ross Webb and I have been chatting about product management career progression.

We cover topics including:

» Thinking of visibility as a strategic competency, not self-promotion

» Controlling your narrative through regular updates

» Building cross-organisational relationships deliberately

» Mapping your stakeholders’ preferred communication styles

A roundtable chat on moving into product leadership

[I Manage Products]

PRODUCTHEAD is a newsletter for product people of all varieties, and is lovingly crafted from all the bags of garden clippings.