PRODUCTHEAD is a regular newsletter of product management goodness,
curated by Jock Busuttil.
fake product trees
tl;dr
Use sensitivity analysis to identify the make-or-break metrics for your business
Your analytics approach needs to change over time to keep up with your evolving product
Blindly copying best practices results in an imperfect copy
Analytics tools don’t do the thinking for you
a favour: please share this with other product people
every PRODUCTHEAD edition is online for you to refer back to
hello
What does my inability to keep houseplants alive teach us about analytics?
Spending more time at home has meant that I can do lots of odd jobs during my regular tea breaks. I decided I would attempt to look after a houseplant. But after a few weeks it was compost. Where was I going wrong?
I had tried to be reasonably methodical so I gave the plant exactly 100ml of water each week to keep it happy. By following that routine, the plant should be neither too dry, nor too water-logged.
And yet: mulch.
It turns out that Lovely Wife was occasionally also watering the plant to be helpful — without me knowing.
My carefully planned watering routine was based on an incomplete — and therefore flawed — view of what was actually happening.
Analytics and measurement are undeniably useful as part of your product management toolkit, and certainly better than hoping for the best and never measuring anything.
However, when measuring something, it’s always worthwhile to consider whether external factors are subtly influencing your results without you knowing.
Imagine you’re running A/B tests on your live site to determine which combination of messaging and layout converts better. Your test concludes, and there’s a clear winner, so you switch over to the winning variant. But after a few weeks, you notice that conversion gets worse, not better as you were expecting. What’s going on?
It turns out that your original A/B test happened to coincide with an AdWords campaign targeting the same pages that skewed your visitors and led you to a false result.
In other words, you didn’t realise someone else was also watering your plants.
So this week I’ve pulled together some good content on how to avoid the pitfalls of analytics. (But sadly no gardening tips.)
Speak to you soon,
Jock
what to think about this week
How data will make you do totally the wrong thing
Jason Cohen is best known for founding WP Engine. He’s also excellent at bringing to life the theory around business analytics. In this talk for Business of Software, he uses a cute hamster to explain statistical significance, debunks Google’s famous ‘41 shades of blue’ test, and reveals which metrics actually matter for your business and more importantly, why.
[JASON COHEN / BUSINESS OF SOFTWARE]
Everything a product manager needs to know about analytics
Analytics isn’t rocket science. But learning analytics from a former space engineer and satellite orbital dynamicist is cool. Simon Cast of ProdPad and Mind The Product fame explains in this article what product managers need to know about analytics and metrics.
What you don’t measure, you can’t improve
[SIMON CAST / MIND THE PRODUCT]
Product Management Coaching
Whether you’re new to product management or have been a product manager for years, a coaching session can help you to step up your career.
We’ve coached people wanting to get into product management, product people with nobody in their organisation to manage them, and experienced product managers preparing to apply for a promotion.
We can help you prepare for your product manager interview, including mock interviews.
A proportion of the fees from every coaching session is donated to charity. Just reply to this email if you’re interested in finding out more.
Death by ‘best practices’: why they can kill your business
Louis Grenier writes on Hotjar’s blog about how he failed to increase conversion rate despite following best practices in analytics from other leading companies. Why? Because copying your competitors will (at best) put you at the level they were at some time ago.
[LOUIS GRENIER / HOTJAR]
12 A/B testing mistakes I see all the time
Founder of CXL Peep Laja finds A/B testing fun. He’s identified the 12 mistakes he sees most often and describes them simply and comprehensively with illustrated examples. Well worth a read if you’re trying to improve how you use A/B testing in your organisation.
There’s more to it than just setting up a test
[PEEP LAJA / CXL]
recent posts
How to start a new product manager job
Starting a new product manager job can be daunting, particularly if you don’t change jobs very often. I work freelance, so I find myself in a new organisation roughly every 3-6 months. Let me share with you my tips for your first few months in a new role.
What to do in the first 30-90 days
[I MANAGE PRODUCTS]
The product leader’s guide to interviewing
Because I tend to help organisations build up their product team from scratch, I’m often involved in the interviewing and hiring process, so I’d like to share with you my product leader’s guide to interviewing product managers.
Countering unconscious bias in recruiting
[I MANAGE PRODUCTS]
Mission to Mars
Imagine you’ve just been told that you’ll be a member of the team responsible for the first manned mission to Mars.
Now imagine someone asks you how much the mission’s going to cost. The whole thing. There and back. By close of business on Thursday.
[I MANAGE PRODUCTS]
upcoming talks and events
5th May 2021, 16:00 GMT
Threads
Online product management round table discussion (topic TBC)
Tickets
can we help you?
Product People is a product management services company. We can help you through consultancy, training and coaching. Just contact us if you need our help!
Helping people build better products, more successfully, since 2012.
PRODUCTHEAD is a newsletter for product people of all varieties, and is lovingly crafted from waterlogged and gently rotting houseplants.