Lo-fi usability testing – Part 1: Background

Lo-fi usability testing – Part 1: Background

I recently spoke at ProductCamp London about conducting lo-fi usability testing, that is, easy, quick and inexpensive usability tests that anyone can run. I did a neater version when I was invited recently to present to the BBC product managers and user experience practitioners, which I’d like to share with you.

Jock Busuttil running a training session on usability testing

Quite a few people are put off usability testing because they think it’s complicated, time-consuming and expensive.  What you may not realise is that you can run a set of usability tests in a single afternoon that will uncover eighty percent of the problems your product has.  And the only specialist equipment you’ll need is a pen, some paper and the computer you need to access the software or website.

This my friends is lo-fi usability testing – a high-return, low-cost method for these cash-strapped times.  In this first instalment, I’ll be discussing what usability is and why testing it is so important.

Our assessment of a product’s usability is subjective.  It depends on many factors: visual cues, familiarity or similarity to something else we’ve experienced, and so on.  While there no absolute measure of usability, what is clear is that some products are far easier and pleasant to use than others.  What is more, these more usable products tend to be the successful ones.

So what do we mean by usability? #

What is meant by usability?
4 More London Riverside by Jock Busuttil, all rights reserved

You know your product is usable when users intuitively understand what it is for and how they should use it.  It is also a good sign when people enjoy using your product.  This is likely to indicate that your product will save the user time or at the very least give them a pleasant distraction from whatever it is they’re meant to be doing, rather than being a source of frustration and delay.

Perfection is unattainable.

Perfection is unattainable
2 More London Riverside by Jock Busuttil, all rights reserved

But don’t let that put you off.  We can nevertheless attempt to get close.

Think about it this way:

  • Market needs and problems change – people weren’t worrying about how to tweet before Twitter existed
  • User expectations change – do you remember how sloooooow everything was before broadband?
  • The bar is continually raised by innovative products and websites – things are generally getting easier to use; hurray!  But if you’re complacent about usability, you’re risking being left behind by your competitors.

What is considered to be good usability is not only subjective but also continually in flux, so we need to keep re-evaluating and improving our products and websites.

Why do we need usability testing?
6 More London Place by Jock Busuttil, all rights reserved

So why do we need to test the usability of our products?

Evidence #

Generally speaking, you’re going to need to convince someone that it will be worthwhile to make changes to your product.  Whether that someone is the budget holder who needs to justify the financial expense of making those changes or the developer who refuses to accept that their ‘baby’ is in any way imperfect, simply saying “this product could be better than it is now” is not going to cut the mustard.  In other words, you’re going to need evidence to convince people a change is needed.

Your opinion is irrelevant #

The next revelation is that we, as product managers, are not right about everything.  Steve Johnson of Pragmatic Marketing fame often uses the phrase, “your opinion, though interesting, is irrelevant”.  Guess what?  He’s pointing at us product managers when he says that.  We’re usually* not the target users and our products are not our vanity design projects.

* There’s always one smart-ass product manager who is responsible for requirements or backlog management software.  Yes yes yes, very good.  Now sit down.

Fresh eyes #

When you come to a product or website you’ve never seen before, it takes you a little while to learn its quirks until you become proficient at using it.  The problem is that you’re now working unconsciously around the usability problems.  So you need to rely on fresh pairs of eyes to spot the things that you, your developers and other proficient users are automatically compensating for.

Don’t assume, check #

We spend our time voicing the needs of our customers within our organisation.  Consequently, it’s all too easy to fall into the trap of thinking we know what our users need without bothering to ask them.  Usability testing forces us to find out how our users are working with our products.  I will guarantee you that every set of usability tests you run will reveal at least one wholly new insight.

We’re moving the goalposts… #

Over time we evolve our products and add new features that we believe are of value to our customers.  However, without understanding the effect these new features will have on the overall usability of the product, there is a risk your ‘improvement’ is in fact the reverse.  One extreme example of this was a data entry product I managed where the addition of a single mouse click in an otherwise keyboard-driven process meant a ten percent reduction in productivity.  Multiplied by hundreds of data entry clerks, the cost implication would have been massive.

We took the mouse-click back out again.

…and so is everyone else #

User needs and problems change.  Consider for example the impact of social media on information exchange.  Even if you’ve made no changes to your product, your users and the other products they use are gradually becoming more sophisticated.  As soon as one product raises the bar for usability, everything else seems second-rate by comparison.

When in Rome #

Different territories, countries and cultures do things in different ways.  They have different keyboard layouts, different expectations and assumptions.  You had better check to see if any of this is going to have an adverse effect on their ability to use your product or website effectively.

Remember: perfection is unattainable #

Our products are not perfect, they can always be improved.

That covers the ‘why’ and ‘what’ of usability testing.  Next time, we’ll get down to the practicalities of what you need to do to prepare for running a set of usability tests.  Thanks for reading.

Read part 2 now!  Preparing to run a usability test

Get articles when they’re published

My articles get published irregularly (erratically, some might say). Never miss an article again by getting them delivered direct to your inbox as soon as they go live.  

Read more from Jock

The Practitioner's Guide to Product Management book cover

The Practitioner's Guide To Product Management

by Jock Busuttil

“This is a great book for Product Managers or those considering a career in Product Management.”

— Lyndsay Denton

Jock Busuttil is a product management and leadership coach, product leader and author. He has spent over two decades working with technology companies to improve their product management practices, from startups to multinationals. In 2012 Jock founded Product People Limited, which provides product management consultancy, coaching and training. Its clients include BBC, University of Cambridge, Ometria, Prolific and the UK’s Ministry of Justice and Government Digital Service (GDS). Jock holds a master’s degree in Classics from the University of Cambridge. He is the author of the popular book The Practitioner’s Guide To Product Management, which was published in January 2015 by Grand Central Publishing in the US and Piatkus in the UK. He writes the blog I Manage Products and weekly product management newsletter PRODUCTHEAD. You can find him on Mastodon, X (formerly Twitter) and LinkedIn.

2 Comments on “Lo-fi usability testing – Part 1: Background

  1. Good stuff. Regarding fresh eyes, yesterday we pulled in a guy from our customer services team to test a new product extension, with little knowledge of what it should do or how it worked. Some frustration in the morning for me as PM (“goddammit do I have to explain everything, can’t he just find the bugs in the specific windows + features we’ve added? Should have take a sales engineer who knows this stuff”), but at the end of the day the testleader came back with a long list of things that had come up during the testing session. Most issues on the list were outside of the extension, but impacted the experience as a whole nonetheless!

  2. It is really hard for product managers to observe people using their product without jumping in to help them all the time. Similarly, it can be hard to take negative criticism: nobody likes being told their baby’s ugly.

    I think about this in a few ways:

    • I really enjoy the ‘lightbulb’ moments when I understand better how people interact with my product
    • My opinion, though interesting, is irrelevant (thanks Pragmatic Marketing)
    • It’s better to get unbiased feedback from lots of people, then see if there are recurring problems

Leave a Reply

Your email address will not be published. Required fields are marked *