Big data – big deal?

Big data – big deal?

Everyone seems to be hyping Big Data right now. I think we’ve reached that slightly scary point where CEOs are aware of Big Data and are beginning to think it a panacea for all business ills. But I’m asking the question: what’s the big deal with Big Data?

We’ve always had more data available to us than we can deal with. Our ability to manage that volume of data has increased over time, although so also has the amount of data we generate. We’re always going to be playing catch-up. But are we any better at turning all this data into useful and valuable meaning?

Very few people seem to be really thinking about the point of Big Data

I attended a recent CloudCamp in January focusing on Big Data and made a few observations:

  • techies love to indulge their hoarding tendencies (Must. Store. More. Data.);
  • armchair lawyers still love to digress on high-impact, low-probability concerns such as the Patriot Act and how likely cloud tech companies were to be raided by the Feds (although this did provoke an amusing slap-down by Werner Vogels, CTO of Amazon); and
  • very few people seem to be really thinking about the ultimate point of being able to store and query such large amounts of data, i.e. what questions will all this data help us to answer?

While being able to store more data and query it more quickly is fundamental to our ability to start deriving meaning from it, I came away from CloudCamp’s Big Data event with the overriding impression that technology vendors in this space are continuing to focus primarily on size and speed (features) rather than facilitating knowledge and insight (benefits).

The problem with this approach is that corporate buyers will likely see the value not so much in Big Data itself, but in the answers to business questions offered by analysis of the Big Data. Organisations want easier solutions to the day-to-day business problems they have such as:

  • Where should I build my next retail store?
  • How can I identify the people most likely to want to consume my products and services?
  • How can I understand more about my current and prospective customers so that I can tailor my content and services to them?

Although the ability to answer these questions absolutely depends on being able to store and query the underlying data, the typical CEO (think: glorified sales person) is not going to appreciate or concern themselves with the capabilities of Hadoop, Hive, noSQL or any other Big Data technology.  In other words, they don’t care how you get to the answer – they just want the answer. We know Einstein’s equation E=mc2, but history is more concerned with its meaning rather the type of paper on which he wrote the proof.

So where does this lead us? Transactional information is becoming less dense in meaning so we need to be able to store and query larger sets of data before we can start to see the underlying picture. Big Data provides us with the ability to do this, which in turn allows us to find and articulate meaning within that data.

What I feel we’re lacking are similarly strong analytics tools, designed to cope with this volume of low-density information. In other words, we can store all this information pretty well, but we can’t unlock the meaning of that information quite as easily.

Big Data is a big deal. But our challenge remains to ask the right questions, understand the answers hiding in our data and then turn those answers into the valuable insight that will then drive those business decisions. Big Data is a great foundation, but I’m much more interested in seeing how Big Analytics turns out.

Are you a product professional working with Big Data? Do you agree or think I’m completely missing the point? Let me know in the comments.


Get articles when they’re published

My articles get published irregularly (erratically, some might say). Never miss an article again by getting them delivered direct to your inbox as soon as they go live.  


Read more from Jock

The Practitioner's Guide to Product Management book cover

The Practitioner's Guide To Product Management

by Jock Busuttil

“This is a great book for Product Managers or those considering a career in Product Management.”

— Lyndsay Denton

Jock Busuttil is a freelance head of product, product management coach and author. He has spent over two decades working with technology companies to improve their product management practices, from startups to multinationals. In 2012 Jock founded Product People Limited, which provides product management consultancy, coaching and training. Its clients include BBC, University of Cambridge, Ometria, Prolific and the UK’s Ministry of Justice and Government Digital Service (GDS). Jock holds a master’s degree in Classics from the University of Cambridge. He is the author of the popular book The Practitioner’s Guide To Product Management, which was published in January 2015 by Grand Central Publishing in the US and Piatkus in the UK. He writes the blog I Manage Products and weekly product management newsletter PRODUCTHEAD. You can find him on Mastodon, Twitter and LinkedIn.

Agree? Disagree? Share your views: