Sergey Nivens - Fotolia

Managing big data: The two biggest mistakes companies make

Many organizations are setting their big data analytics efforts up for failure by plunging ahead without proper planning upfront, says book author Keith B. Carter.

Keith B. Carter is the author of Actionable Intelligence: A Guide to Delivering Business Results with Big Data Fast!, which was published in September 2014 by John Wiley & Sons Inc. This is the first in a series of articles that Carter is writing for SearchBusinessAnalytics based on the chapters in his book.

There's probably not a day -- if not an hour -- that goes by when you don't hear about big data: managing big data, analyzing big data. Indeed, the term has been so widely and, at times, so inappropriately applied that it has started to lose its meaning.

And there's something to watch out for in all the talk about big data and its potential benefits. It's the big data lie -- the notion that big data can automatically solve any number of business problems and position businesses to compete more effectively with their rivals. In my experience as a supply chain executive and, most recently, as a faculty member in the Decision Sciences Department at the National University of Singapore's Business School, I've worked and consulted with dozens of companies and hundreds of IT and business professionals. What I've consistently found is that those who believe the big data lie are making two big mistakes:

The problem is that many companies today are collecting and managing big data with little to no forethought.

Collecting data indiscriminately. Companies are collecting loads and loads of data. That's great. But they have no idea how to effectively make sense of it. They jump in by collecting big data -- extremely large, comprehensive, and often varied and fast-changing data sets -- to investigate and draw conclusions. But raw data says nothing to most business users, and placing even more data into bigger databases or copying increasing amounts of it into spreadsheets typically provides no value in and of itself.

Categorizing data within an inch of its life. Conversely, companies may think they're doing well if they "scrub" all the big data they've collected and then categorize it. Sure, there is a benefit to normalizing data by centralizing, cleansing, classifying and structuring it. But it's hard to make a business case for doing that if, when the cleansing and organizing is done, the data doesn't provide answers to real business questions. This is especially true when a big data management effort turns into an IT-focused project that doesn't gain assurance from business users that the data in question has been validated and is actually usable.

The problem is that many companies today are collecting and managing big data with little to no forethought. And just as planning is key to any strategic business project, forethought is of utmost importance when dealing with big data. All the elements of the process must be purposeful and aligned closely with business goals.

So, what's the right way? You have to start with a strategic business question and then acquire the data needed to answer that question. Only then can you quickly start visualizing your business, performing business discovery and delivering actionable intelligence.

That's the process that tips big data from a checklist item to a true strategic differentiator -- because even in the truest definition of the term, big data isn't a solution to any business problems. At heart, it's just what it sounds like: a collection of large amounts of information from different sources. It's what you do with that data that can make a big difference to your organization, and that begins with avoiding the big data mistakes that other people are making.

About the author:
Keith B. Carter is a visiting senior fellow in the Decision Sciences Department at the National University of Singapore's Business School. He worked previously as global head of supply chain intelligence at The Estee Lauder Companies Inc. and as a consultant at Andersen Consulting (now Accenture).

Email us at [email protected] and follow us on Twitter: @BizAnalyticsTT.

Next Steps

Get a list of big data analytics project management tips from consultant Rick Sherman

Read about how big data initiatives raise the stakes on analytical modeling efforts

See why consultant Wayne Eckerson thinks Hadoop 2 and YARN are primed to shake up the data analytics landscape

Dig Deeper on Big data analytics