Sergey Nivens - Fotolia

Manage Learn to apply best practices and optimize your operations.

Big data benefits begin with business focus in analytical modeling

Big data analytics programs need a strong analytical modeling foundation to deliver the business benefits that companies are looking for. And that starts with tight ties between analytics teams and business units.

Mainstream business intelligence and analytics applications focusing primarily on retrospective reviews of data can provide organizations with valuable information about corporate performance, customers and business trends. But companies looking for competitive advantages over business rivals are increasingly turning their gaze to the future -- and big data is broadening the scope of the predictive analytics they can do in order to see what's coming.

The vast quantities and varied forms of data that many organizations are now collecting give predictive modelers and data miners lots of material to work with in trying to forecast things such as customer behavior and effective business strategies. Unlocking big data benefits isn't just a matter of turning skilled analysts loose with predictive modeling tools, though. In fact, the process of developing useful models to analyze sets of big data isn't what people tend to think it is, according to Mark Pitts, vice president of enterprise informatics, data and analytics at Highmark Inc., a medical insurer and healthcare services provider in Pittsburgh.

The popular conception of predictive modeling involves a bunch of cloistered data scientists and statistical analysts writing algorithms on their own. The algorithms and analytical models do eventually need to be created, tested and run by analytics professionals, but Pitts said the bulk of the work takes place before any data is crunched. "Eighty percent of the work is in gathering the information you need to solve a problem," he said. "And before you start that, you have to be connected to the business."

It's a portfolio approach. You have to expect some things to work and some things to not.
Mark PittsVP of enterprise informatics, data and analytics, Highmark Inc.

To Pitts, it's crucial for the modelers involved in big data analytics initiatives to have business knowledge in addition to their math and science skills. The most elegant algorithm won't be of any use if it doesn't address a specific business need, he said. As a result, Pitts works to ensure that the data analysts on his team are familiar with the business units they're supporting and the issues those units face, so they'll be aware of problems that need solving and opportunities to take advantage of data to improve business operations and strategies.

Stay out of the analytics isolation chamber

Judith Hurwitz, president and CEO of consultancy Hurwitz & Associates, said some big data analytics applications involve exploring data in search of patterns or trends that might be relevant to an organization. But she agreed with Pitts that in most cases, there needs to be a tight connection between modelers and business users. Analytical models "can't be created in isolation," Hurwitz said. "Otherwise, you're just doing a mental exercise."

She added that the situation has evolved as organizations have gained more experience with capturing, storing and analyzing big data. "Before, people were just excited about how much data they could get into a Hadoop system," she said. Now, they're thinking more about how to use the available data "to make a difference and to drive more revenue or save money" -- which means predictive models must be designed based on tangible business goals and objectives.

If that doesn't happen, analytical modeling work can end up being a wasted effort. "Whether it's big data or not, the foundation is still on defining the business problem well," said Dean Abbott, president of consultancy Abbott Analytics Inc. "I've seen a lot of projects fail because the analysts were kind of running amok, building good models but ones that weren't really applicable to the business."

Modeling goal: More wins than losses

At Highmark, once a business issue and relevant data sets have been identified, the nitty-gritty work of developing a predictive model is an iterative process. Pitts said the analysts on his team typically try out several different types of models -- for example, linear regressions or neural networks -- to find the one that best fits the application at hand. While developing models, they also set up separate data sandboxes with relevant information, walled off from the full database, so analysts can explore the different options and test prototype models to ensure that they work properly before deploying and running them for real.

Highmark, which insures more than 5 million people in Pennsylvania, Delaware and West Virginia, uses modeling, data mining and analysis tools from software vendor SAS Institute Inc. Modeling projects primarily focus on analyzing insurance claims and health records to try to identify patients who are likely to need additional care in the future -- people who haven't received recommended vaccinations, screenings and follow-up care, say, or whose symptoms indicate that they may have undiagnosed illnesses such as diabetes or heart disease. The analytics team also looks to pinpoint clinical interventions that could reduce the severity of illnesses, helping to speed patient recovery and reduce healthcare costs.

Because the data analysts try out different approaches to building models, the process can be time-consuming -- and some projects simply fail. "It's a portfolio approach," Pitts said. "You have to expect some things to work and some things to not." That makes it important to get ongoing buy-in on big data analytics programs and investments from upper-level management in order to maintain support for modeling initiatives when failures do occur, he added. Program managers need to manage the expectations of corporate and business executives to make sure they understand that the wins will more than compensate for the losses and produce the kind of big data benefits that organizations are looking for, Pitts said.

Ed Burns is site editor of SearchBusinessAnalytics. Email him at and follow him on Twitter: @EdBurnsTT.

Executive editor Craig Stedman also contributed to this story.

Next Steps

See why consultant Wayne Eckerson says building analytical models is a mix of art and science

Get real-world insight and examples in our guide to big data analytics tools and best practices

Read consultant Rick Sherman's checklist of to-do items for big data analytics project managers

Dig Deeper on Big data analytics

Join the conversation


Send me notifications when other members comment.

Please create a username to comment.

What's the key to success in building effective predictive models for big data analytics and other analytics applications?
In my opinion the key is the same as it's always been; you have to start by knowing what question to ask. Big data enables us to answer questions we've never been able to answer before, but if we can't start by asking the right question, the answers won't have much impact on success. That said, understanding the advantages that big data affords analysts over traditional modeling techniques can help you stretch your expectations of what a model can do.
This sounds wise. It's not hard to imagine some future system that can instantly answer any question we can think of. But if you don't know what you're aiming at and why you won't get very far. 
Totally in agreement! The questions to ask are at the heart of success. But then, also knowing how to better use your data to get good insights: is all data alike, are there subgroups where certain models work better than others, are all attributes informative? Both questions are actually related to the same problem: dimension reduction, which is especially an issue with big data projects but also a great opportunity for getting much more information.