Sergey Nivens - Fotolia
Mainstream business intelligence and analytics applications focusing primarily on retrospective reviews of data can provide organizations with valuable information about corporate performance, customers and business trends. But companies looking for competitive advantages over business rivals are increasingly turning their gaze to the future -- and big data is broadening the scope of the predictive analytics they can do in order to see what's coming.
The vast quantities and varied forms of data that many organizations are now collecting give predictive modelers and data miners lots of material to work with in trying to forecast things such as customer behavior and effective business strategies. Unlocking big data benefits isn't just a matter of turning skilled analysts loose with predictive modeling tools, though. In fact, the process of developing useful models to analyze sets of big data isn't what people tend to think it is, according to Mark Pitts, vice president of enterprise informatics, data and analytics at Highmark Inc., a medical insurer and healthcare services provider in Pittsburgh.
The popular conception of predictive modeling involves a bunch of cloistered data scientists and statistical analysts writing algorithms on their own. The algorithms and analytical models do eventually need to be created, tested and run by analytics professionals, but Pitts said the bulk of the work takes place before any data is crunched. "Eighty percent of the work is in gathering the information you need to solve a problem," he said. "And before you start that, you have to be connected to the business."
Mark PittsVP of enterprise informatics, data and analytics, Highmark Inc.
To Pitts, it's crucial for the modelers involved in big data analytics initiatives to have business knowledge in addition to their math and science skills. The most elegant algorithm won't be of any use if it doesn't address a specific business need, he said. As a result, Pitts works to ensure that the data analysts on his team are familiar with the business units they're supporting and the issues those units face, so they'll be aware of problems that need solving and opportunities to take advantage of data to improve business operations and strategies.
Stay out of the analytics isolation chamber
Judith Hurwitz, president and CEO of consultancy Hurwitz & Associates, said some big data analytics applications involve exploring data in search of patterns or trends that might be relevant to an organization. But she agreed with Pitts that in most cases, there needs to be a tight connection between modelers and business users. Analytical models "can't be created in isolation," Hurwitz said. "Otherwise, you're just doing a mental exercise."
She added that the situation has evolved as organizations have gained more experience with capturing, storing and analyzing big data. "Before, people were just excited about how much data they could get into a Hadoop system," she said. Now, they're thinking more about how to use the available data "to make a difference and to drive more revenue or save money" -- which means predictive models must be designed based on tangible business goals and objectives.
If that doesn't happen, analytical modeling work can end up being a wasted effort. "Whether it's big data or not, the foundation is still on defining the business problem well," said Dean Abbott, president of consultancy Abbott Analytics Inc. "I've seen a lot of projects fail because the analysts were kind of running amok, building good models but ones that weren't really applicable to the business."
Modeling goal: More wins than losses
At Highmark, once a business issue and relevant data sets have been identified, the nitty-gritty work of developing a predictive model is an iterative process. Pitts said the analysts on his team typically try out several different types of models -- for example, linear regressions or neural networks -- to find the one that best fits the application at hand. While developing models, they also set up separate data sandboxes with relevant information, walled off from the full database, so analysts can explore the different options and test prototype models to ensure that they work properly before deploying and running them for real.
Highmark, which insures more than 5 million people in Pennsylvania, Delaware and West Virginia, uses modeling, data mining and analysis tools from software vendor SAS Institute Inc. Modeling projects primarily focus on analyzing insurance claims and health records to try to identify patients who are likely to need additional care in the future -- people who haven't received recommended vaccinations, screenings and follow-up care, say, or whose symptoms indicate that they may have undiagnosed illnesses such as diabetes or heart disease. The analytics team also looks to pinpoint clinical interventions that could reduce the severity of illnesses, helping to speed patient recovery and reduce healthcare costs.
Because the data analysts try out different approaches to building models, the process can be time-consuming -- and some projects simply fail. "It's a portfolio approach," Pitts said. "You have to expect some things to work and some things to not." That makes it important to get ongoing buy-in on big data analytics programs and investments from upper-level management in order to maintain support for modeling initiatives when failures do occur, he added. Program managers need to manage the expectations of corporate and business executives to make sure they understand that the wins will more than compensate for the losses and produce the kind of big data benefits that organizations are looking for, Pitts said.
Executive editor Craig Stedman also contributed to this story.
See why consultant Wayne Eckerson says building analytical models is a mix of art and science
Get real-world insight and examples in our guide to big data analytics tools and best practices
Read consultant Rick Sherman's checklist of to-do items for big data analytics project managers