This content is part of the Essential Guide: Guide to telling stories with data: How to share analytics insights

Effective big data visualization starts with simplicity

Big data analytics offers lots of opportunities to visualize data for business users. But analysts have to be careful not to overload big data visualizations with too much information.

The business stakes of data visualization projects are getting higher, as more and more organizations deploy big data systems and look to take full advantage of the rapidly growing pools of data they're collecting. In the past, data visualization often was more closely associated with self-service business intelligence and data discovery applications run by business users looking to create basic charts on revenue, profits and other key performance indicators. But data visualization tools increasingly are being applied in big data analytics environments as part of efforts to coalesce a diverse mix of data -- often from both internal and external sources -- into actionable information.

Charles Whittaker, Avant Inc.Charles Whittaker

The big challenge facing IT, BI and analytics teams on big data visualization initiatives is how to distill all the data they're dealing with into an easy-to-grasp form that can have a meaningful impact on the decision-making process. It's tempting to turn data visualization software loose on all sorts of data, and build elaborate charts and graphics. But a more measured and methodical approach might get better results, according to analytics managers like Charles Whittaker, head of BI at Chicago-based online lender Avant Inc.

Whittaker said he tries to keep his team from going overboard and building too many data visualizations because he thinks they sometimes can distract from the true objectives of analytics applications, such as improving business processes and enabling better business decisions. "I stress steering clear of vanity metrics," he said, pointing to things like reporting on the number of loans issued. "You don't need fancy visualizations for that."

Don't go too far in visualizing data

In addition, Whittaker preaches the benefits of simplicity for the visualizations that do get built at Avant. He and his team use a BI tool from Looker Data Sciences Inc. to analyze customer data in order to help the company price its financial products more precisely for individual borrowers and segmented groups of customers. They also use Looker, plus the D3.js open source visualization library in some cases, to visualize the data for corporate performance reports delivered to Avant's executive team.

"Most business decisions that will drive growth long-term, you can get out of a pivot table or simple chart," Whittaker said. "There's a million ways I can plot performance data, but what I'm really trying to figure out is what [customer] segments I can better price."

Even when it comes to the more heavy-duty data science work of developing and running predictive models to score new customers on their creditworthiness, he doesn't see the need to create intricate big data visualizations to present the analytical findings. That would be a waste of both time and effort when the goal of the modeling work is simply to understand a correlation between different data elements for specific customers, Whittaker said.

Most business decisions that will drive growth long-term, you can get out of a pivot table or simple chart.
Charles Whittakerhead of business intelligence at Avant Inc.

Paul Bradley, chief data scientist at healthcare administration software vendor ZirMed Inc., based in Louisville, Ky., also makes a point of trying to prevent visualizations from overwhelming hospital officials who use reports that the company sends to its customers. ZirMed's software as a service applications help healthcare providers process medical insurance claims; before passing along the claims to insurers, the vendor runs predictive models against them to check for possible missed billing codes covering treatments that typically would be associated with the listed medical procedures.

As part of its services offering, ZirMed delivers reports to clients with visualizations of commonly missed billing codes and other metrics. Bradley said the company's analysts need to keep in mind that the hospital administrators reading the reports may not have the time or interest to delve into complicated graphs and charts. "We spend so much time with complex relationships in large data sets," he said. "But the main goal of my team is to boil that down to the minimum nuggets people need to do their job."

Inside job on big data visualization

Things are different on development of data visualizations for use by Bradley's own team. Working with a mixed set of data from healthcare organizations and databases at the U.S. Census Bureau and the Centers for Medicare & Medicaid Services, ZirMed's data scientists look for correlations between tens of thousands of variables to get an idea of the medical procedures that healthcare providers tend to cluster together, and the ones that providers most frequently forget to bill patients for. Then, they use the correlations to build and update the predictive models used to check claims.

So much data is involved that the only way to make sense of it is by visualizing it, Bradley said. And in this case, he added, it's reasonable to build more complexity into the data visualizations. The members of his team are used to grappling with complex data, so detailed visualizations don't faze them. Some of the visualization work is done in Excel, but more complicated tasks are handled in Tableau's BI software, enabling the data scientists to take a deep dive into the available info. "My team wants to look at the patterns and trends from that data," Bradley said. "We want to look at all the data elements that describe what happened to the patient while they were in the care of the doctor."

Analytics tools and techniques are advancing rapidly, with Hadoop and other big data technologies helping to push them along. But a predictive model or data mining algorithm can't change business processes on its own. To have a real effect, the findings of big data analytics applications need to be communicated in organizations, and that's where the power of effective big data visualization efforts becomes critical.

And it isn't rocket science, so to speak. "I talk to people who interact with predictive modeling technology every day who don't even know it, because it's embedded in a way that's non-intrusive," Bradley said, referring to things like the product recommendation engines on Amazon and other websites. He added that analysts who are visualizing big data similarly need to find "a clean way to deliver information that comes from really complex analytics."

Next Steps

Visualization tools play an important role in big data success

Go beyond simple visualizations with this advanced analytics guide

Use visualization tools to tell a story with your data

Dig Deeper on Data visualization software