This content is part of the Essential Guide: Guide to using advanced analytics and AI in business applications

Essential Guide

Browse Sections

Siloed data doesn't have to stop you from doing advanced analytics

Breaking down data silos is rarely easy, but doing so can lead to more effective analytics that bring together data from all of a business's operational systems.

BOSTON -- When the U.S. Tennis Association wanted to make more of its operations analytics-driven, it ran into an increasingly common problem: siloed data.

"Anything we did with data was all short-term thinking," said Kent Schacht, senior director of data strategy and customer engagement at the USTA, in a presentation last week at the 2017 Big Data Innovation Summit here. "Everything was a workaround or a quick patch. In the past, nothing came together."

Data silos have always existed. Previously, however, when an enterprise's analytics were mainly focused on business intelligence, IT would launch concerted efforts to bring everything together into a centralized data warehouse.

It would be harder to pull off such a centralized data warehouse in today's big data world. Data is more and more coming from a diverse array of systems, including cloud platforms, analytics tools and Hadoop-based data lakes. To make matters more complicated, lines of business are increasingly deploying report-generating data visualization tools on their own outside of any enterprise data plan.

Bring your data under one roof

To start building a better picture of tennis fans and grow the number of people who play in leagues, the USTA began stitching together its disparate systems. The group had a database of people who currently play in leagues, but this was stand-alone. Schacht said he and his team created a centralized data lake that combined this member data with online website data. It also brings in data from a mobile app used by attendees at the U.S. Open tennis tournament, which is operated by the USTA.

Schacht and his team are now analyzing the data in this centralized data lake to identify young people who might be interested in playing in a USTA league and connect them with available coaches. This wouldn't have been possible in the USTA's past of siloed data.

"Now we can understand how we're touching people," Schacht said. "We've really built the foundation, and we're exploring how we can use it to grow the business."

Bringing all your data together into a centralized system is one way to solve the problem of siloed data, but it's certainly not the only one. Other businesses are using software that pulls in the data that users need for analysis as it's needed, rather than just dumping everything into one place. This approach demands strong database connectors and advanced analytics software.

AI makes sense of different data types

In a presentation at the conference, Lisa Plimpton, director of digital strategy and data innovation at pharmaceuticals maker Pfizer Inc., said different data types can help drug researchers understand the effects of a new medication. Analyzing it can be a challenge, however.

"Today what's most accessible and structured is from medical care, but there's untapped information in other sources of real-world data," she said.

Pfizer is increasingly looking to collect data from wearable fitness trackers, social media platforms and consumer data for patients enrolled in trials to develop a more holistic view of how new medications are impacting patients. In one trial for a new Parkinson's disease medication Pfizer is using motion tracking sensors to monitor Parkinson's patients' movements to measure disease progression.

To analyze some of this data Pfizer has partnered with IBM to employ the Watson cognitive computing platform. Plimpton said that while Watson is not the only useful tool, this kind of advanced analytics is necessary to understanding unstructured data from different sources. The API connectors offered by the platform make connecting to different data sources possible.

"It's imperative that we look at these real-world data types and use AI to bring better medicine to patients faster," she said.

Time to make the most of siloed data

Art Nazzaro, senior manager of digital enterprise transformation at professional services firm Ernst & Young, said businesses are going to have to live with this kind of siloed data for some time to come and find ways to make the most of their data, regardless of the system in which it resides.

He pointed to a Gartner report that found that about 75% of IT shops are what they call bimodal, meaning they have some of their operational data-generating systems in the cloud and others on premises. This isn't going to be solved any time soon, so finding ways to analyze data from different sources is paramount.

"We're still living with the patterns of silos, of operational models that have not been transformed," Nazzaro said. "We might think that everything's in the cloud, but we'll look back and see that that isn't true."

Next Steps

Cloud integration can help tear down data silos

Big data can make the problem of data silos even bigger

How IoT is helping utilities extract value from data silos

Dig Deeper on Advanced analytics software