There may be business value in analyzing "big data," but companies need to make sure they have the right data architecture in place before getting value out of analytics.
In a webinar titled Strategies for Data Exploration and Analysis in the Age of Big Data Analytics, TDWI research director Philip Russom said organizations collect vast amounts of data that may help them develop new insights about their business and customers, but just because they have the data in-house doesn't mean they're ready to analyze it.
"To enter the brave new world of big data analytics you'll probably need to extend your data warehouse environment and make some adjustments to your data warehouse architecture," Russom said.
The point of adding more data platforms to the data warehouse environment is to deal with the diversity of data types and analytic algorithms.
Variety is likely the main reason that traditional data warehouse architectures aren't up to the big data challenge. A lot of data being collected is unstructured, including social media posts or natural language from health records. Similarly, many analytic tools aren't designed to pull data from traditional relational databases. The velocity piece of big data is also a challenge for legacy databases. Relational databases may struggle to keep up with streaming data from machine sensors, for example.
For these reasons, Russom recommended implementing a new data architecture that can keep up with big data demands. This is how businesses move beyond just managing data to extracting value.
"I regularly tell people to never be content to manage big data as a cost center," Russom said. "We want to be sure that we get some business value out it."
Analytics is necessary to pull useful information from big data sets. But Russom said there isn't one tool that performs all analytic functions. Analyzing data starts with capturing it. Then an organization needs to explore it to see what's there. From there, the real analysis happens. Finally, the analysis output must be put into a visualization so executives can make sense of it.
Each step in the process may demand its own tool, Russom said, which may require an organization to build new features into its existing data warehouse architecture. Some data management professionals may scoff at this idea, as it will inevitably add greater complexity and make the architecture more difficult to manage. But Russom said data managers worry too much about this.
"The point of adding more data platforms to the data warehouse environment is to deal with the diversity of data types and analytic algorithms," he said. "Managers have been dealing with complexity for years, so I think the vast majority of professionals are coping with it quite easily."
Read more on managing a big data architecture
See how big data is changing the role of the data architect
Learn why marketing and advertising are the biggest users of big data
Read other tips on planning for big data analytics
Russom provided more tips for managing a growing data warehouse architecture:
- Determine business drivers for new data marts or applications before implementing them.
- The physical layout of the warehouse (for example, where data is stored) often is not as important as the conceptual or logical level, which determines how applications function alongside each other.
- Be aware that the leading barriers to successful architecture development are skills and staffing.
- Address other barriers, like executive sponsorship and funding, early in the project.
- Establish data warehouse standards, but be open to exceptions.