Survey: Big data projects sneak up on basic BI on IT priority list

TechTarget's 2015 IT Priorities Survey shows that while businesses are still investing in basic BI and data warehousing capabilities, big data initiatives are becoming almost as prevalent.

The use of data visualization software is picking up steam at Nationwide Mutual Insurance Co. Wes Hunt, the Columbus, Ohio-based company's chief data officer, said Nationwide also is moving to incorporate more data sources into analytics applications -- for example, unstructured data from call center notes and machine data from server log files, some of the typical grist for the mill of big data projects.

"We're looking at the big data space," Hunt said. "We've been working on that, and it's starting to grow and more use cases are becoming apparent."

Hunt's experience at Nationwide reflects the general state of data management and analytics in the enterprise today, according to TechTarget Inc.'s 2015 IT Priorities Survey. Basic business intelligence, reporting and data warehousing projects are still at the top of the list of planned software initiatives for this year, the survey found. But they're being augmented -- and perhaps in some cases supplanted -- by big data initiatives involving the use of advanced analytics tools.

In the survey, which was conducted late last year, 31% of the 2,212 respondents worldwide said their organizations plan deployments of BI, analytics or data warehousing tools in 2015. That number was down from last year's survey, when 41% of respondents said they were planning such initiatives. But it again was the top response in the software category, just ahead of mobile applications and data integration, which tied for second at 30%.

Advanced data management, analytics on the advance

At the same time, more advanced data processing and analytics initiatives appear to be on the rise. Projects involving advanced database management systems, like NoSQL, columnar and in-memory technology, were up slightly from last year to 15% in the 2015 survey. More impressively, substantial numbers of respondents also said their companies plan to invest in big data analytics (25%) and big data processing and management (21%), neither of which was broken out as a separate response in the previous survey.

"Everyone has plenty of data at this point. Now it's about using it," said Cam Fortin, senior director of product development at Inc., an online wine retailer based in San Francisco.

This year, Fortin and his team are focusing on improving the site's recommendation engine. Using a platform from RichRelevance Inc., they develop algorithms for making product recommendations to customers. Currently, the recommendations are based mostly on similarities between the product a shopper is viewing and others that are available on the site, but Fortin said he thinks they could be improved by incorporating specific customer data, particularly purchase history information.

Such mixing of data sources is the kind of big data initiative that many businesses say they're planning for 2015, according to consultants.

From what big data is to what to do with it

"I think you're really seeing a shift from businesses asking 'what is [big data]' to doing it at scale," said Vince Dell'Anno, managing director of the global big data practice at Accenture's analytics consulting unit. "I think that conversation is forcing companies to rethink what technologies they'll use."

The reason, he said, is that the BI and data warehousing space is relatively mature: Most businesses already have those tools in place. Now many are moving on to more advanced and potentially more valuable technologies, such as Hadoop, Dell'Anno added.

This new technology focus comes with a few words of caution, however. The growing interest in advanced data management and analytics tools could lead businesses into trouble if they don't have the right use case for them.

IT vendors are heavily promoting things like in-memory databases, NoSQL software and especially Hadoop. Jack Phillips, president and co-founder of the International Institute for Analytics, a research and consulting firm, said such technologies can help users do some exciting things. But he has seen CEOs encourage their IT teams to implement new platforms and tools without really understanding their particular strengths and weaknesses.

"With the concept of big data, the expectations are so far out ahead of the reality," Phillips said. "Increasingly, enterprises are trying on these new paradigms -- NoSQL, Hadoop -- to buttress the data warehouse they have. But the truth is Hadoop is by no means a replacement or a free ticket."

Big data variety adds to integration needs

Mixing various data sources for analytics uses also increases data integration requirements, which might help explain integration's high ranking in this year's IT Priorities Survey. The issue of data governance is another key factor in the progression from basic BI and data warehousing to more advanced analytics. The survey showed that the percentage of organizations planning data quality and governance initiatives this year is essentially unchanged from last year, at 20%. But good data governance becomes particularly important as organizations start implementing new tools, including self-service analytics and visualization software that opens up data to more business users for analysis.

Hunt said Nationwide has been expanding its use of data visualization tools over the last couple years and plans to continue to do so in 2015. Along the way, he and his team have had to develop strong data governance policies to ensure that individual end users have access only to the data they need and that all analyses are based on sound information. He said it would have been risky from a data-quality standpoint to introduce self-service BI and visualization capabilities without putting the governance policies in place first.

"Quality is a little bit like clean drinking water: When it's not clean, you feel it, but you don't really want to make these investments in cleaning the water that comes out of the well," Hunt said.

Phillips thinks that as adoption of big data projects continues to grow, more organizations are likely to realize how important good data quality is to the investments they're making. Companies that "have put concepts of data governance aside will eventually decide that in a big data world, governance is critical," he said.

Ed Burns is site editor of SearchBusinessAnalytics. Email him at [email protected] and follow him on Twitter: @EdBurnsTT.

Next Steps

Advanced analytics is not just about big data

Businesses need to get a handle on big data analytics

Financial planning is a big driver of advanced analytics adoption

Hadoop data engines and NoSQL evolve in 2015

Dig Deeper on Business intelligence strategy