Increased adoption of cloud analytics applications and analytics as a service platforms. A bigger role for graph database and analytics technology in big data applications. Consolidation in the NoSQL database market. Some high-profile big data project failures involving still-emerging open source technologies.
Those are some of the business intelligence, analytics and data management developments predicted to happen over the next 12 months by attendees at the 2016 Pacific Northwest BI Summit. The conference, held annually in Grants Pass, Ore., brings together a small group of IT consultants and vendor executives to discuss technology issues and trends, and one of the sessions each year features round-robin predictions by the participants. Here are highlights of the view into the expected BI, analytics and big data future that emerged from this year's event.
More analytics in the cloud. Various participants said they expect to see expanding use of cloud-based analytics systems, including analytics services offerings, as companies look to accelerate the deployment of analytics applications and reduce costs in the process. Hand in hand with that, they forecast growing deployments of data warehouses and big data systems in the cloud. In fact, Gartner analyst Merv Adrian said recent surveys by the consulting and market research company show there's already "a helluva lot more" Hadoop systems running in the Amazon Web Services cloud than was thought previously. Through its Amazon Elastic MapReduce platform, AWS has "more users of Hadoop than all the other vendors in the market combined," Adrian said, putting the number of EMR installations in the thousands.
Data governance to step forward -- maybe. Calls for stronger data governance processes have accompanied the increasing role of BI and analytics applications in driving business decision making, especially with big data environments expanding the amount and types of data that organizations are collecting and using across a larger set of systems. Some of the conference attendees predicted that data governance will finally become a more central focus in many organizations over the next 12 months. Others weren't so sure, though. "Data governance will still be boring to CEOs -- they just don't get it," said Claudia Imhoff, president of consultancy Intelligent Solutions Inc. in Boulder, Colo.
Graph technology trends higher. Graph databases are already getting increased attention from technology vendors and users alike because of their ability to map relationships between different data elements for more insightful analytics in applications that fit the graph model. Donald Farmer, vice president of innovation and design at BI software vendor Qlik, predicted that graph technology will become a central component of all database platforms as part of the big data future. Graph analytics could also aid in basic data management functions, such as metadata management, added Mike Ferguson, managing director of U.K.-based consultancy Intelligent Business Strategies Ltd.
NoSQL market shakeout ahead. In a bit of hyperbole, Ferguson cited a total of 250 NoSQL databases available for use. The actual number might not be that high -- but there are more NoSQL vendors than the market can support, he said in predicting that some consolidation is ahead. Adrian agreed, saying a small number of vendors are getting out in front of the rest of the NoSQL pack. Illustrating that, Gartner included 10 NoSQL software suppliers among the top vendors of operational databases in a Magic Quadrant report published last October, with four of them making the Leaders category. SQL-on-Hadoop query engines are also ripe for a reduction in current offerings, Ferguson said. Without any hyperbole, he noted that he's tracking 23 different SQL-on-Hadoop platforms, most of them emerging technologies that are still maturing. The plethora of choices "has to get simplified," he said.
Big data deployment nightmares. Speaking of emerging technologies, Adrian predicted there will be at least one "catastrophic failure of an open source project" that mainstream enterprises have bet on and deployed early in its development process. Nonetheless, he expects development and adoption of open source technologies to continue to drive the big data future forward. "I don't think that means the fertile soup of open source creation slows down -- in fact, it accelerates," Adrian said. "It's a more important avenue now [for big data technology development] than commercial mechanisms are."
Hadoop's core components may not be so core in future big data systems
Check out our guide to big data analytics technologies, issues and trends
Read more about the future of big data tools from Strata + Hadoop World