The Hadoop distributed computing system will go mainstream in 2011 while predictive analytics capabilities are increasingly embedded with customer relationship management (CRM) software and other business applications, according to James Kobielus, a senior data warehousing and analytics technology analyst with Cambridge, Mass.-based Forrester Research Inc. SearchDataManagement.com got on the phone with Kobielus recently to get his key data warehouse, business intelligence (BI) and advanced analytics technology predictions for 2011. Here are some excerpts from that conversation:
Do you expect to see continued adoption of Hadoop in 2011?
James Kobielus: Hadoop will become a more mainstream data warehousing platform for larger enterprises in 2011. Hadoop has already achieved considerable adoption by enterprises for data warehousing applications in customer analytics, predictive analytics, social media analytics and other compute-intensive “big data” applications. In 2011, we will see a growing range of data warehousing vendors incorporate Hadoop architectures into the core of their product portfolios. IBM, with BigInsights, is there already with a strong enterprise-grade Hadoop platform. Pentaho and MicroStrategy also have strong Hadoop capabilities, and some of the startups are coming along very fast.
What’s going to happen with predictive analytics technology in 2011?
Kobielus: Next-best-action technologies, specifically recommendation engine technologies, will be embedded in most CRM applications to improve customer retention, up-sell, cross-sell, customer experience, optimization, anti-fraud detection and other key functions. Those next-best-action/recommendation engine technologies will incorporate predictive analytics at their very heart, and those predictive analytics models will increasingly be built in MapReduce, or in tools that support MapReduce interfaces. This is because MapReduce is really the industry’s first open, quasi-standard development framework for building these types of models. So, we will see MapReduce models being incorporated into predictive CRM platforms everywhere in 2011.
In what other ways will data warehousing technology change in 2011?
Kobielus: Enterprise data warehousing appliance products [will] begin to drop below $10,000 per terabyte in 2011 as the price war continues in the enterprise data warehouse market. We already see Microsoft, with its Parallel Data Warehouse and its Fast Track products, at between $11,000 and $13,000 for a fully configured appliance. Forrester expects that IBM, Oracle, SAP, Teradata and others will begin to offer at least one appliance-based solution at or around that $10,000 per terabyte range. This is because the appliance is essentially becoming a commodity in many ways. But also because the competition in the data warehousing appliance space is moving away from a laser focus on price/performance to more of sort of a high-level solution focus that is geared toward pre-integrated analytic solution appliances.
How do you see the world of self-service BI progressing in 2011?
Kobielus: Self-service BI in 2011 will become the only BI approach that the new generation of information workers will ever encounter. Fundamentally, the way it’s going for us is that everybody wants to have the prestige clients on their desktop for BI -- an in-memory client like a Tibco or a PowerPivot or a QlikView. So, what we’re going to see are these in-memory clients that support essentially [light] data mining with interactive visualization. [This will allow users] to bring millions and eventually billions of rows into memory and do some really sophisticated analyses. IT very much wants to go this route since IT doesn’t want to have to build cubes any longer. They don’t want to have to build all of the integration logic if the user can be given front-end tools that they can use to build their own visualizations and to pull data from the data warehouse.
Where do you see social media analytics heading this year?
Kobielus: In 2011, business process management (BPM) will incorporate social networking into its exception- and issue-handling tools to allow users, when they have a problem and they’re stumped, to handle a particular exception condition within a given workflow. They will turn to the social crowdsourcing capabilities of their BPM platform to help them quickly elicit and aggregate advice and guidance. In 2011, social-network-style crowdsourcing and analytics -- social network analytics -- will come into the toolkit of BPM professionals as a way of helping people to manage business processes using the best intelligence available within their organization and to do it in an ad hoc, real-time way. We’ve traditionally thought of BI as a one-way flow of intelligence from data sources to data consumers, who are usually human beings. Now BI is evolving into a new model where the primary intelligence that you want to harvest is in the heads of other users, and so you’re using social-network-style architectures to get the best information and guidance.