News Stay informed about the latest enterprise technology news and product updates.

Data mining, predictive analytics: trends, benefits and challenges

Get an overview and assessment of data mining, predictive analytics and related business modeling tools and trends in an interview with James Kobielus of Forrester Research.

Predictive analytics software is getting increasing amounts of attention from technology users, vendors and analysts. The advanced analytics technology is designed to enable organizations to mine data and build predictive models that can help them analyze future business scenarios, such as customer buying behavior or the financial risks of proposed corporate investments.

Until now, data mining, predictive analytics and related business modeling technology has been used almost exclusively by highly skilled – and highly paid – statisticians, mathematicians and quantitative analysts. But that’s changing as business intelligence (BI) and analytics vendors offer more user-friendly predictive analytics tools – or is it? In this interview, conducted via email, Forrester Research Inc. analyst James Kobielus assesses the current state of predictive analytics software and provides an overview of predictive analytics trends and the potential benefits and challenges of using the technology.

There’s a lot of talk that predictive analytics is the next big battleground in the business intelligence market. Do you agree? And if so, why is that? Yes, I agree. The core BI market has become quite crowded with vendors providing solutions that do a great job of supporting rich analysis of historical data. It would be a gross oversimplification to claim that the traditional BI market has become commoditized. However, vendors all over the BI arena are looking to new types of advanced analytics applications as a way of avoiding the “me too” syndrome of look-alike offerings that blur into each other and fail to differentiate in a way that can justify a premium price. Predictive analytics is a natural evolution path for BI offerings, and it’s something that many users want but have often needed to obtain separate from their current BI tools.

From a general standpoint, is predictive analytics software ready for broader use? Or are there limitations that need to be addressed first? Yes and no. Yes, Forrester is seeing an impressive new generation of user-friendly predictive analytics tools that are geared to the needs of the mass market of information workers and other nontraditional users.

But no, traditional predictive analytics tools are still very much the province of a specialized cadre of statistically and mathematically savvy modelers with an academic background in multivariate statistical analysis and data mining – although most of the established predictive modeling vendors have made great progress in rolling out more user-friendly visual tooling. Still, I had to reflect the current state of the industry when I published my Forrester Wave report on predictive analytics and data mining tools in early 2010. I didn’t put a huge emphasis on features geared to business analysts, subject matter experts and other “nontechnical” information workers. The core problem with today’s offerings is that many of them remain power tools with a steep learning curve and a commensurately high price.

What’s happening with predictive analytics software? Can you give us an overview of the key technology trends you’re tracking? The key trend is the move toward user-friendly, self-service, BI-integrated predictive analytics tools that encourage more pervasive adoption. Another is the move toward integrating more predictive analytics functionality into the enterprise data warehouse, through in-database analytics. That’s an approach in which data preparation, statistical analysis, model scoring and other advanced analytics functions can be parallelized and thereby accelerated across one or more data warehouse nodes. In-database analytics also enables flexible deployment of a wide range of resource-intensive functions – such as data mining and predictive modeling – to a cluster, grid or cloud of high-performance analytic databases.

More on advanced analytics

Find out how predictive analysis is crucial to the data scientist role

Learn how in-memory analytics can match big data business need

Get user tips on implementing location intelligence software

We’re also seeing the growing adoption of open frameworks for building predictive analytics models for data mining, text mining and other applications. The principal ones are MapReduce and Hadoop, which have been adopted by a wide range of vendors of analytics tools and data warehouse platforms. In the coming year, we’ll also see the beginning of an industry push toward an open development framework for inline predictive models that can be deployed to complex event processing (CEP) environments for real-time data streaming applications. Still another trend is the embedding of predictive analytics features in customer relationship management (CRM) applications to drive real-time “next best offer” recommendations in call centers and multichannel customer service environments.

Why should prospective users be interested in predictive analytics? What are the potential benefits or competitive advantages that companies can get from it? Business is all about placing bets and knowing if the odds are in your favor. Business success depends on your company being able to predict future scenarios well enough to prepare 

plans and deploy resources so that you can seize opportunities, neutralize threats and mitigate risks. Clearly, predictive analytics can play a pivotal role in day-to-day business operations. It can help you focus strategy and continually tweak plans based on actual performance and likely scenarios. And, as I noted in a Forrester blog post, the technology can sit at the core of your service-oriented architecture strategy as you embed predictive logic deeply into data warehouses, business process management platforms, CEP streams and operational applications.

The grand promise of predictive analytics – still largely unrealized in most companies – is that it will become ubiquitous, guiding all decisions, transactions and applications. For the technology to rise to that challenge, organizations must move toward a comprehensive advanced analytics strategy that integrates data mining, content analytics and in-database analytics. We’ve sketched out a vision of “service-oriented analytics,” under which you break down silos among data mining and content analytics initiatives and leverage these pooled resources across all business processes.

You may agree that this is the right vision but have doubts about whether there is a practical, incremental roadmap for taking your company in that direction. In fact there is, and it starts with reassessing the core of most companies’ predictive analytics capability: your data mining tools. As you plan your predictive analytics initiatives, you should avoid the traditional approach of focusing on tactical, bottom-up, project-specific requirements. You should also try not to shoehorn your requirements into the limited feature set of whatever modeling tool you currently happen to use.

On the flip side, what kind of challenges or issues should people consider and be prepared for when they’re weighing a possible deployment of predictive analytics software? The learning curve, complexity and cost of predictive analytics tools are the principal challenges. Also, if you’re committed to deploying sophisticated predictive analytics, you’ll need to hire specialized, expensive talent to handle data preparation and cleansing, build and score predictive models, and integrate the models and their results into your BI, CRM and other application environments. And if you decide to integrate your predictive analytics initiatives with your data warehouse through in-database analytics, you’ll need to bring the groups who handle those functions together and get them speaking a common language.

Dig Deeper on Predictive analytics

Start the conversation

Send me notifications when other members comment.

Please create a username to comment.