This is a two-part series on in-database analytics
- Is in-database analytics an emerging business intelligence (BI) trend?
Gartner included advanced analytics as one of the Top Ten Strategic Technologies for 2010 at its recent Symposium/IT Expo 2009. It's no wonder, then, that in-database analytics has become an important business intelligence (BI) trend, according to many experts.
Once limited primarily to government organizations and large financial services and insurance companies, advanced analytics is taking off in applications ranging from fraud detection and prevention to targeted marketing to financial strategy and risk management. In-database analytics has significantly bolstered this BI trend by making such applications affordable to organizations that can't afford either supercomputers or on-staff quantitative analysts, according to Merv Adrian, president of IT Market Strategy.
By integrating analytic processes with business applications on a common enterprise data warehouse (EDW) platform, in-database analytics enables companies to leverage increasingly affordable and powerful platforms in a far more cost-effective way. Furthermore, development time for complex analytic applications can go from weeks or days to hours -- or even minutes.
Neil Raden, president of Santa Barbara, Calif.-based Hired Brains, cites one real-life client example. The advertising arm of a major media company typically gathers tens of terabytes of market data over a two-week period. Under the old ETL regimen, the firm's research analytics team spent three or four days exporting, transforming, moving and distributing chunks of market data in order to gain business insights.
Once it had deployed an enterprise data warehouse platform with in-database analytic capabilities, however, the team could run programs directly in the database, eliminating system bottlenecks. Setting up two weeks' worth of data takes about 20 minutes, the director of research analytics reports.
What makes in-database analytics a unique BI trend?
By making database functions transparent, in-database analytics enables BI application developers "who don't know how to do SQL in a complex way" to create new applications involving complex manipulation of databases, without having to depend on the expertise of SQL programmers, Adrian noted.
Indeed, in-database analytics has the potential to level the playing field in BI, enabling agile, smaller companies to compete with the big guys in terms of predicting, identifying and responding to market trends in a timely and effective manner. Models based on in-database analytics are also more flexible than traditional predictive models, whose parameters -- once set -- are difficult to change, Raden said. This is critical in today's business world, where executives and knowledge workers have to make rapid decisions based on a flood of information from disparate sources.
In one case, modeling software based on in-database analytics has enabled a telephony service provider to do online pricing and margin analyses and usage-based micro-segmentation of the subscriber base. Models are regularly updated on the basis of rated call data records, which are continually collected from the carrier's global network.
Furthermore, experts report that consolidating on a single enterprise data warehouse infrastructure can pave the way to top-down governance of all BI and analytic initiatives. Companies can enforce enterprise guidelines for development templates, data cleansing and transformation rules, and they can leverage resources across analytic initiatives enterprise-wide, according to James Kobielus, a research director at Cambridge, Mass.-based Forrester Research.
However, getting to a place where all analytic initiatives use the same, fresh version of data and consistent models may require some hefty housecleaning and consolidating of data, not to mention a major reorientation of end-user and developer mindsets, industry experts cautioned.
In-database analytics a BI trend to approach thoughtfully
Today's predictive analytic initiatives tend to consist of disparate, project- and application-driven data marts, each with its own cadre of analysts and specialists and a different set of often-inconsistent data, warned Shaku Atre, president of Atre Group, Inc. She recommends that companies take a prioritized approach, focusing on key applications and data, rather than trying to do it all at once.
Be aware, too, that the in-database analytics market is young and volatile, with much of its potential still to be realized. Leading database and data warehouse vendors are still in the process of incorporating it into their platforms. Independent software vendors are just beginning to take advantage of the efficiencies of in-database analytics to develop new BI applications, often in partnership with data warehouse platform vendors.
"This should mean that more packaged analytic applications will become available, which is good news for the many companies that can't afford their own in-house quantitative analysts," Hired Brain's Raden said. "For example, a trucking company can buy an application that uses very 'hairy' analytics, and trust that it works, because other people have bought and benefited from it."
Furthermore, broad industry support of standards such as MapReduce and Hadoop holds the promise of bringing more flexible deployment of advanced analytic applications, as well as the ability for models to work with unstructured as well as structured data.
All that said, companies need not wait months or years to start deploying, according to industry experts.
"Look around and see if you have a problem you want to solve and what kind of information you need to solve it," Raden said. If it's an analytical problem involving large quantities of data, developing a model to solve it could be a lot cheaper and quicker with in-database analytics.
Elisabeth Horwitt is a freelance writer.