News Stay informed about the latest enterprise technology news and product updates.

BI search platform eyes middle ground between BI and unstructured data

Find out how BI search platforms are helping companies like AMD bridge the divide between analytics and unstructured data.

Between the world of traditional business intelligence (BI) and the world of advanced or predictive analytics lies...

an elusive “middle ground,” according to Gregg Hansen.

Hansen is vice president of applications at AMD, a maker of PC and server microprocessors. AMD, whose battles with Intel have been well documented, is always looking for ways to gain a competitive advantage, Hansen said.

To that end, AMD has invested in traditional BI tools, including reporting and visualization software from SAP Business Objects and Microsoft, as well as some predictive analytics tools. The company’s executives and engineers use the tools to explore and identify insights into both corporate and product-related data.

But those applications and tools are limited to exploring structured data found in databases and spreadsheets. AMD has a plethora of unstructured data housed in wikis, product specs and emails, Hansen said. If the company could find a way to integrate that unstructured content with its structured data for ad hoc queries, AMD execs and engineers could potentially gain new insights to help them in their battle with Intel.

“Merging structured and unstructured data into a single query interface is huge,” Hansen said.

Traditional BI not suited to unstructured data

AMD’s situation is not unique. It is especially common at companies that sell or service a wide array of products with related data housed in text documents and other unstructured forms, according to Boris Evelson, an analyst with Forrester Research.

In such cases, as at consumer products companies, products are so diverse that each has any number of unique dimensions (size, color, material, etc.), making comparing and analyzing the products against one another difficult for tools designed for normalized, structured data.

Significant jury-rigging is required in order to customize traditional BI and search tools to perform analysis on such disparate data, Evelson said. That takes up valuable time and manpower. “That’s one of the biggest modeling headaches,” he said.

Of course, workers also have the option of sifting through the reams of unstructured content in corporate documents and emails by hand -- sometimes literally -- to find the data they’re looking for. Obviously, such a labor-intensive approach is simply untenable at most organizations.

But there is another option for traversing this middle ground, as AMD’s Hansen calls it. It is a class of enterprise search technology that’s optimized to query both structured and unstructured data that returns search results in the form of tables, graphs and other BI-like interfaces to help workers answer specific, targeted business questions.

The goal is “to deliver a unified view of information by breaking down the wall between the quantitative world of BI and the semistructured world beyond its scope,” Gartner analyst James Richardson wrote in a 2009 report.

One such product is Active Intelligence Engine (AIE) from Newton, Mass.-based Attivio. When a search is performed via AIE’s user interface, the platform -- unlike traditional BI tools and search engines -- queries both an organization’s structured and unstructured data sources to find the most relevant data.

The data is integrated into a single, inverted index, where Attivio’s technology determines relationships between the data and then visually presents the results to the user, explained Sid Probstein, Attivio’s chief technology officer.  

A search for a type of microprocessor, for example, might return a list of documents most relevant to that particular chip, a line graph plotting its sales volume over time, and a pie chart illustrating the chip’s percentage of overall revenue.

As Gartner’s Richardson explained: “Users benefit from the flexibility of this ad hoc approach, dynamically generating queries to join together any set of tables and document collections to use in their analysis."

Other vendors competing with Attivio in the unified information access and analysis market include Cambridge, Mass.-based Endeca Technologies and Exalead, a Paris-based vendor recently acquired by Dassault Systèmes.

Reduce modeling headaches, increase BI adoption

In addition to helping workers gain new insights, technology like AIE could also help make BI more accessible. Because its user interface is modeled after the familiar Web search format, business users who might otherwise be too intimidated to try a traditional ad hoc query tool are likely to take to it, according to Richardson.

“CIOs looking to make BI more pervasive might consider AIE, as it will make it easier for workers not accustomed to traditional BI tools to find the information they need to make decisions,” he wrote.

Forrester’s Evelson agreed, noting that tools like AIE resemble Google but with “a lot more analytics baked in.”

The tool’s single-index approach also eliminates the need to create complex data models and schemas, freeing data architects to work on more pressing projects.

“Endeca and Attivio achieve this by dynamically inferring the schema based on the ingested data,” Evelson wrote in a recent report. “Every element can be potentially used as a fact or dimension and changed on the fly; changes in source system schemas therefore automatically propagate up all the way into the BI application.”

That does not mean, however, that AIE and similar platforms can replace traditional BI and analytics tools. Instead, they are meant to complement them, a point even Attivio’s Probstein stressed. Report building, for example, is better left to a more traditional BI tool like Crystal Reports than to a platform like AIE.

And they are not necessarily a good fit with all organizations. Those companies with mostly structured data housed in traditional databases, for example, would probably find little benefit in a unified information access and analysis platform like AIE.

There are also risks associated with vendor lock-in. AIE, for example, is a proprietary platform, meaning customers cannot customize it with lower-priced databases or hardware, Evelson said.

But those organizations that fit the target profile – lots of structured and unstructured content around diverse data sets – could definitely benefit, he said.

AMD recently purchased Attivio’s platform to tackle a separate issue – improving the company website’s search capabilities. But Hansen is excited by the platform’s analytic possibilities.

He said: “We think there is an opportunity for Attivio to give people a very simplified way, in a simplified syntax, to hit this data and get out of it what they want.”

Dig Deeper on Business intelligence software

Start the conversation

Send me notifications when other members comment.

Please create a username to comment.