"Text analytics is a very deep technology piece that end users never see, whereas BI search is something every report consumer can see and use," Russom said.
Business intelligence search increases productivity and self-service
On the front end, BI search enables productivity and efficiency gains, by helping end users find reports and information faster, Russom said. Some companies seem to agree, with 62% of TDWI's survey respondents rating the perceived business value of BI search as high or very high. Though only 9% of respondents had deployed BI search technology, 15% are in the design or implementation phase, and 40% are in the exploration phase. Russom is "amazed" that more companies haven't implemented BI search.
"Search capabilities are now built into most BI platforms. [Many] have switches -- you just turn on the switch and it enables search capabilities for their BI platform," Russom said. "If it's that easy to implement, just go ahead and do it. In fact, with many licenses, you don't actually pay extra money for it."
Companies that haven't implemented BI search may be strapped for time or may be dealing with competing priorities, he said. But he thinks organizations might move BI search to the top of the queue if they considered its potential long-term benefits, such as reducing end user requests for new reports or tech support.
"BI search enables a wide variety of self-service capabilities that offload work from IT to the report-consumers," Russom said.
Text analytics increases knowledge and insight
Organizations can gain valuable insight by extracting and analyzing information from unstructured data, Russom said. In the BI context, text analytics is on the back end, part of the underlying data integration infrastructure, he explained. It can be complex to implement, in part because of the development work required and the volumes of unstructured data at most companies -- though the volumes may not be as big as previously thought, he found.
An oft-cited statistic is that, on average, unstructured information makes up 80% of the data in an enterprise, Russom said, but this TDWI survey indicated that 31% of enterprise data is unstructured, 22% is semi-structured (as in XML, etc.) and 47% is structured. Though it's a change from previous figures, it doesn't change the importance of unstructured data to BI, he said. A company's view of corporate performance is incomplete without the contextual information that's provided by unstructured and semi-structured data.
One example of this was an insurance company Russom surveyed that uses text analytics to examine call center records. Call center representatives entered a status code (structured) for each call, which helped the company measure metrics such as customer satisfaction. To gain insight beyond the codes, the organization used text analytics to look at the comments field (unstructured text). The company found that customer satisfaction metrics looked very different depending on whether they considered the status code alone or combined that information with data extracted from the text comments field.
"In the text, you could find things about the customer that the structured data from that call did not capture. So they were actually able to create some new metrics that had to do with softer quantification of customer satisfaction," Russom said.
Some companies seem to recognize the importance of using text analytics in BI, the survey found. It said that 51% of respondents rated the business value of text analytics as high or very high. As for usage, 6% reported having text analytics already deployed, while 7% are in the design or implementation phase, and 43% are in the exploration phase. Russom was not that surprised by the low implementation numbers.
"Text analytics is a full-blown development project. It is not the kind of thing that you can install and be using in a week, like BI search. Text analytics takes a lot of design time before you even select a tool. And, because of the development work, it involves a lot of payroll costs," he said.
The costs and complexities of text analytics can be worth it, he said. Early adopters have used it for warranty analysis, fraud detection and call center analysis, but many companies can benefit.
"If you don't have data in your warehouse representing information that originates from unstructured sources, your data warehouse may be a single version of the truth," Russom said. "But it's not a complete version of the truth."