Petya Petrova - Fotolia
As Darth Vader once said, "If you only knew the power of the dark side."
That's the message coming out of Deloitte, whose new report, Dark analytics: Illuminating opportunities hidden within unstructured data, doesn't quite present the drama of a galaxy-wide struggle, but nonetheless hints toward substantial gains from analytics practices that are still a dark art for some companies.
At issue is unstructured data analytics using deep learning techniques. Data types like free text; image, audio and video files; and research papers hosted on nonindexed web servers hold substantial intelligence. But enterprises have traditionally had limited visibility into these dark data types, according to the report, which is part of a broader report on technology trends for 2017.
That is no longer the case for leading businesses, and soon won't be for average enterprises, said report co-author Paul Roma, who heads the company's Deloitte Analytics consulting practice.
It's time to light up dark data
"This notion of unstructured data and the ability to light up this dark data, the time for that is now," Roma said. "The ability to tap into it and mine it and use it for decision-making is becoming mainstream. It's becoming table stakes."
Not all dark data types are equally as accessible for businesses today. Roma said text files are the most easily analyzed. There's no shortage of API services from companies like Microsoft, IBM and Google that can perform advanced sentiment analysis of text files. Audio files are close behind, as transcription software can easily render a text file for analysis.
Images and videos are a bit more out of reach today, Roma said. But they are catching up rapidly. The pace with which image recognition algorithms have achieved human-level results has been rapid. All that's left now is to find the business cases for this kind of unstructured data analytics.
"The expense and time to use these techniques is nowhere near what it was two or three years ago," Roma said.
API services help reap benefits
Part of the reason for this is the availability of API services that let users hand off data to a fully trained cognitive system. This means that typical enterprises don't have to have deep learning experts on staff to reap the benefits of the practice.
Additionally, a growing number of businesses may be using deep learning and AI without even realizing it. Many software companies are building these features into their products to enhance chat or search functions. Roma pointed to Salesforce's incorporation of the AI engine Einstein into its core customer relationship management platform.
None of this means dark analytics is without risk for enterprises. Roma pointed out that deep learning algorithms can often return a high number of false positives and specious correlations. It takes a lot of business and domain knowledge to make sure they stay on track.
Additionally, he's seen a number of companies take on broad, enterprise-wide projects, only to see them bog down after eating up huge budgets without delivering any return on investment. Generally, more targeted unstructured data analytics projects perform better, he said.
But on the whole, businesses need to be aware of this analytics trend and evaluate how it can help their operations.
"The vendors are investing too much and the payback is too great to ignore it," Roma said.
Unstructured data isn't actually all that lacking in structure
With unstructured data analytics, don't forget about governance
Big data isn't always about unstructured data