The third article in our four-part series on setting up and managing data analytics programs provides advice from industry analysts on preparing for deployments of analytics software and challenges that companies can face.
Table of Contents
Analytics skills in demand – and analytics pros demanding top salaries
Data analytics team’s needs: a business home, leeway on tools and data
Getting ready for an advanced business analytics software project
Creating an advanced data analytics business culture: Tips and advice
Companies looking to take advantage of business analytics software should first make sure that their data house is in order, according to industry analysts and other experts on analytics technology.
The first step in any analytics project is to get company data cleansed and profiled so that it can be made available for use by statisticians and other data analysts, according to Dr. Fern Halper, a partner at Hurwitz & Associates, a Needham, Mass.-based consulting and research firm that focuses on emerging software technologies.
“Data quality is always the most important thing, because ‘garbage in, garbage out,’” Halper said. “That is something people have to understand.”
She added that one of the biggest data quality issues affecting organizations with business analytics software initiatives involves joining disparate data sources that may contain inconsistent information
For example, a customer might be listed as “Customer A” in one data source, but the same listing in a different data source might be for its parent company. And perhaps “Customer A” in yet another data source could be a different company entirely. Without proper data cleansing and profiling, Halper said, the customer revenue totals calculated for the first customer would be way off, and analytical models incorporating the information would produce faulty results.
Wayne Eckerson, an analyst at The Data Warehousing Institute (TDWI), noted that data analytics projects can pose even more data quality challenges than conventional business intelligence (BI) deployments because of the ability of business analytics software to examine large data sets in an effort to detect hidden patterns and trends.
Eckerson named data quality as one of the top-four analytics challenges during a presentation at a recent TDWI event in Cambridge, Mass. He recommended that IT and BI teams working on the projects consolidate detailed data in a single data warehouse, then integrate, normalize and cleanse the information and standardize its underlying metadata before providing access to data analysts
Overcoming the challenges of advanced business analytics software tools
Many enterprises are also becoming more interested in advanced analytics technologies, such as predictive analytics software and tools that enable users to analyze voicemail messages, videos and unstructured text found in call center reports and corporate documents and on social media websites.
But the use of those tools requires careful preparation. For example, voice, video and text analytics technologies come with their own set of challenges to overcome, according to Thomas Davenport, co-author of the new book Analytics at Work and a professor of information technology and management at Babson College in Wellesley, Mass.
There are always issues around data quality, data governance and data integration across large organizations. That never goes away.
Thomas Davenport, professor of information technology and management, Babson College
Typically, the biggest problem when deploying unstructured data analysis tools is that words can often have multiple meanings, Davenport said. And being able to discern what a specific word means in a particular usage requires business analytics software and systems to exhibit a human-like understanding of context and inflection.
For example, in a warranty report or on an invoice, the “buyer” could be an individual consumer, a worker within an organization’s purchasing department or perhaps the organization itself. Davenport said making sure that an analytics system can recognize and differentiate among the types of customers in those different scenarios may require some serious technical expertise and development work.
One of the best ways to ensure the success of an advanced analytics programs is to hire smart people with varying forms of expertise. Davenport said that most organizations will need some “hardcore analytical professionals” – people who tend to have a Ph.D. or other advanced degree in statistics, mathematics or quantitative analysis.
Halper agreed. The people who build analytical data models for an organization need “to be really attuned to understanding the ins and outs of data analysis,” she said. “It requires a deep understanding of your data and a certain thought process.”
Data gaps can hinder business analytics software
One of the toughest parts of large-scale analytics programs is integrating data from various departments and supply-chain partners, said Dr. Richard Hackathorn, founder of Bolder Technology Inc., a Boulder, Colo.-based consultancy specializing in analytics, BI and data warehousing.
Hackathorn has been working with a large high-tech manufacturer that recently completed a data warehousing project designed to combine information from its entire distribution network for BI and analytics purposes. As a result, the company can now track a product's origins no matter how long ago it came off the assembly line, Hackathorn said.
“One [thing] they said to me that really stuck is that the real opportunities for improvements are in the cracks – the cracks between functional units,” he noted. “It’s the cross-functional things that really trip you up.”
For example, a manufacturing process may consist of more than 200 individual steps involving different departments within a company as well as external suppliers. “One department may be really doing their job well and taking their responsibilities seriously, and another department is doing likewise,” Hackathorn said. “But it’s the handoffs between the departments where things can go awry.”
That in turn can lead to data problems that may wreak havoc with analytics results. According to Hackathorn and other analysts, one way to overcome cross-functional data issues is to create a unified data management and data governance program with detailed rules related to the handling of data by different departments. Industry analysts say that the data governance policies should be designed to help ensure that gaps don’t arise in data collection, management and use.
The increasing interest in using in-database analytics software, particularly within large organizations, could make it easier to keep tight control over data and data governance practices, Davenport said. In-database analytics allows users to run data analysis applications within a database or data warehouse, also potentially yielding reduced costs and faster development as well as the ability to embed predictive models in business processes and applications more easily.
But don’t expect the data-related problems created by business analytics software deployments to disappear anytime soon.
“There are always issues around data quality, data governance and data integration across large organizations,” Davenport said. “That never goes away.”