News Stay informed about the latest enterprise technology news and product updates.

The real deal on data integration for business intelligence

While shiny dashboards and cutting edge reports grab the limelight, behind-the-scenes data integration is key to BI success.

There are various ways to sabotage a business intelligence (BI) initiative, but perhaps none is more effective than ignoring key data-integration considerations at the outset.

Data integration -- connecting data sources with data consumers -- lies at the very foundation of BI, according to experts. But successful data integration is no easy task. Poorly labeled data fields and siloed data sources, among other obstacles, see to that. BI done right requires carefully selected data sources, particular attention to data quality, and an understanding of the ultimate business uses of data.

"Business intelligence is all about data integration," said William McKnight, senior vice president of information management at East Hanover, N.J.-based consulting firm Conversion Services International. "You can't have effective BI without good data integration or else you'll be running off different sets of books and encountering various semantic and other cultural -- and very real – [obstacles] to success."

Ensuring a successful BI initiative requires attention to some key data integration considerations, according to McKnight and Rick Sherman, founder of Stow, Mass.-based business intelligence consulting firm Athena IT Solutions.

Get a grip on data granularities

BI users often want to drill down deep into data to gain better insights on the business. The first step in any BI data integration project is to understand the granularities of an organization's data in order to agree upon definitions, McKnight said. That way, when users delve into the data, they are all working off a common framework.

More on data integration
See who made the cut in Gartner's latest data integration Magic Quadrant

Explore three paths to real-time data integration

Balance the costs and benefits of data integration software vs. hand coding

In a retail company, for instance, what exactly defines a customer? Is it a single person or the entire household? If another member of that household makes a purchase, is he or she a new customer or part of the existing household customer account? Questions like these must be addressed before integrating data into a BI system, McKnight said. How they are answered will largely depend on a company's business model and corporate culture.

Not all data sources are created equal

The next step is to take an inventory of data sources – databases, data warehouses, spreadsheets, etc. It is important to realize that different departments often have their own internal databases and spreadsheets that contain overlapping information, which can lead to problems with data quality.

"A lot of companies have grown up in silos and now are looking to integrate data to get a holistic view of the business," McKnight said. "That means there are different cuts of the data taken off in different departments, sometimes with their own IT departments or their own rogue IT departments." Before data can be integrated, he said, these various data sources must be reconciled.

Athena's Sherman also stressed the importance of data quality in any integration initiative. Prior to the implementation of a BI system, he said, users in separate departments probably routinely make changes to data to suit their purposes -- changes that are not relayed to other business units. That's fine when the data never leaves a given department, but can wreak havoc on BI systems that access enterprise-wide data.

"They massage the data to make up for data quality issues, or gaps in the data, or things they don't think are right in the data," Sherman said. "Once you [undertake] a BI initiative, then you want to look at more detailed data, from across different systems. That's when you start to see data quality issues."

Foster buy-in from BI stakeholders

In conjunction with addressing data quality issues, executives and analysts should also be consulted to determine how they plan to use the resulting BI reports, analytics and dashboards.

"You need to pick data sources appropriately for the stakeholders of BI," McKnight said. "And they should understand the choices that they have to pick from within the organization."

Understanding how data will be used is essential to successful BI data integration, Sherman agrees, but understanding data lifecycles is equally important.

"The lifecycle of your data is very closely related to your business processes," Sherman said. Data used by a manufacturing company, for instance, undergoes very different processes throughout its lifetime than data used by a retailer. Even within industries, data lifecycles often vary. Both Sherman and McKnight said knowing when to integrate data for BI purposes during its lifecycle is a key success factor.

Only after data source options are clear, data quality issues have been addressed, and BI business uses determined, can an organization actually begin integrating data with BI systems, McKnight and Sherman agreed.

The reason why some BI projects take so long, sometimes nine months to a year or longer, "is due to getting business users to determine how they're going to define and use the data, plus assuring data quality and writing data governance rules," Sherman said. "Business intelligence really raises the bar for data integration."

Dig Deeper on Business intelligence best practices

Start the conversation

Send me notifications when other members comment.

Please create a username to comment.