Tips for successfully consolidating business intelligence systems

In a recent study by Boston-based Forrester Research, close to 80% of companies polled said they use three or more business intelligence (BI) products. According to experts, it is important for organizations to consider consolidating BI systems, but this is not as easy as it sounds. There are many reasons why organizations choose not to consolidate, while doing so effectively is quite a challenge. In this guide, learn how determine whether your organization is ready to consolidate business intelligence tools. Learn about the challenges that lie ahead during the consolidation process and how to plan for those roadblocks. Find tips for getting enterprise buy-in and working with end users on the consolidation project, and discover other best practices that will result in success.

BI platform consolidationFor organizations with multiple business intelligence (BI) systems, there are many reasons to consider consolidating. In this chapter from our guide, read tips on planning a BI consolidation project, learn how to successfully consolidate your BI systems and find advice for managing end-users throughout the consolidation process from expert Rick Sherman.

Don't miss the other installments in this BI platform consolidation guide
* BI consolidation proves as much a political as technological process
* How to decide whether it is time for BI consolidation
* Tips for successfully consolidating business intelligence systems
* Implementing your BI consolidation program
* Making the business case for a BI consolidation project

Although the term "best practice" is used extensively in our industry, it is often more important in BI consolidation programs to consider pragmatic practices. As discussed in the previous chapters, there will be cultural and political roadblocks during a BI consolidation project. BI consolidations that consider what is "best" only from a technical or theoretical perspective are much less likely to succeed than those that are pragmatic and include the human portion of the equation.

The BI consolidation plan

It is easy to lay out a high-level plan of action for BI consolidation. Here is an outline:

  • Make a preliminary survey of existing BI tools and applications.
  • Select BI applications that may be eligible to migrate off older systems.
  • Make a detailed assessment of selected BI applications to determine tasks and calculate cost/benefit analysis of potential migration.
  • Determine business and technical requirements for an enterprise-wide BI tool.
  • Establish the criteria for the selection of a BI tool.
  • Create a BI tool short-list.
  • Evaluate BI tools and select two for proof-of-concept (POC) or prototype. (There is a lot of politics involved in selecting these tools. Often the POC is the only way to settle the argument, since an evaluation checklist is not accepted by the "losing" side.)
  • Perform POC with selected BI application, data and business users.
  • Evaluate the POC and make a selection.
  • Acquire, install and set up the BI environment.
  • Develop a migration program.
  • Establish business user training.
  • Perform migration projects.
  • Conduct new BI application projects with the standard BI tool.

Of course, it is the execution that's the tough part, not formulating the plan. The following represent several recommendations based on real-world experience for BI consolidation projects. You will note that many of these recommendations are not technical in nature because most of the barriers to success are cultural and political. Technology is easy -- people make it hard.

Recommendations for BI tools comparison and evaluation
Standardize on BI tool categories or analysis styles -- not just a single BI tool. Businesspeople need various categories or styles of BI tools based on their job role and analytical abilities. These categories include: reporting, ad-hoc analysis, OLAP, dashboards, spreadsheet integration, data visualization, statistical analysis, and predictive analytics. Some individuals may need only a single type, particularly if they are casual users, but it is more common that businesspeople need multiple BI categories to do their jobs.

A BI suite is the preferred approach to fulfilling multiple BI categories. From an enterprise perspective, almost all businesses need the following set of BI categories: reporting, ad-hoc analysis, OLAP, dashboards, and spreadsheet integration. The good news is that many BI vendors have bundled these categories into a BI suite. To technology purists, picking a BI suite sometimes appears to be a compromise since you are not picking "best of breed." However, having a BI tool that enables businesspeople to analyze consistent data and metrics in the most productive manner at a reasonable TCO is more important than having the most feature-laden tool on an evaluation checklist.

At first, especially if several of the tool categories were the result of acquisitions, BI suites were more of a marketing pitch than truly integrated BI suites. They have now matured to become generally well-integrated and share metadata (business definitions, rules metrics and algorithms), utilities, infrastructure, user interfaces, and development tools. In addition, dashboards usually can display any of the other categories, such as production reports, OLAP or ad hoc, while enabling drill-down, spreadsheet integration, alerts and other functionality.

Selecting an integrated BI suite shifts systems integration from your business and IT people to the BI vendor. Productivity and consistency increases with an integrated BI suite because you can reuse business metrics, transformations and rules across BI categories; and your enterprise goes on one upgrade cycle for the integrated suite, versus having one cycle for each BI tool.

Include spreadsheets as part of your BI tool portfolio. It is a best practice, from an enterprise perspective, to include Microsoft Excel as part of your BI tool portfolio, though this will occasionally produce push-back from the IT team.

Data shadows systems example Data shadow systems example (Click to enlarge image)

Initially, BI tool vendors felt they were in competition with Microsoft Excel, but they gradually accepted that fighting the use of Excel only exacerbated its use and the spread of data shadow systems or spreadmarts BI tools integrated with Excel allow both BI casual and power users to leverage the tools they know and be more productive. The key caveat is to clearly distinguish between using Excel as the final reporting and charting tool, versus using it as a tool to gather and manipulate data as in a data shadow system. If Excel is abused in this manner, data consistency and compliance (i.e., Sarbanes-Oxley, SWIFT or HIPPAA) is called into question, with potentially harmful consequences.

When evaluating BI tools, do not consider just the BI vendors with the largest market share. BI tools are mature in the primary categories so many vendors are able to "keep up with the Joneses" — i.e., examine competitive offerings and ensure that their software provides comparable functionality. This is great for BI tool consumers because it expands the selection pool and means the tool will continually evolve with industry standards. This enables enterprises to examine TCO and other non-technical criteria that may make the BI tool more successful in the enterprise. If you are willing to expand beyond the Big Four of BI, look at options such as on-demand software, open source software, data warehousing appliances, and other innovative approaches that might not force you to lock into a vendor too early.

Do not constrain businesspeople's advanced analytic needs. Data visualization, statistical analysis and predictive modeling are often not part of a BI suite. Generally, these categories of BI tools are needed by just a small percentage of businesspeople, but their work is extremely valuable. It is appropriate to select a standard for each of these categories outside of the BI suite vendor so that all businesspeople who need this functionality can leverage the same tool.

One of your legacy BI tools should be a leading contender. If you are new to BI and have a green field regarding vendors, you can conduct a typical vendor evaluation with the capability "bake-off" to determine the winner, but if you have multiple BI tools in place, you may not have that luxury. If you have a significant number of business users already familiar with particular BI tools and your IT group has expertise in those tools, that has to be a critical selection criterion. This, of course, assumes that the existing BI tool applications are considered successful (at least from a silo perspective) and that the tools with which you are comparing them are comparable. The rationale for this is that you lessen the cultural resistance to a migration to a new BI tool, at least with the businesspeople who are already using that tool; you start off with IT expertise you can leverage and use in planning; and, of course, you reduce potential migration costs. To a technical purist, this may seem like heresy — i.e., "why wouldn't you pick the best?" — but from a pragmatic viewpoint in terms of costs, time and resources, it is the best course of action.

Rick Sherman

About the author: Rick Sherman has more than 20 years of business intelligence and data warehousing experience, having worked on dozens of implementations as a director/practice leader at PricewaterhouseCoopers and while managing his own firm. He is the founder of Athena IT Solutions, a Boston-based consulting firm providing DW and BI consulting, training and vendor services. Rick blogs on performance management, DW and BI at The Data Doghouse. You can reach him at [email protected] or (617) 835-0546.

In addition to teaching at industry conferences, Sherman offers on-site DW & BI training, which can be customized and teaches public courses in the Boston area. He also teaches data warehousing at Northeastern University's graduate school of engineering.

Don't miss the other installments in this BI platform consolidation guide
* BI consolidation proves as much a political as technological process
* How to decide whether it is time for BI consolidation
* Tips for successfully consolidating business intelligence systems
* Implementing your BI consolidation program
* Making the business case for a BI consolidation project


Dig Deeper on Business intelligence best practices