Business intelligence software product purchasing criteria

Learn what questions to ask when purchasing business intelligence (BI) software products. Get expert BI buying criteria and vendor selection advice in this BI software buying guide.

By now, most organizations understand the many competitive advantages of business intelligence (BI) software. BI software can help organizations analyze data to learn more about their business and make better-informed business decisions. But before reaping the benefits of BI, companies must implement software -- an often complex process. readers report that buying business intelligence software products can be confusing and complicated. Many decisions need to be made about technical requirements, RFPs, budgets and, of course, software vendors. This resource is designed for organizations purchasing BI software products for the first time, or for organizations looking to update or optimize their current BI system.

To learn more, spoke with Boris Evelson, principal analyst with Cambridge, Mass.-based Forrester Research Inc.; William McKnight, senior vice president, information management for East Hanover, N.J.-based Conversion Services International Inc. (CSI); and Michael Corcoran, chief marketing officer for New York City-based Information Builders. We asked these experts how to overcome the challenges of the BI software purchasing process and how to streamline a successful BI vendor and product selection process. Since every organization has unique needs and criteria, the BI buying process can vary -- but all buyers can benefit by being well prepared.

Below, there are two lists of questions to get you started. The first is a list of questions that experts recommend organizations ask themselves internally before they begin the BI purchasing process. The second is a suggested list of questions to ask vendors.


  Questions for organizations to ask themselves  


  • Do we even need a new business intelligence product?

    "When you start on the journey of a successful BI implementation, selecting a BI tool is definitely not within the first 10 steps." -- Evelson

    It's easy to get caught up in the idea of buying business intelligence software, but before you advance in the purchasing process, make sure your organization isn't overlooking existing resources. Look closely at your existing enterprise applications to determine whether there is already a BI capability at your disposal, Forrester's Evelson recommended. For small organizations, the reporting capabilities in finance or CRM applications may be sufficient -- eliminating the need for a more complex BI system.


  • Who's going to be on my BI team? Are the evaluation and purchase processes going to be run primarily by IT or by business -- or both?

    Ensure that you have the right people in the organization on the BI team. Ideally, business should take a lead role during the evaluation and selection processes, according to experts. Since a BI application is going to bring overall value to both business and IT, it's prudent to make both groups part of the evaluation process. If IT and business professionals are represented from the beginning, communication between the groups is likely to be more effective down the road, Information Builders' Corcoran said.

    "If you are deploying business intelligence on an enterprise level, then staffing the BI competency center jointly with technical professionals as well as business users and business subject matter experts is really important," advises Cindi Howson, author of Successful Business Intelligence: Secrets to Making BI a Killer App.


  • What should our data architecture look like?

    An analysis of your data volume, organizational needs and existing infrastructure will help dictate the most suitable data architecture for your BI system. Evelson suggests asking (and answering) the following questions internally, before shopping for technology: Will you need to build a new data warehouse? Do you have such a heterogeneous environment that you'll need separate data marts? Do you require a centralized data hub from which you'll spin off your data marts? Do you already have these in place, or will you have to build from scratch?


  • Should we write a request for proposal (RFP)?

    It's always useful to write an RFP, Evelson said. It's the best way to organize your team's thoughts in one document and establish your priorities. Consider all user and business requirements and articulate them clearly. Evelson has helped create RFPs with as many as 600 criteria.


  • What are our business requirements, short and long term?

    An organization's business requirements will not be the same five years down the road, all experts agreed. Try to visualize the evolution of the organization's business requirements and evaluate BI tools accordingly. Needs change, and a BI system must be agile to meet evolving business constraints. If you can't foresee how your users are going to use BI and structure queries in the future, consider making BI search a requirement. Also, make sure business and IT define business terms the same way -- a task sometimes easier said than done.

    "When I get 10 businesspeople in a room and I ask them, 'How do you define customer profitability?' I get 11 answers," Evelson said.

    While this can be a challenging process, analytical master data management, enterprise-wide data dictionaries, data governance programs and communication can help solve disparate terminology problems, experts say.

    "[One should] take the current process of defining requirements -- where usually the IT professionals will ask the business what they need -- and flip it on its head. It's really studying what workers are doing [and] what information they need to do their jobs more efficiently and providing that information to them in a way that's easy, accessible and integrated into their work process," Howson said.


  • Who will the BI end users be, and what will they need?

    Understanding the ultimate end users of the system before selecting the final product can go a long way toward user adoption and the overall success of the system, experts said. Ask such questions as: How many users will BI have now and in the future? Will this BI tool be used outside the firewall by customers or partners? Will the tool be used in the U.S. only, or internationally? Do executives or salespeople need mobile BI capabilities? Will we implement dashboards, scorecards or data visualization tools? How important is real-time BI for operational workers, like customer service staff or tech support? What kinds of ad hoc queries must the tool be able to handle? How many users will be querying the system concurrently? Are the users technically adept or will they need a simple interface?


  • What are my data volumes today? What will they be in the near and far future?

    Changes in data volumes go hand in hand with evolving business requirements, according to Evelson. Examine growth trends over the last year, five years and 10 years to more accurately predict how much data the organization will have several years from now.


  • What's my budget?

    Once a business case and corporate sponsorship is established for the BI purchase, you'll often know your budget. (If you're lucky, you'll be involved in the process of deciding that number.) Staying within a budget can be difficult, especially because it's impossible to predict extra expenses that may accrue from minor issues, extra training, maintenance or other tasks, McKnight said. Consider also the cost of hardware, if applicable. Experts recommend creating a detailed BI budget -- allotting money for each step of the purchasing and implementation process -- and tracking expenses regularly.


  • What's our timeframe for implementation?

    Is there a corporate deadline for implementation? BI can take anywhere from one week to one year to implement. Allow time for your BI team to evaluate vendors, complete a proof of concept, negotiate with the vendor, implement software and hardware, train administrators and users, and troubleshoot problems. Break down each step and develop a detailed project plan, experts recommend.


  • Do we want to do our own proof of concept -- in our environment with our real data?

    Doing a proof of concept (POC) in your own environment can be a critical -- and telling -- step in the BI evaluation process, Evelson said. Today, most vendors will let you try their product in your environment to show real performance capabilities. If you decide to do your own POC, you need to figure out what data to use and how to structure that POC. Create trial dashboards, test different queries, run test reports -- and remember to get your users involved in the POC process as well. When implementing a new data warehouse, hold back some sample queries (don't provide them to the vendor in advance), Gartner analysts advise, so that complex queries can be tested without any system pre-tuning.


  • Is there a pre-existing vendor/technology bias within my organization?

    If your organization has used or is currently using products or systems from other vendors, there may be a vendor bias that could tilt the neutrality of a new vendor evaluation. Try to be aware of this bias, and encourage your team to be open to change.

    "Probably the worst thing for any BI vendor is to have to replace an existing product entirely, because the users want your product to do everything the other vendor's product did," Corcoran said.


  • What is the technical environment that this BI software will need to integrate into (source systems, databases, preferred hardware platforms, etc.)?
  • Many organizations use products from several vendors. Be certain a new BI product will integrate into the existing environment. Experts recommend carefully considering future architectural changes, such as service-oriented architectures, which could affect this integration in the longer term.


  • How will success be measured? Does the organization expect a certain ROI?

    You (and corporate!) will certainly want results that justify the BI purchase. During the product evaluation process, think about the best way to measure success in your organization, whether it's increased profits, happier customers, streamlined processes or some other metric.

    "We don't implement BI systems because they are trendy; we don't do it because the technology is fascinating. We invest the company's money in a BI system because we expect to get more money back, in terms of income or savings, than we invest," said industry expert Mark Whitehorn, in a recent article covering BI software purchases.


  • How will BI users and administrators be trained?

    To best answer this question, get a feel for the learning style of your users and administrators, Evelson recommends. Would they benefit from an all-inclusive training class? Or will you train only administrators and have the administrators teach the others?


      Questions for organizations to ask vendors  


  • What can the product do? How does it meet our requirements?

    If you clearly define your business requirements before meeting with a BI vendor, aligning their product with your organization's needs will be a much smoother process, experts agreed. But flexibility on your end is important, too. Be open-minded about alternatives that meet your needs. Meeting your own requirements should be a priority -- just be aware that different products may have different ways of getting to the same place, Corcoran said.


  • What pricing/licensing options are available?

    Many vendors offer multiple licensing styles. A few of the most popular options are:

    • Concurrent users
    • Role based
    • Server based

    "Vendors do [licensing] differently, and that makes it hard to compare apples to apples," CSI's McKnight said.

    So when selecting the licensing style, consider long-term user requirements. Another  tip from Gartner analysts about negotiating with BI vendors is to ensure that the contract lists every SKU that will be purchased. Not reviewing these SKUs is a surprisingly common mistake, analysts say -- with costly implications. Organizations that skip this step can find out later that a feature they thought they had purchased wasn't included in the contract and will cost more to acquire.

    "There are so many add-ons out there ... and sometimes customers [aren't] thinking about all that stuff when they engage the vendor," McKnight said. "You get to the checkout stand and of course you want [the add-on] ... and it makes it kind of a pressure situation."


  • How will your BI software integrate into our current environment?

    If you have outlined the technical requirements -- source systems, databases, hardware platforms, etc. -- it will be easier for the vendor to discuss the integration process with you. Ask the vendor what the common integration challenges are that he/she frequently encounters and consider longer-term requirements, like SOA functionality.


  • Will we be able to test the product in our environment? Can we get a proof of concept?

    It's highly unlikely that a vendor will not meet this request, according to Corcoran. Just be sure to factor the POC into your implementation timeline.


  • How are support and maintenance contracts set up?

    If you are using a new vendor and a new product, it's likely that your users and administrators are going to need technical support – particularly during the first year of implementation. Ask vendors for specifics and detailed pricing for support, training and maintenance.


  • How long will implementation take? Will we need consultants or vendor services?

    Make sure the implementation time frame given by the vendor is realistic, experts warn, and matches the time frame you have determined for yourself. There are often vendor services teams or independent consultants that can assist you -- either throughout the whole process or for specific decisions/questions.


  • What are your software update cycles like? How easy is it to add on new functions/features?

    When your reporting requirements change at some point down the road, your BI software may have to change, too. Make sure you talk with the vendor about the process and cost of adding on new features at a later date, McKnight said. Be certain you understand software update cycles and potential upgrade costs as well.


  • What happens if you get acquired? How likely is an M&A for you?

    Vendors may be quick to dodge this question, but it's a wise one to ask nonetheless. A lot of things can change with mergers and acquisitions (M&A), so make sure the terms are clear in your contract, to protect yourself.

    "When buying, you need to consider the lifecycle cost of that software [not just its purchase price] and match this against the benefits you confidently feel you will get from using it," advises industry expert Andy Hayler, in his column Risky Business: How to assess risk during software purchases.


  • Can I talk to one of your customers in an industry similar to mine?

    The best insight into vendors and products comes from people with firsthand experience, all experts agreed. Ask your vendor whether you can talk to one of their customers, ideally a similarly sized organization with similar needs.

    "Talk to people in the same industry, with similar types of applications, similar types of users -- that's where the real value comes in," Corcoran said. Good vendors should have nothing to hide and should be happy to connect you to someone.

    "User groups are typically outside the control of the vendor, and members are usually happy to talk candidly about the numbers of people attending," according to Hayler's column. "A product user group with flat attendance -- whether from a small startup or a large vendor -- should be a red flag."

    Editor's note: Did you find this BI software purchasing guide helpful? Are there any questions you think we're missing? Please let us know what you think -- your opinions are valuable to us.

Dig Deeper on Business intelligence software