This article can also be found in the Premium Editorial Download "BI Trends + Strategies: New BI demands pushing data architecture limits."
Download it now to read this article plus other related content.
We all know change is constant in IT. But when we are responding to demands to extract value from new forms of the most important asset in modern competitive business— information—and are busily managing the resulting increased data volumes, change is both constant and fast-paced. Don’t get left behind: With “big data” increasingly taking hold, the five-year future is going to bring major transformations in the way that information systems are built to support business intelligence (BI) and analytics applications. It’s going to be more about the net additive effect of new possibilities than about dismantling current technology investments.
It’s going to be about seriously using information for BI and analytics through the exploitation of all possible data, including data that is larger than anything you’ve experienced to date, providing less value “per byte” and requiring new data management methods. It’s going to be about exploiting information sooner in its lifecycle— as soon as possible.
Maturity in business intelligence is correlated to maturity and success in business. BI maturity is a worthy progression to understand, and working to increase maturity levels is a recommended strategy for every BI team.
Most companies begin their BI journeys because of a need for reporting. In many cases, reporting requirements can overwhelm, either technically or politically, the operational systems executing business transactions. That often leads to the development of additional structures—data warehouses and marts—to house transactional data for reporting and analysis. Integrating that data, regardless of where it comes from, becomes critical, as does improving the overall quality of the information.
Developing a more expansive view of BI
Eventually, monthly, weekly and even day-after reporting becomes insufficient, and accessing the valuable information being consolidated in data warehouses becomes a requirement on an intraday basis. An organization can then go full circle to doing whatever it can to support BI operationally.
With post-operational warehousing systems being bound by the extract, transform and load cycles that need to be performed there, real-time BI brings data analysis directly into operational systems. Business intelligence previously was viewed as “whatever we do to data in the data warehouse,” but now BI is being expanded into the discipline of data usage and exploitation, wherever that data may reside.
For more and more companies, another aspect of BI maturity involves coping with new data types, which usually means harnessing the “big three” forms of unstructured or semi-structured big data: sensor, social media and Web activity data. It also can mean an organization availing itself of the syndicated data marketplace and bringing external information into its BI systems.
Before making big investments, assess your existing information management architecture to determine if it will meet the new BI demands of the enterprise. Suggested questions to ask as part of that assessment include the following:
- Is all company data, including unstructured and semi-structured data, under management?
- Is there a solid data quality program in place to ensure the distribution of high-quality information?
- Are all BI queries processed in post-operational batch systems now? Could they be done, and decisions made, in operational systems instead?
- Do new query requests automatically require using a particular system or type of BI technology, such as multidimensional cubes? n Is the current usage of BI systems going to continue to scale?
- Is it time to consider alternatives?
- Can data marts, data warehouses and cubes with overlapping functionality be consolidated? And have the potential savings of doing so been calculated?
BI Challenges—and Ways Around Them
Maintaining sufficient performance levels is one of the top challenges in BI today, especially as data volumes grow. Alternative technologies for managing structured data—such as columnar databases, in-memory databases and data warehouse appliances (or a combination of them)—can provide additional performance on specific workloads beyond what conventional relational databases are capable of delivering.
Data integration also remains a major challenge. For many organizations, the answer may be the “perpetual short-term” solution of data virtualization, which can bring together data from separate, technically distinct data stores without requiring the information to be consolidated in a data warehouse.
Interest in the cloud is high as a place to store the data behind BI systems and manage the integra- tion and data access components. Security fears and integration with on-premises data remain big concerns for many prospective users. Nonetheless, cloud and virtual deployments undoubtedly will house upwards of one-third of corporate databases within the next five years.
Centralizing management of master data is potentially valuable to BI initiatives as well, because it replaces individual efforts and creates a governed process for master data management (MDM) along with the infrastructure required to support distribution of the master data to different systems. But there are architectural considerations to address when evaluating an MDM program. For example, does each system need to build its own master data? Is one system’s master data worthy of being used more broadly? Are the owners of master data ready for the responsibilities of providing it beyond their individual systems?
While few companies have a budget for “innovation,” the good news is that advanced business intelligence is not really about innovation. It’s about business, and when done well it provides a high return on investment even in the short periods that are demanded of IT investments these days. But there’s the rub—it must be done well.
By asking some vital questions (see “Information Architecture: Preparing for New Requirements,” page 5) before investing in information management technologies to support BI activities, organizations should be well prepared to manage their information assets effectively. Coupled with a focus on enabling better business outcomes as a result of efforts to increase a company’s level of BI maturity, the resulting information management architecture will be able to support the BI data demands of the foreseeable future.
About the author:
William McKnight, president of McKnight Consulting Group, is a strategist and information architect specializing in information management, business intelligence, data warehousing and master data management. McKnight has authored hundreds of articles and white papers and speaks regularly at conferences and seminars. He can be reached at firstname.lastname@example.org.
This was first published in November 2012