News Stay informed about the latest enterprise technology news and product updates.

Operational BI gets a boost from real-time data loading

Real-time data loading allows data warehouses from vendors like Teradata and Oracle to support operational BI environments, but it doesn't come cheap.

Travelocity doesn't like to waste its customers' time when it comes to personalizing the Web experience. In fact, in less than a second after logging on to, registered users are greeted with offers tailored to their unique travel history.

Even non-registered users, who make up the bulk of the site's visitors, almost instantly see offers based on their location, which Travelocity's Web analytics software determines based on IP address.

Supporting the operational business intelligence (BI) project is a 12-terabyte active data warehouse from Teradata that handles, in near real time, more than 1 million queries a day. It is part of the Miamisburg, Ohio-based vendor's "active" data warehouse initiative, which aims to make operational systems smarter, faster and, ultimately, more effective.

Teradata has been in the data warehousing business a long time, supporting traditional business intelligence deployments with multi-terabyte data warehouse appliances and smaller data marts with the goal of getting timely and accurate information to the right people at the right time.

By touting its real-time, or near-real-time, data integration capabilities, the vendor is essentially trying to do for consumer websites, ATMs, call center software and other operational systems (and the workers who use them) what its data warehouses have done for business analysts and other BI power users, said Dave Schrader, Teradata's marketing director.

"Think of the customer touch-points as fingers and the back office as the brain," Schrader said. The goal of the active data warehouse is to get the intelligence traditionally held in the brain out the fingers -- the front-line operational systems and customer reps that interact with customers on a daily basis.

Jim Kobielus, an analyst with Cambridge, Mass.-based Forrester Research, said Teradata's "active" data warehouse -- "active" simply being Teradata's branded marketing term – is highly scalable for large numbers of concurrent users to perform complex and near-real-time queries. It can accommodate both large batch and real-time data loading, he said, making it highly flexible to both operational and more traditional analytic BI environments.

Its ability to integrate operational data in near real time is due in part to its shared-nothing, massive parallel processing (MPP) architecture, Kobielus said. MPP involves the partitioning of databases so that large amounts of data can be queried at one time, or in parallel.

More on business intelligence software
Find out why business intelligence deployments aren't living up to the hype at most organizations  

Get the inside story of SAP's and IBM's business intelligence integration strategies  

Read about efforts to track carbon emissions using business intelligence software

Teradata also uses a custom-built utility it calls TPump "that connects to the JMS or IBM WebSphere MQ products to get data in real time from the EAI message bus," Schrader said. "TPump loads the data straight into the data warehouse within minutes, sometimes seconds, of an update to an operational system."

In contrast, traditional data warehouses usually employ extract, transform, and load (ETL) tools to batch load data, sometimes just once a day (in the evenings, when usage is low). That's fine for a business analyst performing daily or weekly historical analysis, but not for a call center rep handling a call from an irate customer who had a bad experience just minutes earlier.

Teradata is not alone, however. Certain Oracle and IBM data warehouses have similar real-time data loading capabilities, though each varies slightly in how it delivers that functionality, Kobielus said. Smaller vendors like Greenplum and Aster Data Systems also boast near-real-time data loading capabilities for operational environments.

But the "active" data warehouse is not cheap. Not accounting for recent price drops and customer discounts, the average cost of a fully configured Teradata data warehouse, including hardware, software and database tooling, is more than $200,000 per usable terabyte, according to Kobielus. Oracle charges a similar amount for its HP Oracle Database Machine.

The high price hasn't kept some customers -- at least large enterprises -- away, however. Teradata counts Bell Canada, Norfolk Southern, and Network Solutions among its "active" data warehouse customers. Kobielus couldn't say how many of Teradata's overall customers are using the vendor's real-time data warehousing capabilities for operational BI, but he said that "most of their customers are Fortune 1000."

At Travelocity, installing and testing the Teradata warehouse was also an important process to ensure it could handle the company's workload, according to Debbie Doran, technical lead for global CRM development at Travelocity. The process took about nine months, she said.

"That was definitely a big concern. In the very beginning, we had to work with all our systems engineering groups and go through a series of performance tests in our certification environments and our staging environments," Doran said. "We really worked through exactly how we structured that data to meet the performance requirements."

Since deploying the warehouse in 2006, the company has seen a marked increase in business, she said. Click-through rates on the site are up significantly, and bookings have increased to four times what they were before the active data warehouse went live.

Doran wouldn't reveal what Travelocity paid for the "active" data warehouse, however; nor would Teradata.

Dig Deeper on Business intelligence best practices

Start the conversation

Send me notifications when other members comment.

Please create a username to comment.