Real-time analytics is all about speed, but when it comes to setting up your real-time architecture, slow and steady might be the quickest route to the finish line.
QVC, the cable shopping network most known for its infomercial-focused television programming, put together a real-time analytics platform by using some older analytics and data management technologies not typically known for their speed.
"New can always seem brighter and shinier than the old, but we found that there weren't any vendors out there that could hit the threshold we needed," said Peter Goodnough, vice president of consumer insights and analytics at QVC.
The real-time analytics platform, which the company calls DART, short for data analytics response technology, is essentially built around a 15-year-old Teradata data warehouse with Tableau software used as the front end. Goodnough said the real secret sauce that makes this simple combination work for real-time analytics is the data extraction processes his team wrote to get data out of the data warehouse in real time. For proprietary reasons he couldn't detail the jobs any further.
But Goodnough's main point is that smart data engineering and a deep understanding of existing technologies beat out new technology any day when it comes to getting to real-time analytics. "It's kind of proof positive of the varied skill sets that are required in the world of analytics today," he said.
Listening to the customer
The platform is used to pipe customer feedback into sales processes. For example, if the company's customer service platforms are receiving a lot of questions from viewers about pricing for a product that is currently being promoted on the air, the customer service team can get in touch with the studio and work out changes to how the presenter is describing pricing on the fly to make it clearer to the audience.
Or if analysts see from text analysis of television closed captioning feeds that a product in their catalog is being advertised heavily on other channels they can alert the studio team to the opportunity and make sure they feature it on the channel.
Peter Goodnoughvice president of consumer insights and analytics, QVC
And of course, the real-time analytics platform plays a big role in the company's online shopping site. QVC, which is generally known for its TV channel, actually brings in more than half its revenue from its website. It uses the DART platform to monitor customer transaction histories, feedback, social media posts and item views to keep track of what products are hot and ensure the site recommends the right items to the right customers in a timely manner.
"Making sure we have access to all the different behaviors that manifest online is hugely important," Goodnough said.
Before having his team build the platform, Goodnough shopped several vendors in early 2015. He figured it would be easier to find a vendor platform that could do real-time analytics and that turning the project over to an experienced technology company would be the easiest route.
But after evaluating a few products he felt that none could deliver data at the rate he wanted. In this case, Goodnough uses real time to mean analyses that update in less than five seconds on an ongoing basis.
It was at this point that a couple data engineers came to him and said they wanted to start a project looking at real-time transaction data combined with online behavioral data. After a quick three months of development this project became the DART platform.
"When you're buying a new suite of cutting-edge technology from a vendor, that's not always as customizable as some of the older components that you might have already," he said.
A true trial by fire
Once the platform was developed there was no soft roll out. Goodnough decided to go live with it on Black Friday weekend of 2015, traditionally a make-or-break shopping weekend for retailers.
The stakes were high. Analytics experts often recommend teams look for quick wins to start with. These are typically relatively easy projects that analytics teams are highly confident they can accomplish, and even if they fail the impact on the business is minimal. This is supposed to build trust with the business.
But Goodnough said he was so confident in the platform that his team couldn't afford not to roll it out on Black Friday weekend.
"We felt that the use of real time during that super competitive period in retail would give us a competitive advantage and we were able to use it to juice our results," he said.
The move paid off. He counted 40 separate recommendations that came from the DART platform that weekend that led to positive business decisions.
The successes have continued and by now the real-time analytics platform has more than paid for itself, Goodnough said. After all, since it was built around existing software, the platform only cost the time it took the engineers to develop it.
When you avoid unnecessary vendor tools and maximize the functionality of existing software, "There's a really robust return on investment," Goodnough said.
Real time brings analytics straight into the core of business operations
Hadoop might still be too slow for real-time analytics applications
Real-time analytics puts a lot of pressure on business processes