The big data analytics team at Comcast Corp. wants to see a lot of information in real time. The list includes data on the locations of the media and entertainment company's customer service trucks, phone calls to its call center, the performance of set-top boxes and aggregate TV viewing records.
"We want to be able to fix issues before customers notice them," said Kiran Muglurmath, Comcast's executive director of data science and big data analytics, during a session at Strata + Hadoop World 2016 in New York last September. To make that possible, Comcast is one of a growing number of organizations expanding their big data architectures to support data streaming applications.
What's driving such investments is an increased need for decision-making speed -- and for more clarity amid the burgeoning big data clutter. "The operational side of the business is being flooded by data. That's changing the way people manage their businesses," said Andrew Cardno, CTO and co-founder of operational intelligence software developer VizExplorer.
But not all of the data streaming into analytics systems is golden. "There's a lot of noise in streaming event data," said Mark Madsen, president of consultancy Third Nature Inc. Both he and Cardno spoke at the 2017 TDWI Leadership Summit in Las Vegas in February. Madsen noted that the data collection process with Hadoop, Spark and other big data technologies "is so much faster now, but it's not necessarily better," especially if application developers don't focus on data accuracy and consistency.
Madsen's closing advice for companies creating data streaming applications was succinct: "Manage your data, or it will manage you." This handbook offers further insight on how to make streaming analytics initiatives pay off, with more user examples and information on stream processing tools.