Guide to big data analytics tools, trends and best practices
A comprehensive collection of articles, videos and more, hand-picked by our editors
Stream processing vendors tout their ability to handle millions of transactions or records per second -- but applications with a data load of that magnitude are a niche within a niche. Typically, much less information is involved, even in big data applications. "Streaming can be pretty relative," said Nick Heudecker, an analyst at consulting and market research company Gartner Inc. "In reality, probably most applications are in the tens of thousands [of transactions] per second."
Meru Networks is an example of a company that's deploying stream processing technology to help it analyze a relatively small amount of data. The technical support team at Meru, a maker of Wi-Fi networking devices in Sunnyvale, Calif., has been using analytics tools from software vendor Glassbeam Inc. since late 2013 to check log files from network controllers at customer sites in order to help diagnose and resolve problems. The machine data was being processed in batch mode -- but now Meru is adding more real-time analytics capabilities for some of its customers through a pairing of the Glassbeam software and the Apache Spark processing engine's data streaming module.
Joe Limprecht, Meru's technical support manager for the Americas region, said automated scripts running in Spark Streaming will be able to pull data every five to 10 minutes from controllers at Tier 1 support customers experiencing technical problems that have been escalated for higher-level investigations.
The output from the log files isn't huge: Each contains about 60 MB of data, and customers generally have 20 to 30 controllers installed, Limprecht said. But the goal, he added, is to enable Meru support engineers who are trying to find fixes for thorny issues "to essentially see live data when we're talking to customers." For example, they might be able to analyze the streaming data to pinpoint traffic spikes that require the addition of more wireless access points to a customer's network.
Streaming data can help bring big data into real time
Medical company turns to streaming data to spark real-time analytics