This content is part of the Essential Guide: Guide to big data analytics tools, trends and best practices

Essential Guide

Browse Sections

Stream processing not such a big-data deal for some users

Many data streaming applications don't involve huge amounts of information. A case in point: an analytics initiative aimed at speeding the diagnosis of problems with Wi-Fi networking devices.

Stream processing vendors tout their ability to handle millions of transactions or records per second -- but applications with a data load of that magnitude are a niche within a niche. Typically, much less information is involved, even in big data applications. "Streaming can be pretty relative," said Nick Heudecker, an analyst at consulting and market research company Gartner Inc. "In reality, probably most applications are in the tens of thousands [of transactions] per second."

Meru Networks is an example of a company that's deploying stream processing technology to help it analyze a relatively small amount of data. The technical support team at Meru, a maker of Wi-Fi networking devices in Sunnyvale, Calif., has been using analytics tools from software vendor Glassbeam Inc. since late 2013 to check log files from network controllers at customer sites in order to help diagnose and resolve problems. The machine data was being processed in batch mode -- but now Meru is adding more real-time analytics capabilities for some of its customers through a pairing of the Glassbeam software and the Apache Spark processing engine's data streaming module.

Joe Limprecht, Meru's technical support manager for the Americas region, said automated scripts running in Spark Streaming will be able to pull data every five to 10 minutes from controllers at Tier 1 support customers experiencing technical problems that have been escalated for higher-level investigations.

The output from the log files isn't huge: Each contains about 60 MB of data, and customers generally have 20 to 30 controllers installed, Limprecht said. But the goal, he added, is to enable Meru support engineers who are trying to find fixes for thorny issues "to essentially see live data when we're talking to customers." For example, they might be able to analyze the streaming data to pinpoint traffic spikes that require the addition of more wireless access points to a customer's network.

Craig Stedman is executive editor of SearchBusinessAnalytics. Email him at [email protected], and follow us on Twitter: @BizAnalyticsTT.

Next Steps

Streaming data can help bring big data into real time

Medical company turns to streaming data to spark real-time analytics

Dig Deeper on Big data analytics