PRO+ Premium Content/E-Books

A fire is catching in the world of big data processing. Since it was first introduced by The Apache Software Foundation a few years ago, the Spark processing engine has been moving throughout the big data ecosystem, latching onto users ripe for change the way a wildfire takes to some vegetation better than others.

A major appeal of Spark -- which is often paired with the Hadoop framework -- is its speed, especially compared to MapReduce, another Hadoop partner. Of course, the processing engine's youth also means it has areas in need of improvement. Some users struggle to stay on top of Spark updates because the other tools they pair it with don't have the latest version. But the benefits of Spark outweigh the inefficiencies for many users. "It's a stable [technology], and I have no hesitation at all about deploying it," said Peter Crossley, director of product architecture and technology at Webtrends Inc., one of the many users profiled in this e-book chapter written by Executive Editor Craig Stedman.

Webtrends was an early adopter of Spark and recently expanded the processing engine's role in its big data operations. Other users, like cloud software vendor Xactly Corp., are newer to the technology but are already seeing its benefits. One thing is for sure: Spark has caught the attention of companies that want to process information fast.