LAS VEGAS -- “Big data” continues to land big headlines as vendors’ tools promise to dig into the multi-structured data sets multiplying at a rapid pace and analysts chronicle the big data moment. But is all the attention necessary? It depends on whom you ask.
Attendees at The Data Warehousing Institute’s BI Executive Summit represented a full spectrum of thinking surrounding the buzzword -- from those who believe the term was designed to sell a product to those who have seen organizational restructuring because of big data.
“We’re a midsized company,” said Daniel Bolton, a systems analyst at the Bradenton, Fla.-based Intertape Polymer Group, which specializes in products for the packaging industry. “I haven’t the slightest idea what big data is … but my assumption is that it’s a marketing ploy.”
More on “big data”
Find out how skills and training present big pitfalls for big data analytics
Catch up on some of the big data headlines from 2011
Bolton is not alone. Although big data was a major theme of the conference, how big data is defined and whether an organization even has it can be difficult questions for businesses to answer.
A small pool of representatives from the education industry, for example, defined big data as simply “information overload.” A manager for a large, integrated oil company said his company works with 15 terabytes (TB) of data, but because it’s manageable, he doesn’t consider it big data territory. According to one representative from an asset management firm, big data is just “hype” buoyed by Web and social data. Some of the sources were not willing to be named in this report due to their company policies.
Does everyone (need to) have big data?
The asset management representative said social and Web data -- the two pieces he believes are driving the big data discussion -- will not be something his company introduces to the analytics environment.
“It’s not a real concern,” he said. “I don’t know how useful or relevant it is to my business.”
He said he could see how marketers, for example, would be able to leverage social and Web data, but as a financial institution, the legal ramifications and compliance regulations mean taking precautions other businesses don’t have to.
Allianz, a Richmond, Va.-based insurance company, works with about 5 TB of data, but Ron Naumann, vice president of enterprise data management for the company, doesn’t consider his to be a big data environment.
“The business doesn’t see it as an issue as long as the performance is there,” he said. While collecting data is one thing, Naumann said the harder -- and bigger -- question centers on how to determine the data’s value.
“Do we really need 5 terabytes of data?” he asked. “How do you value extra data from the business perspective?”
Betting on big data
Some attendees are still determining if big data is a big enough deal to worry about, but others see its promise. While the oil company’s 15 TB of data are easy to manage, sensor data -- a new data source for the company -- is another story.
“Every five seconds, these sensors spit out the conditions of the pump at that very time,” the manager said. “These new data sources produce so much noise, and the key is picking out the important pieces -- the information rather than the data.”
Yet the value trapped within that sensor data has the potential to save the company millions of dollars by predicting pump failures or providing detailed information on drilling conditions, he said.
“The drill has sensors that can send back data as to what the drill bit is running into,” he said. “We can take that data and compare it to similar wells and tell what kind of rock formations or what kind of a scenario we might run into.”
Because of the overwhelming volume, processing the sensor data means introducing new technology to the ecosystem; the oil company is developing a big data strategy and has plans to evaluate Hadoop, an open source technology used for processing large data sets in a distributed computing environment, “and other MapReduce-type technologies,” the manager said.
Ingram Barge Co., which moves bulk commodities by water and is based in Nashville, Tenn., sees some promise for big data in the future. It is in the beginning stages of formalizing its business intelligence project.
“We’re trying to be more focused strategically with how we provide data to the user community,” said Maria Conatser, general manager of analyst and project administration.
Conatser said her company is now evaluating vendor options, but although it is still rolling out the program, she’s also looking ahead at the untapped possibility big data could provide, especially for a transportation company.
“Last year, I went to the Gartner BI Summit, and UPS won [the Excellence Award],” she said, adding that she’d like to bring some of what they’re doing into her own work environment.