Look to business needs in deciding what big data sets to analyze

• 
  Ed Burns

  Site Editor
• Published: 06 Feb 2015

There are times when it makes sense for data analysts to use an entire set of big data that's at their disposal, according to Vince Dell'Anno, an information management consultant in Accenture's analytics group. But he said there’s no magic formula -- organizations should come to their own conclusions about what data, and how much of it, is needed to get the desired analytical results.

Dell'Anno recommended that analytics teams start by assessing the business problems that need to be solved and then work to identify big data sets that can help them address those problems. In some cases, the answer might involve fully analyzing a relatively small data set, such as some sales info. In others, using data sampling techniques to pare down a larger data set could be the right approach -- for example, in predictive modeling applications where patterns often become apparent after analyzing a subset of the available data. Then there are cases when working with a full data set containing a massive amount of information is necessary, such as in fraud detection.

And sometimes, Dell'Anno said, the wisest approach is to wait until a good business reason presents itself to even start collecting some types of data. For example, data analysts at a retailer might think that social media data is likely to hold valuable insights into customer sentiment and preferences. But, he said, it could end up being a big waste of time to begin pulling in the data if there isn’t a clear idea of how the analytical findings will influence the company's marketing and sales practices.