Essential Guide

Browse Sections


This content is part of the Essential Guide: Guide to big data analytics tools, trends and best practices
Manage Learn to apply best practices and optimize your operations.

Tweet jam offers tips on finding big data skills, managing projects

In our #TalkingBigData tweet jam, big data experts Boris Evelson and Wayne Eckerson tackled issues such as the lack of big data skills in companies.

If you were asked to define "big data," what would you say? In the past, most people probably would have focused on the famous three Vs: volume, velocity and variety. But the talk about what big data is -- and how to take advantage of it -- has moved beyond that.

Earlier this month, SearchBusinessAnalytics held a #TalkingBigData tweet jam to sift through the hype surrounding big data and to address big data skills and project management issues. Wayne Eckerson, an industry analyst at TechTarget and president of consultancy Eckerson Group Inc., and Boris Evelson, an analyst at Forrester Research Inc., joined Executive Editor Craig Stedman, Site Editor Ed Burns and other TechTarget writers and editors to lead the half-hour discussion.

Our first question -- originally intended as a brief intro to the subject -- asked Eckerson and Evelson (and others) for their definitions of big data. Their answers, while quite different, proved helpful in showing just how murky the topic is.

Evelson went on to say that on average organizations currently transform only 12.5% of their raw data into actionable information, according to Forrester surveys. Companies need to use new approaches, such as Hadoop and other big data technologies, to try to make sense of the rest of their data, he added. Eckerson agreed, saying that in addition to being less expensive than traditional data warehousing systems, big data tools "let us tackle more data."

SearchSoftwareQuality Site Editor James Denman was quick to pose a follow-up question:

To which Evelson responded:

Like it or not as a term, though, big data appears to be here to stay. The #TalkingBigData conversation turned to one of the top problems facing many organizations today: how to find workers with the big data management and analytics skills necessary for orchestrating initiatives.

That tweet by Burns voiced a concern facing many businesses. For example, a 2013 survey conducted by software vendor Lavastorm Analytics found that 47.8% of 427 analytics professionals, corporate executives and other respondents felt their organizations lacked statistical, math or other quantitative skills. Such results raise the question of whether companies are prepared to effectively mine mountains of big data.

Evelson said he views big data skills as a combination of business intelligence and data mining know-how, and he suggested nurturing those skills internally:

Eckerson recommended a more unconventional approach for acquiring big data talent:

Rent-a-Data-Scientist? Don't be shocked. Eckerson said it's a viable option for organizations that don't need data scientists on a full-time basis. Especially when, according to Evelson, "it's the IT pro market," with the high salaries that entails.

It would be easy to say that more and more organizations are jumping aboard the big data train -- and you wouldn't be wrong, based on recent big data survey results. But all big data projects are not the same. When it comes to big data strategies, there's no prescribed method of finding the "correct" one. And you aren't alone if you're looking for the answer for your company, according to Eckerson:

Joe Hebert is associate managing editor of e-products for TechTarget's Business Applications and Architecture Media Group. Email him at, and follow him on Twitter: @jhbrt_TT.

Dig Deeper on Big data analytics

Join the conversation


Send me notifications when other members comment.

Please create a username to comment.

What do you think is the best way to build up big data analytics skills: training existing workers or hiring from outside?
Combination . . . neither is sufficient on its own - it seems to me.
I strongly believe per my past experience performing different roles in IT industry that nurturing in-house talent produces cost-effective results than hiring the talent from outside. The trust factor, and the needed business skills will play major role in producing these results through customization of the company needs.
As with all skills shortages, the "hire" approach will ultimately only result in one thing, the migration of staff with the required skills migrating to the employers with the deepest pockets. So, the hire approach is only a short-term fix because you risk losing your hires to better paying firms.
Ultimately, the only solution is to train more staff and to work on retaining staff with the necessary skills once trained. If your approach is to train, and retrain, to keep abreast of technology change, you're more likely to retain staff even if you can't afford to pay top dollar because your staff will feel valued and enjoy working in an environment which is closer to the leading edge and uses its staff to stay there. 
Often IT staff are as least as motivated by technology and work challenges as they are by salary packages and will only look elsewhere when they're bored and under-valued.