Sergey Nvns - Fotolia
One of the biggest things that can put a damper on big data analytics programs has nothing to do with problems in deploying and managing advanced analytics tools: It's the challenge of hiring data scientists who can put the tools you've installed to good use, and then retaining them.
In a survey of business intelligence, analytics and data management professionals conducted by IT research and education provider TDWI in the second half of 2015, the lack of skilled personnel ranked second on a list of top challenges that organizations face in trying to embed analytics processes into their business operations. The skills issue was cited by 38% of the 319 respondents, putting it behind only poor data quality at 40%.
As Hadoop and related technologies are adopted more widely by mainstream enterprises, getting tangible business benefits through effective analytics is becoming a higher priority. And, in many cases, it isn't analytics software that gets in the way of achieving that goal.
"The technology does what it says it will," said Gartner analyst Merv Adrian at the 2016 Pacific Northwest BI Summit in Grants Pass, Ore. But, he added, data integration problems and inadequate skills on both the data management and analytics sides often hold companies back from making good on their advanced analytics investments.
Bill Loconzolo, vice president of data engineering and analytics at Intuit Inc., said his team has had pretty good luck in hiring data scientists. In part, he focuses the recruitment efforts on the benefits that the data scientists can provide to customers who use the Mountain View, Calif., company's finance and accounting software.
Mike Fergusonmanaging director, Intelligent Business Strategies Ltd.
"We talk about what the impact of the work is that they're going to do -- how much money they're going to put in the pockets of people at tax time," Loconzolo said. "That's very attractive to data scientists. They want to solve real problems."
However, even if you do succeed at finding and hiring data scientists with the skills you need, it may be hard to keep them from jumping ship when other companies come calling with more lucrative job offers.
"Data scientists in my part of the world are changing jobs every three months," said Mike Ferguson, managing director of U.K.-based consultancy Intelligent Business Strategies Ltd., also at the BI Summit in Oregon.
As a result, big data analytics applications are being left unfinished in many of the organizations with which Ferguson works. "There are half-written Python and Scala jobs all over the place," he said.
More tips from experienced analytics managers on how to hire data scientists
Does the hype match the reality for data scientist jobs?
Why physicists may be a good fit for data scientist openings