Evaluate Weigh the pros and cons of technologies, products and projects you are considering.

Shortage of data scientists, big data pros vexes IoT efforts

There's an abundance of big data technologies that can support Internet of Things projects, but the pool of IT professionals with the advanced know-how to handle massive amounts of IoT data is limited.

This article can also be found in the Premium Editorial Download: Business Information: IoT applications make advances, but hurdles lie ahead:

It may be the Internet of Things, but it takes people to make IoT data management and analytics projects work. The problem is there's a shortage of data scientists and data management professionals with skills in using Hadoop and other big data technologies -- at least at salary levels that won't bust budgets.

Joe Hsy, director of the cloud services platform and tools at Cisco Systems Inc.'s WebEx conferencing unit, said the competition in the job market is intense on both the processing and analytics sides. WebEx is using Hadoop, Spark, Cassandra and other in-demand technologies to collect and analyze data from the mobile devices and PCs of people using its Web and video conferencing services. To help ensure that his team can use those technologies effectively, Hsy said he is working to expand the skills of existing employees, while "aggressively hiring" new workers when he finds them.

Lack of big data scientists, preventing growth for IOT.

The shortage of data scientists and other workers with big data skills was one of the reasons Automatic Labs Inc. decided to adopt Databricks' cloud-based Spark platform, a change in strategy after the San Francisco company initially began doing its own development with the base Apache Spark open source software. "It's very difficult to find infrastructure engineers in San Francisco," said Rob Ferguson, vice president of software engineering at Automatic Labs. "And the cost is just tremendous -- probably $200,000 a year per person."

It's the same at Rockwell Automation Inc., which used Microsoft's Azure cloud technologies to build a platform for storing and analyzing data captured from manufacturing equipment at companies that use its industrial automation tools. "We don't want to be building infrastructure that's plumbing," said John Dyck, Rockwell's global director of software business development. "We want to focus on manufacturing functionality and analytics because that's our value-add."

Next Steps

The rundown on cloud service management and cloud monitoring tools

Analyzing big data in the cloud

A guide to understanding IoT data

This was last published in December 2015



Find more PRO+ content and other member only offers, here.

Essential Guide

IoT analytics guide: Understanding Internet of Things data

Join the conversation

1 comment

Send me notifications when other members comment.

By submitting you agree to receive email from TechTarget and its partners. If you reside outside of the United States, you consent to having your personal data transferred to and processed in the United States. Privacy

Please create a username to comment.

Is your company struggling to hire qualified data scientists?