It may be the Internet of Things, but it takes people to make IoT data management and analytics projects work. The problem is there's a shortage of data scientists and data management professionals with skills in using Hadoop and other big data technologies -- at least at salary levels that won't bust budgets.
Joe Hsy, director of the cloud services platform and tools at Cisco Systems Inc.'s WebEx conferencing unit, said the competition in the job market is intense on both the processing and analytics sides. WebEx is using Hadoop, Spark, Cassandra and other in-demand technologies to collect and analyze data from the mobile devices and PCs of people using its Web and video conferencing services. To help ensure that his team can use those technologies effectively, Hsy said he is working to expand the skills of existing employees, while "aggressively hiring" new workers when he finds them.
Lack of big data scientists, preventing growth for IOT.
The shortage of data scientists and other workers with big data skills was one of the reasons Automatic Labs Inc. decided to adopt Databricks' cloud-based Spark platform, a change in strategy after the San Francisco company initially began doing its own development with the base Apache Spark open source software. "It's very difficult to find infrastructure engineers in San Francisco," said Rob Ferguson, vice president of software engineering at Automatic Labs. "And the cost is just tremendous -- probably $200,000 a year per person."
It's the same at Rockwell Automation Inc., which used Microsoft's Azure cloud technologies to build a platform for storing and analyzing data captured from manufacturing equipment at companies that use its industrial automation tools. "We don't want to be building infrastructure that's plumbing," said John Dyck, Rockwell's global director of software business development. "We want to focus on manufacturing functionality and analytics because that's our value-add."
The rundown on cloud service management and cloud monitoring tools
Analyzing big data in the cloud
A guide to understanding IoT data
- Big Data Analytics Best Practices –SearchBusinessAnalytics
- Reaping the Benefits of Big Data and Real-time Analytics –SearchBusinessAnalytics
- Accelerate Big Data Analytics to Capture Data in Motion –DataTorrent
- Real-Time Big Data Analytics Applications –DataTorrent