Sky's the limit for cloud big data in online ads

Worldwide spending on cloud-based services supporting online advertising -- the largest segment of the public-cloud market -- is expected to increase 14% to $90.3 billion this year, according to Gartner. And big data is playing an increasing role in the online marketing process, as advertising platform vendors and their corporate clients look to analyze huge volumes of Internet clickstream data and metrics on ad performance. In a survey of 500 IT and marketing professionals conducted in July 2015 by managed cloud services provider 2nd Watch, 50% of the respondents said they were likely to expand their use of big data to help fuel digital marketing programs.

As a result, it's not surprising that cloud big data platforms have become the go-to analytics technology for many online marketing and advertising companies. Sellpoints Inc. is among them -- the Emeryville, Calif., company began using Databricks' cloud-based version of the Spark processing engine in early 2015 after previously running Hadoop and Spark together through a managed cloud service.

Benny Blum, vice president of product and data at Sellpoints, said most of the data the company captures to help serve up ads tailored to individual Internet users comes from system logs on external Web servers. It makes sense, he added, to store and analyze that data in the cloud instead of spending big money to build a sufficient server architecture internally.

Benny Blum

Using the cloud also enables him to scale up or pare down computing and storage resources as needed. Blum said that with IT demands increasing as the company's business grows, it's easy to match needs to resources in the cloud -- and the same goes for downsizing the cloud setup. "Why invest a couple hundred grand in hardware when you're incurring costs that come and go? I can decide I need a 2 terabyte cluster and then just shut it down [when it's no longer necessary]."

The Databricks Spark platform runs in the Amazon Web Services cloud. Blum said Sellpoints uses it to process hundreds of millions of data points streamed into an Amazon Simple Storage Service (S3) data store on a daily basis, converting the raw data into analyzable metrics. The processed data then goes back into S3, where it's mined by the company's data scientists and used by its business intelligence team to build reports and data visualizations for customers in Tableau's BI software.