Data science, when done properly, can give businesses a huge edge on their competition. But there lots of pitfalls along the way to doing effective analytics. Organizations need to be strategic about how they carry out a data science program to make sure they really deliver the impact they're looking for.
"Often, the things that are the hardest to measure are the things that are the most important," said Daniel Mintz, head of analytics at online media company Upworthy, who spoke at the Big Data Innovation Summit in Boston.
You are what you measure
Being a media company, Upworthy -- based in New York -- wanted to measure the degree to which people engage with its content to see what performs the best. Typical Web metrics for gauging success, like page views and shares, didn't really get at this. Just because someone lands on a page and tweets out the link doesn't mean that person actually read the article.
So, to get measurements of engagement, Mintz said he had to work with his team members to develop their own metrics. He said they now look at a complex mix of time on page, mouse scrolls and the timing of clicks to gauge how strongly readers engage with their content and understand what types of content perform the best.
Mintz said this was a challenging process, but that's the whole point of data science. If the best insights came from the most easily answered questions, there would be no need for data scientists. He said businesses need to think about what's most important to their operations and find ways to measure it, even if it's hard. "We have to get past 'we don't measure,'" he said.
Take the long view
Often, analytics professionals advocate for going after quick wins, especially when a data science program is new. This approach helps bring the business along and convinces the organization of the value of data-driven decision making. But the pitfall here is that the quick-wins approach becomes entrenched as the model for data science.
Silvanus Lee, head of product analytics at ride-sharing company Uber, said in a presentation it's important to guard against this approach. He said he and his team are constantly trying to balance the desire for short-term gains against the need for long-term projects that may take a while to pay off, but ultimately could be game changers.
Silvanus Leehead of product analytics, Uber
For example, he talked about how Uber is currently developing a service that would allow passengers to pool rides. It could maximize the efficiency of drivers' trips and reduce the cost for riders, while also increase the total number of trips taken on the Uber network. The company is still working on the algorithm that pools riders and matches them to drivers. The service has yet to deliver any tangible return on investment, but the company thinks it could eventually be a popular service, so Lee and his team are continuing to work on it.
"You want to make sure you're looking at the long-view, not just the short-view, metrics," Lee said.
Ultimately, whether something is worth pursuing over the long term may come down to judgment. He talked about how when Facebook first launched the news feed feature, which is currently the standard view for all users and on which users spend most of their time on Facebook, it was almost universally hated. If the company was measuring for short-term payoffs, Facebook might have scrapped the news feed. But the company knew it would open up huge advertising opportunities and eventually be accepted by users, so it stuck with the effort. Now, it's hard to imagine the company being profitable without it.
"Data science is not just statistics," Lee said. "It's statistics plus a lot of judgment. Data science brings a data-driven voice to the table, but it shouldn't start or stop the conversation."
Target specific business problems
For the Boston Red Sox, one of the biggest business problems is mispriced tickets. Several years ago when all tickets had the same face value, seats to highly desirable games used to sell for huge markups on the secondary market while fans commonly felt they were being overcharged for less prime matchups. "We knew this, but we didn't have it quantified," Tim Zue, vice president of business development for the Red Sox, said in a presentation.
In 2011, the team started working on a dynamic pricing model that would allow it to match prices to demand. Zue and his crew collected six years' worth of data from StubHub.com, one of the leading secondary-market ticket sellers, and looked for patterns that would allow them predict future demand for specific games. They found that the day of the week and the month in which the game was played were some of the strongest predictors of demand. Weekends in July saw high demand, while weekdays in April had low demand, for example. The team now bases prices on this information.
Zue said losing potential revenue to the secondary market due to tickets being priced below the demand level was considered a major problem, but once his workers had the data, it became clear how to address it through dynamic pricing. It all starts with the data science program knowing the business and where there is room for improvement.
"The challenge is the 'so what,'" Zue said. "How do you use data to make decisions that help your business? You have to use your data to make these decisions."
Data science programs need to explain their benefit to the business
Jobs in data science may not be as plentiful as thought
How to develop a big data strategy with data science in mind