Nate Silver came to prominence as a blogger and data journalist, but now as the founder and editor in chief at the site FiveThirtyEight, he's seeing data-driven business problems from the same perspective as most other executives. Things like where analytics fit in the organizational decision-making process, what tools to use for data analyses and how to make sure the site capitalizes on analytics take up as much of his time as reading the tea leaves of poll numbers.
One reason these tasks are so time-consuming is that big data as a discipline, and business analytics more specifically, have yet to develop industry standards and best practices for analyzing, visualizing and interpreting data, according to Silver.
"I've been having the same conversations with people for the last couple years, and people are getting frustrated," he said during a presentation at the HP Big Data Conference in Boston.
Biases stymie data-driven business
This lack of standardization and best practices rears its head in the form of biased analyses and misleading visualizations, Silver said. Not that any of this is intentional, but in his view people are hard-wired to see connections that may not exist or apply causality to an event for which there may only be a correlation. Too often assumptions get suppressed in order to make an analysis look valid, when instead the preconceptions should be stated transparently.
The problem of biases can become even more pronounced, Silver said, working in a true big data environment. For him, analytics is all about understanding the relationships between various events. But as you record more events, relationships between events increase exponentially. Understanding them all becomes even harder, and this is where biases or assumptions are likely to creep in. When no single relationship stands out as especially relevant, it's easy for people to pick out the one or two that make the most sense to them and say their decision is supported by the numbers, even though it may not be.
"The more data you have, the easier it is to reflect bias," Silver said.
Data visualizations often fail, in his view, because people pack too much data into them, making them impossible to interpret, or clean them up too much, stripping them of context that makes them meaningful. Graphs should be clean and simple and use consistent measurement scales and visual elements. Silver said trouble comes when someone outside of the analytics team gets hold of a visualization and, in an effort to make it more visually appealing, removes too much of the context that underpins the analysis.
"There are a lot of bad visualizations produced," Silver said. "Don't let your PR department do a visualization."
Let human intuition guide decisions, too
Nate SilverEditor in chief, FiveThirtyEight
One way to address these pitfalls of the data-driven business is to give data a clearly defined role in the decision-making process, Silver said. An organization doesn't have to turn 100% of the process over to data. In fact, he recommends letting data guide the first 80% of the process and have a person take it from there. This gives analyses a reality check that may correct for biases built into models. If the final decision maker sees that a model's recommendations strongly contradict conventional wisdom, it may be a sign that there are biases in the model or bad data.
This is all part of working toward best practices for analytics, which Silver thinks are coming. For most businesses, a data-driven approach to decision making is a relatively new thing. But as practices progress they will become more standardized.
"We're still learning about best practices," Silver said. "We're still thinking about standards. We're early in the process."
MIT says more data means better decisions
Data-driven culture drives more value from analytics teams
Use caution when letting data drive your decision-making process