Ivelin Radkov - Fotolia

Manage Learn to apply best practices and optimize your operations.

The analytics team shouldn't always pursue quick wins

Analytics teams typically look for quick wins to prove their value. But experts say this approach should not become the guiding strategy for data product development.

A common piece of advice says that an analytics team -- particularly a new one that's eager to prove its value -- should aim for quick wins. By delivering a data product or report quickly, the team can get executives and business partners on board and lay the groundwork for future support, even if the first deliverable isn't exactly groundbreaking.

But not everybody agrees with this advice.

"Quick wins are not always long-term wins," said Ahmad Anvari, head of Messenger business and platform analytics at Facebook, in a presentation at the recent Big Data Innovation Summit in Boston. "I've seen this many times in my career."

Think before you report

For Anvari, focusing too much on short-term projects can distract from long-term strategy and cause teams to miss big opportunities. It's something akin to an investor making stock trades every day to maximize daily returns without thinking about retirement planning.

For example, Anvari said when Instagram, which is owned by Facebook, initially launched in-app advertising about two years ago, there was a lot of excitement about the first data to come out. Compared with other features in the app, users were clicking on ads at high rates. But advertisers weren't happy because users typically left the page immediately after clicking on an ad.

Eventually someone realized the number of people clicking on ads looked good because in the main photo stream users see, ads are the only content type that is clickable. Native content is not. So, many people who clicked on ads did so accidentally. The app now has a feature that asks users to confirm that they intentionally clicked on an ad before sending them to the advertiser's page, and the data is much more meaningful.

Minimally viable can be minimally valuable

The minimally viable product is another way analytics teams look for quick wins, but it can lead to its own share of problems. Dhruv Bhargava, global head of data science at gaming company Zynga Inc., said in a presentation at the conference that while it's important to encourage data scientists and product managers to get usable applications in front of users frequently and support developers' creativity, just getting something out the door shouldn't be an end unto itself.

The minimally viable product strategy is often seen as a good thing because it forces developers and analysts to think about how users will interact with a report or application. It also allows them to continue refining the product based on user feedback.

It can be problematic, though, when developers never update the product after releasing it, which Bhargava said he's seen happen. Product management teams rush to get something basic out with the intention to come back to it to refine it. But then they don't, which means users have nothing but suboptimal data applications.

"If you say we need to come back to something and improve it, do it," he said.

Don't train for the wrong game

Using the wrong data to build data products can also lead development projects astray. Steve Carter, chief scientist for online dating site eHarmony Inc., said data scientists and members of the analytics team will often use whatever data they can get their hands on at early stages of projects. Typically, this will be some kind of historical data. But Carter said if they are building a product that will be used in production -- in the case of eHarmony, the main product is a predictive engine that matches users based on profile characteristics -- the product should be trained on the same data it will use once in production.

It can be challenging for data scientists to get their hands on production data. This could have to do with governance or engineering restrictions, and it is particularly problematic for a data scientist looking for quick wins.

But Carter said a model trained on historical data will often fail when put in production and used against current data. This is why data scientists should always work in conjunction with data engineers to make sure whatever they are building will hold up. It may take longer, but the end result will be better.

"When you start envisioning a process of modeling a data product, start with the engineers," Carter said.

Next Steps

You don't necessarily need a data scientist on your analytics team

Everything you need to know about what your analytics workers need

Execs talk about their approach to hiring data scientists

Dig Deeper on Advanced analytics software

Join the conversation


Send me notifications when other members comment.

Please create a username to comment.

How much emphasis does your analytics team place on quick wins?

Good comments. I think the best approach is a blend of quick wins with a long term solution in mind. A lot of the foundation work to build a responsive, high value BI shop take a lot of planning and work. So, these infrastructure like projects cannot typically be built in quick wins.

Ideally the quick wins should be stepping stones to achieve the bigger overall solution, but one shouldn't be so focused on only delivering long-term solutions that you abdicate your responsibility/ability to provide interim and ad hoc support to your organization.

BI is a journey, not a destination, so we have an opportunity to educate and bring the organization along with our evolving BI maturity (and long term plan). And, in my opinion, there aren't any shortcuts to jump steps in your organization's growth. Take advantage or your agility and ability to provide quick wins and leverage that political capital to build out the longer solutions. If your executive see the value you can provide they will be more accepting of providing you the latitude to build out some longer term solutions.

One of the great opportunities BI has is that it is nimble and responsive. Use these characteristics to your advantage and apply an agile project delivery methodology for report building for your organization. You need to help your organization understand the value BI is bringing to them. Using the agile/iterative methodology allows the organization to understand and build their reporting requirements over time. Anyone that believes a business user can clearly articulate all of their reporting requirements at the onset of a project--especially in new system implementations--is not only fooling themselves, but also providing a disservice to their organization.

My argument against big, long BI implementations is the last thing you want to do is to turn your BI shop into a bureaucratic, black hole IT shop that people avoid like the plague because you won't deliver anything until it is perfect and meets every last requirement of every individual in the organization.