Essential Guide

Guide to big data analytics tools, trends and best practices

Experts share perspectives and identify best practices for big data analytics projects in this Essential Guide.


By now, many companies have decided that big data is not just a buzzword, but a new fact of business life -- one that requires having strategies in place for managing large volumes of both structured and unstructured data. And with the reality of big data comes the challenge of analyzing it in a way that brings real business value. Business and IT leaders who started by addressing big data management issues are now looking to use big data analytics to identify trends, detect patterns and glean other valuable findings from the sea of information available to them.

It can be tempting to just go out and buy big data analytics software, thinking it will be the answer to your company's business needs. But big data analytics technologies on their own aren't sufficient to handle the task. Well-planned analytical processes and people with the talent and skills needed to leverage the technologies are essential to carry out an effective big data analytics initiative. Buying additional tools beyond an organization's existing business intelligence and analytics applications may not even be necessary depending on a project's particular business goals.

This Essential Guide consists of articles and videos that offer tips and practical advice on implementing successful big data analytics projects. Use the information resources collected here to learn about big data analytics best practices from experienced users and industry analysts -- from identifying business goals to selecting the best big data analytics tools for your organization's needs.

1Business benefits-

Real-world experiences with big data analytics tools

Technology selection is just part of the process when implementing big data projects. Experienced users say it's crucial to evaluate the potential business value that big data software can offer and to keep long-term objectives in mind as you move forward. The articles in this section highlight practical advice on using big data analytics tools, with insights from professionals in retail, healthcare, financial services and other industries.


Data streaming projects aren't always a big-data deal

Many data streaming applications don't involve huge amounts of information. A case in point: an analytics initiative aimed at speeding the diagnosis of problems with Wi-Fi networking devices. Continue Reading


Stream processing boosts online ad analytics

Online advertising platform providers Altitude Digital and Sharethrough are both tapping Apache Spark's stream processing capabilities to support more real-time analysis of ad data. Continue Reading


Health claims processor taps Spark Streaming for better analytics

To give healthcare providers a real-time view of the claims processing operations its systems support, RelayHealth is augmenting its Hadoop cluster with Spark's stream processing module. Continue Reading


Don’t let big data complexity deter you

Complexity can seem like a burden to already overworked IT departments -- but when it comes to your organization's big data implementation, there's a good reason for all those systems. Continue Reading


Don’t let your analytics projects fall victim to big data myths

A number of myths about big data have proliferated in recent years. Don't let these common misperceptions kill your analytics project. Continue Reading


Companies opt for long view on managing big data projects

Learn how health system UPMC and financial services firm CIBC are adopting long-term strategies on their big data programs, buying tools as needed to support analytics applications. Continue Reading


Look at the big picture prior to adopting big data tools

An executive from Time Warner Cable explains why it's important to evaluate how big data software fits into your organization's larger business goals. Continue Reading


Grocery co-op looks to big data system to improve sales analysis

Allegiance Retail Services, a mid-Atlantic supermarket co-operative, is deploying a cloud-based big data platform in place of a homegrown system that fell short on analytics power. Continue Reading


Clear heads, open eyes required on big data technology decisions

Users and analysts caution that companies shouldn't plunge into using Hadoop or other big data technologies before making sure they're a good fit for business needs. Continue Reading


Food service company mines big data to reduce theft by workers

Compass Group Canada has started mining pools of big data to help identify ways to stop employee theft, which is a major cause of inventory loss at its food service locations. Continue Reading


Careful data analysis vital to deriving value from big data projects

Big data projects must include a well-thought-out plan for analyzing the collected data in order to demonstrate value to business executives. Continue Reading


Get big data analytics results with small sample sizes

Data analysts often can find useful information by examining only a small sample of available data, streamlining the big data analytics process. Continue Reading


Carpet maker weaves together data sets to help optimize pricing

Shaw Industries had all the data it needed to track and analyze the pricing of its commercial carpeting, but integrating the information was a tall order. Continue Reading

2New developments-

Opportunities and evolution in big data analytics processes

As big data analytics tools and processes mature, organizations face additional challenges but can benefit from their own experiences, helpful discoveries by other users and analysts, and technology improvements. Big data environments are becoming a friendlier place for analytics because of upgraded platforms and a better understanding of data analysis tools. In this section, dig deeper into the evolving world of big data analytics.


Real-time data streaming can help organizations get ahead of events

Technologies that support real-time data streaming and analytics aren't for everyone, but they can aid organizations that need to quickly assess large volumes of incoming information. Continue Reading


Big data trends to keep an eye on

Although the main trends in big data for 2015 may not be a huge departure from the previous year, businesses should still understand what's new in the world of big data analysis techniques. Continue Reading


Predictive models require the right understanding of big data analytics

Before starting the analytical modeling process for big data analytics applications, organizations need to have the right skills in place -- and figure out how much data needs to be analyzed to produce accurate findings. Continue Reading


Big data project helps to improve farm irrigation efforts

The Flint River Partnership is testing technology that analyzes a variety of data to generate localized weather forecasts for farmers in Georgia. Continue Reading


Utility turns to sensors to gain big data edge

Big data analytics processes on data from sensors and log files can propel users to competitive advantages, but a lot of refining is required first. Continue Reading


To-do items for big data analytics project managers

Consultant Rick Sherman offers a checklist of recommended project management steps for getting big data analytics programs off to a good start. Continue Reading


Made for each other: Big data and analytics

Big data analytics initiatives can pay big business dividends. But pitfalls can get in the way of their potential, so make sure your big data project is primed for success. Continue Reading


New analytics needs call for additions to data warehouse architectures

Consultants Claudia Imhoff and Colin White outline an extended business intelligence and analytics architecture that can accommodate big data data analysis tools. Continue Reading


Experts tweet on best ways to inject big data skills into workforce

Big data experts Boris Evelson and Wayne Eckerson shared ideas for addressing the widespread lack of big data skills in a tweet jam hosted by SearchBusinessAnalytics. Continue Reading


Hadoop 2's improved framework better supports data analytics uses

In the Hadoop 2 framework, resource and application management are separate, which facilitates analytics applications in big data environments. Continue Reading


Access, analyze big data with the right SQL-on-Hadoop engine

It's important to carefully evaluate the differences between the growing number of query engines that access Hadoop data for analysis using SQL, says consultant Rick van der Lans. Continue Reading


Marketers gain competitive edge with big data, data discovery tools

Marketers have a new world of opportunities thanks to big data, and data discovery tools can help them take advantage, according to Babson professor Tom Davenport. Continue Reading


Track the maturity of your big data analytics program

The Data Warehousing Institute has created a Big Data Maturity Model that lets companies benchmark themselves on five specific dimensions of the big data management and analytics process. Continue Reading

3News stories-

News and perspectives on big data analytics technologies

Big data analysis techniques have been getting lots of attention for what they can reveal about customers, market trends, marketing programs, equipment performance and other business elements. For many IT decision makers, big data analytics tools and technologies are now a top priority. These stories highlight trends and perspectives to help you manage your big data implementation.


Opinions vary on data security laws proposed by Obama

President Barack Obama has introduced proposals for data security, but not everyone thinks they will address key questions for businesses. Continue Reading


Teradata rolls out new big data application suite

Teradata is the latest technology vendor making a play for businesses looking for easy-to-implement big data apps. Continue Reading


In-memory software offers possible big data analytics path

The faster analytics performance spurred by in-memory technology can help companies capitalize on big data, but there are barriers to be aware of. Continue Reading


Marketing, advertising remain hotspots for big data analytics

Marketing and advertising are the sweet spots for big data, but those who serve up marketing insights and ad space will need to evolve to survive. Continue Reading


Big data continues to expand from original roots

What is "big data" and why is the IT industry talking about it so much? IT pros offer insights into the origins of the popular and much-hyped phrase. Continue Reading


PayPal data scientist looks beyond big data analytics 'limbo state'

PayPal's Mok Oh says big data analytics will have arrived when people like him aren't needed. Continue Reading


Conference speaker: Big data analytics requires revolutionary approach

'Big data' analytics could fundamentally change how businesses operate, according to a keynote speaker at the SAS Analytics conference. Continue Reading


Hadoop and big data technology: Benefits versus pitfalls

Hadoop and its variants offer analysis for “big data” sets, but there are integration and availability limits, according to a panel at the Enzee Universe 2011 conference. Continue Reading


Privacy policies essential for data mining in big data era

The U.S. Supreme Court’s ruling on its first data mining case raises questions on advanced analytics practices and policies, especially in the era of “big data.” Continue Reading


The future of big data: Analytics

Usama Fayyad, a featured speaker at Netezza's Enzee Universe user conference, came armed with information on how the future of "big data" lies in analytics. Continue Reading


Survey finds split on view of analytics from execs, staffs

In this news recap, executives assess analytic success and marketers realign their technology spending priorities. Continue Reading

4Best practices-

Best practices for implementing big data analytics projects

The stories in this section offer a closer look at what makes a big data implementation work -- and what doesn't. Experts share advice for selecting tools used in big data analytics, tips for identifying business goals, best practices for existing resources and insights into how to avoid common mistakes.


Don't think of it as just big data -- think wide

To get maximum business value from big data analytics efforts, users should look to incorporate a mix of structured and unstructured information -- think of it as wide, not merely big, data. Continue Reading


What to use -- and not to use -- leads big data challenges

Businesses looking to get real value out of big data, and avoid overwhelming their systems, need to be selective about what they analyze. Continue Reading


Big data for big data's sake is a bad idea

IT observers caution organizations against data for data’s sake, urging instead a focus on the best technology for the business problem at hand. Continue Reading


Podcast: How does the term ‘big data’ affect analytics tools?

The market for big data analytics tools remains fragmented, but users are expecting consolidation in the near future. Continue Reading


Big data volume must be balanced with analytics needs

Many businesses implementing big data analytics applications focus on the size of their data sets, when what they should really focus on is whether they have the right data. Continue Reading


Data-driven decision making must be part of big data mix

C-level execs need to rely more on data and less on instincts to capitalize on big data technology, said speakers at the 2013 MIT Sloan CIO Symposium. Continue Reading


Strong foundation required for in-memory analytics on big data

Using in-memory tools to analyze pools of big data raises system design, scalability and data integration issues that must be addressed up front. Continue Reading


Gartner's to-do list for unlocking big data's business value

A breakthrough like big data comes around only once in a blue moon. Here's what business executives need to do to capitalize on it. Continue Reading


Tech, business savvy both needed on big data analytics programs

Consultant Rick Sherman details the skill sets and roles that he thinks are vital to the success of "big data" analytics initiatives. Technical skills alone aren't enough, he says. Continue Reading


Worst practices: What to avoid in big data analytics initiatives

Consultant Rick Sherman details the biggest mistakes to avoid in planning and managing deployments of big data analytics tools, from focusing on the technology to overselling projects. Continue Reading


Initiating a big data analytics project: Five steps to take

Choosing the right technology is only half the startup battle on "big data" analytics. Get a list of deployment tips from consultant Lyndsay Wise to help set your organization on the right path. Continue Reading


Turning talk into action on big data analytics projects

Putting an effective "big data" analytics plan in place can be a challenging proposition. Consultant Lyndsay Wise offers her advice on what to consider and how to get started. Continue Reading


Build on familiar disciplines for big data analytics best practices

The terrain may seem foreign, but many proven data management and business intelligence best practices translate well to big data analytics programs, according to analysts. Continue Reading


Big data analytics videos

Consultants and experienced users discuss big data analytics technologies and trends in the following videos. Speakers include Colin White, president and founder of BI Research; William McKnight, president of McKnight Consulting Group; and Wayne Eckerson, director of TechTarget's BI Leadership Research unit.


Shifting the conversation from big data management to analytics

Industry analyst Colin White is tired of all the talk about the three V's of "big data." He instead wants to focus on the use cases -- the analytics -- of big data.


Big data analytics helps trucking company track its fleet

In a video Q&A, Timothy Leonard of U.S. Xpress explains how the trucking company uses big data analytics, real-time BI software and mobile BI apps in unison to make important business decisions.


Getting started with big data analytics

In a video interview, consultant William McKnight provides an explanation of "big data" basics: what big data is and the issues that are involved in managing and analyzing it.


Big data analytics combines mix of old, new technologies

In a video interview, business intelligence analyst Wayne Eckerson offers tips on using big data analytics software and related big data technology. He also gives his view of what big data really is.


Is there something real behind the big data analytics hype?

Is there real value to be gained by analyzing "big data"? And what challenges does big data present to organizations? Consultant Jill Dyche answers those questions and more in a video interview.


Tips for effective management of analytics programs

Successful analytics managers mix business blue and IT red skills, TechTarget's Wayne Eckerson says in discussing his new book on analytical leaders.


Important terms related to big data analytics

Read the definitions below to learn more about commonly used terms related to big data and the distinctions among them.


Big data analytics quiz

Take this brief quiz to test what you've learned about big data analytics best practices.

Take This Quiz