By now, many companies have decided that big data is not just a buzzword, but a new fact of business life -- one that requires having strategies in place for managing large volumes of both structured and unstructured data. And with the reality of big data comes the challenge of analyzing it in a way that brings real business value. Business and IT leaders who started by addressing big data management issues are now looking to use big data analytics to identify trends, detect patterns and glean other valuable findings from the sea of information available to them.
It can be tempting to just go out and buy big data analytics software, thinking it will be the answer to your company's business needs. But big data analytics technologies on their own aren't sufficient to handle the task. Well-planned analytical processes and people with the talent and skills needed to leverage the technologies are essential to carry out an effective big data analytics initiative. Buying additional tools beyond an organization's existing business intelligence and analytics applications may not even be necessary depending on a project's particular business goals.
This Essential Guide consists of articles and videos that offer tips and practical advice on implementing successful big data analytics projects. Use the information resources collected here to learn about big data analytics best practices from experienced users and industry analysts -- from identifying business goals to selecting the best big data analytics tools for your organization's needs.
Real-world experiences with big data analytics tools
Technology selection is just part of the process when implementing big data projects. Experienced users say it's crucial to evaluate the potential business value that big data software can offer and to keep long-term objectives in mind as you move forward. The articles in this section highlight practical advice on using big data analytics tools, with insights from professionals in retail, healthcare, financial services and other industries.
Many data streaming applications don't involve huge amounts of information. A case in point: an analytics initiative aimed at speeding the diagnosis of problems with Wi-Fi networking devices. Continue Reading
To give healthcare providers a real-time view of the claims processing operations its systems support, RelayHealth is augmenting its Hadoop cluster with Spark's stream processing module. Continue Reading
A number of myths about big data have proliferated in recent years. Don't let these common misperceptions kill your analytics project. Continue Reading
Learn how health system UPMC and financial services firm CIBC are adopting long-term strategies on their big data programs, buying tools as needed to support analytics applications. Continue Reading
An executive from Time Warner Cable explains why it's important to evaluate how big data software fits into your organization's larger business goals. Continue Reading
Allegiance Retail Services, a mid-Atlantic supermarket co-operative, is deploying a cloud-based big data platform in place of a homegrown system that fell short on analytics power. Continue Reading
Users and analysts caution that companies shouldn't plunge into using Hadoop or other big data technologies before making sure they're a good fit for business needs. Continue Reading
Compass Group Canada has started mining pools of big data to help identify ways to stop employee theft, which is a major cause of inventory loss at its food service locations. Continue Reading
Big data projects must include a well-thought-out plan for analyzing the collected data in order to demonstrate value to business executives. Continue Reading
Data analysts often can find useful information by examining only a small sample of available data, streamlining the big data analytics process. Continue Reading
Shaw Industries had all the data it needed to track and analyze the pricing of its commercial carpeting, but integrating the information was a tall order. Continue Reading
Opportunities and evolution in big data analytics processes
As big data analytics tools and processes mature, organizations face additional challenges but can benefit from their own experiences, helpful discoveries by other users and analysts, and technology improvements. Big data environments are becoming a friendlier place for analytics because of upgraded platforms and a better understanding of data analysis tools. In this section, dig deeper into the evolving world of big data analytics.
Technologies that support real-time data streaming and analytics aren't for everyone, but they can aid organizations that need to quickly assess large volumes of incoming information. Continue Reading
Before starting the analytical modeling process for big data analytics applications, organizations need to have the right skills in place -- and figure out how much data needs to be analyzed to produce accurate findings. Continue Reading
The Flint River Partnership is testing technology that analyzes a variety of data to generate localized weather forecasts for farmers in Georgia. Continue Reading
Consultant Rick Sherman offers a checklist of recommended project management steps for getting big data analytics programs off to a good start. Continue Reading
Consultants Claudia Imhoff and Colin White outline an extended business intelligence and analytics architecture that can accommodate big data data analysis tools. Continue Reading
Big data experts Boris Evelson and Wayne Eckerson shared ideas for addressing the widespread lack of big data skills in a tweet jam hosted by SearchBusinessAnalytics. Continue Reading
In the Hadoop 2 framework, resource and application management are separate, which facilitates analytics applications in big data environments. Continue Reading
It's important to carefully evaluate the differences between the growing number of query engines that access Hadoop data for analysis using SQL, says consultant Rick van der Lans. Continue Reading
Marketers have a new world of opportunities thanks to big data, and data discovery tools can help them take advantage, according to Babson professor Tom Davenport. Continue Reading
The Data Warehousing Institute has created a Big Data Maturity Model that lets companies benchmark themselves on five specific dimensions of the big data management and analytics process. Continue Reading
News and perspectives on big data analytics technologies
Big data analysis techniques have been getting lots of attention for what they can reveal about customers, market trends, marketing programs, equipment performance and other business elements. For many IT decision makers, big data analytics tools and technologies are now a top priority. These stories highlight trends and perspectives to help you manage your big data implementation.
President Barack Obama has introduced proposals for data security, but not everyone thinks they will address key questions for businesses. Continue Reading
The faster analytics performance spurred by in-memory technology can help companies capitalize on big data, but there are barriers to be aware of. Continue Reading
Marketing and advertising are the sweet spots for big data, but those who serve up marketing insights and ad space will need to evolve to survive. Continue Reading
PayPal's Mok Oh says big data analytics will have arrived when people like him aren't needed. Continue Reading
'Big data' analytics could fundamentally change how businesses operate, according to a keynote speaker at the SAS Analytics conference. Continue Reading
Hadoop and its variants offer analysis for “big data” sets, but there are integration and availability limits, according to a panel at the Enzee Universe 2011 conference. Continue Reading
The U.S. Supreme Court’s ruling on its first data mining case raises questions on advanced analytics practices and policies, especially in the era of “big data.” Continue Reading
In this news recap, executives assess analytic success and marketers realign their technology spending priorities. Continue Reading
Best practices for implementing big data analytics projects
The stories in this section offer a closer look at what makes a big data implementation work -- and what doesn't. Experts share advice for selecting tools used in big data analytics, tips for identifying business goals, best practices for existing resources and insights into how to avoid common mistakes.
Businesses looking to get real value out of big data, and avoid overwhelming their systems, need to be selective about what they analyze. Continue Reading
The market for big data analytics tools remains fragmented, but users are expecting consolidation in the near future. Continue Reading
Many businesses implementing big data analytics applications focus on the size of their data sets, when what they should really focus on is whether they have the right data. Continue Reading
C-level execs need to rely more on data and less on instincts to capitalize on big data technology, said speakers at the 2013 MIT Sloan CIO Symposium. Continue Reading
Using in-memory tools to analyze pools of big data raises system design, scalability and data integration issues that must be addressed up front. Continue Reading
A breakthrough like big data comes around only once in a blue moon. Here's what business executives need to do to capitalize on it. Continue Reading
Consultant Rick Sherman details the skill sets and roles that he thinks are vital to the success of "big data" analytics initiatives. Technical skills alone aren't enough, he says. Continue Reading
Consultant Rick Sherman details the biggest mistakes to avoid in planning and managing deployments of big data analytics tools, from focusing on the technology to overselling projects. Continue Reading
Choosing the right technology is only half the startup battle on "big data" analytics. Get a list of deployment tips from consultant Lyndsay Wise to help set your organization on the right path. Continue Reading
Putting an effective "big data" analytics plan in place can be a challenging proposition. Consultant Lyndsay Wise offers her advice on what to consider and how to get started. Continue Reading
The terrain may seem foreign, but many proven data management and business intelligence best practices translate well to big data analytics programs, according to analysts. Continue Reading
Big data analytics videos
Consultants and experienced users discuss big data analytics technologies and trends in the following videos. Speakers include Colin White, president and founder of BI Research; William McKnight, president of McKnight Consulting Group; and Wayne Eckerson, director of TechTarget's BI Leadership Research unit.
Industry analyst Colin White is tired of all the talk about the three V's of "big data." He instead wants to focus on the use cases -- the analytics -- of big data.
In a video Q&A, Timothy Leonard of U.S. Xpress explains how the trucking company uses big data analytics, real-time BI software and mobile BI apps in unison to make important business decisions.
In a video interview, consultant William McKnight provides an explanation of "big data" basics: what big data is and the issues that are involved in managing and analyzing it.
In a video interview, business intelligence analyst Wayne Eckerson offers tips on using big data analytics software and related big data technology. He also gives his view of what big data really is.
Is there real value to be gained by analyzing "big data"? And what challenges does big data present to organizations? Consultant Jill Dyche answers those questions and more in a video interview.
Successful analytics managers mix business blue and IT red skills, TechTarget's Wayne Eckerson says in discussing his new book on analytical leaders.
Important terms related to big data analytics
Read the definitions below to learn more about commonly used terms related to big data and the distinctions among them.
Big data analytics quiz
Take this brief quiz to test what you've learned about big data analytics best practices.Take This Quiz