Sergey Nivens - Fotolia
Earlier this year, Japanese retailer Muji created a new membership rewards program called Passport. It allows customers to scan in-store displays, check in at different locations and rate products from their smartphones, all of which earns them points to use toward future purchases. Muji collects the data to do audience segmentation, A/B testing, marketing campaign optimization and targeted promotions. However, disparities between all these data types made it difficult to process in a relational database.
When Muji first started collecting and analyzing customer data several years ago, the operation was pretty straightforward. Point-of-sale and e-commerce data went into an Amazon Redshift cloud storage system, accessed and analyzed either in a Tableau system or Microsoft Excel. But Passport made things more complicated. Takashi Okutani, general manager of Web and business development at Muji, said the Redshift database could not handle the massive amounts of streaming and semi-structured data.
"Especially for weblogs, we have so much data, like 900 million records per year," Okutani said. "Some of it includes 400 columns. That means we need huge processing machines."
For this role, Okutani chose in May a system from Treasure Data, a cloud-hosted database that he said can handle various data types coming in at high speed.
Of course, there are certain drawbacks to going with a cloud database. For one thing, businesses face the daunting prospect of sending all their data to the cloud. Security is another major concern.
Despite these potential snags, Okutani said going with cloud storage made sense for the use case. The data feeding into the database is all created online, so it's easy to send it straight to the cloud. Security also isn't a great concern in this case because the program doesn't capture sensitive customer information like credit card numbers.
Takashi Okutanigeneral manager of Web and business development, Muji
"We don't need any kind of personal data, so to be honest I don't really mind too much those kind of security concerns," Okutani said. He added that even on-site databases get hacked. As long as valuable data is being stored, thieves will try to grab it.
Additionally, the cloud storage system was easier to get up and running and to maintain with a small staff than some other options. For example, he looked at a Hadoop cluster for the project, but felt it would be too difficult to find the right staff to manage. Currently, Muji employs a relatively small team of one data scientist and one data warehouse technician. Treasure Data is responsible for all of the technical maintenance of the database, which allows Muji to keep its technical team small.
"We don't have to worry so much about management," Okutani said. "When it comes to the human resources point of view, it's easier compared to a Hadoop system."
The Passport program has led to some promising sales numbers for Muji. Okutani said about 15% of Muji's customers participate in the Passport program, and these customers spend about $40 US dollars per trip, compared to the overall average of about $20 per trip.
"So that means the purchases are nearly doubled, and they'll come to the stores every day," he said.
Cloud storage providers discuss challenges of big data in the cloud
Big data and the cloud aren't always a match made in heaven
Where Hadoop enters the cloud picture