Blog

We spend a lot of time with customers discussing how DMExpress can be used to pre-process data prior to loading into their data warehouse or analytics system, and get the most up to the minute information to their business users in the shortest possible time. For company's that obtain a growing percentage of their business from the web, we find DMExpress is generally the best tool for processing Clickstream and Web Log data (ie. web server logs from Apache, IIS etc) due to the huge volumes of data that are involved.

For example, Overstock.com is a leading on-line retailer of high-quality, low-cost goods. Without a brick and mortar presence, the company relies on quickly making 30 GB of daily clickstream data available for analysis by brand and category managers. This analysis supports strategies for marketing spend, offerings, pricing, and campaigns aimed at increasing website traffic. The process previously involved using lengthy, complex, custom coding to move data from the web analytics platform, into a data warehouse powered by Teradata. The code, used for parsing, manipulation, and cleansing of data prior to loading, was time-consuming to maintain.

“We need an environment that supports agile development and rapid delivery of information,” said Sam Peterson, SVP Technology at Overstock.com. “Time to deployment and the ability to do file-based processing was important to us. DMExpress was fast and efficient.”

Another customer - comScore is a global leader in measuring the digital world and preferred source of digital marketing intelligence. Through a powerful combination of behavioural and survey insights, comScore enables clients to better understand, leverage and profit from the rapidly evolving worldwide web and mobile arena. comScore collects terabytes of clickstream data relating to activities conducted by its 2 million panellists.

The company processes over 10.3 billion new master records per week (about 8.5 terabytes of compressed data) and captures about 6.1 million unique domain names per month. The analysis of the raw clickstream data was an overwhelming undertaking as the volume and unformatted nature of clickstream data required very complex parsing techniques. Hand-coded solutions did not have the capability or reliability to effectively address this problem.

Michael Brown, EVP of Software Engineering said, “The performance and ease of use of DMExpress technology positively impacts our bottom line. Our analysts are able to deliver timely and accurate solutions to our clients and meet service level agreements, as DMExpress technology is able to convert raw clickstream data into valuable granular information at lightning speed. In fact, we saw a 500% improvement in data integration throughput after deploying DMExpress technology in our environment..."

In Australia, a consumer information company uses DMExpress as the basis for clickstream analysis services it provides to large finance and retail customers. DMExpress is the only product on the market that can process the large volumes of data involved in a commercially viable time frame.

To download the comScore case study, click here >>

To find out more about how we can help process your clickstream and web log data faster Contact Us >>

UPDATE: We also have a White Paper available on this subject, click here >>

Add a Comment

Fields marked with an * are required