Building an End-to-End Marketing Analysis and Attribution Solution

by auriqadmin on

Objective

Providing marketing attribution modeling at big data scale is difficult for most data analysis platforms. Ingesting a variety of data from different sources in multiple formats can be a data integration nightmare as there are multiple levels of ETL to perform, then joining the disparate data into a cohesive customer journey table, and finally applying rules based or algorithmic attribution models against the combined data.

Our solution engineers built the platform, that later evolved to become the Essentia Marketing SaaS, to be able to connect, cleanse, process and analyze enormous amounts of diverse marketing data in a flexible, fast and iterative way. The basic workflow is as follows:

1. Connecting to the Data

Marketing data is fetched on regularly scheduled basis from our customers’ servers directly via FTP/SFTP as well as from third party vendors.

In addition, some customers utilize a tag script that we provide and manage to collect data from various advertising and marketing channels. Collected data is stored on the cloud in Amazon S3, and left in their raw original state. Typically data is in the form of delimited file types that have been compressed in either zip or gzip format.

2. Data Categorization

The data scanner module is used to characterize the types of log data collected, which is then organized into categories based on simple rules.

The Essentia approach minimizes management as more data is added, since files do not have to be copied to specific locations or modified in any way. Because of this ‘data lake’ philosophy, your data is left in its original state, and when analysis needs to be performed, Essentia references the category database to find which files to include in its analysis.

3. Data Preparation

Essentia’s ETL tools automatically cleanse, dedupe, and transform raw data in a schema-on-read process, streaming the now structured data into an in-memory database.

In addition to structuring the raw data, different data sources can be blended and merged at the same time. A combination of key-value matching and fuzzy logic allows Essentia to join disparate data into a cohesive and comprehensive customer journey table that contains the record of all events and interactions for each user.

Analysis and Visualization

Customers work with our data scientists and engineers to define their analysis requirements. From there Essentia can analyze millions to billions of records and provides various reporting options including marketing attribution analysis, conversion path analysis, behavioral analysis, and much more.

Marketing Attribution

Essentia SaaS has a library of attribution models that can be applied to customer data. Rules based models are applied as baselines for comparison against any customized or data driven models. Attribution analysis is rarely a one size fits all solution, so our data scientists and engineers work with our customers to find the right models to apply to their data.

Conversion Path Analysis

Understanding the most efficient paths to conversion is always a key concern of any marketer. The customer journey table that Essentia can generate allows our customers to get a complete view of all possible paths to conversion and rank them. External ad impressions & clicks, organic search activity, and in-site actions can all be combined and segmented for a more thorough understanding of what drives traffic and conversion.

Campaign and Site Trends

Trend data for all your data sources can be viewed from the Essentia dashboard. Ad server logs, web logs, application logs, and more can all be added. View the number of event and unique user counts and quickly see any correlation of general trends between data sources. For each data source, you can also further segment for a more granular view of campaign and site performance.

Customer data is analyzed on a regularly scheduled basis and visualized through custom dashboards. Any new analysis can be easily provided with short turn-around times due to the flexibility and scalability of the Essentia platform.