Case Study: Massive Data Integration and Analytics for Large Airline Company

In 2016, AuriQ Systems was engaged by a large airline company to assist in a massive data integration and analytics project. AuriQ is a Select Technology Partner with Amazon Web Services (AWS).

The company wanted to integrate all their customer touchpoint data to better understand the behavior and intent of their customers. These data sources included all their online sites and services, email, reservation and boarding systems, as well as third party data from partners. Before engaging AuriQ, they had tried several approaches with multiple large IT vendors, but were unable to find a suitable solution that could deal with the challenging volume and disparity of their data.

In late 2016, AuriQ demonstrated an initial proof of concept (POC) to the client. The solution utilized AuriQ’s cloud based software, Essentia, that runs on Amazon EC2 instances with S3 object storage serving as a data lake. Essentia is a proprietary software developed by AuriQ Systems that is designed to perform massively parallel data processing jobs. This includes data preprocessing/preparation as well as data analysis and loading. It’s power allows users to analyse billions of data in real-time, with a simple to use Excel-like user interface.

During the POC, Essentia was used for data exploration, data preparation (ETL), integration and analysis of a large sample of data. This was accomplished within a week while prior attempts by the client with other solutions had taken up to 6 months.

In 2017, AuriQ worked with the client to open 3 different AWS accounts for three different business units within the company. This was done so that each unit could analyze their data independently and for compliance with their data privacy policy. For each unit, AuriQ processed and integrated over 10 billion events of the client’s data and made it available for analysis through interactive real-time dashboards, pivot tables and charts. The Essentia solution allowed each of the client’s business units to make changes to their own analysis criteria on the fly and have results returned within a few seconds.

After the successful POC, ownership for all 3 AWS accounts created were transferred to the client in early 2018. Since then the client has moved it to an official project, adding a variety of other data to integrate and analyze, with typical AWS usage charges of more than $20k per month. AuriQ continues to assist them to integrate and analyze their massive volumes of data, as well as help to apply other AWS services to their data such as Amazon’s machine learning service, Sagemaker.

To find out more about this case study, contact us at