Case Study - Real-Time Data Monitoring
A Fortune 50 company came to us needing an enterprise-wide, real-time data platform. A system that could both secure and support access to highly controlled commercial and military flight data. They had hundreds of data sources with hundreds of thousands of tables and data types and a need to ingest up to a petabyte of data a day. We broke these challenges down to develop a solution that was ideal for the client
- Client
- Global Aerospace Company
- Year
- Service
- Strategy, Data Ingest/ETL, Data Warehousing, Visualizations
The Challenge
Security, Classification and Location
Data must be secure, classified and in some cases limited to where it can be physically stored. It must conform to GDPR and ITAR requirements.
Massive Data Consumption
Hundreds of terabytes of batch and streaming data per day needed to be classified, transformed and then stored.
Data Inconsistency
Data was plagued with multiple sources of truth and managed through 100’s of custom ETLs and stored throughout an enterprise.
Real-time Data Ingest and Transformation
Data transformation was needed to support business intelligence and operations often requiring massive chunks of data analyzed quickly.
The Solution
We delivered a data platform that is an easily maintainable, scalable, and highly secure data management and governance system built on proven non-proprietary industry standard tools the client was already familiar with.
Scalability
Ingest of streaming and bulk data, scalable to hundreds of terabytes a day.
A Platform for Business Intel & Operations
It provides real-time access to many data types and management and analysis of petabytes of data.
Easy ETL Services and Governance
Easily manage and monitor thousands of simple but powerful ETL jobs in one place.
What we did
- Data Preparation
- Data Cleansing and Normalization
- Data Warehousing
- Custom Dashboards & Visualizations
- Predictive Analysis
- Sensor Data Ingested Daily
- 450 Terabytes
- Increased Transparency & Visibility
- 5x
- Efficiency Gains
- 10x