Logistics - Data Hub - Techmango

Background

Our customers are from logistic domain dealing with sensitive data covers customs and other information. This required higher level of security standards. On the other hand business demands on getting faster insights to data and a scalable data platform. This required the architecture team to build a cloud based data hub which enabled them to bring the data of huge volume as quick as possible. Ability to scale quickly, horizontally as well as vertically also one of the main objective. Compliance required the customer to process and archive huge volume of data with secured access

Solution Implemented

Parameter driven ETL framework
S3 based Data lake, Glue and PySpark for scalable and serverless solution
Fine grained security (glue database level as well as for S3 folders) using AWS IAM
AWS KMS for Encryption (S3 and Glue)
EDI Integration (IBM MQ)

Outcomes

With the ETL framework and other implemented agile methodologies, a robust and scalable cloud data hub was built. It helped the end-user with seamless data handling and effective data processing. AWS KMS encryption approach gave top-notch security standards. The product totally impressed the user with its best-in-class performance.

Download Case Study