Case Study
Cutting Data Engineering Delivery Time by 40% for a Top 20 Global Pharmaceutical Company
Fresh Gravity helped a global pharmaceutical company modernize its data ingestion and engineering lifecycle using its proprietary Agentic Intelligent Data Engineering (AiDE) Accelerator. The solution reduced end-to-end development time by 40% and data profiling effort by 60%, while creating a more flexible data foundation to support the client’s evolving data needs
About the client
The client is one of the world’s top 20 pharmaceutical companies, producing active pharmaceutical ingredients and finished products across Human Pharma and Animal Health.
To manage master data for client and competitor products, the organization implemented Semarchy xDM to develop the Reporting Product Master (RPM), a platform designed to capture, master, and manage product data and hierarchies. RPM integrates multiple data sources to support downstream analytical operations.
Problem
As the client’s portfolio of applications and data volumes grew, its existing data ingestion approach presented challenges in flexibility, transparency, and scalability. Even minor modifications required significant manual intervention, adding to engineering effort and Total Cost of Ownership (TCO).
The existing architecture also limited visibility into data lineage, quality, and transformation logic. Manual data profiling and hard-coded pipelines required substantial engineering effort, while adapting the ingestion process to new business requirements could result in delays.
The client needed a modernized, cloud-native data foundation that could streamline ingestion and transformation while supporting advanced analytics and operational master data management.
Solution
Fresh Gravity designed a modern data stack anchored by automation, using its proprietary Agentic Intelligent Data Engineering (AiDE) Accelerator to automate key elements of data profiling and pipeline generation.
AiDE dynamically profiled source data and generated Source-to-Target Mappings (STTMs), reducing the manual analysis required during the engineering process. Snowflake provided centralized storage and elastic compute, while dbt was used to orchestrate version-controlled data transformations directly within Snowflake.
SnapLogic and AWS Glue pipelines were used to extract curated data from Snowflake and insert it into Semarchy landing zones for master data management.
Together, these components enabled the client to move from fragmented ingestion processes to a more cohesive, automated pipeline, standardize transformations within Snowflake, and establish reliable outbound feeds to its master data management platform.
Technology stack: AiDE | Snowflake | dbt | SnapLogic | AWS Glue | Semarchy
Impact
The modernized approach delivered measurable improvements across the data engineering lifecycle. End-to-end development time was reduced by 40%, enabling the business to use its data assets faster. Automated data profiling also reduced profiling effort by 60%, lowering engineering overhead and improving resource allocation.
The new Snowflake- and dbt-based architecture provided a more scalable and flexible data foundation that could accommodate new integrations. Despite systemic bottlenecks encountered within the complex enterprise environment, project milestones were met on schedule
The result was a more transparent and automated data engineering capability, with reduced development and profiling effort and greater flexibility to support the client’s evolving data requirements.