Session
(Recordings Available)
Data validation is complex and there are no generic frameworks which can suite all the data validation requirements on a file based target system. There is a need to create a generic framework which can take care of automating validations which are common across different data pipelines.
The framework is built using native AWS services and is available as a CDK. The core for the framework is the Data Quality feature in AWS Glue.
This framework ensures that the basic data checks(null check, file structure check, valid value check, duplicate check) are validated on the target file system by using an automated process.
Manoj Kumar Anand
Experienced Software Quality Assurance Engineer with 10 years of experience in ETL, ELT, Data Warehouse, Salesforce and SAP system. Strong SQL, AWS and Test Automation skills. Problem solver and data enthusiast.
Pranavi Kandagadla Prasad
Pranavi is a Quality Engineering Manager with over 13 years of experience. She excels in ensuring the delivery of top-notch software products by leveraging her deep understanding of industry best practices and cutting-edge testing methodologies. Pranavi has a proven track record of implementing effective testing strategies and leading cross-functional teams to streamline processes and foster continuous improvement. Skilled in various testing methodologies, she is adept at developing test plans, managing defects, and collaborating with development teams. Committed to staying current with industry trends, Pranavi is dedicated to delivering high-quality software solutions that surpass customer expectations.
Testµ Conference is a virtual or online-only conference to define the future of testing. Join over 30,000+ software testers, developers, quality assurance experts, industry experts, and thought leaders for 3 days of learning, testing, and networking at Testμ Conference 2024 by LambdaTest.