ETL Tester

Data Engineering QA Build automated validation for ETL ELT across batch and streaming implement schema contract testing versioning backward compatibility and data quality frameworks profiling anomaly detection completeness uniqueness

Databricks and Delta Lake Develop test suites for performance partitioning and ACID guarantees validate Unity Catalog permissions lineage and governance assure Feature Store integrity definitions freshness drift

ML Model Testing Design and execute tests for classification regression and time series models including champion challenger evaluations calibration checks counterfactual optimization output verification and drift stability monitoring

MLOps and MLflow Use MLflow for experiment tracking model versioning reproducibility and audit trails enforce reproducible pipelines artifacts and deployment gates

Python QA Engineering Build robust Python test harnesses pandas numpy scikit learn for data matrix checks statistical validations synthetic data generation implement unit integration and end to end tests with CICD