Sr Data QA

6️⃣ Data Quality Automation Analyst (2) (Databricks-Centric)

Exp: 6 - 9 years

Role Overview

Design and automate data validation within Databricks pipelines to ensure accurate migration and ongoing data reliability.

Key Responsibilities

• Build automated data quality checks in PySpark
• Develop reconciliation framework between:
  • Legacy SQL outputs
  • New Databricks outputs
• Implement rule-based validation for healthcare datasets
• Automate profiling and anomaly detection
• Integrate quality checks into pipeline workflows
• Build DQ scorecards and dashboards
• Support cutover validation

Required Skills

• Strong SQL & PySpark
• Experience with Databricks validation frameworks
• Experience implementing automated DQ checks
• Strong reconciliation expertise
• Healthcare data validation experience preferred