Data Engineer

4️⃣ Technical Lead (Databricks Engineering)

Exp: 6 - 9 years

Role Overview

Lead development of Databricks pipelines replacing SQL Server cleansing processes.

Key Responsibilities

• Lead development of API/FTP ingestion pipelines
• Oversee Spark-based cleansing & enrichment implementation
• Review code and enforce standards
• Optimize Delta tables & Spark jobs
• Guide migration from SQL logic to PySpark logic
• Implement monitoring & logging
• Troubleshoot production pipeline issues

Required Skills

• Strong Databricks hands-on experience
• Strong PySpark & Spark SQL
• Experience converting SQL procedures into Spark logic
• Experience with API ingestion
• Strong performance tuning expertise
• Healthcare data experience preferred