Data Engineer

Key Responsibilities

The following are the key responsibilities of the position. It is expected

that most, if not all of these, are met by the candidate:

Design, build, and optimize data pipelines using Azure Data Factory and

Databricks (pyspark/spark SQL).

Develop robust ETL/ELT processes to ingest, transform, and validate large

volumes of operational, telemetry, incident, and transactional data.

Implement scalable workflows leveraging Azure services such as Data Lake

Storage, SQL Databases, Key Vault, Logic Apps, and Functions.

Develop clean, maintainable, and well-documented Python code for data

processing, automation, and model-serving pipelines.

Build efficient SQL queries and stored procedures across Oracle and SQL

Server to support the ODS, EDW, and analytics layer.

Contribute to data engineering standards, best practices, reusable

templates, and version control via Git.

Required Skills and Qualifications

Bachelor’s/master’s degree in Comp Science, Data Analytics, Engineering,

Mathematics, or related field.

5+ years of experience in data engineering roles.

Strong expertise with Azure Data Factory (ADF) pipelines, triggers, mapping

data flows, and orchestrations.

Advanced experience with Azure Databricks (pyspark, spark SQL, Delta Lake,

notebooks, clusters).

High proficiency in Python for ETL, automation, and data transformation.

Strong SQL skills across Oracle and SQL Server, including query tuning and

complex transformations.

Solid understanding of data modelling, warehousing, star/snowflake schema,

and distributed data processing.

Experience handling large-scale datasets and complex data domains.

Familiarity with CI/CD practices, Git branching, and DevOps pipelines.

Experience with data governance, data cataloging, and managing

structured/unstructured data in cloud environments.