Azure Data Engineer

JD#1

Data Engineer with Azure Databricks (R2DS - JV Project)

Mandatory Skills

Data Warehousing, Azure Data Engineer, Azure Data Lake, Data Bricks, Pyspark, Python

Job Description

- Proficiency in Databricks and experience with Spark for big data processing. - Solid programming skills in Python or Scala for developing data pipelines. - Knowledge of SQL and experience with relational databases and data warehousing concepts.

- Familiarity with cloud platforms - Azure, particularly in relation to data storage and processing.

- Understanding of data modeling, ETL processes, and data integration methodologies. - Excellent problem-solving skills and attention to detail.

- Strong communication and collaboration abilities to work effectively in a team-oriented environment.

Skill Area Tools & Technologies

Data

Engineering

Databricks, Apache Spark, Delta Lake

Programming Python, SQL, PySpark

Cloud Platforms Azure

Education Qualificaiton

B Tech

Roles & Responsibilities

- Develop and implement scalable data processing solutions using Azure Databricks, including ETL processes that efficiently handle large data volumes.

- Collaborate with cross-functional teams to understand data requirements and deliver solutions that meet business needs and objectives.

- Optimize data storage and retrieval processes to enhance performance and reduce latency. - Ensure data quality and integrity by implementing best practices for data governance and

validation.

- Stay updated with the latest trends and technologies in data engineering, and evaluate their applicability to the organization's needs.