Senior Data Engineer

Acts as an Azure Data Factory (ADF) Developer on an agile development team that supports a variety of work types.
This role will be responsible for creating Data orchestration with Azure Data Factory Pipelines & Dataflows. Key role is to understand the business requirements and implement the requirements using Azure Data Factory.
Expert level knowledge on Azure Data Factory
Advance knowledge of Azure ADF, SQL DB, Blog Storage, Synapse Analytics, and other Azure Native Data Services.
Should be able to analyze and understand complex data
Knowledge of Azure data lake is required and Azure Services like Analysis Service, SQL Databases is a must.
Knowledge of master data management, data warehousing and business intelligence architecture.
Experience in data modeling and database design with excellent knowledge of SQL Server best practices.
Excellent interpersonal/communication skills (both oral/written) with the ability to communicate at various levels with clarity & precision.
Should have clear understanding of DW lifecycle and contribute in preparing Design documents, Unit Test plans, Code review reports.
Experience working in Agile environment (Scrum, Lean, Kanban) is a plus
Knowledge of Big data technologies - Spark Framework, NoSQL, Azure Data Bricks and Haqoop Ecosystem- Hive, Impala, HDFS, YARN, Pig, Oozie is a plus
Writes high quality code within prescribed iterations
Integrate data from different data sources: databases, csv files, xml files, queues
Integrate data for batch, near real-time, real-time latencies
Organize data at different aggregation levels, hierarchies
Write complex data transformation logic in ADF and SQL
Proactively monitor the performance of data jobs and tune the jobs as needed
Make the data integration jobs fault tolerant and scalable
Experience/Knowledge of agile methodology for entire development
Ability to work independently with good verbal & written communication skills
Performs unit testing and acts as a subject matter expert to QA and Business Testers through the testing lifecycle
Develops ADF Jobs while adhering to best practices.
Documents the data extracting/loading mapping processes; documents the data transforming processes; documents the data relationships and maintaining the consistency of the data.
Perform analysis, design, development, and configuration functions (includes defining technical requirements) with a high degree of accuracy and speed, operating as an individual contributor to team goals.
Work independently to tackle well-scoped and loosely scoped problems.
Seek opportunities to expand technical knowledge and capabilities.
Provide technical guidance and mentorship to less experienced employees.