Microsoft Fabric Architect

Job Summary:

We are looking for a highly skilled and experienced Microsoft Fabric Engineer / Architect to design, build, and maintain our enterprise-grade data and machine learning solutions. The ideal candidate will possess a strong background in data engineering, with extensive hands-on experience across the Microsoft Fabric ecosystem. A deep understanding of MLOps principles and experience with Azure Databricks are highly preferred. This role will be instrumental in enabling our data scientists and analysts to efficiently develop, deploy, and manage data assets and machine learning models at scale, within a robust and governed environment.

Responsibilities:

• Design, implement, and optimize scalable and secure data warehousing, Lakehouse, and analytics solutions within Microsoft Fabric (e.g., Lakehouse, Data Warehouse, Pipelines, Dataflows Gen2, Notebooks).
• Develop and manage data ingestion, transformation, and storage solutions for analytical and machine learning workloads, leveraging various Fabric components and Azure data services (e.g., Azure Data Lake Storage Gen2, Azure SQL Database).
• Collaborate closely with data scientists, machine learning engineers, and business stakeholders to understand data and analytical requirements and translate them into efficient technical designs within Fabric.
• Implement and maintain CI/CD pipelines for data solutions and, where applicable, machine learning models, ensuring automated testing, deployment, and monitoring within Azure DevOps or similar tools.
• Optimize data processing and query performance within Fabric, utilizing Spark, SQL, and KQL where appropriate.
• Implement data governance, security, and compliance standards (e.g., Microsoft Purview) across the Fabric environment, ensuring data quality, lineage, and access control.
• Monitor the performance and health of deployed data solutions and underlying Fabric infrastructure, implementing robust logging and alerting.
• Troubleshoot and resolve complex technical issues related to Microsoft Fabric components and data pipelines.
• Stay current with the latest Microsoft Fabric features, Azure updates, and MLOps best practices, advocating for and implementing new technologies to enhance our data capabilities.
• If applicable, design and implement data pipelines and machine learning workflows utilizing Azure Databricks, optimizing for performance and cost.
• Provide technical guidance and mentorship to junior engineers and data analysts.

Required Qualifications:

• Bachelor's or Master's degree in Computer Science, Engineering, Data Science, or a related quantitative field.
• 5-10 years of hands-on experience in Data Engineering, with a strong focus on building and maintaining large-scale data pipelines and data platforms.
• Extensive hands-on experience with Microsoft Fabric, including a deep understanding and practical application of its various components (e.g., Lakehouse, Data Warehouse, Data Pipelines, Notebooks, Dataflows Gen2).
• Proficiency in SQL, Python, and/or PySpark for data manipulation, scripting, and development within Fabric notebooks.
• Strong understanding of cloud architecture principles and best practices on Microsoft Azure.
• Experience with Infrastructure as Code (IaC) tools such as Azure Resource Manager (ARM) templates or Terraform.
• Solid understanding of data warehousing concepts, dimensional modeling, and ETL/ELT processes.
• Excellent problem-solving skills and the ability to troubleshoot complex technical issues.
• Strong communication and collaboration skills, with the ability to work effectively with cross-functional teams.

Preferred Qualifications:

• Hands-on experience with Azure Databricks, including Spark optimization, Delta Lake, and Databricks notebooks for advanced data engineering and machine learning workloads.
• Experience with Power BI for data visualization and reporting, and connecting to Fabric data sources.
• Familiarity with Azure DevOps for source control, CI/CD, and project management.
• Experience with other Azure data services such as Azure Data Factory, Azure Synapse Analytics, or Azure Stream Analytics.
• Microsoft Certifications relevant to data and analytics (e.g., Microsoft Certified: Fabric Analytics Engineer Associate, Azure Data Engineer Associate, Azure Solutions Architect Expert).