Senior Data Engineer

Resposibilities:

Develop efficient Data Lake frameworks using Scala, PySpark, or Python, with robust data quality and security controls.
Create reusable libraries and config-driven pipelines for enterprise-wide ingestion and integration. 
Ensure compliance with data governance and security standards using tools like AWS DataZone, Lake Formation, AWS Glue and MWAA
Communicate effectively with internal stakeholders and contribute to team knowledge sharing. 

Required Skills:
Demonstrable in Python, SQL, Spark, and Scala for distributed data processing.
Demonstrable experience with AWS Glue, Lambda, Redshift, Athena, Kinesis, and EMR. 
Exposure to AWS Marketplace, API Gateway, and data monetisation strategies.
Performance tuning of ETL jobs and query optimisation for cost efficiency. 
Design and develop next-generation data platforms and pipeline solutions.

Experience with CI/CD tools such as GitHub, GitHub Actions, ArgoCD, and automated deployment pipelines.
Experience with observability tools like Cloudwatch, Prometheus, or Grafana. 
Exposure to AI/ML technologies, with the ability to integrate or leverage AI models to enhance data processing, automation, or system intelligence
Experience using DevSecOps methodologies and best practice to automate deployments and testing. 
Build and maintain backend services and integrations using Java Spring Boot, ensuring performance, scalability, and security.
Collaborate with engineering peers to co-create and contribute to shared practices and frameworks. 
Apply modern coding standards to deliver high-quality, reliable solutions that address core business needs and reduce risk.
Strong experience in Java, Spring Boot, and RESTful API development.