Python - PySpark Tech Lead

Job Description:

We are hiring Python - PySpark Sr Developers having at least 6+ years of experience in:

• Develop and maintain scalable data pipelines using Python and Pyspark.
• Proven experience as a Python developer with expertise in PySpark.
• Strong understanding of big data technologies and frameworks.
• Experience with distributed computing and parallel processing.
• Proficiency in SQL and experience with database systems.
• Collaborate with data engineers and data scientists to understand and fulfil data processing needs.
• Optimize and troubleshoot existing PySpark applications for performance improvements.
• Write clean, efficient, and well-documented code following best practices.
• Participate in design and code reviews.
• Develop and implement ETL processes to extract, transform, and load data.
• Ensure data integrity and quality throughout the data lifecycle.
• Stay current with the latest industry trends and technologies in big data and cloud computing.