Big Data Engineer

We are seeking a Senior Big Data Engineer with deep expertise in on premises Big Data platforms, specifically the Cloudera ecosystem, and a strong background in the telecom domain. The ideal candidate will have 12 to 15 years of experience building and managing large scale data lakes, developing batch and real time pipelines, and working with distributed data systems in telecom environments.
While the primary focus is on on-premises Cloudera based data lakes, familiarity with cloud data services is considered an added advantage. Design, build, and maintain on premises data pipelines using Apache Spark, Hive, and Python on Cloudera.
Develop real-time data ingestion workflows using Kafka, tailored for telecom datasets
Manage job orchestration and scheduling using Oozie, with data access enabled via Hue and secured through Ranger.
Implement and manage data security policies to ensure compliance and controlled access.
Develop and expose REST APIs for downstream integration and data access.
Ensure performance tuning, resource optimization, and high availability of the Cloudera platform.
Collaborate with data architects, engineers, and business stakeholders to deliver end to end solutions in a telecom context.
Support data migration or integration efforts with cloud platforms where applicable is added advantage. 12 to 15 years of experience in Big Data engineering, with hands on focus on on premises data lake environments.
Extensive Telecom domain knowledge including data models and pipelines related to CDRs, BSS OSS, customer and network data.
Strong practical experience with
Cloudera components Spark, Hive, HDFS, HBase, Impala
Kafka: configuration, topic management, producer consumer setup o Python for data transformations and automation
Job orchestration via Oozie
Access control and metadata management using Ranger
Proficient in performance tuning, resource management, and security hardening of Big Data platforms.
Experience with API development and integration for data services. Strong problem solving and troubleshooting skills in distributed data systems.
Ability to work independently in a fast paced project environment.
Effective communicator with both technical and business teams.
Experience in mentoring junior engineers or leading small technical teams is a plus.