Design and execute test plans for ETL processes, ensuring data accuracy, completeness, and integrity.Develop automated test scripts using Python or R for data validation and reconciliation.Perform source-to-target data verification, transformation logic testing, and regression testing.Collaborate with data engineers and analysts to understand business requirements and data flows.Identify data anomalies and work with development teams to resolve issues.Maintain test documentation, including test cases, test results, and defect logs.Participate in performance testing and optimization of data pipelines. Required Skills & Qualifications:Strong experience in ETL testing across various data sources and targets.Proficiency in Python or R for scripting and automation.Solid understanding of SQL and relational databases.Familiarity with data warehousing concepts and tools (e.g., Power BI, QlikView, Informatica, Talend, SSIS).Experience with test management tools (e.g., JIRA, TestRail).Knowledge of data profiling, data quality frameworks, and validation techniques.Excellent analytical and communication skills.
We are seeking a highly skilled and motivated Big Data Engineer to join our data engineering team. The ideal candidate will have hands-on experience with Hadoop ecosystem, Apache Spark, and programming expertise in Python (PySpark), Scala, and Java. You will be responsible for designing, developing, and optimizing scalable data pipelines and big data solutions to support analytics and business intelligence initiatives.
