Collaborate with business stakeholders to identify and define use casesDesign develop and execute automated tests using Python to ensure the functionality and performance of the product Conduct manual testing where necessary to complement automated tests Write complex SQL queries for test data generation and validation Test and validate the product developed by the developers ensuring it meets the specified requirements Stay informed and proficient in GenAI evaluation frameworks applying them effectively to assess the product Perform rigorous testing of machine learning ML and deep learning DL components Knowledge of Synthetic data generation techniques Demonstrate in depth knowledge of evaluation based frameworks such as RAGAS Retrieval Augmented Generation for Automated Systems and a basic understanding of how RAG Retrieval Augmented Generation works Define and implement testing strategies Exhibit excellent communication and documentation skills to clearly report issues document test cases and articulate the results to technical and non technical stakeholders Apply analytical skills to troubleshoot and resolve complex problems ensuring the highest quality standards are metSkills and Qualifications Proven experience in testing AI ML based solutions particularly generative AI GenAI Proficiency in automated testing with Python Familiarity with manual testing processes Strong knowledge of SQL for data manipulation and testing purposes Demonstrated experience with GenAI evaluation frameworks and testing of ML DL solutions Expertise in prompt engineering for LLMs including question generation and evaluation metrics Comprehensive knowledge of RAGAS and RAG frameworks Experience with evaluating chatbot use cases and ensuring their effectiveness and relevance Exceptional communication and documentation abilities Strong analytical and problem solving skillsEducation and Experience Bachelors degree or above
