Match Score
Against your primary resumeLogin to See Your Match Score
Create a free account or log in to unlock your CV match score across:
Protocol Intelligence
Data-driven signals on your job's competitivenessLog in to see why each signal reads the way it does.
Job Description
Structured overview of role & requirementsAbout This Role
Lead design and execution of comprehensive test strategies for AI/ML solutions focusing on accuracy, bias, robustness, and regression.
Develop and implement rubric-based evaluation frameworks and automated validation suites for GenAI systems, integrating with CI/CD pipelines.
Partner cross-functionally to ensure AI data quality, security, compliance, and continuous iteration of evaluation methodologies.
Minimum Requirements
Proficiency in programming languages such as Python or TypeScript used in AI/test automation.
Experience with rubric-based evaluation and LLM evaluation frameworks or benchmarking tools (e.g., LangSmith, Confident AI).
Knowledge of GenAI solutions including Retrieval Augmented Generation and cloud AI platforms (Azure OpenAI, Vertex AI, ChatGPT Enterprise).
Work Experience Required: Not explicitly mentioned in the JD.
Ideal Candidate Profile
Experienced in designing and operationalizing AI/ML testing strategies with a focus on GenAI and large language models.
Comfortable collaborating across IT, engineering, compliance, and risk teams to enforce AI governance and ethical standards.
Detail-oriented analytical thinker capable of detecting subtle AI model issues like hallucinations and documenting findings precisely.