Match Score
Against your primary resumeLogin to See Your Match Score
Create a free account or log in to unlock your CV match score across:
Protocol Intelligence
Data-driven signals on your job's competitivenessLog in to see why each signal reads the way it does.
Job Description
Structured overview of role & requirementsAbout This Role
Lead and define the test strategy for AI/ML features, including model output validation, regression testing, and edge case discovery.
Develop and automate evaluation frameworks for AI outputs focusing on accuracy, bias, hallucination detection, and safety compliance, integrating these into CI/CD pipelines.
Lead and mentor a QA team, establish AI testing standards, and adapt test coverage based on production performance and real-world AI failures.
Minimum Requirements
8+ years of QA/test engineering experience with at least 3 years focused on AI, ML, or data-driven application testing.
3+ years experience with AI/LLM evaluation frameworks including prompt regression and detection of hallucinations and bias.
5+ years of Python experience for test automation and evaluation scripting; proficiency with UI automation frameworks (Selenium, Playwright, or Cypress).
Strong SQL and data validation skills; experience testing REST/GraphQL APIs; knowledge of InfoSec testing for AI applications and compliance frameworks such as SOC 2, ISO 27001, NIST AI RMF, GDPR.
Ideal Candidate Profile
Experienced in leading AI feature quality assurance with a deep understanding of AI/ML model evaluation and testing methodologies.
Proficient in bridging AI product requirements to comprehensive and automatable test scenarios with strong scripting and data validation skills.
Skilled in guiding QA teams on AI testing standards and evolving test strategies in response to real-world AI application behavior and compliance demands.