AI Quality Engineer, Safety and RAG - Assistant Vice President
CitiMatch Score
Against your primary resumeLogin to See Your Match Score
Create a free account or log in to unlock your CV match score across:
Protocol Intelligence
Data-driven signals on your job's competitivenessLog in to see why each signal reads the way it does.
Job Description
Structured overview of role & requirementsAbout This Role
Own the design and execution of comprehensive quality engineering strategies for Agentic AI and RAG pipelines, including testing agent reasoning, tool use, memory, and multi-agent orchestration.
Develop, maintain, and integrate automated test frameworks and evaluation pipelines for continuous validation of AI models focusing on safety, performance, security, and reliability at scale.
Lead AI safety and security testing efforts including red-teaming for vulnerabilities, output guardrails, bias and fairness assessments, and performance and reliability testing under load, collaborating with cybersecurity and MLOps teams.
Minimum Requirements
Bachelor's degree in Computer Science, Engineering, or related field; Master's degree is a plus.
8+ years of experience in software or AI/ML quality engineering.
3+ years of hands-on experience with RAG systems or Agentic AI quality engineering.
Proven expertise in building automated test frameworks for non-deterministic AI systems; strong background in performance testing and AI safety/security assessments.
Ideal Candidate Profile
Experienced in end-to-end AI quality engineering with deep knowledge of RAG architectures and agent design patterns like ReAct, Plan-and-Execute, Reflexion.
Skilled in automated test development integrated with CI/CD for highly complex autonomous AI systems operating at scale.
Familiarity with AI safety and alignment principles, adversarial testing, and performance/reliability engineering in production AI systems, ideally within regulated industries.