AI QA Engineer (GenAI & ML Systems)

Own the quality, reliability, and safety of GenAI and ML-driven engineering applications across Powertrain Engineering.

Design evaluation frameworks and automated evaluation pipelines for LLM, RAG, and agentic systems covering accuracy, hallucination, consistency, latency, and cost.

Define quality gates and release criteria; monitor production drift and lead root-cause analysis for AI features.

4-8 years of QA/SDET experience including 1-2+ years testing ML or GenAI systems.

Strong Python skills and test automation expertise (pytest, API testing).

Hands-on experience with LLM evaluation tools such as Ragas, DeepEval, promptfoo, or LangSmith.

Working knowledge of LLM behavior, RAG architectures, embeddings, prompt engineering, CI/CD (GitHub Actions or Azure DevOps), Git, and cloud fundamentals (preferably Azure).

Experience in designing tests for non-deterministic AI systems with a focus on statistical thinking and metric design.

Comfortable with adversarial testing including prompt injection, jailbreaks, data leakage, bias, and toxicity assessment.

Able to collaborate closely with AI engineers, communicate quality metrics clearly, and manage quality risks in complex engineering environments.