IN_Associate_Agentic AI Testing_Digital Integration_Advisory_Kolkata
PwC IndiaMatch Score
Against your primary resumeLogin to See Your Match Score
Create a free account or log in to unlock your CV match score across:
Protocol Intelligence
Data-driven signals on your job's competitivenessLog in to see why each signal reads the way it does.
Job Description
Structured overview of role & requirementsAbout This Role
Validate the entire agentic AI lifecycle including user intent, prompting, agent reasoning, routing, and MCP tool execution.
Utilize observability and automation tools to monitor, test, and enhance AI workflows and performance.
Develop and apply AI quality assessment methodologies including prompt testing strategies and AI metrics validation.
Minimum Requirements
4+ years of relevant experience in Agentic AI testing and validation.
Bachelor's degree in Computer Science, IT, or related field (Bachelor of Engineering preferred).
Hands-on expertise with MCP testing covering tool invocation, payload/schema validation, orchestration, error handling, retries, and API/integration testing.
Experience with AI observability and tracing tools (e.g., LangFuse, Phoenix) and API testing automation frameworks (e.g., Postman, Katalon, Playwright).
Ideal Candidate Profile
Experienced in designing test strategies and evaluation rubrics for AI systems, including multi-turn and ambiguous prompt scenarios.
Skilled in collaborating across teams to refine validation methodologies and AI quality assessment.
Familiar with AI quality metrics such as hallucination detection, accuracy, consistency, determinism, and latency for comprehensive AI system evaluation.