Back
Known MNC brand and Bangalore metro increase applicant density, but seniority and niche LLM QA skills moderate competition.
Core QA automation skills are transferable, but GenAI/LLM evaluation expertise increases domain specificity.
Explicit 8+ years and mandatory Python, Pytest, Playwright, JMeter, and GitHub Actions create high filtering.
Login to See Your Match Score
Create a free account or log in to unlock your CV match score across:
Develop and own evaluation framework for GenAI solutions focusing on Faithfulness, Relevancy, and Hallucination detection using LLM-as-a-judge frameworks.
Architect and lead implementation of a dual-layered automation suite including deterministic E2E UI and API testing, and probabilistic automated evaluation of non-deterministic LLM outputs.
Embed automated quality checks within GitHub Workflows for CI/CD and lead JMeter-based performance testing.
8+ years of experience in Software QA.
Bachelor’s or Master’s degree in Computer Science, Software Engineering, or related field.
Expert-level Python skills for building custom test tooling and automation scripts.
Hands-on experience with Pytest (API testing), Playwright (E2E UI testing), JMeter (performance testing), and advanced experience designing and maintaining GitHub Actions/Workflows for automated test execution.
Strong expertise in AI and LLM evaluation concepts with ability to rapidly learn and apply metrics like Faithfulness, Relevancy, and Groundedness.
Familiarity with LLM operations including prompting and context windows and experience implementing LLM-as-a-Judge strategies.
Data-driven mindset toward quality monitoring beyond binary pass/fail, focusing on probabilistic quality metrics and tools like Langfuse for live trace analysis.