Match Score
Against your primary resumeLogin to See Your Match Score
Create a free account or log in to unlock your CV match score across:
Protocol Intelligence
Data-driven signals on your job's competitivenessLog in to see why each signal reads the way it does.
Job Description
Structured overview of role & requirementsAbout This Role
Design and build scalable backend services and ML pipelines in Python to power AI agent evaluation, observability, and quality measurement at Intuit scale.
Own and evolve core evaluation platform components including golden datasets, LLM-as-judge pipelines, scoring mechanisms, and regression detection across agent releases.
Drive end-to-end project ownership from design through production, collaborating with product teams and mentoring engineers on platform architecture and AI evaluation concepts.
Minimum Requirements
Bachelor’s degree in Computer Science or equivalent practical experience.
10+ years experience building and operating distributed backend systems in production.
Strong proficiency in Python and experience with ML pipeline orchestration tools like Kubeflow Pipelines or Argo Workflows, deployed on Kubernetes.
Hands-on experience with LLM/agent systems (e.g., prompt engineering, LLM-as-judge evaluation) and observability/tracing systems (e.g., Langfuse, OpenTelemetry).
Ideal Candidate Profile
Experienced in delivering backend infrastructure for AI/ML products at scale with deep expertise in distributed systems and ML pipelines.
Comfortable working cross-functionally with data scientists and product managers to define quality metrics and observability for AI agents.
Able to thrive in a fast-moving, ambiguous AI landscape with minimal supervision and a strong focus on shipping high-quality production code.