Match Score
Against your primary resumeLogin to See Your Match Score
Create a free account or log in to unlock your CV match score across:
Protocol Intelligence
Data-driven signals on your job's competitivenessTier-1 brand and metro location increase competition, balanced by seniority and niche platform+LLM requirements.
Requires niche platform and LLM production expertise, limiting transferability across industries.
Explicit 10+ years plus mandatory Python, Kubernetes, LLM production and platform operations make filters strict.
Job Description
Structured overview of role & requirementsAbout This Role
Lead deployment, operations, and security of a self-hosted AI evaluation platform (LangSmith) including Kubernetes infrastructure and data management.
Develop evaluation tooling such as eval harnesses, scoring pipelines, golden datasets, and integrate with CI for AI model validation.
Own AI production observability covering tracing, metrics, alerting, and reliability with responsibility for operational standards and on-call practices.
Minimum Requirements
10+ years production software experience, notably on platform or infrastructure teams.
Strong Python proficiency required due to Python-native ecosystem for LLM evaluation and observability.
Hands-on technical leadership experience managing small teams while actively coding.
Experience with Kubernetes, cloud infrastructure, self-hosted platform operations, and established on-call operational practices.
Ideal Candidate Profile
Proven track record building internal platforms with successful adoption across engineering teams.
Experience working with LLM production challenges such as prompt/model versioning, tracing, retrieval behavior, and regression analysis.
Capable of cross-functional influence working alongside product leadership and distributed teams without direct authority.