Senior ML Engineer - MLOps and AI Evaluation
ELLKAY, LLC (company site, opens in a new tab)RemoteDirect career pageEst. 28–48 LPA
Posted 1d ago
Reach people at ELLKAY, LLC
People at this company you can reach
Log in to see who works here
Create a free account or log in to see the people at ELLKAY, LLC you could write to:
- Recruiters & HR
- People in this function
- Work addresses
Your match
Against your primary resume
Log in to see your match score
Create a free account or log in to unlock your CV match score across:
- Role fit
- Experience
- Industry relevance
Protocol Intelligence
How competitive this role is
Competition
Background fit
Strictness
Log in to see why each signal reads the way it does.
Job description
Structured overview of role & requirements
About This Role
- Own development and maintenance of ML operations and AI evaluation infrastructure including training/deployment pipelines, model and prompt versioning, and telemetry.
- Implement CI/CD evaluation gates and quality controls for multiple AI services (mapping, clinical extraction, recommendations, RAG, agents) with validation against SME labels.
- Optimize and monitor deployed AI models for drift, latency, failure rates, unit cost, supporting rollback, shadow testing, and cost-efficient inference.
Minimum Requirements
- Experience in building reproducible ML training and deployment pipelines with version control for models, prompts, and datasets.
- Knowledge of CI/CD processes applied to AI/ML model evaluation and release gates.
- Familiarity with monitoring ML lifecycle metrics including drift, quality, latency, failure rates, and unit cost.
- Work Experience Required: Not explicitly mentioned in the JD
Ideal Candidate Profile
- Experienced working with classical ML models, large language models (LLMs), and AI agent services in production environments.
- Ability to collaborate cross-functionally with security, data platform, clinical SMEs, QA, and product engineering teams.
- Strong operational mindset focused on reliability, reproducibility, compliance (e.g. PHI-safe telemetry), and cost optimization for AI lifecycles.