Senior ML Engineer - Inference and Model Serving
YAL.ai (company site, opens in a new tab)HyderabadOn-siteDirect career pageEst. 30–45 LPA
Posted 1d ago
Reach people at YAL.ai
People at this company you can reach
Log in to see who works here
Create a free account or log in to see the people at YAL.ai you could write to:
- Recruiters & HR
- People in this function
- Work addresses
Your match
Against your primary resume
Log in to see your match score
Create a free account or log in to unlock your CV match score across:
- Role fit
- Experience
- Industry relevance
Protocol Intelligence
How competitive this role is
Competition
Background fit
Strictness
Log in to see why each signal reads the way it does.
Job description
Structured overview of role & requirements
About This Role
- Own production deployment of language and speech models ensuring fast, reliable inference within tight latency budgets.
- Optimize GPU usage for cost-efficient model serving across cloud, on-premises, and restricted-network environments including automated CI/CD pipelines for model deployment.
- Build and maintain secure, multi-tenant metering, access control, autoscaling, and failover mechanisms to ensure reliability and cost tracking per request and tenant.
Minimum Requirements
- Strong Python and working C++ skills.
- Hands-on experience serving language models in production with vLLM, TensorRT-LLM, SGLang, or Triton Inference Server.
- Experience serving or optimizing streaming speech models under real-time latency constraints.
- Bachelor’s degree in CS, Computer or Electrical Engineering, or related field, or equivalent practical experience.
Ideal Candidate Profile
- Deep expertise in GPU architecture, performance profiling (e.g. Nsight Systems), quantization, batching, and parallelism for scalable inference.
- Experience managing production-grade GPU workloads on Kubernetes with CI/CD, monitoring, autoscaling, and multi-tenant security design.
- Proven ability to measure and balance trade-offs between latency, throughput, cost, and quality under strict SLAs in real-time conversational systems.