LLMOps Engineer — LLM Inference & Model Optimization
Gnani.ai (Gnani Innovations Private Limited) (company site, opens in a new tab)3–6 yrsBengaluruOn-siteDirect career pageEst. 20–35 LPA
Posted on 8 Sept
Reach people at Gnani.ai (Gnani Innovations Private Limited)
People at this company you can reach
Log in to see who works here
Create a free account or log in to see the people at Gnani.ai (Gnani Innovations Private Limited) you could write to:
- Recruiters & HR
- People in this function
- Work addresses
Your match
Against your primary resume
Log in to see your match score
Create a free account or log in to unlock your CV match score across:
- Role fit
- Experience
- Industry relevance
Protocol Intelligence
How competitive this role is
Competition
Background fit
Strictness
Log in to see why each signal reads the way it does.
Job description
Structured overview of role & requirements
About This Role
- Own and optimize large language model serving stack on NVIDIA GPU clusters ensuring low-latency and high-quality voice AI inference.
- Drive performance engineering across serving engine, model optimization (quantization, distillation, pruning), and custom kernel development to reduce cost per million tokens.
- Manage production inference operations including Kubernetes orchestration, monitoring latency metrics, load testing, capacity planning, and incident response.
Minimum Requirements
- 3 to 6 years experience in ML infrastructure, model serving, or performance engineering; at least 2 years focused on LLM inference in production.
- Hands-on experience with at least two serving engines among vLLM, SGLang, TensorRT-LLM, NVIDIA Dynamo, including understanding of schedulers and KV cache designs.
- Demonstrated quantization expertise (FP8, INT8, INT4) on real models with calibration and accuracy impact measurement.
- Proficiency in Python, CUDA fundamentals, multi-GPU serving, Kubernetes/Docker in production, and observability tools like Prometheus/Grafana.
Ideal Candidate Profile
- Deep technical expertise with at least two domains among serving stack tuning, model quantization/distillation/pruning, and custom kernel engineering.
- Experienced in operating large-scale LLM inference platforms with real-time latency constraints and cost optimization focus relevant to voice AI.
- Familiar with complex distributed GPU environments and proficient in profiling and eliminating performance bottlenecks.