Research Engineer, AI Evaluations
Karya (company site, opens in a new tab)4+ yrsBengaluruOn-siteDirect career pageEst. 12–20 LPA
Posted 3h ago
Reach people at Karya
People at this company you can reach
Log in to see who works here
Create a free account or log in to see the people at Karya you could write to:
- Recruiters & HR
- People in this function
- Work addresses
Your match
Against your primary resume
Log in to see your match score
Create a free account or log in to unlock your CV match score across:
- Role fit
- Experience
- Industry relevance
Protocol Intelligence
How competitive this role is
Competition
Background fit
Strictness
Log in to see why each signal reads the way it does.
Job description
Structured overview of role & requirements
About This Role
- Own end-to-end design, build, and execution of AI model evaluations including benchmark design, model deployment, statistical analysis, and evaluation engineering.
- Develop and maintain modular evaluation pipelines and leaderboard infrastructure for inference, scoring, and reporting at scale with reproducible results.
- Prototype new evaluation methods and produce publications, technical reports, and public releases based on results.
Minimum Requirements
- 4+ years of experience in ML engineering or applied ML research with production-quality system delivery.
- Strong software engineering skills, especially in Python, data pipelines, and ML pipelines.
- Master's degree in Computer Science, Machine Learning, NLP or related field preferred; equivalent industry/research experience accepted.
- Work Experience Required: 4+ years
Ideal Candidate Profile
- Experience building AI benchmarks or evaluation systems, especially with human judgment data.
- Background in speech or multilingual NLP including Indian languages is a strong plus.
- Comfortable working across interdisciplinary teams handling ambiguous problems and messy data with rigorous statistical and engineering practices.