Senior Software Engineer - AI Gateway & Agentic AI Solutions
EXLMatch Score
Against your primary resumeLogin to See Your Match Score
Create a free account or log in to unlock your CV match score across:
Protocol Intelligence
Data-driven signals on your job's competitivenessLog in to see why each signal reads the way it does.
Job Description
Structured overview of role & requirementsAbout This Role
Design and build an Azure-native LLM gateway in C# with features like intelligent routing, fallback/load balancing, rate limiting, token-quota enforcement, semantic caching, and centralized API key management.
Implement observability and governance layers including request/response tracing, logging, token and cost metering, and latency dashboards using Azure Monitor and related tooling.
Lead end-to-end technical delivery on client engagements including architecture proposals, hands-on builds, demos, production readiness, and packaging reusable engineering assets for redeployment.
Minimum Requirements
12-15+ years of deep C#/.NET production-grade API/service development experience including async patterns, dependency injection, and testing discipline.
Hands-on Azure platform expertise: Azure API Management with GenAI/LLM policies, Azure OpenAI/AI Foundry, Azure Monitor/Application Insights, Key Vault, and service hosting via Azure Functions/App Service or AKS.
Working knowledge of LLM gateway and observability platforms such as LiteLLM, Langfuse, Portkey, or Kong AI Gateway to translate their concepts for Azure/C# implementation.
Experience with Microsoft Agent Framework or demonstrated capability with Semantic Kernel and/or AutoGen, involving multi-agent orchestration and advanced agent communication protocols.
Ideal Candidate Profile
Senior engineer capable of independently architecting and delivering complex Azure-based AI gateway solutions without a large platform team.
Experienced client-facing solution engineer skilled in translating ambiguous business requirements into demoable AI agentic scenarios and defending design decisions live.
Strong practitioner of multi-agent orchestration and AI tooling integration, familiar with Microsoft Agent Framework or its predecessors, oriented towards reusable engineering asset creation.