Match Score
Against your primary resumeLogin to See Your Match Score
Create a free account or log in to unlock your CV match score across:
Protocol Intelligence
Data-driven signals on your job's competitivenessMid-level ML/LLM roles with cloud and LLM skills attract moderate competition, balanced by niche vector/LLM expertise.
Role requires specialized LLM, vector search, and cloud production experience, limiting cross-industry transferability.
Many mandatory technical stacks (Airflow, AWS, MLflow, pgvector, Databricks) increase screening rigor and resume filters.
Job Description
Structured overview of role & requirementsAbout This Role
Lead integration and optimization of AI/LLM inference pipelines for NLP tasks such as document classification, entity extraction, and de-identification within AWS EKS using Airflow.
Design, develop, and maintain large-scale multilingual document processing and data pipelines handling 300+ GB data, with batch orchestration across AWS services including S3, Athena, Glue, Fargate, SQS, and Step Functions.
Deploy, version, and monitor AI/ML models using MLflow and Databricks, manage schema evolution on Aurora PostgreSQL, and ensure production pipeline quality and scalability to 500K+ claims and hundreds of millions of text chunks.
Minimum Requirements
Proficiency in Python and SQL with experience in deploying AI/NLP solutions such as classification, NER, PII masking, de-identification, hybrid search, and LLM API integrations.
Hands-on experience with Apache Airflow for orchestrating batch data pipelines and familiarity with AWS cloud services including S3, Athena, Glue, Fargate, EKS, SQS, and Step Functions.
Experience with large-scale document processing of multilingual unstructured data and managing databases with PostgreSQL/Aurora, pgvector, GIN indexes, and full-text search.
Bachelor's degree in Computer Science, IT, Data Science, AI, Statistics, Mathematics, or related field. Work Experience Required: Not explicitly mentioned in the JD.
Ideal Candidate Profile
Strong background in operationalizing complex AI and NLP pipelines in a cloud environment with AWS and Kubernetes (EKS).
Skilled at handling large-scale, multilingual unstructured data and building scalable document processing systems with complex JSON and embedded documents.
Experienced in full ML lifecycle management including model deployment, monitoring, and schema migrations with emphasis on production readiness and data integrity.