Senior Data Engineer
Caterpillar Inc. (company site, opens in a new tab)BengaluruOn-siteDirect career pageEst. 22–40 LPA
Posted 7d ago
Reach people at Caterpillar Inc.
People at this company you can reach
Log in to see who works here
Create a free account or log in to see the people at Caterpillar Inc. you could write to:
- Recruiters & HR
- People in this function
- Work addresses
Your match
Against your primary resume
Log in to see your match score
Create a free account or log in to unlock your CV match score across:
- Role fit
- Experience
- Industry relevance
Protocol Intelligence
How competitive this role is
Competition
Background fit
Strictness
Log in to see why each signal reads the way it does.
Job description
Structured overview of role & requirements
About This Role
- Build and maintain scalable batch and near-real-time AWS pipelines and connectors feeding Coveo and downstream systems.
- Own pipeline data quality—completeness, freshness, consistency, accuracy, schema validation, and anomaly detection—and implement checks, alerts, and monitoring.
- Analyze indexed datasets and work with search and platform teams to improve data accuracy and search relevance; contribute to CI/CD, documentation, and enterprise security standards.
Minimum Requirements
- Work Experience Required: Experience as a Data Engineer working on AWS is required; required years are Not explicitly mentioned in the JD.
- Hands-on data pipeline experience with AWS services such as Glue, Bedrock, Lambda, Step Functions, S3, and SNS/SQS, or similar; proficiency in Python and/or PySpark.
- Strong understanding of data quality practices, ETL/ELT patterns, data modeling, and integrating data from APIs, databases, or files.
- Experience with pipeline logging, monitoring, and alerting; familiarity with CI/CD and Git.
Ideal Candidate Profile
- Has built production AWS ingestion pipelines and can make data quality and reliability measurable through validation rules, monitoring, and remediation.
- Coveo, search-platform, or content-indexing experience is a strong advantage for improving indexed data quality and search performance.
- Likely to fit an enterprise, multi-team environment involving data, search, and platform teams; Snowflake experience is an additional advantage.