Back
Login to See Your Match Score
Create a free account or log in to unlock your CV match score across:
Metro location and common Data Engineer title increase applicant density, while some niche skills moderate competition.
Core AWS, Spark, and Python skills are transferable, but finance reference-data context requires moderate domain knowledge.
Extensive mandatory AWS, Spark, Iceberg, Starburst, Kafka and CI/CD requirements imply high technical filtering.
Design, implement, and optimize scalable, secure, and cost-efficient AWS data architectures for Financial Master & Reference Data Management.
Develop, maintain, and unit test ETL pipelines and data applications using AWS Lambda, AWS Glue ETL, Apache Spark (PySpark), and integrate federated querying with Starburst/Trino.
Manage data lakes with Apache Iceberg on AWS, orchestrate event-driven workflows using AWS Step Functions and EventBridge, and implement CI/CD pipelines for automated deployment.
Hands-on experience with AWS services: Lambda, Glue ETL, Athena, S3, DynamoDB, Step Functions, EventBridge, SNS, and SQS.
Proficient in Apache Spark (PySpark) development including unit testing and performance tuning.
Experience with Apache Iceberg for data lake design and optimization, including table compaction.
Work Experience Required: Not explicitly mentioned in the JD
Experienced in building real-time and API-driven data applications and event-driven architectures on AWS.
Strong Python development skills with relevant libraries (pandas, requests, json, awswrangler) and unit testing (PyTest).
Familiarity with Apache Kafka and Confluent Kafka, along with performance optimization techniques for big data processing.