Back
Metro location, popular data-engineer title, broad tech stack, and a well-known employer increase candidate competition.
Core data engineering skills are transferable across industries but need specific Spark and AWS experience.
Explicit 8–12 years requirement plus mandatory Python/PySpark/Spark/AWS/Kafka tech stack enforces strict shortlisting.
Login to See Your Match Score
Create a free account or log in to unlock your CV match score across:
Design, develop, and maintain batch and streaming data pipelines using Python, PySpark, and Apache Spark.
Ensure data quality and implement ETL processes for migrating and deploying data across various enterprise systems.
Collaborate with cross-functional teams to deliver scalable data solutions that meet business requirements.
8 to 12 years of relevant work experience in Python Spark SQL and AWS data engineering.
Proficiency in Python, PySpark, Apache Spark, and SQL including advanced features like CTEs and window functions.
Hands-on experience with AWS services such as EMR, S3, Lambda, EC2, and Athena.
Bachelor’s degree in Computer Science, Information Technology, Engineering, or related discipline.
Strong understanding of Spark architecture, Spark DStreams, and Spark Structured Streaming indicating deep domain expertise.
Experience implementing and supporting both batch and streaming data pipelines operating at enterprise scale.
Familiarity with big data tools and frameworks like Kafka, data warehousing, data lakes, and Splunk as part of an advanced data engineering environment.