Senior Software Engineer - Python, Data Engineering, AI
ZenotiMatch Score
Against your primary resumeLogin to See Your Match Score
Create a free account or log in to unlock your CV match score across:
Protocol Intelligence
Data-driven signals on your job's competitivenessLog in to see why each signal reads the way it does.
Job Description
Structured overview of role & requirementsAbout This Role
Design, build, and operate scalable batch and incremental data pipelines using Python and Spark on AWS/Azure cloud environments.
Lead data pipeline health management including data-quality framework extension, anomaly detection, alerting, and on-call responsibilities.
Collaborate with cross-functional teams to define metrics and expose data for BI and AI consumption, as well as mentor junior engineers and enforce best practices.
Minimum Requirements
5-7 years of hands-on data engineering experience building and operating production data pipelines (ETL/ELT) beyond BI or low-code tools.
Strong Python 3.x skills including pandas/pyarrow, modular/testable code development, not just notebooks.
Proficiency with Apache Spark/PySpark, advanced SQL on distributed engines, and lakehouse technologies (Parquet, Apache Iceberg or Delta Lake).
Experience with cloud data platforms on AWS or Azure covering object storage, serverless/managed compute, data catalogs, and basic IAM/RBAC.
Ideal Candidate Profile
Engineer with deep expertise in large-scale data pipeline architecture and cloud-native data engineering (AWS or Azure).
Experienced in managing data quality, observability, and reliability in operational data environments with alerting and troubleshooting.
Capable of engaging with product and analytics teams for metric definitions and data consumption in BI/AI, and comfortable leading engineering workstreams and mentoring peers.