Match Score
Against your primary resumeLogin to See Your Match Score
Create a free account or log in to unlock your CV match score across:
Protocol Intelligence
Data-driven signals on your job's competitivenessLog in to see why each signal reads the way it does.
Job Description
Structured overview of role & requirementsAbout This Role
Design, build, and own production-grade Python ETL/ELT pipelines and data platform components using Databricks, Delta Lake, and Spark.
Optimize and profile data pipelines for performance, migrating workloads to Databricks/Spark when Python scalability limits are reached.
Lead dimensional data modeling decisions and enforce Python engineering standards including code reviews and testing frameworks.
Minimum Requirements
8-12 years of data engineering experience, with at least 2 years in technical leadership or mentoring roles.
Proficient in Python (OOP, multiprocessing, asyncio, decorators, generators), Pytest, and production ETL/ELT pipeline development.
Strong hands-on experience with Databricks (Delta Lake, Structured Streaming, Auto Loader, Unity Catalog) and Spark performance tuning.
Bachelor’s or Master’s degree in Computer Science, Engineering or related field, or equivalent practical experience.
Ideal Candidate Profile
Experienced in scalable data engineering platforms with advanced dimensional modeling knowledge (star/snowflake schemas, SCD types, Data Vault, Kimball, Inmon).
Demonstrated ability to communicate complex technical trade-offs to both engineering and business stakeholders while leading design decisions.
Familiarity with orchestration tools like Databricks Workflows, Airflow, Azure Data Factory, and experience leading or mentoring teams of 3+ engineers.