Match Score
Against your primary resumeLogin to See Your Match Score
Create a free account or log in to unlock your CV match score across:
Protocol Intelligence
Data-driven signals on your job's competitivenessLog in to see why each signal reads the way it does.
Job Description
Structured overview of role & requirementsAbout This Role
Design, develop, and maintain scalable ETL/ELT data pipelines using Databricks Lakehouse platform with PySpark and SQL.
Integrate data from multiple sources including REST APIs, databases, and cloud storage (AWS S3) while managing Delta Lake tables using Medallion Architecture.
Optimize Spark workloads and Databricks Jobs/Workflows for performance, reliability, and scalability; ensure data quality through validations and monitoring.
Minimum Requirements
9+ years of experience in data engineering, including 3+ years hands-on with Databricks.
Strong expertise in Python, PySpark, Advanced SQL, and Databricks features including Unity Catalog, Delta Lake, and Medallion Architecture.
Experience with ETL/ELT pipeline development, REST API integrations, data modeling (Star/Snowflake schema), and Spark performance tuning.
Work Experience Required: 9+ years in Data Engineering, Prior Lead/Technical Lead experience preferred.
Ideal Candidate Profile
Experienced technical lead with proven ability to guide data engineering teams and drive architecture/technical decisions.
Deep knowledge of Databricks ecosystem and hands-on experience optimizing data pipelines in large-scale production environments.
Familiarity with modern data ingestion and workflow orchestration tools such as Auto Loader, Kafka, Airflow, or dbt is advantageous but not mandatory.