Back
Databricks data-engineer role in metro attracts many qualified applicants, balanced by seniority and non-remote shift requirement.
Core data engineering skills transfer across industries, but Databricks/Azure specialization raises domain specificity moderately.
Requires specific Databricks, Delta Lake, PySpark, Azure Data Factory, and lead-level technical depth making screening strict.
Login to See Your Match Score
Create a free account or log in to unlock your CV match score across:
Design, develop, and optimize end-to-end batch and streaming data pipelines using Azure Databricks and Spark.
Implement Medallion Architecture to structure data layers (raw, enriched, curated) and oversee data quality and governance via tools like Azure Purview and Unity Catalog.
Collaborate with analysts, architects, and business teams to deliver scalable, efficient data solutions and optimize Spark jobs, Delta Lake tables, and SQL queries for cost-effectiveness.
Hands-on experience with Azure data stack: Databricks, Data Factory, and Delta Lake.
Proficiency in Python, PySpark, SQL with strong query optimization skills.
Experience building scalable ETL/ELT pipelines and applying Lakehouse architecture (Medallion design patterns).
Rotational shift with 2 weeks night shift per quarter included; Work Experience Required: Not explicitly mentioned in the JD.
Experienced data engineer with deep expertise in Azure Databricks ecosystem and Lakehouse architecture.
Familiar with DevOps practices including Git, CI/CD pipelines, Agile methodologies, and data governance.
Comfortable working in environments with rotational night shifts and collaborating cross-functionally to deliver end-to-end data solutions.