Match Score
Against your primary resumeLogin to See Your Match Score
Create a free account or log in to unlock your CV match score across:
Protocol Intelligence
Data-driven signals on your job's competitivenessMid-level, popular Data Engineer role with common Databricks/PySpark skills increases candidate competition.
Databricks-focused data engineering requires domain-specific tooling, moderately reducing cross-industry transferability.
Explicit 4+ years requirement and mandatory Databricks/PySpark/AWS skills make filtering highly strict.
Job Description
Structured overview of role & requirementsAbout This Role
Design, develop, and maintain scalable ETL/ELT pipelines on Databricks Lakehouse Platform using PySpark, SQL, and Python.
Integrate data from multiple sources including databases, Amazon S3, and REST APIs, applying business logic and dimensional modeling (Medallion Architecture).
Schedule, monitor, optimize Databricks Jobs and Workflows ensuring data quality, error handling, and Spark performance tuning.
Minimum Requirements
Strong expertise in Python, PySpark, advanced SQL, and hands-on experience with Databricks Lakehouse Platform.
Experience with Unity Catalog, Delta Lake, Databricks Workflows/Jobs, and Medallion Architecture for data pipelines.
Work Experience Required: Minimum 4 years in Data Engineering with at least 2 years using Databricks.
Experience with REST API integrations and knowledge of ETL/ELT development, batch processing, incremental loading, and data modeling (Star Schema, SCD).
Ideal Candidate Profile
Proven track record of delivering production-ready data pipelines in environments relying on Databricks and Spark performance optimization.
Experienced in handling large-scale structured and semi-structured data in cloud environments (AWS preferred).
Familiar with version control (Git), CI/CD best practices, and preferably with Databricks certification or experience using related tools like Auto Loader, Kafka, or Airflow.