Data Engineer (Databricks)
Onit (company site, opens in a new tab)5+ yrsPuneHybridDirect career pageEst. 12–22 LPA
Posted 4h ago
Reach people at Onit
People at this company you can reach
Log in to see who works here
Create a free account or log in to see the people at Onit you could write to:
- Recruiters & HR
- People in this function
- Work addresses
Your match
Against your primary resume
Log in to see your match score
Create a free account or log in to unlock your CV match score across:
- Role fit
- Experience
- Industry relevance
Protocol Intelligence
How competitive this role is
Competition
Background fit
Strictness
Log in to see why each signal reads the way it does.
Job description
Structured overview of role & requirements
About This Role
- Design, build, and maintain scalable batch and incremental data pipelines using Databricks technologies (PySpark, Spark SQL, Delta Lake) following a medallion architecture.
- Implement and maintain canonical data models to ensure data consistency and build production entity resolution pipelines to reconcile duplicate/conflicting records.
- Develop and enforce automated data quality checks, monitor pipeline performance and reliability, optimize Spark cost and performance, and ensure governance and privacy compliance using Unity Catalog and multi-tenant data isolation.
Minimum Requirements
- 5+ years of experience in data engineering or analytics engineering building production data pipelines in cloud environments.
- Hands-on experience with Databricks including PySpark, Spark SQL, Delta Lake, Databricks Workflows/Jobs, and Unity Catalog.
- Strong skills in SQL and Python with solid data modeling experience, and experience with deduplication/entity resolution techniques at scale.
- Experience with cloud platforms (AWS preferred), Git version control, CI/CD for data pipelines, and implementing automated data quality checks and pipeline monitoring.
Ideal Candidate Profile
- Experienced in operating data pipelines at scale using Databricks in cloud environments, with practical knowledge of medallion/lakehouse architectures and performance optimization.
- Able to collaborate effectively with data scientists, data architects, and engineering teams to productionize models and expose curated data to products.
- Has background or interest in regulated or enterprise SaaS contexts, preferably legal technology sector, and familiarity with data governance, privacy, and anonymization techniques.