Copy of Lead Databricks Engineer (Senior Software Engineer II)
Nielsen (company site, opens in a new tab)8+ yrsBengaluruHybridDirect career pageEst. 30–50 LPA
Posted on 25 Sept
Reach people at Nielsen
People at this company you can reach
Log in to see who works here
Create a free account or log in to see the people at Nielsen you could write to:
- Recruiters & HR
- People in this function
- Work addresses
Your match
Against your primary resume
Log in to see your match score
Create a free account or log in to unlock your CV match score across:
- Role fit
- Experience
- Industry relevance
Protocol Intelligence
How competitive this role is
Competition
Background fit
Strictness
Log in to see why each signal reads the way it does.
Job description
Structured overview of role & requirements
About This Role
- Own end-to-end architecture and operational maturity of Nielsen's Databricks Lakehouse platform, including workspace design, cluster policies, compute strategy, and multi-cloud deployments.
- Define and enforce data governance via Unity Catalog, including access controls, schemas, lineage and audit logging to maintain a secure and scalable platform.
- Lead platform reliability efforts with automation (Infrastructure-as-Code, CI/CD), cost governance, and enable governed AI use cases embedded in the data platform.
Minimum Requirements
- 8–10+ years of experience in Senior or Lead Data Engineering/Data Platform roles with direct ownership of production Databricks environments.
- Hands-on expertise in Databricks platform components: Unity Catalog, Delta Lake, Delta Live Tables, Databricks Workflows, Databricks SQL, and cluster/compute policy design.
- Proficiency in Python, SQL, Apache Spark (batch and streaming), and infrastructure-as-code tools like Terraform; knowledge of data security and governance within governed lakehouses.
- Work Experience Required: 8–10+ years; Education/Certification: Not explicitly mentioned as mandatory; Preferred certifications include Databricks Certified Data Engineer Professional or Platform Architect.
Ideal Candidate Profile
- Experienced leader with a strategic focus on scalable, governed, multi-cloud Databricks Lakehouse platform design and operation.
- Technical owner comfortable bridging platform engineering and data governance, with clear communication skills to engage both technical and non-technical stakeholders.
- Proven ability to embed AI-native capabilities and AI-assisted engineering practices within large-scale data platforms to drive platform innovation and efficiency.