Copy of Lead Databricks Engineer (Senior Software Engineer II)
NielsenMatch Score
Against your primary resumeLogin to See Your Match Score
Create a free account or log in to unlock your CV match score across:
Protocol Intelligence
Data-driven signals on your job's competitivenessNiche Databricks senior role but metro location and recognizable brand produce medium competition.
Highly platform-specific Databricks, Unity Catalog, and governance expertise limits cross-industry transferability.
Explicit 8+ years, 3+ years Databricks experience and specific tech stack make shortlisting highly strict.
Job Description
Structured overview of role & requirementsAbout This Role
Own end-to-end architecture and operational maturity of Nielsen's Databricks Lakehouse platform, including workspace design, cluster policies, compute strategy, and multi-cloud deployments.
Define and enforce data governance via Unity Catalog, including access controls, schemas, lineage and audit logging to maintain a secure and scalable platform.
Lead platform reliability efforts with automation (Infrastructure-as-Code, CI/CD), cost governance, and enable governed AI use cases embedded in the data platform.
Minimum Requirements
8–10+ years of experience in Senior or Lead Data Engineering/Data Platform roles with direct ownership of production Databricks environments.
Hands-on expertise in Databricks platform components: Unity Catalog, Delta Lake, Delta Live Tables, Databricks Workflows, Databricks SQL, and cluster/compute policy design.
Proficiency in Python, SQL, Apache Spark (batch and streaming), and infrastructure-as-code tools like Terraform; knowledge of data security and governance within governed lakehouses.
Work Experience Required: 8–10+ years; Education/Certification: Not explicitly mentioned as mandatory; Preferred certifications include Databricks Certified Data Engineer Professional or Platform Architect.
Ideal Candidate Profile
Experienced leader with a strategic focus on scalable, governed, multi-cloud Databricks Lakehouse platform design and operation.
Technical owner comfortable bridging platform engineering and data governance, with clear communication skills to engage both technical and non-technical stakeholders.
Proven ability to embed AI-native capabilities and AI-assisted engineering practices within large-scale data platforms to drive platform innovation and efficiency.