Databricks Data Engineer - Assistant Manager
State Street Corporation (company site, opens in a new tab)BengaluruOn-siteDirect career pageEst. 16–30 LPA
Posted 10h ago
Reach people at State Street Corporation
People at this company you can reach
Log in to see who works here
Create a free account or log in to see the people at State Street Corporation you could write to:
- Recruiters & HR
- People in this function
- Work addresses
Your match
Against your primary resume
Log in to see your match score
Create a free account or log in to unlock your CV match score across:
- Role fit
- Experience
- Industry relevance
Protocol Intelligence
How competitive this role is
Competition
Background fit
Strictness
Log in to see why each signal reads the way it does.
Job description
Structured overview of role & requirements
About This Role
- Lead development and implementation of large-scale data engineering solutions on Databricks platform using Azure and AWS cloud services.
- Manage and optimize Databricks components including Notebooks, Clusters, Jobs, Delta Lake, Unity Catalog, and RBAC configurations.
- Develop, debug, and optimize data processing pipelines using Scala, Python, PySpark, SQL, and implement CI/CD with Azure DevOps.
Minimum Requirements
- Proven expertise in Databricks platform features and Azure cloud services including Azure Data Factory, Blob Storage, and Azure DevOps for CI/CD.
- Strong programming skills in Scala, Python, PySpark, and SQL with ability to optimize Apache Spark performance.
- Work Experience Required: Not explicitly mentioned in the JD.
- Work Schedule: Hybrid
Ideal Candidate Profile
- Experienced in managing end-to-end data engineering projects in cloud environments (Azure primarily, AWS knowledge is a plus).
- Capable of handling secure coding, data governance, compliance standards, and RBAC configurations on Databricks.
- Skilled in debugging and performance optimization of PySpark and SQL code with hands-on experience in CI/CD pipelines.