Senior Databricks Data Engineer - Assistant Manager
State Street CorporationMatch Score
Against your primary resumeLogin to See Your Match Score
Create a free account or log in to unlock your CV match score across:
Protocol Intelligence
Data-driven signals on your job's competitivenessLog in to see why each signal reads the way it does.
Job Description
Structured overview of role & requirementsAbout This Role
Design, develop, and optimize high throughput, configurable data pipelines using Databricks (SQL & Scala/PySpark) for processing large volumes of data in near real-time.
Build and maintain end-to-end data ingestion pipelines (streaming, events, batch) and deploy scalable ETL pipelines on cloud infrastructure (Azure/AWS).
Own the full software development lifecycle including design, coding, testing, deployment, troubleshooting, and root cause analysis of data engineering solutions.
Minimum Requirements
7+ years overall IT experience with 5+ years in Data Engineering working on large datasets.
5+ years hands-on experience with Databricks, specifically using SQL & Scala/PySpark.
5+ years experience in cloud-based development using Azure and/or AWS.
Bachelor’s degree in Computer Science, IT, or related field.
Ideal Candidate Profile
Experienced in end-to-end data lakehouse solutioning on Databricks with strong command over data engineering pipeline design and optimization.
Proficient in troubleshooting DevOps pipelines and Azure services, with knowledge of Microservices and API/Event driven architectures.
Capable of collaborating across geographies and stakeholders in a financial industry environment with aggressive timelines and agile methodologies.