Back
Senior, niche Spark/Databricks manager at Tier-1 metro reduces applicant density despite company brand.
Spark/Databricks and cloud skills are transferable, but leadership and streaming specialization moderately restrict fit.
Explicit 13+ years and 6+ years Spark/Scala plus cloud and tooling mandates strict filters.
Login to See Your Match Score
Create a free account or log in to unlock your CV match score across:
Lead a team of Software Development Engineers in designing, developing, and optimizing large-scale streaming data pipelines using Apache Spark.
Manage projects from design through testing, including new programs, enhancements, and modifications, ensuring performance standards are met.
Collaborate across teams to implement data quality, governance, monitoring, and compliance standards, and drive continuous improvements in Spark-based data engineering processes.
13+ years experience in engineering roles, with at least 6 years focused on Spark/Scala development.
Strong proficiency in Spark, Scala or Java, and experience with cloud platforms (AWS EMR, Azure Databricks, GCP Dataproc).
Bachelor's or Master's degree in Computer Science, Engineering, or related field (or equivalent combination of coursework and experience).
Experience with CI/CD pipelines and Infrastructure as Code tools (Terraform, CloudFormation).
Experienced in building and optimizing performant Spark-based streaming data pipelines on cloud and on-premises systems.
Capable of managing cross-functional collaboration with data scientists, analysts, and stakeholders to deliver reliable datasets and meet compliance standards.
Proficient in implementing automation and best practices for distributed data processing, monitoring, and operational maintenance.