Senior Databricks Apache Spark -SQL Engineer - Assistant Vice President
CitiMatch Score
Against your primary resumeLogin to See Your Match Score
Create a free account or log in to unlock your CV match score across:
Protocol Intelligence
Data-driven signals on your job's competitivenessLog in to see why each signal reads the way it does.
Job Description
Structured overview of role & requirementsAbout This Role
Lead modernization and refactoring of legacy Hadoop Spark pipelines into Databricks native architectures on AWS.
Design, develop, and optimize scalable data processing solutions using Apache Spark (Java/PySpark), Databricks features, and Delta Lake.
Contribute to architectural decisions ensuring performance optimization, maintainability, and production readiness while collaborating with senior architects and platform teams.
Minimum Requirements
8+ years of experience in data engineering or distributed systems.
Strong expertise with Apache Spark (JavaSpark/PySpark), Databricks on AWS, Delta Lake, and SQL.
Bachelor’s degree or equivalent experience.
Experience modernizing legacy data platforms to cloud-based architectures on AWS.
Ideal Candidate Profile
Proven ability to translate high-level architecture into detailed technical designs for data processing platforms.
Experience with Spark performance tuning and large-scale batch optimization in time-bound, high-impact environments.
Operates well within collaborative, multi-geography teams involving architects and DevOps, providing technical leadership on complex distributed systems.