Match Score
Against your primary resumeLogin to See Your Match Score
Create a free account or log in to unlock your CV match score across:
Protocol Intelligence
Data-driven signals on your job's competitivenessLog in to see why each signal reads the way it does.
Job Description
Structured overview of role & requirementsAbout This Role
Lead design, development, and deployment of scalable batch and real-time data pipelines and lakehouse solutions using Databricks and AWS.
Own integration frameworks, data governance, metadata management, and performance optimization of Spark workloads and cloud resources.
Provide technical leadership including code reviews, mentoring engineers, collaborating across teams, and managing delivery risk in Agile frameworks.
Minimum Requirements
Master’s degree in Computer Science, Engineering, IT, Data Science or related and at least 7 years relevant experience; OR Bachelor’s degree with at least 9 years experience.
Advanced hands-on expertise with Databricks, Apache Spark, PySpark, Spark SQL, Delta Lake, Python, SQL.
Experience in production-grade batch and streaming pipeline design and operation, with AWS services and orchestration tools like Databricks Workflows.
Strong knowledge of data governance, metadata management, RBAC, security controls, CI/CD pipelines, and Agile or Scaled Agile environments.
Ideal Candidate Profile
Experienced technical leader comfortable guiding complex, scalable data solutions in regulated industries such as biotech or pharmaceuticals.
Proven ability to define reusable engineering standards, optimize Spark and cloud infrastructure for cost and performance, and lead cross-functional teams.
Experienced working with enterprise data architectures including lakehouses, metadata-driven frameworks, and data governance in cloud-native environments.