Match Score
Against your primary resumeLogin to See Your Match Score
Create a free account or log in to unlock your CV match score across:
Protocol Intelligence
Data-driven signals on your job's competitivenessLog in to see why each signal reads the way it does.
Job Description
Structured overview of role & requirementsAbout This Role
Design, build, and maintain reliable, scalable distributed data pipelines on Databricks and AWS for bp pulse platforms.
Model data for analysis ensuring accuracy, governance, and usability while writing secure, tested software meeting compliance requirements.
Mentor engineers, lead informally within the squad, improve CI/CD and infrastructure as code, and manage pipeline monitoring and support.
Minimum Requirements
Strong hands-on experience with Databricks, PySpark, Spark SQL, Delta Lake, Unity Catalog and AWS data/serverless services (Lambda, Glue, S3, Redshift, Kinesis, SNS/SQS).
Proficient in Python programming and advanced SQL for production-level data modelling.
Experience with CI/CD pipelines, infrastructure as code tools and mentoring other engineers.
Degree in computer science or related field, or equivalent knowledge and experience; Work Experience Required: Not explicitly mentioned in the JD.
Ideal Candidate Profile
Experienced in building and running complex data pipelines and products in large-scale environments with a focus on data lakehouse technologies.
Operates as an informal technical leader with strong mentoring skills and ability to positively influence cross-functional teams.
Comfortable working in hybrid settings and collaborating across business units to deliver dependable data products aligned with compliance and governance.