Match Score
Against your primary resumeLogin to See Your Match Score
Create a free account or log in to unlock your CV match score across:
Protocol Intelligence
Data-driven signals on your job's competitivenessLog in to see why each signal reads the way it does.
Job Description
Structured overview of role & requirementsAbout This Role
Design, build, and optimize ETL/ELT data pipelines using Python and PySpark within Azure Databricks and Azure Data Factory.
Develop and manage data ingestion from multiple sources and implement monitoring solutions for data quality and pipeline health.
Manage cloud infrastructure related to data on OpenShift, utilize CI/CD pipelines via GitHub Actions, and optimize SQL queries for performance.
Minimum Requirements
6–8 years of relevant experience in data engineering or related fields.
Strong proficiency in Python, PySpark, and SQL with experience in ETL/ELT pipeline development.
Hands-on experience with Azure data platform tools including Azure Databricks, Azure Data Factory, Azure SQL Server, Azure Key Vault.
Familiarity with OpenShift, HELM, GitHub Actions for CI/CD, and monitoring tools such as Grafana.
Ideal Candidate Profile
Experienced in designing and maintaining scalable data pipelines in a complex cloud environment (Azure and OpenShift).
Proficient in both development (Python, PySpark, SQL) and operational aspects (CI/CD, container orchestration, monitoring).
Capable of collaborating across teams (data scientists, analysts, engineers) to deliver reliable data infrastructure.