Lead Data Engineer
Algoworks (company site, opens in a new tab)10+ yrsRemoteDirect career pageEst. 30–45 LPA
Posted on 24 Sept
Reach people at Algoworks
People at this company you can reach
Log in to see who works here
Create a free account or log in to see the people at Algoworks you could write to:
- Recruiters & HR
- People in this function
- Work addresses
Your match
Against your primary resume
Log in to see your match score
Create a free account or log in to unlock your CV match score across:
- Role fit
- Experience
- Industry relevance
Protocol Intelligence
How competitive this role is
Competition
Background fit
Strictness
Log in to see why each signal reads the way it does.
Job description
Structured overview of role & requirements
About This Role
- Build and maintain scalable enterprise data pipelines using Azure Databricks and Azure Data Factory focusing on Bronze, Silver, and curated data layers.
- Optimize Delta Lake tables and Spark jobs for performance and cost efficiency while ensuring data quality and reliability.
- Collaborate with DWH and reporting teams to provide optimized datasets for downstream reporting and analytical workloads.
Minimum Requirements
- 10+ years of work experience in data engineering or related roles.
- Strong hands-on experience with Azure Databricks, Azure Data Factory, PySpark, SQL, and Delta Lake.
- Proven skills in data pipeline development including batch and incremental processing, data transformation, and performance tuning of Spark and Delta Lake.
- Work Experience Required: 10+ years
Ideal Candidate Profile
- Experienced senior-level data engineer capable of independently designing and developing complex pipelines in enterprise Azure environments.
- Skilled in performance, scalability, reliability, and ensuring data quality across pipeline components integrated with downstream DWH and reporting systems.
- Strong ownership with ability to troubleshoot production data issues and implement best practices including CI/CD and Agile process adherence.