Data Engineer Consultant II
Allstate (company site, opens in a new tab)4+ yrsHybridDirect career pageEst. 12–20 LPA
Posted on 29 Sept
Reach people at Allstate
People at this company you can reach
Log in to see who works here
Create a free account or log in to see the people at Allstate you could write to:
- Recruiters & HR
- People in this function
- Work addresses
Your match
Against your primary resume
Log in to see your match score
Create a free account or log in to unlock your CV match score across:
- Role fit
- Experience
- Industry relevance
Protocol Intelligence
How competitive this role is
Competition
Background fit
Strictness
Log in to see why each signal reads the way it does.
Job description
Structured overview of role & requirements
About This Role
- Design, build, and maintain scalable batch and streaming data pipelines using Apache Spark and cloud-native platforms.
- Develop and optimize ETL/ELT workflows to create analytics-ready datasets supporting reporting, dashboards, and ML workloads.
- Ensure data quality, performance, scalability, and operational excellence across data platforms including Microsoft Fabric / One Lake.
Minimum Requirements
- Proven experience as a Data Engineer building and operating production data pipelines, preferably 4+ years.
- Hands-on proficiency with Apache Spark, Python, SQL, and cloud-based data platforms (e.g., Data Lakes, Lakehouse).
- Experience designing and optimizing data models and implementing CI/CD pipelines for data workloads.
- Familiarity with Microsoft Fabric or One Lake is a strong plus; Work Experience Required: Not explicitly mentioned in the JD.
Ideal Candidate Profile
- Experienced in large-scale distributed data processing and performance tuning in cloud environments.
- Skilled in creating reusable frameworks and collaborating across technical and non-technical teams for data solutions.
- Comfortable with Agile methodologies and partnering with analytics, BI, and data science teams to deliver data products.