Data Engineer - PySpark
YASH TechnologiesMatch Score
Against your primary resumeLogin to See Your Match Score
Create a free account or log in to unlock your CV match score across:
Protocol Intelligence
Data-driven signals on your job's competitivenessLog in to see why each signal reads the way it does.
Job Description
Structured overview of role & requirementsAbout This Role
Design and develop scalable data processing solutions using PySpark within Microsoft Fabric.
Build and optimize ETL/ELT pipelines for large-scale structured and semi-structured data using Spark DataFrames and Spark SQL.
Develop data ingestion, transformation, and aggregation workflows leveraging Fabric Notebooks, Lakehouse, and Data Factory Pipelines.
Minimum Requirements
3-5 years of hands-on experience with PySpark and Microsoft Fabric data engineering.
Experience with designing and optimizing ETL/ELT pipelines using Spark technology.
Experience with large-scale structured and semi-structured datasets processing.
Work Experience Required: 3-5 years.
Ideal Candidate Profile
Experienced in working with Microsoft Fabric ecosystem including Fabric Notebooks, Lakehouse, and Data Factory Pipelines.
Capable of handling end-to-end data engineering tasks involving ingestion, transformation, and aggregation.
Comfortable working in an agile, flexible, and collaborative technology environment.