Data Engineer - IT
The Guardian Life Insurance Company of AmericaMatch Score
Against your primary resumeLogin to See Your Match Score
Create a free account or log in to unlock your CV match score across:
Protocol Intelligence
Data-driven signals on your job's competitivenessLog in to see why each signal reads the way it does.
Job Description
Structured overview of role & requirementsAbout This Role
Develop and maintain scalable data pipelines using Databricks and Apache Spark for structured and semi-structured data.
Optimize performance of Spark jobs and SQL queries within data lake/lakehouse architecture.
Apply ETL, data warehousing, and data modeling expertise to support data platform reliability and efficiency.
Minimum Requirements
Strong hands-on experience with Databricks (Lakehouse platform, notebooks, jobs, clusters) and PySpark/Apache Spark.
Advanced SQL skills including joins, window functions, and performance tuning.
Solid understanding of ETL concepts, data warehousing, and data modeling.
Work Experience Required: Not explicitly mentioned in the JD.
Ideal Candidate Profile
Experienced in building and scaling data pipelines specifically within Databricks and Spark ecosystems.
Familiarity with data lake/delta lake architectures and performance optimization techniques.
Knowledge of version control (Git) and deployment practices, preferably with some exposure to cloud platforms (AWS) and CI/CD pipelines.