Match Score
Against your primary resumeLogin to See Your Match Score
Create a free account or log in to unlock your CV match score across:
Protocol Intelligence
Data-driven signals on your job's competitivenessLog in to see why each signal reads the way it does.
Job Description
Structured overview of role & requirementsAbout This Role
Design, develop, and optimize end-to-end data pipelines using Databricks with streaming and batch processing (PySpark, Delta Lake).
Build and maintain real-time streaming ingestion pipelines primarily from Kafka and other event-driven sources.
Lead and mentor team members, owning module delivery and collaborating across distributed teams while supporting cloud migration and CI/CD integration.
Minimum Requirements
8+ years of total Data Engineering experience with at least 3 years hands-on Databricks experience.
Strong expertise in Databricks streaming and batch, PySpark, Spark SQL, Spark Streaming, and Kafka.
Experience with AWS services (S3, Airflow, Lambda), CI/CD, Git, and deployment automation.
Domain experience in Healthcare Payer (Claims, Membership, Coverage) with knowledge of data privacy and governance (PII/HIPAA).
Ideal Candidate Profile
Experienced data engineering leader capable of independently owning modules and mentoring teams in a distributed environment.
Hands-on expertise in real-time data streaming pipelines and layered data architecture (Bronze, Silver, Gold) suited for healthcare datasets.
Familiarity with healthcare data standards like FHIR and strong focus on compliance with healthcare data security and governance.