Match Score
Against your primary resumeLogin to See Your Match Score
Create a free account or log in to unlock your CV match score across:
Protocol Intelligence
Data-driven signals on your job's competitivenessLog in to see why each signal reads the way it does.
Job Description
Structured overview of role & requirementsAbout This Role
Lead engineering and modernization of Databricks data processing platform on AWS, transitioning from legacy Cloudera Hadoop.
Refactor and optimize Spark pipelines (JavaSpark/PySpark) for enhanced performance and simplification using Databricks native features like Delta Lake and Workflows.
Drive technical design, including scalable data models and reusable components, ensuring production-ready, maintainable solutions aligned with engineering standards.
Minimum Requirements
12+ years experience in data engineering or distributed systems.
Strong expertise in Apache Spark (JavaSpark/PySpark), Databricks on AWS, Delta Lake, and SQL.
Experience modernizing legacy data platforms to cloud-based architectures and Spark performance tuning.
Bachelor’s degree or equivalent experience.
Ideal Candidate Profile
Hands-on Spark engineer capable of complex implementation and architectural contributions in distributed, large-scale batch processing environments.
Experienced in translating high-level architecture into detailed technical designs with focus on scalability and maintainability.
Comfortable operating under time-bound, high-impact conditions with strong problem-solving mindset and cross-team collaboration skills.