Back
Mid-level, metro role but niche Databricks/Spark skills limit applicant density.
Databricks and Spark skills transfer across industries, though platform and Azure specifics increase specialization.
Several mandatory Databricks, Spark, Unity Catalog, cloud, and security requirements make filters stringent.
Login to See Your Match Score
Create a free account or log in to unlock your CV match score across:
Manage, configure, and optimize Databricks platform components including workspaces, clusters, SQL Warehouses, Unity Catalog, and jobs ensuring high availability, performance, security, and cost-efficiency.
Develop and maintain production-grade Spark-based ETL pipelines using Delta Lake and Delta Live Tables; provide support in debugging, performance tuning, and code reviews for Spark jobs.
Implement infrastructure governance including RBAC, access controls, secrets management, CI/CD pipelines, monitoring, alerting, and compliance audits for the Databricks environment.
4–6 years experience with Databricks administration or data engineering.
Strong proficiency in Apache Spark programming (PySpark preferred; SQL or Scala is a plus).
Hands-on experience with Databricks Jobs, cluster configuration, SQL Warehouses, and Unity Catalog.
Experience with Azure cloud platform and familiarity with IAM, networking, monitoring, and security patterns.
Technically adept in both platform administration and Spark development with a focus on optimizing performance and cost in cloud-based data engineering environments.
Experienced in implementing governance, security, and compliance controls within Databricks and cloud infrastructure.
Capable of owning end-to-end Databricks environment health including automation, monitoring, and collaboration with cross-functional teams on cloud networking and security issues.