Back
Senior, specialized Spark role in a metro at a recognizable global firm yields moderate competition.
Deep Spark, distributed systems, and cloud platform expertise limits cross-domain transferability.
Explicit 10+ years, deep Spark/distributed systems and architecture leadership make hiring filters highly selective.
Login to See Your Match Score
Create a free account or log in to unlock your CV match score across:
Design scalable and resilient distributed data processing architectures using Apache Spark and modern data platform technologies.
Provide technical leadership and architecture guidance across engineering teams for Spark-based batch and streaming data solutions, including performance optimization and operational best practices.
Collaborate with stakeholders to translate business requirements into scalable technical solutions and contribute to data platform modernization and cloud transformation initiatives.
10+ years of experience in software engineering, data engineering, or distributed systems development.
Strong hands-on expertise with Apache Spark, including Spark SQL, Structured Streaming, DataFrames, and Dataset APIs.
Proficiency in Scala, Java, or Python and experience with technologies such as Hadoop, Hive, Iceberg, AWS EMR, AWS Glue, GCP Dataproc, and BigQuery.
Hybrid work model with minimum two days in-person per week at the assigned TransUnion office location.
Experienced individual contributor with architecture leadership in distributed data platforms and Spark-based solutions within large-scale environments.
Strong background in Spark optimization techniques and cloud-native architecture, with demonstrated ability to guide engineering teams and drive performance improvements.
Familiarity with AWS and GCP cloud services for deploying and managing large data processing pipelines at terabyte-to-petabyte scale.