Back
Mid-level data engineer in Bangalore with broad, popular tech requirements increases applicant competition.
Core data engineering skills are highly transferable across industries despite pharma preference.
Explicit multi-year requirements for PySpark and Snowflake plus mandatory CI/CD and cloud skills increase shortlisting strictness.
Login to See Your Match Score
Create a free account or log in to unlock your CV match score across:
Design and implement high-performance data ingestion pipelines from multiple sources ensuring data quality and consistency.
Develop scalable and reusable frameworks for data ingestion and integrate end-to-end pipelines from source systems to target repositories.
Evaluate and present proofs of concept for technology components using event-based/streaming technologies and propose solutions for new requirements.
Bachelor’s degree in computer science, engineering, or related field.
Minimum 5 years hands-on experience with PySpark, Python, Spark, Scala, including data validation, delivery, quality, and integrity.
Minimum 5 years experience with Snowflake/Databricks cloud data warehouse implementations handling various file formats like XML, JSON, Avro, Parquet.
Minimum 2 years experience with ETL from multiple data sources and formats, Big Data modeling with Python/Spark, and CI/CD tools including Jenkins, Terraform, Docker, AWS, Kubernetes.
Proven ability to design and optimize large-scale data pipelines and implement end-to-end data solutions using modern distributed systems (Hadoop, Spark, Kafka).
Experienced in cloud technologies (AWS services including EC2, S3, Lambda, SQS, SNS) and automation of deployment pipelines with CI/CD practices.
Background with data warehousing concepts, strong SQL/PLSQL programming skills, and ability to work with business intelligence and visualization tools (Tableau, Power BI).