Senior Data Engineer - Data Bricks, Airflow, Snowflake, AI
Optum (company site, opens in a new tab)4+ yrsHyderabadOn-siteDirect career pageEst. 12–22 LPA
Posted on 3 Sept
Reach people at Optum
People at this company you can reach
Log in to see who works here
Create a free account or log in to see the people at Optum you could write to:
- Recruiters & HR
- People in this function
- Work addresses
Your match
Against your primary resume
Log in to see your match score
Create a free account or log in to unlock your CV match score across:
- Role fit
- Experience
- Industry relevance
Protocol Intelligence
How competitive this role is
Competition
Background fit
Strictness
Log in to see why each signal reads the way it does.
Job description
Structured overview of role & requirements
About This Role
- Design, develop, and maintain scalable data pipelines using Databricks (Spark/PySpark), orchestrate workflows with Apache Airflow, and optimize ELT pipelines in Snowflake.
- Develop and deploy production-quality Python code and APIs using modern frameworks and Azure-native services, ensuring cloud best practices for security, scalability, and cost optimization.
- Troubleshoot complex production issues, participate in architecture design and technical decision-making, and collaborate with cross-functional teams to support downstream data consumption.
Minimum Requirements
- Bachelor's degree in B.Tech, MCA, or Graduation.
- Minimum 4 years of total professional experience.
- Hands-on experience with Databricks (Spark/PySpark), Apache Airflow, Snowflake, Python programming, and Microsoft Azure cloud services.
- Experience building and hosting REST APIs with Python frameworks; familiarity with Azure Functions, App Services, and API Management.
Ideal Candidate Profile
- Experienced in end-to-end design and independent delivery of complex data engineering solutions in cloud-native environments (Azure).
- Strong operational exposure to data warehousing, ELT pipeline development, workflow orchestration, and performance tuning within Snowflake and Databricks ecosystems.
- Technically skilled in Python API development, CI/CD best practices, and managing production data pipelines with observability and reliability focus.