Lead Software Engineer - Data Ingestion

Configure, develop, and maintain data ingestion pipelines integrating diverse data sources like relational databases, Apache Kafka, AWS S3, and Elasticsearch.

Monitor, troubleshoot, and resolve operational issues related to data flow and pipeline execution, supporting platform maintenance and incident management.

Support release, governance, and compliance processes including deployment, configuration reviews, and documentation maintenance.

Minimum 2+ years of Python development experience.

Practical hands-on experience with Apache NiFi and Apache Airflow is mandatory.

Experience integrating data from relational databases and knowledge of ETL/ELT performance tuning.

Basic experience with Apache Kafka, Elasticsearch, SQL knowledge, and understanding of incident and release management processes.

Medior-level engineer comfortable working independently on standard integration tasks within an enterprise environment with strong governance and security.

Strong analytical and troubleshooting skills to identify root causes of data integration and connectivity issues.

Experience collaborating across multiple teams including architects, security, audit, and infrastructure for release and platform operations.