We're looking for a highly skilled and experienced Python Cloud Data Pipeline Engineer to provide production support, feature implementation...
We're looking for a highly skilled and experienced Python Cloud Data Pipeline Engineer to provide production support, feature implementation, and rapid problem-solving if data ingestion pipeline breaks. This is an excellent opportunity to work in a highly collaborative Agile environment, leveraging modern technologies and tools.
We can facilitate w2 and corp-to-corp consultants. For our w2 consultants, we offer a great benefits package that includes Medical, Dental, and Vision benefits, 401k with company matching, and life insurance.
Responsibilities of the Python Cloud Data Pipeline Engineer:
- Provide production support and participate in on-call rotation
- Monitor ingestion pipeline and solve breaks for a critical system
- Collect and add metrics to existing Datadog dashboard
- Monitor Slack channel and answer customer/user questions about data and latency
Requirements of the Python Cloud Data Pipeline Engineer:
- Excellent expertise in Python
- Basic understanding of data modeling and good SQL skills
- Experience building data pipelines to migrate data from on-premise to AWS cloud
- Experience with Hadoop (YARN, Hdfs, Hive, distcp) and AWS Cloud (S3 , IAM, Kubernetes).
- Experience in containerization (Docker, EKS)
- Experience collecting metrics and building dashboards in Datadog
- Expert in Scheduling Complex DAG's in Airflow
- Understanding of version control like Git and CI/CD workflow
Keywords: Python Cloud Data Pipeline Engineer, SQL, AWS, Hadoop, CI/CD, Datadog
Job ID: 353215