Python Cloud Data Pipeline Engineer

Company: Eliassen Group
Location: Cupertino, California, United States
Type: Full-time
Posted: 11.AUG.2021
< >

Summary

We're looking for a highly skilled and experienced Python Cloud Data Pipeline Engineer to provide production support, feature implementation...

Description

We're looking for a highly skilled and experienced Python Cloud Data Pipeline Engineer to provide production support, feature implementation, and rapid problem-solving if data ingestion pipeline breaks. This is an excellent opportunity to work in a highly collaborative Agile environment, leveraging modern technologies and tools.


We can facilitate w2 and corp-to-corp consultants. For our w2 consultants, we offer a great benefits package that includes Medical, Dental, and Vision benefits, 401k with company matching, and life insurance.


Responsibilities of the Python Cloud Data Pipeline Engineer:


  • Provide production support and participate in on-call rotation
  • Monitor ingestion pipeline and solve breaks for a critical system
  • Collect and add metrics to existing Datadog dashboard
  • Monitor Slack channel and answer customer/user questions about data and latency


Requirements of the Python Cloud Data Pipeline Engineer:


  • Excellent expertise in Python
  • Basic understanding of data modeling and good SQL skills
  • Experience building data pipelines to migrate data from on-premise to AWS cloud
  • Experience with Hadoop (YARN, Hdfs, Hive, distcp) and AWS Cloud (S3 , IAM, Kubernetes).
  • Experience in containerization (Docker, EKS)
  • Experience collecting metrics and building dashboards in Datadog
  • Expert in Scheduling Complex DAG's in Airflow
  • Understanding of version control like Git and CI/CD workflow


Keywords: Python Cloud Data Pipeline Engineer, SQL, AWS, Hadoop, CI/CD, Datadog


Job ID: 353215

 
Apply Now

Share

Flash-bkgn
Loader2 Processing ...