Senior Data Engineer (Python, Spark & AWS)

Company: ICONMA
Location: Mc Lean, Virginia, United States
Type: Full-time
Posted: 01.MAY.2021

Summary

Location: McLean, VA Duration: 8 months with possible extension Description: Technical Must Haves: AWS experience - must have experience...

Description

Location: McLean, VA

Duration: 8 months with possible extension

Description:

Technical Must Haves:

AWS experience - must have experience supporting an AWS environment. Must have Python and Spark. Strong preference for Java. Must also be proficient in SQL.

Position Overview:

Client's Investments & Capital Markets Division is currently seeking a Senior Full Stack Engineer who enjoys data and building data storage platforms from ground up using microservices architecture. The ideal candidate has a passion for data analysis, technology and helping people leverage the technology to transform their business processes and analytics. As a Data Engineer, you will be part of a team responsible for supporting a wide range of internal customers. You will draw on all the skills in your toolkit to analyze, design, and develop data storage and data analytic solutions using data lake patterns, that help our customers run more effective operations and make better business decisions.

Your Work Falls Into Two Primary Categories:

- Develop data filtering, transformational and loading requirements

- Define and execute ETLs using Apache Spark among other Data technologies

- Determine appropriate translations and validations between source data and target databases

- Implement business logic to cleanse & transform data

- Design and implement appropriate error handling procedures

- Develop project, documentation and storage standards in conjunction with data architects

- Monitor performance, troubleshoot and tune ETL processes as appropriate using tools like in the AWS ecosystem.

- Execution of end to end implementation of underlying data ingestion workflow.

Operations and Technology:

- At least 5 years of experience developing in Python and Spark (Java is preferred)

- At least 3 years' experience in one of the following Cloud technologies: Amazon Web Services (AWS), Google

- Experience working with different Databases and understanding of data concepts (including data warehousing, data lake patterns, structured and unstructured data)

- Implementation and tuning experience specifically using Amazon Elastic Map Reduce (EMR).

- Implementing AWS services in a variety of distributed computing, enterprise environments.

- Experience writing automated unit, integration, regression, performance and acceptance tests

- Good understanding of the current trends in Data Engineering domain

- Solid understanding of software design principles



Job Requirements:

Technical Must Haves:

- Must have Python and Spark.

- AWS experience - must have experience supporting an AWS environment.

- Must also be proficient in SQL.

- Experience with S3, Lambda & Snowflake is critical for this role

- Strong preference for Java but not required.

- Big Preference to candidates coming from Financial Services/Mortgage Industry background.

As an equal opportunity employer, ICONMA prides itself on creating an employment environment that supports and encourages the abilities of all persons regardless of race, color, gender, age, sexual orientation, citizenship, or disability.

 
Apply Now

Share

Flash-bkgn
Loader2 Processing ...