Data Engineer(Python+Pyspark)

Position Responsibilities: * Work on palantir- foundry platform * Understands business requirements and applies them to complex programming...

Position Responsibilities:

• Work on palantir- foundry platform
• Understands business requirements and applies them to complex programming and analysis.
• Communicates (oral and written) recommendations and mentors/provides guidance to less experienced colleagues.
• Partners with various departments to understand and incorporate standards information and requirements into work procedures.
• Identifies, analyzes and provides feedback to departmental standards, norms, and new goals/objectives.
• Analyzes existing applications and systems and formulates logic for new systems, devises logic procedures, logical database design, performs coding and tests/debugs programs with an operational mindset.
• Works on complex data & analytics-centric problems having broad impact that require in depth analysis and judgment to obtain results or solutions.
• Designs and deploys new complex Enterprise systems and enhancements to existing systems ensuring compatibility and inter-operability.
• Resolves application programming analysis problems of broad scope within procedural guidelines. May seek assistance from the supervisor or more skilled programmers/analysts on unusual or especially complex problems that cross multiple functional/technology areas.
• Delivers best-in-class software as part of a software delivery team.
• Conceptualizing and generating infrastructure that allows big data to be accessed and analyzed
• Plans work to meet assigned general objectives; progress is reviewed upon completion and solutions may provide an opportunity for creative/non-standard approaches.

Required Skills:

• Experience in data acquisition, data set process, improving data reliability, quality and efficiency
• Strong Experience around ETL with Python, validation/cleaning/engineering with Python/PySpark - centric should be the focus.
• Must have good experience developing in Python using PySpark and other data tools
• Must have experience with large data sets to address business issues
• Experience with modern software delivery practices, including source control, testing, continuous delivery
• Experience delivering product with Agile methodologies