Python PySpark Data Engineer

5 days ago

Winnipeg MB, Winnipeg Census Division, MB; Manitoba, Canada LeverageTek Staffing Solutions Full-time
Python and PySpark expertise define this Ottawa-based contract for a Data Engineer supporting an existing customer remotely. Convert code and optimize nightly ETL pipelines. LeverageTek seeks a Data Engineer for a five-month contract with an Ottawa-based customer. The work requires intermediate to senior hands-on experience with Apache Spark, PySpark, and Python. You will also use GitLab or GitHub while improving existing ETL architecture, pipeline accuracy, and efficiency.

Key Responsibilities
Convert Python packages to PySpark Test, validate, and integrate packages into nightly ETL Identify and implement performance optimizations Write and validate code improvements Document ETL processes when time permits

Requirements:
Intermediate to senior Apache Spark experience Intermediate to senior Python experience Experience converting and optimizing code GitLab or GitHub experience Ability to work independently remotely Apply Python, PySpark, testing, and optimization expertise to strengthen nightly ETL pipeline performance for LeverageTek's customer.