Python PySpark Data Engineer

4 days ago

Toronto ON, Toronto Census Division, ON; Ontario, Canada LeverageTek Staffing Solutions Full-time

Python and PySpark expertise define this Ottawa-based contract for a Data Engineer supporting an existing customer remotely. Convert code and optimize nightly ETL pipelines.

LeverageTek seeks a Data Engineer for a five-month contract with an Ottawa-based customer. The work requires intermediate to senior hands-on experience with Apache Spark, PySpark, and Python. You will also use GitLab or GitHub while improving existing ETL architecture, pipeline accuracy, and efficiency.

Key Responsibilities:

  • Convert Python packages to PySpark
  • Test, validate, and integrate packages into nightly ETL
  • Identify and implement performance optimizations
  • Write and validate code improvements
  • Document ETL processes when time permits

Requirements:

  • Intermediate to senior Apache Spark experience
  • Intermediate to senior Python experience
  • Experience converting and optimizing code
  • GitLab or GitHub experience
  • Ability to work independently remotely

Apply Python, PySpark, testing, and optimization expertise to strengthen nightly ETL pipeline performance for LeverageTek's customer.

#J-18808-Ljbffr