Python Databricks Data Engineer
19 hours ago
Toronto ON, Toronto Census Division, ON; Ontario, Canada
Electric Mind
Full-time
Free with email or Google
Save this job and keep your search organized
Create a free account to save jobs, create alerts and return to this listing from your dashboard.
Free with email or Google
By continuing, you agree to our Terms & Privacy Policy.
Build Python and PySpark data pipelines at Electric Mind, supporting a financial services platform across development, quality assurance, and user acceptance testing environments.
This developer role is available permanently or by contract. You will maintain Databricks pipelines using SQL, medallion architecture, and curated data models. Working alongside a senior engineer, you will write clean transform code, resolve issues, and learn Databricks internals while supporting reliable platform operations.
Key Responsibilities
Write PySpark transforms for silver and gold data layers Apply Auto CDC and SCD Type 2 merge patterns Query Unity Catalog using SQL warehouses Debug pipeline logs, schema drift, and data quality issues Maintain tests, scripts, documentation, and GitLab merge requests
Requirements:
Strong Python fundamentals, including functions, modules, and basic object-oriented programming Databricks experience or willingness to learn quickly Apache Spark, PySpark, or strong SQL exposure Working knowledge of joins, aggregations, and window functions Basic Git workflow and methodical debugging skills Develop Python, PySpark, SQL, and Databricks expertise while improving financial services data pipelines at Electric Mind. #J-18808-Ljbffr
Key Responsibilities
Write PySpark transforms for silver and gold data layers Apply Auto CDC and SCD Type 2 merge patterns Query Unity Catalog using SQL warehouses Debug pipeline logs, schema drift, and data quality issues Maintain tests, scripts, documentation, and GitLab merge requests
Requirements:
Strong Python fundamentals, including functions, modules, and basic object-oriented programming Databricks experience or willingness to learn quickly Apache Spark, PySpark, or strong SQL exposure Working knowledge of joins, aggregations, and window functions Basic Git workflow and methodical debugging skills Develop Python, PySpark, SQL, and Databricks expertise while improving financial services data pipelines at Electric Mind. #J-18808-Ljbffr