Python Data Platform Engineer
2 weeks ago
Job Title: Python Data Platform Engineer
Montreal, Quebec, Canada. (Day 1 Onsite - Hybrid)
12 Months contract
Job Duties:
In this role you will be primarily responsible for contributing to the development of a unified data pipeline framework written in Python utilizing technologies such as Airflow, DBT, Spark and Snowflake. You will also be responsible for contributing to the integration of this framework with existing internal platforms for data quality, data cataloging, data discovery, incident logging, and metric generation. You will be working closely with data warehousing leads, data analysts, ETL developers, infrastructure engineers, and data analytics teams to facilitate the implementation of this data platform and data pipeline framework.
Key responsibilities:
- To develop various components in Python of our unified data pipeline framework.
- To contribute towards the establishment of best practices for the optimal and efficient usage of Airflow, DBT and Snowflake.
- To assist with the testing and deployment of our data pipeline framework utilizing standard testing frameworks and CI/CD tooling.
- To monitor the performance of queries and data loads and perform tuning as necessary.
- To provide assistance and guidance during the QA & UAT phases to quickly confirm the validity of potential issues and to determine the root cause and best resolution of verified issues.
Minimum Skills Required:
- Bachelor's degree in Computer Science, Software Engineering, Information Technology, or related field required.
- At least 7 years of experience in data development and solutions in highly complex data environments with large data volumes.
- At least 7 years of SQL / PLSQL experience with the ability to write ad-hoc and complex queries to perform data analysis.
- At least 5 years of experience developing data pipelines and data warehousing solutions using Python and libraries such as Pandas, NumPy, PySpark, etc.
- At least 3 years of experience developing solutions in a hybrid data environment (on-Prem and Cloud)
- At least 3 years of experience developing Airflow DAGs to orchestrate data pipelines that utilize branching, dynamic DAG / task generation, and error handling.
- Hands on experience with developing data pipelines for structured, semi-structured, and unstructured data and experience integrating with their supporting stores (e.g. RDBMS, NoSQL DBs, Document DBs, Log Files, etc.)
- Hands on experience with Snowflake a must.
- Hands on experience with Apache Spark a must.
- Hands on experience with DBT preferred.
- Experience with performance tuning SQL queries, Spark job, and stored procedures.
- An understanding of E-R data models (conceptual, logical, and physical).
- Understanding of advanced data warehouse concepts (Factless Fact Tables, Temporal \ Bi-Temporal models, etc.) a plus.
-
Canada Data Elephant Full timeAbout Data ElephantWe're a boutique data and analytics consulting firm based in Vancouver, BC, helping Canadian organizations develop data strategies, modernize their data platforms, and unlock the power of data, analytics, ML, and GenAI.Our MissionWe deliver strategic solutions that create real business impact. Our team thrives on solving complex...
-
Python Data Engineer
3 weeks ago
Canada High 5 Games Full timePosition Overview:As a Python Data Engineer at High 5 Casino, you will be responsible for designing, implementing, and maintaining data pipelines, databases and our in-house real-time player interaction software. You will collaborate with cross-functional teams to ensure seamless data integration, support data-driven decision-making, and contribute to the...
-
Python Data Engineer
1 week ago
canada | ca High 5 Games Full timePosition Overview:As a Python Data Engineer at High 5 Casino, you will be responsible for designing, implementing, and maintaining data pipelines, databases and our in-house real-time player interaction software. You will collaborate with cross-functional teams to ensure seamless data integration, support data-driven decision-making, and contribute to the...
-
Data Engineering Specialist
4 days ago
Canada Data Elephant Full timeAbout UsData Elephant is a boutique data and analytics consulting firm based in Vancouver, BC. We help Canadian organizations develop data strategies, modernize their data platforms, and unlock the power of data, analytics, and AI.We're looking for a Cloud Data Solutions Architect to join our growing team. In this role, you'll contribute to the design,...
-
Databricks Data Engineer
7 days ago
Canada Data Elephant Full timeAbout UsWe're a boutique data and analytics consulting firm based in Vancouver, BC, helping Canadian organizations develop data strategies, modernize their data platforms and unlock the power of data, analytics, ML and GenAI. As a certified partner with leading cloud platforms, including AWS, Azure and Databricks, we deliver strategic solutions that create...
-
Databricks Data Engineer
5 days ago
Canada Data Elephant Full timeAbout Us We're a boutique data and analytics consulting firm based in Vancouver, BC, helping Canadian organizations develop data strategies, modernize their data platforms and unlock the power of data, analytics, ML and GenAI. As a certified partner with leading cloud platforms, including AWS, Azure and Databricks, we deliver strategic solutions that create...
-
Data Engineer
3 weeks ago
Canada hireVouch Full timeOur client is a leading data and cloud services provider that works with enterprise companies to build systems and infrastructures for them. As a Data Engineer, you'd be responsible for designing, building, and optimizing data pipelines, ensuring seamless data flow and analytics capabilities. This is a 6-month contract, with a possibility of extension. ...
-
Data Engineer
3 days ago
Canada KData AI Full timeCompany Description KData AI is a leading firm in Data and AI Engineering in the cloud, dedicated to delivering exceptional results for businesses worldwide. We specialize in transforming data into actionable insights through innovative AI and engineering solutions. We are looking for an experienced Data Architect with a minimum of 5 years in designing...
-
Data Engineer
20 hours ago
Canada KData AI Full timeCompany Description KData AI is a leading firm in Data and AI Engineering in the cloud, dedicated to delivering exceptional results for businesses worldwide. We specialize in transforming data into actionable insights through innovative AI and engineering solutions. We are looking for an experienced Data Architect with a minimum of 5 years in designing...
-
Platform Engineer
4 weeks ago
Canada Curative Full timeOur Platform Engineering team plays a crucial role in supporting this transformation by ensuring our cloud infrastructure, backend services, and developer tools are scalable, secure, and high-performing. As we integrate data-driven decision-making and AI/ML automation, we are also expanding our infrastructure capabilities to support AI workloads and...
-
Platform Engineer
4 weeks ago
Canada Curative Full timeOur Platform Engineering team plays a crucial role in supporting this transformation by ensuring our cloud infrastructure, backend services, and developer tools are scalable, secure, and high-performing. As we integrate data-driven decision-making and AI/ML automation, we are also expanding our infrastructure capabilities to support AI workloads and advanced...
-
Data Engineer
3 weeks ago
Canada hireVouch Full timeOur client is a leading data and cloud services provider that works with enterprise companies to build systems and infrastructures for them.As a Data Engineer, you'd be responsible for designing, building, and optimizing data pipelines, ensuring seamless data flow and analytics capabilities.This is a 6-month contract, with a possibility of...
-
Data Platform Architect and Developer
9 hours ago
Canada KData AI Full timeAt KData AI, we believe that data is the lifeblood of any organization. That's why we're committed to delivering exceptional results for our clients by transforming data into actionable insights through innovative AI and engineering solutions. Our team of experts specializes in designing and implementing enterprise-scale data platforms in the cloud using...
-
Data Scientist-SQL/Python
3 weeks ago
Canada Insight Global Full timeWe are seeking a Data Protection Expert with deep expertise in Protegrity to design, implement, and manage data security solutions. The ideal candidate will ensure compliance with data protection regulations while securing sensitive data in Azure DevOps, Azure Data Lake Gen2, Azure Data Factory (ADF), Snowflake, and Collibra environments. Data Protection...
-
Data Engineer III
3 weeks ago
Canada OnX Full timeOverview In today's rapidly evolving environment, organizations need to make data-driven decisions that deliver enterprise value. Our OnX Cloud and Artificial Intelligence practitioners design, develop, and implement large-scale data ecosystems, leveraging cloud-based platforms to integrate structured and unstructured data. We utilize automation, cognitive,...
-
Data Engineer III
3 weeks ago
Canada OnX Full timeOverview In today's rapidly evolving environment, organizations need to make data-driven decisions that deliver enterprise value. Our OnX Cloud and Artificial Intelligence practitioners design, develop, and implement large-scale data ecosystems, leveraging cloud-based platforms to integrate structured and unstructured data. We utilize automation, cognitive,...
-
Data Engineer IV
4 weeks ago
Canada OnX Full timeOverview In today's rapidly evolving environment, organizations need to make data-driven decisions that deliver enterprise value. Our OnX Cloud and Artificial Intelligence practitioners design, develop, and implement large-scale data ecosystems, leveraging cloud-based platforms to integrate structured and unstructured data. We utilize automation, cognitive,...
-
Data Engineer
7 days ago
Canada Haptiq Full timeOverview Haptiq Technology and Solutions, a leader in uniting companies and their investors through innovative technology, is looking for a Principal Fullstack Engineer to join our dynamic team in Canada. Haptiq excels in transforming potential into tangible performance, offering a suite of meticulously designed software solutions that cater to a wide...
-
Senior Data Engineer
7 days ago
Canada Newport Williams Full timeTitle: Senior Data EngineerLocation: 100% remote, open to anyone in Canada with valid work authorization.Requirements: This work will require Enhanced Reliability status.Description of ServiceThe Senior Data Engineer Service Provider will be responsible for leading the delivery of data on data and analytics platform. These services are at the forefront of...
-
Senior Data Engineer
5 days ago
Canada Newport Williams Full timeTitle: Senior Data EngineerLocation: 100% remote, open to anyone in Canada with valid work authorization.Requirements: This work will require Enhanced Reliability status.Description of ServiceThe Senior Data Engineer Service Provider will be responsible for leading the delivery of data on data and analytics platform. These services are at the forefront of...