Lead Data Engineer
1 week ago
Lead Data Engineer Python, PySpark & SQL Location: Canada Job Type: Full time contract We are looking for a strong Lead Data Engineer with deep experience in Python, PySpark, SQL, and AWS to design, develop, and optimize large-scale data pipelines. This role requires strong hands‑on coding skills, the ability to validate and process complex raw data, and expertise in running and tuning PySpark jobs on EMR. Responsibilities Build scalable data ingestion and transformation pipelines using Python, PySpark, and SQL. Process raw CSV/text files from AWS S3, including validating headers, schema checks, and malformed file detection. Convert raw data into structured DataFrames and implement reusable data quality checks. Develop advanced transformations using SQL/PySpark (window functions, LAG(), grouping logic, date gap detection, etc.). Deploy and tune PySpark applications on AWS EMR, optimizing executor memory, cores, shuffle behavior, and cluster performance. Work with AWS services such as S3, EMR, Glue, Lambda, IAM. Debug performance issues (OOM errors, shuffle spill, etc.) and improve pipeline reliability. Lead design discussions, code reviews, and mentor junior engineers. Required Skills 8+ years of experience in Data Engineering. Expert Python (file processing, scripting, validation automation). Strong PySpark (DataFrames, job tuning, distributed processing). Advanced SQL (analytical functions, performance tuning). Hands‑on with AWS data stack: S3, EMR, Glue, Lambda. Strong understanding of Spark memory allocation, YARN container usage, and EMR resource tuning. Excellent debugging, communication, and problem‑solving skills. Nice to Have Airflow or Databricks experience. Terraform or CloudFormation. Experience with data lake formats (Delta, Iceberg, Hudi). #J-18808-Ljbffr
-
Manager, Data Centre Facilities Engineering
1 week ago
Toronto, ON MW G, Canada NTT DATA Full time $120,000 - $180,000 per yearMake an impact with NTT DATAJoin a company that is pushing the boundaries of what is possible. We are renowned for our technical excellence and leading innovations, and for making a difference to our clients and society. Our workplace embraces diversity and inclusion – it's a place where you can grow, belong and thrive.Your day at NTT DATAThe Manager, Data...
-
Data Engineer Lead
4 weeks ago
Toronto, Canada Compunnel, Inc. Full timeWe are seeking a Data Engineer Lead with strong expertise in relational and NoSQL databases, big data technologies, and cloud-native architectures. The ideal candidate will lead data engineering initiatives, design scalable solutions, and guide the team in implementing best practices. Key Responsibilities Lead the design and development of data pipelines and...
-
Data Engineer Lead
4 weeks ago
Toronto, Canada Compunnel, Inc. Full timeWe are seeking a Data Engineer Lead with strong expertise in relational and NoSQL databases, big data technologies, and cloud-native architectures. The ideal candidate will lead data engineering initiatives, design scalable solutions, and guide the team in implementing best practices. Key Responsibilities Lead the design and development of data pipelines and...
-
Data Engineer Lead
4 weeks ago
Toronto, Canada Compunnel, Inc. Full timeWe are seeking a Data Engineer Lead with strong expertise in relational and NoSQL databases, big data technologies, and cloud-native architectures. The ideal candidate will lead data engineering initiatives, design scalable solutions, and guide the team in implementing best practices. Key Responsibilities Lead the design and development of data pipelines and...
-
Data Engineer Lead
3 weeks ago
Toronto, Canada Compunnel, Inc. Full timeWe are seeking a Data Engineer Lead with strong expertise in relational and NoSQL databases, big data technologies, and cloud-native architectures. The ideal candidate will lead data engineering initiatives, design scalable solutions, and guide the team in implementing best practices. Key Responsibilities - Lead the design and development of data...
-
Lead Data Engineer
5 days ago
Toronto, Canada Gala Solutions Inc Full timeJob Title: Lead Data Engineer / Data Architect Location: Canada (Hybrid Toronto) About the Role: We are seeking an experienced Lead Data Engineer / Data Architect to spearhead the design, development, and implementation of our enterprise data platform. This role will be instrumental in shaping and driving our data engineering roadmap, ensuring...
-
Lead Data Engineer
3 days ago
Toronto, Canada Gala Solutions Full timeJob Title: Lead Data Engineer / Data Architect Location: Canada (Hybrid Toronto) About the Role: We are seeking an experienced Lead Data Engineer / Data Architect to spearhead the design, development, and implementation of our enterprise data platform. This role will be instrumental in shaping and driving our data engineering roadmap, ensuring scalability,...
-
Lead Data Engineer
3 days ago
Toronto, Canada Mastercard Full timeOur Purpose Mastercard powers economies and empowers people in 200+ countries and territories worldwide. Together with our customers, we’re helping build a sustainable economy where everyone can prosper. We support a wide range of digital payments choices, making transactions secure, simple, smart and accessible. Our technology and innovation, partnerships...
-
Lead Data Engineer
3 days ago
Toronto, Canada Mastercard Full timeOur Purpose Mastercard powers economies and empowers people in 200+ countries and territories worldwide. Together with our customers, we’re helping build a sustainable economy where everyone can prosper. We support a wide range of digital payments choices, making transactions secure, simple, smart and accessible. Our technology and innovation, partnerships...
-
Lead Data Engineer
5 days ago
Toronto, Canada Gala Solutions Full timeJob Title: Lead Data Engineer / Data ArchitectLocation: Canada (Hybrid Toronto)About the Role:We are seeking an experienced Lead Data Engineer / Data Architect to spearhead the design, development, and implementation of our enterprise data platform. This role will be instrumental in shaping and driving our data engineering roadmap, ensuring scalability,...
