Big Data Lead
Save this job and keep your search organized
Create a free account to save jobs, create alerts and return to this listing from your dashboard.
By continuing, you agree to our Terms & Privacy Policy.
JD: Develop and maintain scalable data pipelines using PySpark and Spark SQL for processing large datasets efficiently. Write clean, reusable, and optimized code in Python for data manipulation, analysis, and automation tasks. Design and implement ETL workflows to extract, transform, and load data from various structured and unstructured sources. Collaborate with data engineers, analysts, and stakeholders to understand data requirements and deliver solutions. Optimize Spark jobs for performance tuning, resource utilization, and minimizing execution time. Work with distributed computing frameworks to process and analyze big data in a cloud or on-premises environment. Develop and maintain unit tests to ensure the accuracy and reliability of data pipelines and transformations. Utilize Spark SQL for querying and managing large datasets stored in distributed systems like Hadoop or cloud storage. Monitor and troubleshoot data pipeline issues, ensuring reliability and timely delivery of data. Stay updated with the latest advancements in PySpark, Spark SQL, and big data technologies to improve existing systems.
-
Remote Data Engineer: Big Data Pipelines
3 days ago
canada | ca Capgemini Engineering Full-timeCapgemini Engineering in Canada is seeking a highly skilled Software/Data Engineer to build scalable data processing solutions and maintain data pipelines using Apache, SQL, Python, and Scala.You will optimize ETL workloads, work with Parquet/Hive/HDFS, and collaborate across data, software, and product teams to deliver data-driven solutions. Remote work in...
-
Senior Data Architect
3 days ago
canada | ca Rackspace Technology Full-time6 days ago Be among the first 25 applicantsAs a full spectrum AWS integrator, we assist hundreds of companies to realize the value, efficiency, and productivity of the cloud. We take customers on their journey to enable, operate, and innovate using cloud technologies – from migration strategy to operational excellence and immersive transformation.If you...
-
Data Tech Lead
1 day ago
US/ Canada Mitra Innovation Full-timeLead / Principal Data Engineer – Snowflake & dbtLocation: Remote (Offshore / Onshore)Experience: 12–15 years preferredEngagement: ContractProject: Data Engineering & Modernisation – (Client Based in USA)Job OverviewWe are seeking a highly experienced, hands-on Lead / Principal Data Engineer with deep expertise in Snowflake, dbt, advanced SQL, and...
-
Big Data Specialist Hadoop Spark Scala
4 days ago
canada | ca LTM Full-timeJoin LTIMindtree as a Big Data Specialist, focusing on Hadoop, Spark, and Scala. Utilize your expertise in data application development to create scalable solutions for large data sets. With 3 to 5 years of experience, you will play a critical role in designing and developing big data applications. Your proficiency in Scala, along with your ability to work...
-
Data Integration Lead
4 days ago
Canada Hexaware Full-timeDescriptionHands-on experience with Azure Databricks for data processing, data engineering, or analytics workflows. o Experience in pyspark, ADF, o Python experience o Hands-on experience with SQL, Specialized in modernizing legacy on-premise systems into cloud-native architectures using Azure Data Factory, Databricks and PySpark, resulting in significant...
-
Account Director
3 days ago
canada | ca Storyteller Full-timeA dynamic B2B SaaS company is seeking an experienced Account Director to manage a key client in US sports. The role is fully remote within Canada and involves leading client relationships, ensuring exceptional service, and driving account performance. Ideal candidates should possess strong account management experience, exceptional communication skills, and...
-
Senior Software Engineer: Big Data Spark
4 days ago
canada | ca Capgemini Engineering Full-timeA leading engineering consultancy in Canada is looking for a Senior Software Engineer specialized in Java and Spark. In this role, you will work on a large social media platform processing petabytes of data. The position involves optimizing data systems, designing solutions, and collaborating with cross-functional teams. Ideal candidates should have over 5...
-
Data Governance, Lead
3 days ago
canada | ca Interac Corp. Full-timeWho We Are:Every transaction matters. Every Canadian matters. At Interac, we protect both — driving trust, security, and inclusion, so our digital economy thrives. Founded in 1984, Interac connects Canadians through secure digital payments, advanced identity verification and industry-leading fraud protection. Connecting banks, businesses, and individuals,...
-
Lead Data Engineer
3 days ago
canada | ca FalconSmartIT Full-timeJob Title: Lead Data EngineerLocation: Canada (Remote)Job Type: C2CJob Description:Required QualificationsExperience:8+ years of experience in data engineering, software engineering, or enterprisedata platform development.3+ years of technical leadership experience leading engineering teams or large-scale data initiatives.Proven experience designing and...
-
Lead Data Engineer
3 days ago
southwestern ontario, on, canada | ca, Ontario Sidian Inc. Full-timeSidian is building the company brain for engineering firms. As Lead Data Engineer, you will design and build pipelines that ingest drawings, BIM files, PDFs, spreadsheets, documents, emails, and project history, then unify them in a structured, queryable knowledge graph. What you will own Build multimodal ingestion pipelines for engineering data. Architect...