Expert Data Engineer for Large-Scale Databricks Solutions

4 days ago


Toronto, Ontario, Canada Rubicon Path Full time
About the Role
Rubicon Path is seeking an experienced Data Engineer to join our team. As a key member of our engineering group, you will design, develop, and implement large-scale data pipelines using Databricks.

Job Description
In this role, you will be responsible for developing and managing ETL pipelines, jobs, and workflows in Databricks. You will work closely with cross-functional teams to ensure the efficient transformation and movement of data to meet business needs. Your expertise in Delta Lake, Databricks Workflows, and Python/PySpark will enable you to automate ETL pipelines and handle Change Data Capture (CDC) for incremental data loads.

Key Responsibilities
  • Develop and manage ETL pipelines, jobs, and workflows in Databricks.
  • Collaborate with cross-functional teams to design, develop, and implement large-scale data pipelines.
  • Automate ETL pipelines using Delta Live Tables and handle Change Data Capture (CDC) for incremental data loads.
  • Ensure the efficient transformation and movement of data to meet business needs.
  • Work closely with data architects and business teams to optimize data quality and performance.

Requirements
To be successful in this role, you will need:
  • 7+ years of experience using ETL tools such as Microsoft SSIS, stored procedures, T-SQL.
  • 2+ years of experience working with Delta Lake, Databricks, and Azure Databricks pipelines.
  • Strong knowledge of Delta Lake for data management and optimization.
  • Familiarity with Databricks Workflows for scheduling and orchestrating tasks.
  • 2+ years of experience working with Python and PySpark.
  • Solid understanding of the Medallion Architecture (Bronze, Silver, Gold) and experience implementing it in production environments.
  • Hands-on experience with CDC tools (e.g., GoldenGate) for managing real-time data.
  • SQL Server, Oracle experience.

Salary Range
$140,000 - $170,000 per year based on experience and qualifications.
  • Technical Director

    5 days ago


    Toronto, Ontario, Canada Databricks Full time

    About DatabricksDatabricks is a leading cloud-based data and AI platform that enables businesses to unlock the full potential of their data. Our team of experts activates and accelerates the value customers get from their data and AI, securely delivering AI's potential at scale.Salary Range: $180,000 - $220,000 per annumJob Description: As a dynamic Solution...


  • Toronto, Ontario, Canada Fluent, LLC Full time

    About FluentAt Fluent, we're revolutionizing the advertising industry with cutting-edge technology and innovative solutions. Our goal is to build a seamless and relevant experience for online shoppers, leveraging machine learning and artificial intelligence to drive business growth.Job OverviewWe're seeking an experienced Data Engineer to join our team,...


  • Toronto, Ontario, Canada Quantum World Technologies Inc. Full time

    Quantum World Technologies Inc. is looking for a skilled Big Data Engineering Expert to join our team in Toronto, ON.We offer a competitive salary of $120,000 per year, plus benefits.About the Role:The successful candidate will have 6-12 years of experience in big data engineering and will be responsible for designing, implementing, and maintaining...

  • Databricks Engineer

    2 weeks ago


    Toronto, Ontario, C6A, Ontario, Canada Smart IT Frame LLC Full time

    Hi There,We have a contract role with one of our clients. Please find the below details.Role: Databricks EngineerLocation: Toronto, Ontario (Hybrid)Type: ContractRequired Skills:* Proficiency in Databricks and Apache Spark.* Experience with one or more cloud platforms (Azure Data Lake, AWS S3, GCP BigQuery).* Strong programming skills in Python, Scala, or...

  • Data Engineer

    4 weeks ago


    Toronto, Ontario, Canada Rubicon Path Full time

    About the RoleRubicon Path is seeking an experienced Data Engineer to join our team. As a key member of our engineering team, you will be responsible for designing, developing, and maintaining large-scale data pipelines using Databricks, Delta Lake, and Azure.Key ResponsibilitiesDesign and implement ETL processes using Databricks, Delta Lake, and Azure Data...

  • Data Engineer

    2 days ago


    Toronto, Ontario, Canada Capgemini Engineering Full time

    About Capgemini EngineeringCapgemini Engineering is a global leader in engineering and technology consulting. We help organizations accelerate their digital transformation by providing innovative solutions that drive business value.About the JobWe are currently seeking a highly skilled Data Engineer - Large Scale Big Data Infrastructure to join our team. As...


  • Toronto, Ontario, Canada Rubicon Path Full time

    **Job Summary:**We are seeking a highly skilled Senior Data Engineer with expertise in designing, developing, and implementing large-scale data pipelines using Databricks and related tools. As an ETL Specialist at Rubicon Path, you will play a critical role in ingesting, transforming, and storing large-scale datasets for our business.Key...


  • Toronto, Ontario, Canada Rubicon Path Full time

    Rubicon Path Job DescriptionJob Title: Data Engineering Specialist - Cloud Data ArchitectureEstimated Salary Range: $140,000 - $200,000 per yearJob SummaryRubicon Path is seeking an experienced Data Engineering Specialist to join our team. As a key member of our cloud data architecture team, you will be responsible for designing, developing, and maintaining...

  • Databricks Developer

    4 weeks ago


    Toronto, Ontario, Ontario, Canada Tata Consultancy Services Full time

    Inclusion without Exception:Tata Consultancy Services (TCS) is an equal opportunity employer, and embraces diversity in race, nationality, ethnicity, gender, age, physical ability, neurodiversity, and sexual orientation, to create a workforce that reflects the societies we operate in. Our continued commitment to Culture and Diversity is reflected in our...

  • Data Engineer

    2 days ago


    Toronto, Ontario, Canada Strategic Solutions Technology Staffing Inc. Full time

    About Our ClientStrategic Solutions Technology Staffing Inc. is a multi-strategy alternative asset manager with global expertise in private equity, venture capital, credit, and more.We are seeking an experienced Data Engineer to join our data-driven team. The ideal candidate will have a strong background in building and managing large-scale data pipelines...


  • Toronto, Ontario, Canada Rubicon Path Full time

    Job Summary: We are seeking an experienced Enterprise Architecture Expert to join our team at Rubicon Path. As a key member of our architecture team, you will be responsible for designing and implementing large-scale solutions that meet the complex needs of our clients.About Us: Rubicon Path is a leading provider of innovative technology solutions. Our team...


  • Toronto, Ontario, Canada NearSource Full time

    About NearSource TechnologiesWe're a forward-thinking company driving innovation in data-driven decision-making across our digital platform.Job SummaryThis is a unique opportunity to contribute to impactful machine learning initiatives as part of a collaborative team.Compensation and Benefits$120,000 - $160,000 per year, depending on experience. We offer a...


  • Toronto, Ontario, Canada Q1 Technologies, Inc. Full time

    Key ResponsibilitiesWe are seeking an experienced Data Engineer to join our team at Q1 Technologies, Inc. The ideal candidate will have a strong background in cloud-based data engineering and experience with AWS technologies.Key SkillsAWS Glue, Python, AWS Redshift, Airflow, and AWS CloudWatchGood experience in cloud data engineeringThe successful candidate...


  • Toronto, Ontario, Canada S.i. Systems Full time

    S.i. Systems Seeks Experienced Salesforce ExpertAt S.i. Systems, we are committed to delivering exceptional solutions that meet our clients' needs. We are currently seeking a highly skilled Sr. Salesforce Solution Architect with experience in Service Cloud and Experience Cloud to support a large-scale Salesforce implementation.Key Responsibilities:Develop...


  • Toronto, Ontario, Canada Fluent, LLC Full time

    Job Description:A Senior Data Engineer opportunity has arisen at Fluent, LLC, a high-flying performance marketing company. As part of our team, you will play a crucial role in building data products to power our business lines using your expertise in Databricks pipeline development.The successful candidate will be responsible for developing, deploying, and...


  • Toronto, Ontario, Canada Bitstrapped Full time

    Bitstrapped is a fast-growing data engineering and cloud consulting company.We help companies invest in forward-thinking data, machine learning solutions, and cloud-native infrastructure.Job SummaryWe are seeking an experienced Data Engineer to work with our team on developing and deploying large language models.The ideal candidate will have strong expertise...

  • Cloud Data Engineer

    2 days ago


    Toronto, Ontario, Canada Maarut Inc Full time

    Job SummaryMaarut Inc is seeking a highly skilled Cloud Data Engineer - Senior to join our team. The successful candidate will be responsible for designing, developing, and implementing large-scale cloud-based data systems, with a focus on scalability, performance, and reliability.About UsMaarut Inc is a leading provider of innovative solutions in the field...

  • Cloud Data Engineer

    4 weeks ago


    Toronto, Ontario, Canada Tata Consultancy Services Full time

    About the RoleWe are seeking a highly skilled Cloud Data Engineer to join our team in Canada. In this role, you will be responsible for designing, implementing, and managing large-scale data platforms on Azure.Key ResponsibilitiesDesign and develop scalable data pipelines using Azure Data Factory and DatabricksWork with cross-functional teams to implement...


  • Toronto, Ontario, Canada Disability Solutions Full time

    Job OverviewThe position of Data Scientist and Innovator is a key role within our organization, focused on leveraging data science methodologies to drive business growth and improvement. As a member of our team, you will collaborate with cross-functional groups to identify opportunities for innovation and develop strategies for extracting insights from...


  • Toronto, Ontario, Canada Royal Bank of Canada Full time

    About the Role:This is a unique opportunity to lead a team of developers in delivering high-performing cloud-based big data solutions. As a Senior Data Architecture Lead, you will be responsible for designing and implementing data pipelines using big data technologies such as Spark and Databricks on medium to large scale datasets.Key Responsibilities:-...