Senior Databricks Data Engineer

24 hours ago
Toronto ON, Toronto Census Division, ON; Ontario, Canada Raise - find a more meaningful working experience Full-time
Senior Databricks Data Engineer
  • Location: Toronto (Hybrid - minimum 2 days per week onsite at the client's downtown Toronto office)
  • Contract Length: 2 Months Key ResponsibilitiesDatabricks Engineering and Migration
    • Migrate and refactor existing ETL/ELT pipelines, scripts and processing workloads into Databricks.
    • Develop production-quality data pipelines using Python, PySpark and SQL.
    • Translate existing processing and business logic into scalable Databricks-native implementations.
    • Develop and maintain ingestion, transformation and metric-calculation logic across multiple enterprise data sources.
    • Modernize existing workflow and orchestration processes, including scheduled jobs and Airflow/DAG-based processing.
    • Implement reusable engineering patterns to accelerate migration of additional workloads.
    Data Integration
    • Integrate data from enterprise applications, APIs, databases, files and other source systems.
    • Work with source-system SMEs to understand existing data structures, interfaces and processing logic.
    • Implement source-to-target mappings based on approved requirements and target data models.
    • Support integration with enterprise platforms such as ServiceNow and cybersecurity data sources.
    • Troubleshoot connectivity, ingestion, schema and data-quality issues.
    Data Quality, Testing and Reconciliation
    • Develop automated developer tests and data-validation routines.
    • Validate source-to-target transformations and data movement through bronze, silver and gold processing layers where applicable.
    • Perform reconciliation between legacy processing outputs and Databricks-generated results.
    • Identify, document and resolve data defects, processing exceptions and reconciliation discrepancies.
    • Produce clear validation evidence to support business and technical acceptance.
    Engineering Quality and DevOps
    • Develop maintainable, reusable and well-documented production code.
    • Follow enterprise coding, security, architecture and release-management standards.
    • Use Git-based repositories and established CI/CD and deployment processes.
    • Participate in peer reviews and contribute to improving engineering standards and reusable patterns.
    • Support code check-in, deployment readiness, defect remediation and production-transition activities.
    • Monitor and troubleshoot pipeline execution and processing failures.
    Agile Delivery and Collaboration
    • Work within an Agile delivery team executing against two-week sprint commitments.
    • Participate in sprint planning, backlog refinement, technical design sessions, defect triage and sprint reviews.
    • Estimate engineering effort and identify technical dependencies, blockers and risks.
    • Work closely with architects and technical leads to ensure alignment with enterprise architecture and Databricks standards.
    • Collaborate directly with client engineering teams and transfer knowledge throughout the engagement.
    • Produce technical documentation, source-to-target mappings, implementation notes and handover materials.
    Required Skills and Experience
    • 7+ years of experience in data engineering, software engineering or data-platform development.
    • Strong hands-on experience building production workloads on Databricks.
    • Advanced development skills in:
    • Python
    • PySpark / Apache Spark
    • SQL
    • Experience designing and implementing ETL/ELT and large-scale data-processing pipelines.
    • Experience migrating legacy data pipelines or processing applications to modern cloud data platforms.
    • Strong understanding of data ingestion, transformation, modelling, source-to-target mapping, data quality and reconciliation.
    • Experience with workflow orchestration technologies such as Apache Airflow or equivalent.
    • Experience integrating data through APIs, databases, files and enterprise applications.
    • Experience with Git, CI/CD and enterprise software-development practices.
    • Strong troubleshooting skills across data, code, integration and environment issues.
    • Ability to understand existing code and processing logic and efficiently refactor it into a modern architecture.
    • Experience working within structured Agile/Scrum delivery environments.
    • Strong written and verbal communication skills with the ability to work directly with client technical teams.
    • Must be based in the Greater Toronto Area and available to work onsite in downtown Toronto a minimum of two days per week.
    Preferred Qualifications
    • Databricks certification such as Databricks Certified Data Engineer Associate or Professional.
    • Experience delivering d

  • Senior Databricks Data Engineer

    24 hours ago

    Ottawa, ON, Ottawa Census Division, ON; Ontario, Canada Celebal Technologies Full-time

    Shape Databricks and AWS data foundations as a hands-on Senior Data Engineer building scalable pipelines and trusted data products. This senior position transforms complex operational data into reusable products for analytics, AI, machine learning, and self-service decisions. You will develop lakehouse solutions with Spark, SQL, Python, Delta Lake, and...

  • Senior Databricks Data Engineer

    8 hours ago

    Toronto, Ontario, Canada BDO Canada LLP Full-time

    Build production-grade data pipelines with Databricks and Azure at BDO Digital, using a blended office, client-site, and virtual model.As a Senior Data Engineer in Technology Advisory Services, you will lead hands-on engineering and client engagements. Apply Databricks, Delta Lake, Apache Spark, and Azure Data Lake to create scalable data solutions....

  • Senior Databricks Data Engineer

    19 hours ago

    Toronto, ON, Canada Raise - find a more meaningful working experience Full-time €100,000 - €167,000 Contract

    Senior Databricks Data Engineer - Location: Toronto (Hybrid - minimum 2 days per week onsite at the client's downtown Toronto office) - Contract Length: 2 Months Key Responsibilities Databricks Engineering and Migration - Migrate and refactor existing ETL/ELT pipelines, scripts and processing workloads into Databricks. - Develop production-quality data...

  • Senior Databricks Data Engineer

    2 days ago

    Toronto, Ontario, Canada BDO Canada Full-time

    Lead Databricks and Azure data engineering at BDO Digital, building scalable pipelines and guiding client engagements within Technology Advisory Services.As a Senior Data Engineer, you will combine technical leadership with hands-on delivery for production-grade data solutions. Your work spans Databricks, Delta Lake, Apache Spark, Azure Data Lake, and...

  • Senior Azure Databricks Data Engineer

    24 hours ago

    Toronto, ON, Toronto Census Division, ON; Ontario, Canada Recruiting in Motion Full-time

    Build Azure Databricks Lakehouse pipelines as a Senior Data Engineer in the Greater Toronto Area, working four days onsite and one remotely. This senior role develops the production data and AI foundation on Azure Databricks. You will scale Medallion architecture across Bronze, Silver, and Gold layers while optimizing clusters hosted in Azure. The work...

  • Senior Azure Databricks Data Engineer

    19 hours ago

    Toronto, ON, Canada Recruiting in Motion Full-time €120,000 - €140,000 Contract

    Build Azure Databricks Lakehouse pipelines as a Senior Data Engineer in the Greater Toronto Area, working four days onsite and one remotely. This senior role develops the production data and AI foundation on Azure Databricks. You will scale Medallion architecture across Bronze, Silver, and Gold layers while optimizing clusters hosted in Azure. The work...

  • Senior Databricks Data Platform Engineer

    8 hours ago

    Toronto, Ontario, Canada ShyftLabs Full-time

    Lead Databricks data platform architecture at ShyftLabs from our downtown Toronto office three days weekly, using Spark and Python.As a Senior / Lead Data Engineer, you will guide enterprise-scale platforms for Fortune 500 clients. Apply Databricks, Apache Spark, Python, SQL, Delta Lake, and cloud platforms to modernize data environments. Own delivery from...

  • Senior Databricks Data Engineer

    7 days ago

    Toronto, ON, Canada Ian Martin Group Full-time €100,000 - €167,000 Contract

    Senior Databricks Data Engineer - Location: Toronto (Hybrid - minimum 2 days per week onsite at the client's downtown Toronto office) - Contract Length: 2 Months Key Responsibilities Databricks Engineering and Migration - Migrate and refactor existing ETL/ELT pipelines, scripts and processing workloads into Databricks. - Develop production-quality data...

  • Senior AWS Databricks Data Engineer

    2 days ago

    Toronto, ON, Canada Akkodis Full-time $80,000 - $100,000 Contract

    A leading IT consultancy is seeking a Mid-Senior Level Data Engineer for a contract role in Toronto. Candidates will design, develop, and maintain scalable data pipelines, ensuring data quality and collaborating closely with data scientists. Ideal applicants should have 5+ years of experience, strong SQL skills, and proficiency in Databricks. This hybrid...

  • Senior Azure Databricks Data Engineer

    6 days ago

    Toronto, ON, Canada Recruiting In Motion - Eastern Ontario Full-time €120,000 - €140,000 Contract

    Build Azure Databricks Lakehouse pipelines as a Senior Data Engineer in the Greater Toronto Area, working four days onsite and one remotely. This senior role develops the production data and AI foundation on Azure Databricks. You will scale Medallion architecture across Bronze, Silver, and Gold layers while optimizing clusters hosted in Azure. The work...

Similar jobs in Toronto ON

Top cities