Azure Data Engineer

2 weeks ago


Toronto, Ontario, Canada Eviden Full time
Role : Azure Data Engineer (Databricks Expert) Location : Toronto, ON
Hire Type : Full-Time with Eviden

We are seeking a highly skilled and experienced Senior Azure Data Engineer to join team and tackle critical performance challenges within Azure data platform. You will play a pivotal role in optimizing and re-architecting key components of data infrastructure, directly impacting the performance and scalability of core data marts. this is a chance to make a significant impact by redesigning and implementing solutions for a high-visibility project.
Performance Optimization: Lead the performance tuning and optimization of Azure-based data pipelines, specifically addressing the current bottleneck where processing 1000 records takes 40 minutes. This requires a deep understanding of Azure Data Factory (ADF), Databricks (Spark), and data processing best practices.
Evaluate the existing Azure data platform architecture, identify bottlenecks and design flaws, and propose and implement solutions for a more efficient and scalable system. This will involve working with Delta tables and optimizing data storage and retrieval.
Utilize your advanced PySpark skills, particularly with structured streaming, to re-engineer data transformation and processing logic within Databricks. The ideal candidate will have a proven track record of optimizing Spark jobs for performance.
Data Integration: Work with Azure Data Lake Storage (ADLS), Azure Data Factory (ADF), Azure Database (DB), and file exchange processes (batch mode) to ensure seamless data flow and integration.
MDM Integration (Desirable): Contribute to the ongoing development and optimization of Master Data Management (MDM) system, which follows a hub-and-spoke architecture. Experience with large-scale MDM implementations is highly desirable.
Collaboration: Work closely with other engineers, business stakeholders, and vendors to understand requirements, communicate progress, and ensure successful project delivery. Minimum experience of implementing 2 projects on Azure data platform
~ A proven track record of success in optimizing and scaling Azure data platforms, particularly with Databricks and Spark. We are looking for an expert who can quickly diagnose and resolve complex performance issues.
~ Extensive experience with Azure Data Factory (ADF), Azure Data Lake Storage (ADLS), Azure Synapse, Event hub, and other relevant Azure services. Demonstrated ability to write and optimize complex Spark queries for performance.
~ Data Warehousing and MDM: Solid understanding of data warehousing principles and experience with Master Data Management (MDM) implementations, preferably with a hub-and-spoke architecture.
~ Problem-Solving Skills: Exceptional analytical and problem-solving skills, with the ability to identify root causes of performance bottlenecks and design effective solutions.
~ Excellent communication and collaboration skills, with the ability to work effectively 1 with both technical and non-technical 2 stakeholders.
  • Azure Data Engineer

    3 weeks ago


    Toronto, Ontario, Canada Diverse Lynx Full time

    Azure Data Engineer(Hybrid) TorontoTech lead: Experienced Azure Data Engineer Lead, Data Modeler, and Solution Designer. This hybrid role will combine leadership, data engineering, and solution design responsibilities. The ideal candidate will have extensive expertise in building and managing cloud-based data solutions, designing complex data models, and...

  • Azure Data Engineer

    4 days ago


    Toronto, Ontario, Canada Diverse Lynx Full time

    Azure Data Engineer(Hybrid) TorontoTech lead: Experienced Azure Data Engineer Lead, Data Modeler, and Solution Designer. This hybrid role will combine leadership, data engineering, and solution design responsibilities. The ideal candidate will have extensive expertise in building and managing cloud-based data solutions, designing complex data models, and...

  • Azure Data Engineer

    4 weeks ago


    Toronto, Ontario, Canada Eviden Full time

    Role : Azure Data Engineer (Databricks Expert)Location : Toronto, ONHire Type : Full-Time with EvidenJob Description:About the Role:We are seeking a highly skilled and experienced Senior Azure Data Engineer to join team and tackle critical performance challenges within Azure data platform. You will play a pivotal role in optimizing and re-architecting key...

  • Azure Data Engineer

    2 weeks ago


    Toronto, Ontario, Canada Eviden Full time

    Role : Azure Data Engineer (Databricks Expert) Location : Toronto, ON Hire Type : Full-Time with Eviden Job Description: About the Role: We are seeking a highly skilled and experienced Senior Azure Data Engineer to join team and tackle critical performance challenges within Azure data platform. You will play a pivotal role in optimizing and...

  • Azure Data Engineer

    2 weeks ago


    Toronto, Ontario, Canada BetasharpInc Full time

    BetasharpInc is seeking an experienced Azure Data Engineer to lead the development and implementation of a scalable data hub built on the Azure stack.The ideal candidate should possess a strong background in data engineering, cloud data platforms, and project management.Key Responsibilities:Develop and implement data ingestion, transformation, and...

  • Azure Data Engineer

    3 weeks ago


    Toronto, Ontario, Canada Themesoft Inc. Full time

    Position : Azure Data Engineer Location : Toronto, On – Hybrid Job Description : Lead the analysis and interpretation of complex data to support business objectives. Oversee the development and implementation of data models using Azure Databricks. Provide expertise in Databricks SQL to optimize data queries and enhance data processing...

  • Azure Data Engineer

    3 weeks ago


    Toronto, Ontario, Canada Themesoft Inc. Full time

    Position: Azure Data EngineerLocation: Toronto, On – HybridJob Description:Lead the analysis and interpretation of complex data to support business objectives.Oversee the development and implementation of data models using Azure Databricks.Provide expertise in Databricks SQL to optimize data queries and enhance data processing efficiency.Develop and manage...


  • Toronto, Ontario, Canada Tiger Analytics, LLC Full time

    Data Engineering and Architecture:Tiger Analytics is a leading provider of AI and analytics solutions. We are looking for a skilled Azure Data Engineer Role to join our team. As a Principal Data Engineer (Azure), you will have hands-on experience working on Azure as cloud, Databricks and some exposure/experience on Data Modelling. You will build and learn...


  • Toronto, Ontario, Canada Bevertec Full time

    Job DescriptionDataOps/Cloud Data Engineer – Senior Candidate is required to come in office 3 days a week.Expert experience with Microsoft Azure technologies, including Azure pipeline, DACPAC, storage account, disk, file share, Azure key vault, Azure portal, virtual machine, Azure DevOps, Data Lake, Data Lakehouse, Azure monitoring tools.Experience with...


  • Toronto, Ontario, Canada Manulife Financial Full time

    About the OpportunityWe are seeking a highly skilled Associate Data Engineer to join our Canadian data office at Manulife Financial. As an Associate Data Engineer, you will play a vital role in enabling the building, maintenance, enhancements, and support of new and existing Azure data pipelines for various analytical use cases.Main...


  • Toronto, Ontario, Canada Compunnel Inc. Full time

    Compunnel Inc. is seeking an experienced Senior Data Engineer to lead our Azure Data Engineering efforts. This role involves designing and developing data warehouses and ETL tools, focusing on Azure Data Engineering for our client in the financial service industry.Role OverviewThis position requires strong expertise in Azure Databricks, Azure Data Lake,...


  • Toronto, Ontario, Canada Atlantis IT Group Full time

    Atlantis IT Group is seeking an experienced Microsoft Azure Data Engineer to join our team.In this role, you will be responsible for building, testing, and deploying scalable data solutions on Microsoft Azure, utilizing technologies such as Azure Data Factory, Azure Storage, and Azure Synapse Analytics.You will work closely with development teams to ensure...


  • Toronto, Ontario, Canada Compunnel Inc. Full time

    Typical day in the role/Responsibilities: Hands-on technical skills in developing batch and real-time data pipelines, DevOps pipelines, and end-to-end processes in Microsoft Azure data and analytics PaaS services. Lead the agile software development process, including roadmap planning, cross-team coordination, task assignments, code reviews, sprint demos,...


  • Toronto, Ontario, Canada AXIS Capital Full time

    This is your opportunity to join AXIS Capital – a trusted global provider of specialty lines insurance and reinsurance. We stand apart for our outstanding client service, intelligent risk taking and superior risk adjusted returns for our shareholders. We also proudly maintain an entrepreneurial, disciplined and ethical corporate culture. As a member of...


  • Toronto, Ontario, Canada AXIS Capital Full time

    This is your opportunity to join AXIS Capital – a trusted global provider of specialty lines insurance and reinsurance. We stand apart for our outstanding client service, intelligent risk taking and superior risk adjusted returns for our shareholders. We also proudly maintain an entrepreneurial, disciplined and ethical corporate culture. As a member of...


  • Toronto, Ontario, Canada Recrute Action Full time

    Senior Azure Data Engineer We are hiring for two exciting opportunities with our client in the insurance industry, supporting high-impact data modernization initiatives. These roles offer the chance to lead the development of scalable, cloud-based data pipelines and platforms in a collaborative, forward-thinking environment. If you are passionate about...


  • Toronto, Ontario, Canada Tata Consultancy Services Full time

    Job DescriptionThis position involves working as a Data Engineer with a focus on big data processing and/or relational databases with a good understanding of SQL. The successful candidate will have experience working with Microsoft Azure Data Platform, specifically Azure SQL Database, Azure Data Factory, ADLS storage account, Azure Data Bricks, and...


  • Toronto, Ontario, Canada Rubicon Path Full time

    **Job Title:** Azure Data Solutions Engineer - LeadRubicon Path is a forward-thinking company seeking an experienced Azure Data Solutions Engineer - Lead to lead the design, development, and implementation of cloud-based data solutions using Microsoft Azure technologies.Responsibilities:Oversee the development of cloud data pipelines using Azure...

  • Azure Data Engineer

    2 weeks ago


    Toronto, Ontario, Canada Navacord Full time

    Professional Development Courses & Designations100% tuition re-imbursement for business relevant courses and trainingHybrid remote work flexibilityA volunteer day to make a difference and give back to your communityVary days – including a day off for your birthdayPOSITION OVERVIEW:Navacord is hiring technical talent in the insurance industry We are...


  • Toronto, Ontario, Canada Tata Consultancy Services Full time

    Data Engineering Expert on Azure PlatformWe are looking for a highly skilled Data Engineering Expert on Azure Platform who can design, develop, and deploy scalable and secure data architectures on Azure. The ideal candidate will have experience working with cloud-based data platforms such as Azure Synapse Analytics, Azure Cosmos DB, and Azure Data Lake...