Azure Data Engineer
2 weeks ago
Hire Type : Full-Time with Eviden
We are seeking a highly skilled and experienced Senior Azure Data Engineer to join team and tackle critical performance challenges within Azure data platform. You will play a pivotal role in optimizing and re-architecting key components of data infrastructure, directly impacting the performance and scalability of core data marts. this is a chance to make a significant impact by redesigning and implementing solutions for a high-visibility project.
Performance Optimization: Lead the performance tuning and optimization of Azure-based data pipelines, specifically addressing the current bottleneck where processing 1000 records takes 40 minutes. This requires a deep understanding of Azure Data Factory (ADF), Databricks (Spark), and data processing best practices.
Evaluate the existing Azure data platform architecture, identify bottlenecks and design flaws, and propose and implement solutions for a more efficient and scalable system. This will involve working with Delta tables and optimizing data storage and retrieval.
Utilize your advanced PySpark skills, particularly with structured streaming, to re-engineer data transformation and processing logic within Databricks. The ideal candidate will have a proven track record of optimizing Spark jobs for performance.
Data Integration: Work with Azure Data Lake Storage (ADLS), Azure Data Factory (ADF), Azure Database (DB), and file exchange processes (batch mode) to ensure seamless data flow and integration.
MDM Integration (Desirable): Contribute to the ongoing development and optimization of Master Data Management (MDM) system, which follows a hub-and-spoke architecture. Experience with large-scale MDM implementations is highly desirable.
Collaboration: Work closely with other engineers, business stakeholders, and vendors to understand requirements, communicate progress, and ensure successful project delivery. Minimum experience of implementing 2 projects on Azure data platform
~ A proven track record of success in optimizing and scaling Azure data platforms, particularly with Databricks and Spark. We are looking for an expert who can quickly diagnose and resolve complex performance issues.
~ Extensive experience with Azure Data Factory (ADF), Azure Data Lake Storage (ADLS), Azure Synapse, Event hub, and other relevant Azure services. Demonstrated ability to write and optimize complex Spark queries for performance.
~ Data Warehousing and MDM: Solid understanding of data warehousing principles and experience with Master Data Management (MDM) implementations, preferably with a hub-and-spoke architecture.
~ Problem-Solving Skills: Exceptional analytical and problem-solving skills, with the ability to identify root causes of performance bottlenecks and design effective solutions.
~ Excellent communication and collaboration skills, with the ability to work effectively 1 with both technical and non-technical 2 stakeholders.
-
Azure Data Engineer
3 weeks ago
Toronto, Ontario, Canada Diverse Lynx Full timeAzure Data Engineer(Hybrid) TorontoTech lead: Experienced Azure Data Engineer Lead, Data Modeler, and Solution Designer. This hybrid role will combine leadership, data engineering, and solution design responsibilities. The ideal candidate will have extensive expertise in building and managing cloud-based data solutions, designing complex data models, and...
-
Azure Data Engineer
4 days ago
Toronto, Ontario, Canada Diverse Lynx Full timeAzure Data Engineer(Hybrid) TorontoTech lead: Experienced Azure Data Engineer Lead, Data Modeler, and Solution Designer. This hybrid role will combine leadership, data engineering, and solution design responsibilities. The ideal candidate will have extensive expertise in building and managing cloud-based data solutions, designing complex data models, and...
-
Azure Data Engineer
4 weeks ago
Toronto, Ontario, Canada Eviden Full timeRole : Azure Data Engineer (Databricks Expert)Location : Toronto, ONHire Type : Full-Time with EvidenJob Description:About the Role:We are seeking a highly skilled and experienced Senior Azure Data Engineer to join team and tackle critical performance challenges within Azure data platform. You will play a pivotal role in optimizing and re-architecting key...
-
Azure Data Engineer
2 weeks ago
Toronto, Ontario, Canada Eviden Full timeRole : Azure Data Engineer (Databricks Expert) Location : Toronto, ON Hire Type : Full-Time with Eviden Job Description: About the Role: We are seeking a highly skilled and experienced Senior Azure Data Engineer to join team and tackle critical performance challenges within Azure data platform. You will play a pivotal role in optimizing and...
-
Azure Data Engineer
2 weeks ago
Toronto, Ontario, Canada BetasharpInc Full timeBetasharpInc is seeking an experienced Azure Data Engineer to lead the development and implementation of a scalable data hub built on the Azure stack.The ideal candidate should possess a strong background in data engineering, cloud data platforms, and project management.Key Responsibilities:Develop and implement data ingestion, transformation, and...
-
Azure Data Engineer
3 weeks ago
Toronto, Ontario, Canada Themesoft Inc. Full timePosition : Azure Data Engineer Location : Toronto, On – Hybrid Job Description : Lead the analysis and interpretation of complex data to support business objectives. Oversee the development and implementation of data models using Azure Databricks. Provide expertise in Databricks SQL to optimize data queries and enhance data processing...
-
Azure Data Engineer
3 weeks ago
Toronto, Ontario, Canada Themesoft Inc. Full timePosition: Azure Data EngineerLocation: Toronto, On – HybridJob Description:Lead the analysis and interpretation of complex data to support business objectives.Oversee the development and implementation of data models using Azure Databricks.Provide expertise in Databricks SQL to optimize data queries and enhance data processing efficiency.Develop and manage...
-
Azure Data Engineer Role
5 days ago
Toronto, Ontario, Canada Tiger Analytics, LLC Full timeData Engineering and Architecture:Tiger Analytics is a leading provider of AI and analytics solutions. We are looking for a skilled Azure Data Engineer Role to join our team. As a Principal Data Engineer (Azure), you will have hands-on experience working on Azure as cloud, Databricks and some exposure/experience on Data Modelling. You will build and learn...
-
Senior Data Engineer
1 week ago
Toronto, Ontario, Canada Bevertec Full timeJob DescriptionDataOps/Cloud Data Engineer – Senior Candidate is required to come in office 3 days a week.Expert experience with Microsoft Azure technologies, including Azure pipeline, DACPAC, storage account, disk, file share, Azure key vault, Azure portal, virtual machine, Azure DevOps, Data Lake, Data Lakehouse, Azure monitoring tools.Experience with...
-
Data Engineer for Azure Solutions
2 weeks ago
Toronto, Ontario, Canada Manulife Financial Full timeAbout the OpportunityWe are seeking a highly skilled Associate Data Engineer to join our Canadian data office at Manulife Financial. As an Associate Data Engineer, you will play a vital role in enabling the building, maintenance, enhancements, and support of new and existing Azure data pipelines for various analytical use cases.Main...
-
Azure Data Engineering Lead Opportunity
4 days ago
Toronto, Ontario, Canada Compunnel Inc. Full timeCompunnel Inc. is seeking an experienced Senior Data Engineer to lead our Azure Data Engineering efforts. This role involves designing and developing data warehouses and ETL tools, focusing on Azure Data Engineering for our client in the financial service industry.Role OverviewThis position requires strong expertise in Azure Databricks, Azure Data Lake,...
-
Microsoft Azure Data Engineer
2 weeks ago
Toronto, Ontario, Canada Atlantis IT Group Full timeAtlantis IT Group is seeking an experienced Microsoft Azure Data Engineer to join our team.In this role, you will be responsible for building, testing, and deploying scalable data solutions on Microsoft Azure, utilizing technologies such as Azure Data Factory, Azure Storage, and Azure Synapse Analytics.You will work closely with development teams to ensure...
-
Senior Azure Data Engineer
1 week ago
Toronto, Ontario, Canada Compunnel Inc. Full timeTypical day in the role/Responsibilities: Hands-on technical skills in developing batch and real-time data pipelines, DevOps pipelines, and end-to-end processes in Microsoft Azure data and analytics PaaS services. Lead the agile software development process, including roadmap planning, cross-team coordination, task assignments, code reviews, sprint demos,...
-
Senior Azure Data Engineer
2 weeks ago
Toronto, Ontario, Canada AXIS Capital Full timeThis is your opportunity to join AXIS Capital – a trusted global provider of specialty lines insurance and reinsurance. We stand apart for our outstanding client service, intelligent risk taking and superior risk adjusted returns for our shareholders. We also proudly maintain an entrepreneurial, disciplined and ethical corporate culture. As a member of...
-
Senior Azure Data Engineer
1 week ago
Toronto, Ontario, Canada AXIS Capital Full timeThis is your opportunity to join AXIS Capital – a trusted global provider of specialty lines insurance and reinsurance. We stand apart for our outstanding client service, intelligent risk taking and superior risk adjusted returns for our shareholders. We also proudly maintain an entrepreneurial, disciplined and ethical corporate culture. As a member of...
-
Senior Azure Data Engineer
3 weeks ago
Toronto, Ontario, Canada Recrute Action Full timeSenior Azure Data Engineer We are hiring for two exciting opportunities with our client in the insurance industry, supporting high-impact data modernization initiatives. These roles offer the chance to lead the development of scalable, cloud-based data pipelines and platforms in a collaborative, forward-thinking environment. If you are passionate about...
-
Azure Big Data Engineer
6 days ago
Toronto, Ontario, Canada Tata Consultancy Services Full timeJob DescriptionThis position involves working as a Data Engineer with a focus on big data processing and/or relational databases with a good understanding of SQL. The successful candidate will have experience working with Microsoft Azure Data Platform, specifically Azure SQL Database, Azure Data Factory, ADLS storage account, Azure Data Bricks, and...
-
Azure Data Solutions Engineer
4 days ago
Toronto, Ontario, Canada Rubicon Path Full time**Job Title:** Azure Data Solutions Engineer - LeadRubicon Path is a forward-thinking company seeking an experienced Azure Data Solutions Engineer - Lead to lead the design, development, and implementation of cloud-based data solutions using Microsoft Azure technologies.Responsibilities:Oversee the development of cloud data pipelines using Azure...
-
Azure Data Engineer
2 weeks ago
Toronto, Ontario, Canada Navacord Full timeProfessional Development Courses & Designations100% tuition re-imbursement for business relevant courses and trainingHybrid remote work flexibilityA volunteer day to make a difference and give back to your communityVary days – including a day off for your birthdayPOSITION OVERVIEW:Navacord is hiring technical talent in the insurance industry We are...
-
Toronto, Ontario, Canada Tata Consultancy Services Full timeData Engineering Expert on Azure PlatformWe are looking for a highly skilled Data Engineering Expert on Azure Platform who can design, develop, and deploy scalable and secure data architectures on Azure. The ideal candidate will have experience working with cloud-based data platforms such as Azure Synapse Analytics, Azure Cosmos DB, and Azure Data Lake...