Azure Data Engineer
2 weeks ago
Hire Type : Full-Time with Eviden
We are seeking a highly skilled and experienced Senior Azure Data Engineer to join team and tackle critical performance challenges within Azure data platform. You will play a pivotal role in optimizing and re-architecting key components of data infrastructure, directly impacting the performance and scalability of core data marts. this is a chance to make a significant impact by redesigning and implementing solutions for a high-visibility project.
Performance Optimization: Lead the performance tuning and optimization of Azure-based data pipelines, specifically addressing the current bottleneck where processing 1000 records takes 40 minutes. This requires a deep understanding of Azure Data Factory (ADF), Databricks (Spark), and data processing best practices.
Evaluate the existing Azure data platform architecture, identify bottlenecks and design flaws, and propose and implement solutions for a more efficient and scalable system. This will involve working with Delta tables and optimizing data storage and retrieval.
Utilize your advanced PySpark skills, particularly with structured streaming, to re-engineer data transformation and processing logic within Databricks. The ideal candidate will have a proven track record of optimizing Spark jobs for performance.
Data Integration: Work with Azure Data Lake Storage (ADLS), Azure Data Factory (ADF), Azure Database (DB), and file exchange processes (batch mode) to ensure seamless data flow and integration.
MDM Integration (Desirable): Contribute to the ongoing development and optimization of Master Data Management (MDM) system, which follows a hub-and-spoke architecture. Experience with large-scale MDM implementations is highly desirable.
Collaboration: Work closely with other engineers, business stakeholders, and vendors to understand requirements, communicate progress, and ensure successful project delivery. Minimum experience of implementing 2 projects on Azure data platform
~ A proven track record of success in optimizing and scaling Azure data platforms, particularly with Databricks and Spark. We are looking for an expert who can quickly diagnose and resolve complex performance issues.
~ Extensive experience with Azure Data Factory (ADF), Azure Data Lake Storage (ADLS), Azure Synapse, Event hub, and other relevant Azure services. Demonstrated ability to write and optimize complex Spark queries for performance.
~ Data Warehousing and MDM: Solid understanding of data warehousing principles and experience with Master Data Management (MDM) implementations, preferably with a hub-and-spoke architecture.
~ Problem-Solving Skills: Exceptional analytical and problem-solving skills, with the ability to identify root causes of performance bottlenecks and design effective solutions.
~ Excellent communication and collaboration skills, with the ability to work effectively 1 with both technical and non-technical 2 stakeholders.
-
Azure Data Engineer
2 weeks ago
Toronto, Ontario, Canada Diverse Lynx Full timeAzure Data Engineer(Hybrid) TorontoTech lead: Experienced Azure Data Engineer Lead, Data Modeler, and Solution Designer. This hybrid role will combine leadership, data engineering, and solution design responsibilities. The ideal candidate will have extensive expertise in building and managing cloud-based data solutions, designing complex data models, and...
-
Azure Data Engineer
3 weeks ago
Toronto, Ontario, Canada Eviden Full timeRole : Azure Data Engineer (Databricks Expert)Location : Toronto, ONHire Type : Full-Time with EvidenJob Description:About the Role:We are seeking a highly skilled and experienced Senior Azure Data Engineer to join team and tackle critical performance challenges within Azure data platform. You will play a pivotal role in optimizing and re-architecting key...
-
Azure Data Engineer
2 weeks ago
Toronto, Ontario, Canada Eviden Full timeRole : Azure Data Engineer (Databricks Expert) Location : Toronto, ON Hire Type : Full-Time with Eviden Job Description: About the Role: We are seeking a highly skilled and experienced Senior Azure Data Engineer to join team and tackle critical performance challenges within Azure data platform. You will play a pivotal role in optimizing and...
-
Azure Data Engineer
1 week ago
Toronto, Ontario, Canada BetasharpInc Full timeBetasharpInc is seeking an experienced Azure Data Engineer to lead the development and implementation of a scalable data hub built on the Azure stack.The ideal candidate should possess a strong background in data engineering, cloud data platforms, and project management.Key Responsibilities:Develop and implement data ingestion, transformation, and...
-
Azure Data Engineer
3 weeks ago
Toronto, Ontario, Canada Themesoft Inc. Full timePosition : Azure Data Engineer Location : Toronto, On – Hybrid Job Description : Lead the analysis and interpretation of complex data to support business objectives. Oversee the development and implementation of data models using Azure Databricks. Provide expertise in Databricks SQL to optimize data queries and enhance data processing...
-
Azure Data Engineer
2 weeks ago
Toronto, Ontario, Canada Themesoft Inc. Full timePosition: Azure Data EngineerLocation: Toronto, On – HybridJob Description:Lead the analysis and interpretation of complex data to support business objectives.Oversee the development and implementation of data models using Azure Databricks.Provide expertise in Databricks SQL to optimize data queries and enhance data processing efficiency.Develop and manage...
-
Senior Data Engineer
3 days ago
Toronto, Ontario, Canada Bevertec Full timeJob DescriptionDataOps/Cloud Data Engineer – Senior Candidate is required to come in office 3 days a week.Expert experience with Microsoft Azure technologies, including Azure pipeline, DACPAC, storage account, disk, file share, Azure key vault, Azure portal, virtual machine, Azure DevOps, Data Lake, Data Lakehouse, Azure monitoring tools.Experience with...
-
Data Engineer for Azure Solutions
6 days ago
Toronto, Ontario, Canada Manulife Financial Full timeAbout the OpportunityWe are seeking a highly skilled Associate Data Engineer to join our Canadian data office at Manulife Financial. As an Associate Data Engineer, you will play a vital role in enabling the building, maintenance, enhancements, and support of new and existing Azure data pipelines for various analytical use cases.Main...
-
Microsoft Azure Data Engineer
1 week ago
Toronto, Ontario, Canada Atlantis IT Group Full timeAtlantis IT Group is seeking an experienced Microsoft Azure Data Engineer to join our team.In this role, you will be responsible for building, testing, and deploying scalable data solutions on Microsoft Azure, utilizing technologies such as Azure Data Factory, Azure Storage, and Azure Synapse Analytics.You will work closely with development teams to ensure...
-
Senior Azure Data Engineer
2 days ago
Toronto, Ontario, Canada Compunnel Inc. Full timeTypical day in the role/Responsibilities: Hands-on technical skills in developing batch and real-time data pipelines, DevOps pipelines, and end-to-end processes in Microsoft Azure data and analytics PaaS services. Lead the agile software development process, including roadmap planning, cross-team coordination, task assignments, code reviews, sprint demos,...
-
Senior Azure Data Engineer
1 week ago
Toronto, Ontario, Canada AXIS Capital Full timeThis is your opportunity to join AXIS Capital – a trusted global provider of specialty lines insurance and reinsurance. We stand apart for our outstanding client service, intelligent risk taking and superior risk adjusted returns for our shareholders. We also proudly maintain an entrepreneurial, disciplined and ethical corporate culture. As a member of...
-
Senior Azure Data Engineer
3 days ago
Toronto, Ontario, Canada AXIS Capital Full timeThis is your opportunity to join AXIS Capital – a trusted global provider of specialty lines insurance and reinsurance. We stand apart for our outstanding client service, intelligent risk taking and superior risk adjusted returns for our shareholders. We also proudly maintain an entrepreneurial, disciplined and ethical corporate culture. As a member of...
-
Senior Azure Data Engineer
2 weeks ago
Toronto, Ontario, Canada Recrute Action Full timeSenior Azure Data Engineer We are hiring for two exciting opportunities with our client in the insurance industry, supporting high-impact data modernization initiatives. These roles offer the chance to lead the development of scalable, cloud-based data pipelines and platforms in a collaborative, forward-thinking environment. If you are passionate about...
-
Azure Big Data Engineer
1 day ago
Toronto, Ontario, Canada Tata Consultancy Services Full timeJob DescriptionThis position involves working as a Data Engineer with a focus on big data processing and/or relational databases with a good understanding of SQL. The successful candidate will have experience working with Microsoft Azure Data Platform, specifically Azure SQL Database, Azure Data Factory, ADLS storage account, Azure Data Bricks, and...
-
Azure Data Engineer
7 days ago
Toronto, Ontario, Canada Navacord Full timeProfessional Development Courses & Designations100% tuition re-imbursement for business relevant courses and trainingHybrid remote work flexibilityA volunteer day to make a difference and give back to your communityVary days – including a day off for your birthdayPOSITION OVERVIEW:Navacord is hiring technical talent in the insurance industry We are...
-
Azure Data Engineer Specialist
18 hours ago
Toronto, Ontario, Canada Bevertec Full timeData and Analytics at BevertecWe are looking for a highly skilled Azure Data Engineer Specialist to join our data and analytics team. As a key member of our team, you will be responsible for designing, building, and configuring data pipelines and architectures using Azure Synapse, Data Lakes, Warehouses, and other related services.The ideal candidate will...
-
Data Engineering Expert on Azure
14 hours ago
Toronto, Ontario, Canada Tata Consultancy Services Full timeAzure Data Platform SpecialistWe are seeking an experienced Cloud Data Architect to join our team. The successful candidate will be responsible for providing technical pre-sales enablement covering data architecture, data engineering, data modelling, data consumption, data platform migration, and data governance focusing on Azure data platform.About the...
-
Data Scientist and Azure Engineer
4 days ago
Toronto, Ontario, Canada Manulife Financial Full timeJob Description:The OpportunityManulife Financial is seeking a highly skilled data engineer to join its Canadian data office in Toronto, Ontario. This exciting role offers the chance to enable the building, maintenance, enhancements, and support of new and existing Azure data pipelines for various analytical use cases, inclusive of AI/ML/BI/GAN and...
-
Azure Data Engineer
3 weeks ago
Toronto, Ontario, Canada Navacord Full timeWE OFFER:Great learning and growth opportunitiesA competitive compensation packageFull Benefits PackageRRSP Matching ProgramProfessional Development Courses & Designations100% tuition re-imbursement for business relevant courses and trainingHybrid remote work flexibilityA volunteer day to make a difference and give back to your communityVary days –...
-
Azure Data Engineer
1 week ago
Toronto, Ontario, Canada Tata Consultancy Services Full timeInclusion without Exception:Tata Consultancy Services (TCS) is an equal opportunity employer, and embraces diversity in race, nationality, ethnicity, gender, age, physical ability, neurodiversity, and sexual orientation, to create a workforce that reflects the societies we operate in. Our continued commitment to Culture and Diversity is reflected in our...