Data Integration Expert

7 days ago


Waterloo, Ontario, Canada Mcafee Full time

About the Role

The Senior Data Integration Engineer is responsible for integrating data from various sources, managing ETL/ELT pipelines on Databricks, and ensuring data pipeline scalability and compliance.

  • Integrate data from a variety of source systems and knowledge to tailor integration approaches on a per-system basis.
  • Liaise with Technical Data Steward and understanding business requirements for ingestion and integration.
  • Document ingestion and integration flows and pipelines.
  • Use Airflow to schedule and automate ingestion jobs.
  • Manage metadata and master data in technical data catalog.
  • Manage, maintain, and oversee ETL/ELT pipelines on the Databricks platform.
  • Ensure ELT/ETL design meets required security and compliance guidelines, and ensure PII management, flagging, and risk assessment during ingestion and integration.
  • Conduct pipeline monitoring.
  • Ensure adherence to SDLC best practices.

About You

  • Databricks: Expertise in managing and maintaining ETL/ELT pipelines on the Databricks platform.
  • Spark: Proficiency in using Apache Spark for big data processing and analytics.
  • Python: Strong programming skills in Python for data manipulation, analysis, and automation.
  • SQL: Advanced knowledge of SQL for querying.
  • AWS: Experience with AWS services for data integration and management.
  • Cloud: Familiarity with cloud computing concepts and services.
  • Big Data: Knowledge of big data technologies and frameworks.
  • Data Integration: Experience in integrating data from various source systems and tailoring integration approaches.
  • ETL/ELT Design: Ability to design and optimize ETL/ELT pipelines for scalability and speed.
  • CI/CD: Familiarity with continuous integration and continuous deployment practices.
  • System Integration Tools: Knowledge of tools and techniques for system integration.
  • Data Pipeline Management: Expertise in managing data pipelines and ensuring they meet security and compliance guidelines.
  • Metadata and Master Data Management: Skills in managing metadata and master data in the technical data catalog.


  • Waterloo, Ontario, Canada Mcafee Full time

    Job DescriptionThe Senior Data Integration Engineer will be responsible for designing, developing, and implementing data integration solutions that integrate data from various sources, manage ETL/ELT pipelines on Databricks, and ensure data pipeline scalability and compliance.The role involves collaboration with multiple departments and requires expertise in...


  • Waterloo, Ontario, Canada Huawei Technologies Canada Co., Ltd. Full time

    Huawei Canada has an immediate  permanent  opening for a Technical ExpertAbout the team:The Data and Privacy Protection Technology Lab is dedicated to ensuring user data flows while maintaining privacy. Researchers focus on key areas such as user identity authentication, data integrity, privacy protection, extensive model privacy assessment, multi-modal...


  • Waterloo, Ontario, Canada Sun Life Financial Full time

    About the PositionWe are looking for a highly skilled Operational Data Insights Expert to join our team. As an Operational Data Insights Expert, you will play a crucial role in analyzing and modeling business process data to inform strategic decisions.You will work closely with operational partners to develop capacity plans, including timing of hiring and...


  • Waterloo, Ontario, Canada Mcafee Full time

    Senior Data Integration Engineer page is loadedSenior Data Integration EngineerApply locations Canada, Waterloo Canada time type Full time posted on Posted 17 Days Ago job requisition id JR0031697Role Overview:The Senior Data Integration Engineer is responsible for integrating data from various sources, managing ETL/ELT pipelines on Databricks, and ensuring...


  • Waterloo, Ontario, Canada Verily Full time

    Job OverviewVerily is a subsidiary of Alphabet that uses data to change the way people manage their health and deliver healthcare. We focus on generating and activating data from various sources, including clinical, social, behavioral, and real-world data.Data Governance RoleThe Precision Health Platform is at the core of our research and care. It comprises...


  • Waterloo, Ontario, Canada Tbwa ChiatDay Inc Full time

    Company OverviewCarta is a leading provider of purpose-built software solutions that transform traditional accounting into a powerful growth engine.The company's world-class fund administration platform supports nearly 7,000 funds and SPVs, representing nearly $130 billion in assets under management in venture capital and private equity.With a global...


  • Waterloo, Ontario, Canada ROSS Full time

    ROSS is a leading provider of HR and Recruitment solutions, and we are seeking an experienced Integration Specialist to join our team. The successful candidate will be responsible for implementing and integrating our client's software solutions, utilizing their knowledge of industry-standard protocols and technical expertise.About ROSSAt ROSS, we pride...


  • Waterloo, Ontario, Canada OneSeven Technology (formerly 17 Web Dev) Full time

    About the JobWe are seeking an experienced UX/UI designer to join our team and help us deliver impactful solutions faster than traditional design processes allow.About the TeamOur team is composed of talented experts who share a passion for building digital products and helping companies succeed in their market. Working at OneSeven is an excellent...


  • Waterloo, Ontario, Canada Equitable Life of Canada Full time

    Data Security Architect Waterloo, ON, Canada Req #2241 Monday, April 29, 2024 At Equitable, we realize that your work life is not just about performing a job; it's about being part of a workplace that helps you grow and reach your full potential. Within our friendly and collaborative work environment, we recognize that the key to our growth and success...


  • Waterloo, Ontario, Canada ROSS Full time

    Our client, a world-leading provider of healthcare solutions, requires an experienced Integration Specialist to assist with the implementation and integration of their software solutions.About ROSSROSS is a team of People Specialists dedicated to helping companies align their People Strategy with their business growth. We provide remote or onsite HR and...


  • Waterloo, Ontario, Canada Verily Full time

    Company OverviewVerily is a subsidiary of Alphabet that uses a data-driven approach to transform healthcare management and delivery. Launched from Google X in 2015, Verily's mission is to bring the promise of precision health to everyone, every day.At the core of research and care lies the Precision Health Platform, which underpins software to support...


  • Waterloo, Ontario, Canada ROSS Full time

    Our client, a worldwide leader in healthcare solutions has engaged ROSS (www.getROSS.com) to help them recruit an Integration Specialist. ROSS is a team of People Specialists helping companies align their People Strategy with their business growth. We provide remote or onsite HR and Recruitment solutions complemented by leading edge open-source HRIS...

  • Integration Tech Lead

    4 weeks ago


    Waterloo, Ontario, Canada Agilus Work Solutions Full time

    Agilus is recruiting for an Integration Tech Lead in the IT Sector in a hybrid work environment in Waterloo, Ontario.Responsibilities:1. Serve as a hands-on technical lead in designing and implementing cloud-native and hybrid integration solutions using a variety of patterns, including APIs, microservices, and event-based architectures.2. Lead the...


  • Waterloo, Ontario, Canada Equitable Life of Canada Full time

    Intermediate Technology Specialist, Integration Waterloo, ON, Canada Req #2372 Thursday, July 25, 2024 At Equitable, we realize that your work life is not just about performing a job; it's about being part of a workplace that helps you grow and reach your full potential. Within our friendly and collaborative work environment, we recognize that the key...


  • Waterloo, Ontario, Canada Mcafee Full time

    Solution Design ExpertWe are looking for an experienced Solution Design Expert to join our Enterprise Data Platform team.In this role, you will collaborate with data stewards and data engineers on solution design, develop and maintain physical data models for the Databricks platform, and manage metadata in data catalog tools, such as Collibra.Key...

  • Data Analyst

    7 days ago


    Waterloo, Ontario, Canada CARFAX Full time

    At CARFAX, we pride ourselves on being mission-focused on helping to grow a brand built on accuracy and integrity.We care deeply about our products and our customers. We know that our teammates are our most valuable asset, and we value a balanced life while tackling challenging projects in a fast-paced environment.This role will be working in the data...


  • Waterloo, Ontario, Canada Company 1 - The Manufacturers Life Insurance Company Full time

    Description We are seeking a dynamic and well versed Senior Data Engineer to orchestrate our Data Lake team, with a focus on data curation, report creation, and data feed management. The ideal candidate will possess the qualifications to champion the development and implementation of end-to-end data pipelines using advanced data engineering skills to...


  • Waterloo, Ontario, Canada Carta Full time

    We believe that data should be used to drive business decisions, not just inform them. As a Senior Data Scientist, you'll play a key role in helping us build a data-driven culture that drives business growth and improvement.About the RoleDevelop and implement data-driven solutions to drive business outcomes and improve operational efficiencyCollaborate with...


  • Waterloo, Ontario, Canada CarePartners Full time

    Who We AreCarePartners is one of Ontario's largest accredited home health care providers, providing nursing, personal support, therapy and rehabilitation support services for patients of all ages. We care for approximately 72,000 patients each year in homes, schools, clinics, workplaces and retirement homes, through our 24 locations and 22 nursing clinics....


  • Waterloo, Ontario, Canada CarePartners Full time

    Who We AreCarePartners is one of Ontario's largest accredited home health care providers, providing nursing, personal support, therapy and rehabilitation support services for patients of all ages. We care for approximately 72,000 patients each year in homes, schools, clinics, workplaces and retirement homes, through our 24 locations and 22 nursing clinics....