Sr. Data Engineer

7 days ago


Waterloo, Canada Dematic Ltd. Full time

We are seeking a dynamic and highly skilled Senior Data Engineer who has extensive experience building self-service enterprise scale data platforms with microservices architecture and lead these foundational efforts. This role demands someone who not only possesses a profound understanding of the data engineering landscape but also has a very strong software engineering background specially building microservices frameworks and architectures. The ideal candidate will be an individual contributor as well as the technical lead and contribute significantly to platform development and actively shape our data ecosystem.

What we offer:
  • Career Development
  • Competitive Compensation and Benefits
  • Pay Transparency
  • Global Opportunities
Tasks and Qualifications:

This is What You Will do in This Role:

  • As a senior engineer, you will be responsible for ideation, architecture, design and development of our enterprise data platform.
  • Architect and design core components with a microservices architecture, abstracting platform, and infrastructure intricacies.
  • Create and maintain essential data platform SDKs and libraries, adhering to industry best practices.
  • Design and develop connector frameworks and modern connectors to source data from disparate systems both on-prem and cloud.
  • Design and optimize data storage, processing, and querying performance for large-scale datasets using industry best practices while keeping costs in check.
  • Design and develop data quality frameworks and processes to ensure the accuracy and reliability of data.
  • Design and develop microservices based semantic layer and metadata management components.
  • Collaborate with data scientists, analysts, and cross functional teams to design data models, database schemas and data storage solutions.
  • Design and develop advanced analytics and machine learning capabilities on the data platform.
  • Design and develop observability and data governance frameworks and practices.
  • Stay up to date with the latest data engineering trends, technologies, and best practices.
  • Drive the deployment and release cycles, ensuring a robust and scalable platform.

What We are Looking For:

  • Bachelor's or Master's degree in Computer Science, Engineering, or related field.
  • 8-10 years of proven experience in modern cloud data engineering, broader data landscape experience and exposure and solid software engineering experience.
  • Prior experience architecting and building successful self-service enterprise scale data platforms in a green field environment with microservices based architecture.
  • Proficiency in building end to end data platforms and data services in GCP is a must.
  • Proficiency in tools and technologies: BigQuery, Cloud Functions, Cloud Run, Dataform, Dataflow, Dataproc, SQL, Python, Airflow, PubSub.
  • Experience with Microservices architectures - Kubernetes, Docker. Our microservices are build using TypeScript, NestJS, NodeJS stack. Prefer candidates with this experience.
  • Experience building Symantec layers.
  • Proficiency in architecting and designing and development experience with batch and real time streaming infrastructure and workloads.
  • Solid experience with architecting and implementing metadata management including data catalogues, data lineage, data quality and data observability for big data workflows.
  • Hands-on experience with GCP ecosystem and data lakehouse architectures.
  • Strong understanding of data modeling, data architecture, and data governance principles.
  • Excellent experience with DataOps principles and test automation.
  • Excellent experience with observability tooling: Grafana, Datadog.

Nice to have:

  • Experience with Data Mesh architecture
  • Experience building Semantic layers for data platforms.
  • Experience building scalable IoT architectures
#J-18808-Ljbffr

  • Waterloo, Canada Data Spree GmbH Full time

    Dematic Ltd.Waterloo, CanadaRole HighlightsGOSQLPythonData ArchitectData GovernanceAPIData QualityTest AutomationData WarehousingDatabase SchemaMicroservicesInfrastructureCloudStorageDataSetsReliabilityDatabaseDeploymentInternet Of ThingsTools, Libraries and FrameworksSDKGCPDockerAirflowKafkaGrafanaDataDogDescriptionDematic is seeking skilled Senior and...


  • Waterloo, Canada Data Spree GmbH Full time

    Dematic Ltd.Waterloo, CanadaRole HighlightsGOSQLPythonData ArchitectData GovernanceAPIData QualityTest AutomationData WarehousingDatabase SchemaMicroservicesInfrastructureCloudStorageDataSetsReliabilityDatabaseDeploymentInternet Of ThingsTools, Libraries and FrameworksSDKGCPDockerAirflowKafkaGrafanaDataDogDescriptionDematic is seeking skilled Senior and...

  • Sr. Software Engineer

    1 month ago


    Waterloo, Canada MCAP Full time

    Sr. Software Engineer MCAP at a Glance Joining MCAP means you will be a part of our diverse workforce of highly talented individuals who are recognized for their expertise and success! At MCAP, your professional expertise, commitment to teamwork and passion for service excellence are recognized and rewarded with competitive total rewards offering, a career...

  • Sr. Software Engineer

    1 month ago


    Waterloo, Canada MCAP Full time

    Sr. Software Engineer MCAP at a Glance Joining MCAP means you will be a part of our diverse workforce of highly talented individuals who are recognized for their expertise and success! At MCAP, your professional expertise, commitment to teamwork and passion for service excellence are recognized and rewarded with competitive total rewards offering, a career...

  • Sr. Software Engineer

    3 weeks ago


    Waterloo, Canada MCAP Full time

    Sr. Software Engineer MCAP at a Glance Joining MCAP means you will be a part of our diverse workforce of highly talented individuals who are recognized for their expertise and success! At MCAP, your professional expertise, commitment to teamwork and passion for service excellence are recognized and rewarded with competitive total rewards offering, a career...

  • Sr. Software Engineer

    3 weeks ago


    Waterloo, Canada MCAP Full time

    Sr. Software Engineer MCAP at a Glance Joining MCAP means you will be a part of our diverse workforce of highly talented individuals who are recognized for their expertise and success! At MCAP, your professional expertise, commitment to teamwork and passion for service excellence are recognized and rewarded with competitive total rewards offering, a career...

  • Sr. Software Engineer

    3 weeks ago


    Waterloo, Canada MCAP Full time

    Sr. Software Engineer MCAP at a Glance Joining MCAP means you will be a part of our diverse workforce of highly talented individuals who are recognized for their expertise and success! At MCAP, your professional expertise, commitment to teamwork and passion for service excellence are recognized and rewarded with competitive total rewards offering, a career...


  • Waterloo, Canada MCAP Full time

    Sr. Software Engineer MCAP at a Glance Joining MCAP means you will be a part of our diverse workforce of highly talented individuals who are recognized for their expertise and success! At MCAP, your professional expertise, commitment to teamwork and passion for service excellence are recognized and rewarded with competitive total rewards offering, a career...


  • Waterloo, Canada MCAP Full time

    Sr. Software Engineer MCAP at a Glance Joining MCAP means you will be a part of our diverse workforce of highly talented individuals who are recognized for their expertise and success! At MCAP, your professional expertise, commitment to teamwork and passion for service excellence are recognized and rewarded with competitive total rewards offering, a career...


  • Waterloo, Ontario, M2L, City of Toronto, Canada Data Spree GmbH Full time

    Dematic Ltd.Waterloo, CanadaRole HighlightsGOSQLPythonData ArchitectData GovernanceAPIData QualityTest AutomationData WarehousingDatabase SchemaMicroservicesInfrastructureCloudStorageDataSetsReliabilityDatabaseDeploymentInternet Of ThingsTools, Libraries and FrameworksSDKGCPDockerAirflowKafkaGrafanaDataDogDescriptionDematic is seeking skilled Senior and...


  • Waterloo, Canada Constant Contact, Inc. Full time

    Our mobile team has an opening for a talented Sr. Manager, Engineering Mobile to play a key role in leading and developing captivating applications for phones, tablets, and watches. This is an exciting opportunity to contribute to the forefront of mobile technology in the digital marketing sector. What you’ll do: Manage complex technical projects and...


  • Waterloo, Canada Constant Contact, Inc. Full time

    Our mobile team has an opening for a talented Sr. Manager, Engineering Mobile to play a key role in leading and developing captivating applications for phones, tablets, and watches. This is an exciting opportunity to contribute to the forefront of mobile technology in the digital marketing sector. What you’ll do: Manage complex technical projects and...


  • Waterloo, Canada Tbwa ChiatDay Inc Full time

    Sr Software Development Engineer in TestWhy PlayStation?PlayStation isn’t just the Best Place to Play — it’s also the Best Place to Work. Today, we’re recognized as a global leader in entertainment producing The PlayStation family of products and services including PlayStation5, PlayStation4, PlayStationVR, PlayStationPlus, acclaimed PlayStation...

  • Data Engineer

    2 months ago


    Waterloo, Canada Epitec Inc Full time

    JOB SUMMARY FOR Data Engineer: The Fully Networked Vehicle Analytics Data team is looking for a passionate team member to join the organization. Our mission is to provide the best possible user experience for our customers, by providing invaluable wisdom from data to engineering, sales, and executive decision makers. POSTION IS HYBRID ...

  • Sr. Software Engineer

    3 weeks ago


    Waterloo, Canada MCAP Group of Companies Full time

    Joining MCAP means you will be a part of our diverse workforce of highly talented individuals who are recognized for their expertise and success! At MCAP, your professional expertise, commitment to teamwork and passion for service excellence are recognized and rewarded with competitive total rewards offering, a career with continuous learning and development...

  • Sr. Software Engineer

    2 months ago


    Waterloo, Canada MCAP Group of Companies Full time

    Joining MCAP means you will be a part of our diverse workforce of highly talented individuals who are recognized for their expertise and success! At MCAP, your professional expertise, commitment to teamwork and passion for service excellence are recognized and rewarded with competitive total rewards offering, a career with continuous learning and development...

  • Sr. Software Engineer

    2 months ago


    Waterloo, Canada MCAP Group of Companies Full time

    Joining MCAP means you will be a part of our diverse workforce of highly talented individuals who are recognized for their expertise and success! At MCAP, your professional expertise, commitment to teamwork and passion for service excellence are recognized and rewarded with competitive total rewards offering, a career with continuous learning and development...

  • Data Engineer

    4 weeks ago


    Waterloo, Canada Manulife Insurance Malaysia Full time

    Description d’emploiAs a Data Engineer at Manulife, you would play a critical role in development and maintenance of the company's data systems and architecture. You would collaborate with other highly skilled data engineers, data architects and governance team in designing, developing, and deploying data pipelines, ETL processes, and other data...

  • Data Engineer

    4 weeks ago


    Waterloo, Canada Manulife Insurance Malaysia Full time

    Description d’emploiAs a Data Engineer at Manulife, you would play a critical role in development and maintenance of the company's data systems and architecture. You would collaborate with other highly skilled data engineers, data architects and governance team in designing, developing, and deploying data pipelines, ETL processes, and other data...

  • Sr Java Developer

    1 day ago


    Waterloo, Canada Saransh Inc Full time

    Role: Sr Engineer (Java Developer)Location: Waterloo, ON, Canada (Onsite from Day 1)Job Type: ContractPrimary Skills: Java, Spring bootSecondary Skills: Azure Kubernetes Service (AKS)Required Experience:Work with architecture and business sponsor(s) to assess, design and estimate solutions, assuming end-to-end technical responsibility.Lead complex group...