Lead Data Engineer

3 weeks ago

Cambridge, Ontario, Canada Acerta Full time

About Acerta:

Acerta helps manufacturers optimize the quality, safety, and reliability of the products they make. Our machine-learning solutions use industrial data to deliver valuable insights that improve the entire product lifecycle—resulting in greater efficiency and satisfied customers.

Job Summary:

We are seeking a talented and experienced Lead Data Engineer to join our growing team. The Lead Data Engineer will play a key role in designing, building, and maintaining our data infrastructure, ensuring scalability, reliability, and performance with a team of data engineers around them. This position offers an exciting opportunity to work with large-scale datasets, cutting-edge technologies, and a passionate team of data professionals.

Key Responsibilities:

  • Architect and Develop Data Ingestion to Acerta Platform: Architect, develop, and maintain scalable data pipelines and ETL processes to ingest, process, and analyze large volumes of structured and unstructured data from diverse sources.
  • Collaborate with Cross-Functional Teams: Collaborate with cross-functional teams to understand data requirements, identify opportunities for data-driven insights, and develop solutions to address business challenges.
  • Optimize Data Pipelines: Optimize and tune data pipelines and database queries for performance, efficiency, and reliability.
  • Implement Data Governance: Implement data governance and security best practices to ensure data integrity, privacy, and compliance with regulatory requirements.
  • Support Machine Learning Initiatives: Work closely with data scientists to support their data needs, provide access to relevant datasets, and enable advanced analytics and machine learning initiatives.
  • Stay Abreast of Industry Trends: Stay abreast of industry trends, emerging technologies, and best practices in data engineering and analytics, and apply them to improve our data infrastructure and processes.

Skills and Qualifications:

  • Previous experience in ETL/ELT Architecture: Expertise in designing and optimizing ETL/ELT pipelines to handle various data formats (CSV, JSON, Parquet, etc.) and integrating data from multiple sources (e.g., APIs, cloud storage, client Databricks instances).
  • Strong understanding of REST API principles, experience with high-volume API requests, and ability to optimize API calls and data ingestion strategies.
  • Automation and Orchestration: Proficiency in using orchestration tools like Apache Airflow, or similar tools to automate and manage data workflows.
  • Proficiency using Unity Catalog to establish a consistent security and governance layer, manage access control, and ensure regulatory compliance.
  • Expertise in building efficient ETL/ELT workflows to enable scalable feature engineering.
  • Previous experience working closely with ML/DS teams, understanding of data modeling required for various teams and products.
  • Previous experience in performance testing and optimization (data load testing/performance tuning/monitoring) for various databases, and ETL pipelines.
  • Technical Mentorship: Leadership in coaching junior engineers, setting best practices for coding standards, and fostering a collaborative environment for knowledge sharing and troubleshooting.

Technical stack:

Programming Languages: Python, JavaScript/Node.js, Spark

  1. Big Data experience: Databricks
  2. Data Orchestration and Automation: K8, Apache Airflow or similar.
  3. APIs and Data Ingestion: Strong understanding of REST API principles, experience with high-volume API requests, and ability to optimize API calls and data ingestion strategies.
  4. Large data workflow optimization: Optimizations of Spark jobs
  5. Bonus - experience with Vector Databases.

Why Join Acerta?

  • Competitive Salary: Competitive salary and performance-based incentives.
  • Comprehensive Benefits: Comprehensive benefits package, including health insurance, and more.
  • Professional Growth: Opportunities for professional growth and career advancement.
  • Collaborative Environment: Collaborative and inclusive work environment with a diverse team of professionals.

If you are passionate about data engineering and eager to contribute to Acerta's mission, we invite you to apply for this exciting opportunity. Apply now and be part of Acerta's mission to shape the future of automotive intelligence

Acerta Analytics Solutions Inc. is an equal-opportunity employer. We celebrate diversity and are committed to creating an inclusive environment for all employees.

  • Engineering Design Team Lead

    6 days ago

    Cambridge, Ontario, Canada Meritech Engineering Full time

    As a leading civil engineering firm, Meritech Engineering is committed to delivering high-quality projects that exceed client expectations. We are seeking a talented and experienced Engineering Design Team Lead to lead and manage our design services group, ensuring the successful delivery of complex land development projects.The ideal candidate will have a...

  • Principal Systems Engineer

    8 hours ago

    Cambridge, Ontario, Canada Spire Global, Inc. Full time

    As a Principal Systems Engineer, you will be responsible for the technical outcome of complex space missions. During mission contracting and requirements development, you may work with the sales engineering org to ensure mission requirements are valid and mission scope is adequately defined. Systems Engineers translate higher level requirements down to...

  • Principal Systems Engineer

    4 weeks ago

    Cambridge, Ontario, Canada Spire Global, Inc. Full time

    **Job Title:** Principal Systems EngineerAs a critical member of our team, you will be responsible for the technical outcome of complex space missions. Your expertise will play a vital role in ensuring that we deliver successful missions from contracting and requirements development through on-orbit verification.Your responsibilities will include managing...

  • Master Data Expert

    6 hours ago

    Cambridge, Ontario, Canada Aecon Concessions Full time

    Company OverviewAecon Concessions is a leading player in the infrastructure development industry, delivering projects that shape the future of our communities. We prioritize safety, integrity, and accountability in everything we do.Job DescriptionWe are seeking an experienced Master Data Analyst to join our Corporate Procurement Data team. The successful...

  • Lead Engineer, Tooling Design

    4 days ago

    Cambridge, Ontario, Canada BWXT Medical Ltd. Full time

    Join to apply for the Lead Engineer, Tooling Design role at BWXT Medical.BWXT Canada Ltd. (BWXT Canada) has over 60 years of expertise and experience in the design, manufacturing, commissioning and service of nuclear power generation equipment. This includes steam generators, nuclear fuel and fuel components, critical plant components, parts and related...

  • Lead Engineer, Tooling Design

    4 weeks ago

    Cambridge, Ontario, Canada BWX Technologies, Inc. Full time

    BWXT Canada Ltd. (BWXT Canada) has over 60 years of expertise and experience in the design, manufacturing, commissioning and service of nuclear power generation equipment. This includes steam generators, nuclear fuel and fuel components, critical plant components, parts and related plant services. BWXT Canada's subsidiary, BWXT Medical Ltd. (BWXT Medical)...

  • Analyst, Master Data

    8 hours ago

    Cambridge, Ontario, Canada Aecon Concessions Full time

    It has come to our attention that various international organizations or individuals have been offering false employment opportunities at Aecon Group Inc. Aecon Group Inc. employment policies and processes involve interviews, and candidates who seek employment are never required to pay us any sum of money. To do so would be contrary to our business conduct...

  • Project Procurement Lead

    3 days ago

    Cambridge, Ontario, Canada ATS Automation Full time

    Job Description Reporting to the Supply Chain Manager, the Project Procurement Lead will be the technical liaison between suppliers, Designers and Supply Chain to achieve project and departmental goals, objectives, and best practices.  Responsibilities include: • Track, monitor and maintain purchase order deliveries to drive on-time delivery to...

  • Front End Engineer

    12 hours ago

    Cambridge, Ontario, Canada The 401 Group Full time

    The 401 Group of Companies is a privately owned and 100% Canadian organization that strives to hire exceptional employees who see themselves as trailblazers, trendsetters, creators, do-ers, catalysts, go-getters, and influential leaders in all aspects of who they aspire to be daily.This role is an In Person Position located at 2200 Eagle Street North,...

  • Lead Engineering Project Manager

    5 days ago

    Cambridge, Ontario, Canada Meritech Engineering Full time

    Mission Statement:Meritech Engineering is committed to providing innovative engineering solutions that meet our clients' needs while contributing to the growth and development of our community.About Us:We are a dynamic civil engineering firm based in the Waterloo Region, known for our expertise in municipal design, construction, and project management. Our...

  • Mechanical Engineer

    4 weeks ago

    Cambridge, Ontario, Canada IFAB Engineering Partners Ltd. Full time

    IFAB Engineering Partners Ltd. is seeking serious applicants to fill the role of Mechanical Engineer at the intermediate to senior level. Work is performed in collaboration with a full complement of designers and engineers from a variety of disciplines. Reporting to the Mechanical Engineering Manager, the Mechanical Engineer leads on an array of mechanical...

  • Automation Lead

    6 days ago

    Cambridge, Ontario, Canada Engineering Search Firm Inc Full time

    Company OverviewOur client, an Original Equipment Manufacturer (OEM), specializes in engineering, design, manufacturing, assembly, and programming of automated manufacturing solutions. They provide a strong engineering approach in solving complex manufacturing challenges using automation, robotics, and state-of-the-art technology.Job DescriptionWe are...

  • Mechanical Engineer

    2 weeks ago

    Cambridge, Ontario, Canada IFAB Engineering Partners Ltd. Full time

    IFAB Engineering Partners Ltd. is seeking serious applicants to fill the role of Mechanical Engineer at the intermediate to senior level. Work is performed in collaboration with a full complement of designers and engineers from a variety of disciplines. Reporting to the Mechanical Engineering Manager, the Mechanical Engineer leads on an array of mechanical...

  • Mechanical Engineer

    2 weeks ago

    Cambridge, Ontario, Canada IFAB Engineering Partners Ltd. Full time

    IFAB Engineering Partners Ltd. is seeking serious applicants to fill the role of Mechanical Engineer at the intermediate to senior level. Work is performed in collaboration with a full complement of designers and engineers from a variety of disciplines. Reporting to the Mechanical Engineering Manager, the Mechanical Engineer leads on an array of mechanical...

  • Mechanical Engineer

    4 weeks ago

    Cambridge, Ontario, Canada IFAB Engineering Partners Ltd. Full time

    IFAB Engineering Partners Ltd. is seeking serious applicants to fill the role of Mechanical Engineer at the intermediate to senior level. Work is performed in collaboration with a full complement of designers and engineers from a variety of disciplines. Reporting to the Mechanical Engineering Manager, the Mechanical Engineer leads on an array of...

  • Process Engineer

    5 days ago

    Cambridge, Ontario, Canada IFAB Engineering Partners Ltd. Full time

    Job DescriptionWe are seeking a skilled Process Engineer to join our team at IFAB Engineering Partners Ltd. in Canada.Lead process design efforts for new and/or modified food and beverage production facilities across Canada.Work closely with designers, fellow engineers, technical specialists, and project managers to design optimized systems, structures, and...

  • RPA Team Lead

    3 days ago

    Cambridge, Ontario, Canada Farm Mutual Re Full time

    We are seeking a talented Digital Transformation Manager to join our Process Improvement team at Farm Mutual Re. As a leader in the mutual insurance sector, we offer unique reinsurance solutions tailored to various insurance lines, providing financial stability and capacity to mutual insurers.In this role, you will drive digital transformation by identifying...

  • Welding / Manufacturing Engineering Supervisor

    4 weeks ago

    Cambridge, Ontario, Canada BWX Technologies, Inc. Full time

    Welding / Manufacturing Engineering SupervisorBWXT Canada Ltd. (BWXT Canada) has over 60 years of expertise and experience in the design, manufacturing, commissioning and service of nuclear power generation equipment. This includes steam generators, nuclear fuel and fuel components, critical plant components, parts and related plant services. BWXT Canada's...

  • Senior Data Analyst

    1 week ago

    Cambridge, Ontario, Canada Gore Mutual Insurance Full time

    The Senior Data Analyst is a hands-on role responsible for developing and maintaining user-centric reporting and providing subject matter expertise to business leaders in alignment with business strategy and organizational needs. This role involves curating, monitoring, tracking, and notifying on data and insights across the enterprise. The Senior Data...

  • Senior Data Analyst

    4 days ago

    Cambridge, Ontario, Canada Gore Mutual Insurance Full time

    The Senior Data Analyst is a hands-on role responsible for developing and maintaining user-centric reporting and providing subject matter expertise to business leaders in alignment with business strategy and organizational needs. This role involves curating, monitoring, tracking, and notifying on data and insights across the enterprise. The Senior Data...