Data Lake Architect Lead

1 week ago


Canada Amicon Hub Services Full time

About Amicon Hub Services

We are a leading provider of innovative data modernization, analytics and AI solutions. Our team specializes in leveraging cutting-edge technologies to help businesses build and manage scalable data lakes and data lakehouses.

Job Description

This role involves designing, implementing, and optimizing our next-generation data lake architecture. You will lead the development of scalable data platforms using Apache Spark and modern table formats like Apache Iceberg.

We are looking for a Data Lake Architect Lead to join our team.

Key Responsibilities

Data Lake Architecture:

  • Design and implement modern data lake architectures using best practices for data organization, partitioning, and storage optimization.

Spark Development:

  • Lead the development of data processing applications using PySpark, optimizing for performance and resource utilization across large-scale Spark clusters.

Table Format Implementation:

  • Design and implement data storage strategies using Apache Iceberg, enabling ACID transactions, time travel, and schema evolution capabilities.

Data Pipeline Development:

  • Create robust and scalable data pipelines for batch and streaming workflows, implementing proper error handling and data quality checks.

Performance Optimization:

  • Tune Spark applications and cluster configurations for optimal performance, implementing partitioning strategies and query optimization techniques.

Data Governance:

  • Establish and maintain data governance frameworks specific to data lake environments, including access controls, audit logging, and compliance measures.

Infrastructure as Code:

  • Develop and maintain infrastructure as code for automated deployment of data lake components and Spark clusters.

Cross-functional Leadership:

  • Collaborate with data engineers, data scientists, and business stakeholders to ensure the data lake architecture meets both technical and business requirements.
Requirements

You will need to have:

  • Bachelor's or Master's degree in Computer Science, Data Engineering, or related field.
  • 12+ years of experience in data engineering, with at least 3 years focused on data lake architecture and implementations.
  • Deep expertise in Apache Spark and PySpark, including complex ETL development, performance tuning and optimization, Spark SQL and DataFrame APIs, Spark cluster management and monitoring.
  • Strong experience with Apache Iceberg, including table management and optimization, schema evolution, time travel operations, and partition evolution.
  • Proficiency in Python programming and software engineering best practices.
  • Experience with cloud platforms (AWS, Azure, or GCP) and their respective data lake services.
  • Strong understanding of data modeling concepts for both structured and unstructured data.
  • Experience with version control systems (Git) and CI/CD pipelines.
Salary

The estimated salary for this role is around $175,000 per year, depending on location and qualifications.

We offer a comprehensive benefits package, including health insurance, professional development support, and a remote-friendly work environment with flexible scheduling options.



  • Canada PrecisionERP Incorporated Full time

    Job OverviewPrecisionERP Incorporated is seeking an experienced Senior Data Architect to lead our cloud data platform initiatives.About the RoleThis full-time, 35-hour per week position offers a hybrid work arrangement with 2 days on-site in downtown Montreal and 3 days remotely.Estimated Salary: $120,000 - $160,000 per year, based on industry standards and...


  • Canada OnX Full time

    Job OverviewWe are seeking an experienced Lead Data Engineering Architect to join our team at OnX in Calgary. As a key member of our data engineering group, you will be responsible for driving the technical vision and strategy for our data platform.About the RoleThe successful candidate will have a minimum of 7 years of experience in a data engineering role,...

  • Lead Data Architect

    4 months ago


    Canada Hillsdale Investment Management Inc. Full time

    LEAD DATA ARCHITECTHillsdale Investment Management Inc. (Hillsdale) is an independent, employee-controlled, and client-aligned investment boutique managing investments on behalf of a select group of sophisticated institutional and private wealth investors. We engage in disciplined active investment management, specializing in equities, alternatives, ESG, and...

  • Lead Data Architect

    4 months ago


    Canada Hillsdale Investment Management Inc. Full time

    LEAD DATA ARCHITECTHillsdale Investment Management Inc. (Hillsdale) is an independent, employee-controlled, and client-aligned investment boutique managing investments on behalf of a select group of sophisticated institutional and private wealth investors. We engage in disciplined active investment management, specializing in equities, alternatives, ESG, and...


  • Canada Amicon Hub Services Full time

    About QSpark Group :- QSpark Group is a leading provider of innovative solutions in the field of data modernization, analytics and AI. We specialize in leveraging cutting-edge technologies to help businesses build and manage scalable data lakes and data lakehouses, enabling them to harness the full potential of their data assets for strategic...


  • Canada, CA Amicon Hub Services Full time

    About QSpark Group :- QSpark Group is a leading provider of innovative solutions in the field of data modernization, analytics and AI. We specialize in leveraging cutting-edge technologies to help businesses build and manage scalable data lakes and data lakehouses, enabling them to harness the full potential of their data assets for strategic...


  • Canada Amicon Hub Services Full time

    About QSpark Group :- QSpark Group is a leading provider of innovative solutions in the field of data modernization, analytics and AI. We specialize in leveraging cutting-edge technologies to help businesses build and manage scalable data lakes and data lakehouses, enabling them to harness the full potential of their data assets for strategic...

  • Data Architect Lead

    2 days ago


    Canada Resonaite Full time

    About the RoleWe are seeking an experienced Data Architect to lead our end-to-end data architecture efforts for a critical system rebuild project. As a key member of our team, you will be responsible for designing and implementing scalable, future-ready data integration solutions.The successful candidate will have expertise in SAP and a proven track record...


  • Canada Procom Full time

    Job Overview:We are seeking a skilled Cloud Data Architect to join our Procom team.About the Role:The Cloud Data Architect will be responsible for designing and implementing comprehensive Azure solutions, considering infrastructure, applications, data, security, and ETL development. This includes Azure administration skills and experience with Azure Data...

  • Lead Data Architect

    6 days ago


    Canada, CA Themesoft Inc. Full time

    Position: Lead Data ArchitectLocation: Canada – RemoteJob Description:Work independently to solve open-ended questions.Help lead and manage the design, testing, and implementation of databases (Azure SQL and/or Snowflake) that support productionized BI dashboards (PowerBI, Tableau) and custom analytic applications.Focus on the coding visuals for the...


  • Canada TES The Employment Solution Full time

    Job DescriptionWe are seeking an experienced Data Storage Architect Lead to join our team at TES The Employment Solution. In this role, you will be responsible for monitoring, supporting, and enhancing associated storage, backups, and data replication for a large number of Sysplexes.You will provide support for Disaster Recovery Exercises and new data center...


  • Canada Amicon Hub Services Full time

    About Amicon Hub Services">We are a leading provider of innovative solutions in the field of data modernization, analytics and AI. We specialize in leveraging cutting-edge technologies to help businesses build and manage scalable data lakes and data lakehouses, enabling them to harness the full potential of their data assets for strategic decision-making.

  • Data Architect Lead

    1 week ago


    Canada Hillsdale Investment Management Inc. Full time

    Hillsdale Investment Management Inc. is a leading investment boutique managing investments on behalf of sophisticated institutional and private wealth investors.Competitive Salary: $120,000 - $180,000 per yearWe engage in disciplined active investment management, specializing in equities, alternatives, ESG, and customized mandates. Our team is committed to...

  • Data Architect Lead

    4 days ago


    Canada Themesoft Inc. Full time

    Lead Data ArchitectWe are seeking a skilled Lead Data Architect to join our team at Themesoft Inc. in Canada, with the opportunity to work remotely.Job OverviewThe ideal candidate will have experience in designing, testing, and implementing databases that support productionized BI dashboards and custom analytic applications.Key ResponsibilitiesDevelop...

  • Cloud Architect

    1 week ago


    Quebec, Canada, Canada Klanik Full time

    Position: Senior Cloud Architect (Azure, Network, & Security)Location: Remote (North America) – Must be able to work during Mountain Time Zone hoursClient Location: Salt Lake City, UtahStart Date: Late NovemberDuration: 6+ months, with potential for extensionHours: 30-40 hours per week (TBD)Company OverviewWe are a leading provider of IT consulting and...


  • Canada Agility Partners Full time

    Job DescriptionAgility Partners is seeking a seasoned Cloud Data Architect Lead to spearhead data pipeline development, model and prepare data, perform complex data analysis to address business queries, build and automate data pipeline and quality framework to empower self-service data pipelines, and assist in operationalizing AI/ML engineering...


  • Canada Intuitive Full time

    About UsWe are Intuitive.Cloud, a leading Cloud & SDx solution and services company that supports enterprise customers globally. Our engineering company delivers measurable value and key business outcomes.Job OverviewWe are seeking an experienced Data Architect and Engineer to design and implement scalable AI microservices and robust data pipelines. This...


  • Canada TEEMA Full time

    Cloud Expert WantedTEEMA is seeking a highly skilled Technical Lead to join our team.This is a 75% hands-on and 25% architect role, requiring expertise in Azure DevOps and CICD pipeline management.50% Technical Support: The ideal candidate will provide expert-level support, troubleshooting issues, and conducting in-depth analysis to ensure system smooth...


  • Canada Themesoft Inc. Full time

    About the Role:We are seeking a seasoned Lead Data Architect to join our team at Themesoft Inc. in Canada.As a key member of our organization, you will play a crucial role in designing, implementing, and managing databases that support our productionized Business Intelligence (BI) dashboards and custom analytic applications.Main Responsibilities:Work...


  • Canada IMCS Group Full time

    Job Title: Data Solutions ArchitectAbout the Role:We are seeking an experienced Data Solutions Architect to join our team at IMCS Group in Edmonton, AB. This is a 6-month contract position with possible extension.About the Job:The proposed resource must meet or exceed one of the following criteria:University graduation in business, management, or a related...