Lead Data Engineer

3 weeks ago

Toronto, Canada National Consultants Professionals Ltd Full time

The Lead Data Engineer will be responsible for leading a team of data engineers that build and deliver the components of an effective Data Warehouse, leveraging Azure Data Lake, Azure Synapse, Snowflake for data storage, utilising Azure Data Factory and Synapse Pipelines for data integration, Databricks for data transformation and PowerBI for end-user data analytics and reporting. This role will play a critical part in the architectural design and implementation of these elements - capable of meeting a wide range of financial and underwriting reporting/analytics needs. This role will encompass a mix of hands-on technical leadership, team management, strategy development, and stakeholder relations; supporting Sunwing to continue its journey to develop its data platform on next-generation technology.

The Lead Data Engineer will:

  • data engineering through both day-to-day data management (BAU) activities and strategic development projects, defining and orchestrating tasks and deliverables.
  • develop, and hold accountability for core data engineering services, encompassing the end-to-end data lifecycle journey from ingestion through to the Warehouse.
  • as a senior custodian of the Data Platform; administering CICD pipelines, completing peer/code reviews, ensuring the integrity of data, and holding accountability for access and security permissions.
  • and develop ETL/ELT processes, data lake storage architecture, data warehouse, data marts; in-line with the company data management framework and enterprise data strategy.
  • and develop models and databases both relational and dimensional to support reporting requirements.
  • reporting solutions in line with standards, architectural principles, and practices.
  • change impact assessment on live systems and processes.
  • in an agile, proactive manner, delivering high quality solutions in agreed timescale.
  • and support junior data engineers/developers in the team Principal Accountabilities:
  • as a senior custodian of the Data Platform, with performance, security, and data integrity at the forefront of all decisions, ensure that adequate steps are taken to protect data and the data platform.
  • in collaboration with Data Architects and act as a Senior contributor on the design, build, and management of the Data Platform; taking direct ownership of data engineering activities and responsibilities and ensure that all new data requirements are met using the most appropriate architecture components and engineering practices.
  • leadership of both senior and general data engineers.
  • ownership of incidents and define remediation plans, with the support of the wider team, to ensure production issues are resolved in line with expected RTO.
  • and enforce SDLC best practices, coding standards and design principles covering all aspects of data lifecycle management and BI solutions.
  • business requirements and build data solutions to meet business needs.
  • in developing data management framework policies, processes, and standards.
  • abreast of technology trends, state-of-the-art platforms, data engineering techniques/standards and industry trends to provide input on strategic roadmap.
  • maintains a low technical debt & identify area(s) for automation and continuous improvement.
  • Proof of Concepts to prove that business requirements can be met by technical solutions.
  • with building the team and hiring activities.


Education, Qualifications, Knowledge, Skills, and Experience:

Required Skills & Knowledge

  • hands-on experience with Azure Data Lake (incl. Blob Storage, Azure Synapse, Azure Data Factory, Azure Synapse and Snowflake
  • experience of SQL/Azure SQL data warehousing
  • experience of data engineering activities such as release management, environment controls, CICD pipeline orchestration
  • working with Python and Spark
  • ability of Microsoft BI software development including database / reporting development, full development cycle
  • data modelling, problem solving, information analysis, attention to detail, flexibility of approach.
  • ability to deal directly with business functions and interpret their thoughts into successful data & analytics solutions.
  • ability to design and develop complex BI solutions in line with agreed architectural principles and toolsets.
  • of working closely with the business and delivering complex requirements to tight timescales.
  • interest in latest data engineering technologies, best practises, techniques and trends in the data industry Highly Desirable Skills and Knowledge
  • experience in the design and build of data warehouse/data lake.
  • of Synapse.
  • development techniques (SCRUM, Kanban etc.) Required Toolsets.
  • Data Lake Storage, Analysis Services (Multidimensional/Tabular), Power BI, Azure Data Factory, Synapse pipelines, Azure Databricks, Spark.
  • SQL Server or Azure SQL DB.
  • knowledge of data modelling techniques is Desirable Toolsets.
  • working with Snowflake.
  • with API, Connectors, etc. is desirable.

  • Lead Data Engineer

    7 days ago

    Toronto, Canada Insight Global Full time

    Must-haves - 10-12 years experience working within data engineering roles - 5-7 years experience with AWS cloud services including: EMR, MWAA, LakeFormation, Glue, DMS, QuickSight, and AWS Athena - 8+ years experience with big data technologies including: Apache Spark, Apache Hudi, Hive, Airflow, Trino, and Databricks - CICD - Terraform for sure ,...

  • Manager/Lead- Data Engineering

    4 weeks ago

    Toronto, Canada hireVouch Full time

    Our client is a Toronto-based GCP consulting company that is focused on helping a variety of businesses through cloud-based solutions. They are growing their team and are looking for a passionate Data Engineering Manager/Lead to continue to guide them to success. In this role, you will be responsible for leading a team of data engineers in designing,...

  • Manager/Lead- Data Engineering

    4 weeks ago

    Toronto, Canada hireVouch Full time

    Our client is a Toronto-based GCP consulting company that is focused on helping a variety of businesses through cloud-based solutions. They are growing their team and are looking for a passionate Data Engineering Manager/Lead to continue to guide them to success. In this role, you will be responsible for leading a team of data engineers in designing,...

  • Manager/Lead- Data Engineering

    4 weeks ago

    Toronto, Canada hireVouch Full time

    Our client is a Toronto-based GCP consulting company that is focused on helping a variety of businesses through cloud-based solutions. They are growing their team and are looking for a passionate Data Engineering Manager/Lead to continue to guide them to success. In this role, you will be responsible for leading a team of data engineers in designing,...

  • Manager/Lead- Data Engineering

    4 weeks ago

    Toronto, Canada hireVouch Full time

    Our client is a Toronto-based GCP consulting company that is focused on helping a variety of businesses through cloud-based solutions. They are growing their team and are looking for a passionate Data Engineering Manager/Lead to continue to guide them to success.In this role, you will be responsible for leading a team of data engineers in designing,...

  • Manager/Lead- Data Engineering

    4 weeks ago

    Toronto, Canada hireVouch Full time

    Our client is a Toronto-based GCP consulting company that is focused on helping a variety of businesses through cloud-based solutions. They are growing their team and are looking for a passionate Data Engineering Manager/Lead to continue to guide them to success.In this role, you will be responsible for leading a team of data engineers in designing,...

  • Data Engineering Practice Lead

    4 weeks ago

    Toronto, Canada McMillan Shakespeare Limited Full time

    The McMillan Shakespeare Group (MMS) is a trusted provider of salary packaging, novated leasing, disability plan management and support co-ordination, asset management and related financial products and services. From our origins in 1988 when we created Australia’s salary packaging industry to today, MMS has a proud history of innovation and exceptional...

  • Sr/Lead Data Engineer

    1 month ago

    Toronto, Canada Asset Management Industry Full time

    Good day Data Engineers ! Insight Global is looking for 6 Sr or Lead Data Engineers to join a large Asset Management company in Toronto on a full time basis. The successful candidates will be responsible for leading projects end-to-end within the Data Distribution Team or Asset Management Team . T

  • Sr/Lead Data Engineer

    1 month ago

    Toronto, Canada Asset Management Industry Full time

    Good day Data Engineers!Insight Global is looking for 6 Sr or Lead Data Engineers to join a large Asset Management company in Toronto on a full time basis. The successful candidates will be responsible for leading projects end-to-end within the Data Distribution Team or Asset Management Team. This role is heavily hands on and requires strong experience with...

  • Sr/Lead Data Engineer

    4 weeks ago

    Toronto, Canada Asset Management Industry Full time

    Good day Data Engineers ! Insight Global is looking for 6 Sr or Lead Data Engineers to join a large Asset Management company in Toronto on a full time basis. The successful candidates will be responsible for leading projects end-to-end within the Data Distribution Team or Asset Management Team . This role is heavily hands on and requires strong...

  • Sr/Lead Data Engineer

    1 month ago

    Toronto, Canada Asset Management Industry Full time

    Good day Data Engineers!Insight Global is looking for 6 Sr or Lead Data Engineers to join a large Asset Management company in Toronto on a full time basis. The successful candidates will be responsible for leading projects end-to-end within the Data Distribution Team or Asset Management Team. This role is heavily hands on and requires strong experience with...

  • Sr/Lead Data Engineer

    4 weeks ago

    Toronto, Canada Asset Management Industry Full time

    Good day Data Engineers ! Insight Global is looking for 6 Sr or Lead Data Engineers to join a large Asset Management company in Toronto on a full time basis. The successful candidates will be responsible for leading projects end-to-end within the Data Distribution Team or Asset Management Team . This role is heavily hands on and requires strong...

  • Sr/Lead Data Engineer

    4 weeks ago

    Toronto, Canada Asset Management Industry Full time

    Good day Data Engineers ! Insight Global is looking for 6 Sr or Lead Data Engineers to join a large Asset Management company in Toronto on a full time basis. The successful candidates will be responsible for leading projects end-to-end within the Data Distribution Team or Asset Management Team . This role is heavily hands on and requires strong...

  • Sr/Lead Data Engineer

    4 weeks ago

    Toronto, Canada Asset Management Industry Full time

    Good dayData Engineers !Insight Global is looking for 6 Sr or LeadData Engineersto join a large Asset Management company in Toronto on a full time basis. The successful candidates will be responsible for leading projects end-to-end within theData Distribution TeamorAsset Management Team . This role is heavily hands on and requires strong experience with...

  • Sr/Lead Data Engineer

    1 month ago

    Toronto, Canada Asset Management Industry Full time

    Good day Data Engineers!Insight Global is looking for 6 Sr or Lead Data Engineers to join a large Asset Management company in Toronto on a full time basis. The successful candidates will be responsible for leading projects end-to-end within the Data Distribution Team or Asset Management Team. This role is heavily hands on and requires strong experience with...

  • Sr/Lead Data Engineer

    1 month ago

    Toronto, Canada Asset Management Industry Full time

    Good day Data Engineers ! Insight Global is looking for 6 Sr or Lead Data Engineers to join a large Asset Management company in Toronto on a full time basis. The successful candidates will be responsible for leading projects end-to-end within the Data Distribution Team or Asset Management Team . This role is heavily hands on and requires strong experience...

  • Sr/Lead Data Engineer

    1 month ago

    Toronto, Canada Asset Management Industry Full time

    Good day Data Engineers!Insight Global is looking for 6 Sr or Lead Data Engineers to join a large Asset Management company in Toronto on a full time basis. The successful candidates will be responsible for leading projects end-to-end within the Data Distribution Team or Asset Management Team. This role is heavily hands on and requires strong experience with...

  • Lead Data Engineer

    3 weeks ago

    Toronto, Canada Crypto Recruiters Full time

    We are actively searching for a Lead Data Engineer to join our team on a permanent basis. This role does require onsite work as you will be leading a team that is building a data product that will give insights to investors in the crypto and Web3 industry. In this role, you will be responsible for designing and building cutting-edge data analysis systems,...

  • Lead Data Engineer

    3 weeks ago

    Toronto, Canada Crypto Recruiters Full time

    We are actively searching for a Lead Data Engineer to join our team on a permanent basis. This role does require onsite work as you will be leading a team that is building a data product that will give insights to investors in the crypto and Web3 industry. In this role, you will be responsible for designing and building cutting-edge data analysis systems,...

  • Lead Data Engineer

    3 weeks ago

    Toronto, Canada Crypto Recruiters Full time

    We are actively searching for a Lead Data Engineer to join our team on a permanent basis. This role does require onsite work as you will be leading a team that is building a data product that will give insights to investors in the crypto and Web3 industry. In this role, you will be responsible for designing and building cutting-edge data analysis systems,...