Data Engineer
2 weeks ago
Our client is a VC-backed high-growth startup building software to make access to knowledge and support easy and instant for the Franchising Industry. The software can look through all information stored across operating manuals, file management systems, training videos, chat groups, etc., and provide summarized answers to franchisee operational questions instantly. However, their end goal is to become the universal interface into all elements of their customer's tech stack. Enabling our customers to query their data, launch workflows and run their operations using simple natural language from one place.
As the Data Engineer, you would own the ingestion process and the challenging problem of reliably converting various data formats into a canonical and well-organized standard. Moreover, you would own our orchestration pipeline for the entire data scraping/ingestion process with an eye toward building a reliable and robust system that can scale effectively to hundreds/thousands of customer organizations.
Overview of Responsibilities:
- Own and lead implementation for ingesting scraped customer data effectively and reliably into information amenable to use in a Retrieval Augmented Generation (RAG) pipeline.
- Own and upgrade our orchestration pipeline with an eye towards rapid and reliable refreshes of customer data and clear visibility into the setup for business purposes.
- Collaborate with ML Engineers to design and implement novel techniques to surface valuable information from user documentation using LLMs, multimodal models etc.
- Play an influential role in engineering discussions by lending technical expertise and contributing to planning on other projects.
Required skills and experience:
- Prior experience working as a data engineer or backend engineer with a heavy focus on data engineering/orchestration.
- Strong experience with Python programming.
- Experience working with SQL-based databases such as PostgreSQL.
- Have set up and overseen an orchestration pipeline (preferably Airflow)
- Ability to make informed decisions trading off speed versus stability depending on business needs.
Desirable skills and experience:
- Experience working with messy customer data including PDFs, Excel sheets, scraped website information etc.
- Prior experience working with AWS and setting up Lambdas, or Elastic Container Service clusters.
- History of working on other DevOps tasks such as setting up CI/CD pipelines, setting up infrastructure as code, security, etc.
- Experience working in conjunction with information retrieval/machine learning teams.
*** Experience working in Seed or Series A level companies is a must have requirement***
-
Databricks Data Engineer
22 hours ago
Canada Data Elephant Full timeAbout UsWe're a boutique data and analytics consulting firm based in Vancouver, BC, helping Canadian organizations develop data strategies, modernize their data platforms and unlock the power of data, analytics, ML and GenAI. As a certified partner with leading cloud platforms, including AWS, Azure and Databricks, we deliver strategic solutions that create...
-
Data Engineering Lead for Cloud Data Platforms
19 hours ago
Canada Data Elephant Full timeAbout Data ElephantWe're a boutique data and analytics consulting firm based in Vancouver, BC, helping Canadian organizations develop data strategies, modernize their data platforms, and unlock the power of data, analytics, ML, and GenAI.Our MissionWe deliver strategic solutions that create real business impact. Our team thrives on solving complex...
-
Data Engineer
3 weeks ago
Canada KData AI Full timeCompany DescriptionKData AI is a leading firm in Data and AI Engineering in the cloud, dedicated to delivering exceptional results for businesses worldwide. We specialize in transforming data into actionable insights through innovative AI and engineering solutions. Follow us on Twitter: @KDataAI.Role DescriptionThis is a full time role for a Data Engineer....
-
Data Engineer
2 weeks ago
Canada KData AI Full timeCompany DescriptionKData AI is a leading firm in Data and AI Engineering in the cloud, dedicated to delivering exceptional results for businesses worldwide. We specialize in transforming data into actionable insights through innovative AI and engineering solutions. Follow us on Twitter: @KDataAI.Role DescriptionThis is a full time role for a Data Engineer....
-
Data Engineer
2 weeks ago
Canada Summer Full timeTitle: Data EngineerLocation: Remote CanadaAbout this Role: As a Data Engineer, you will play a crucial role in designing, building, and optimizing our data infrastructure. You'll be responsible for developing scalable data pipelines, implementing ETL processes, and ensuring the quality and integrity of our data. Your expertise will contribute to the success...
-
Data Engineer
1 week ago
Canada Summer Full timeTitle: Data Engineer Location: Remote Canada About this Role: As a Data Engineer, you will play a crucial role in designing, building, and optimizing our data infrastructure. You'll be responsible for developing scalable data pipelines, implementing ETL processes, and ensuring the quality and integrity of our data. Your expertise will contribute to the...
-
Data Engineer
2 days ago
Canada United Software Group Inc Full timeJob Title: Data EngineerRemote work12 Months ContractPosition overview : Help us create our Financial Data Warehouse using dbt, Dagster, Snowflake and Power BIResponsibilities :Build our data pipelines for our Financial Data WarehouseBuild power bi reports and dashboards associated to this processContribute to our strategy for new data pipelines and data...
-
Data Engineer
4 weeks ago
Canada hireVouch Full timeOur client is a leading data and cloud services provider that works with enterprise companies to build systems and infrastructures for them. As a Data Engineer, you'd be responsible for designing, building, and optimizing data pipelines, ensuring seamless data flow and analytics capabilities. This is a 6-month contract, with a possibility of extension. ...
-
Data Engineer
2 weeks ago
Canada hireVouch Full timeOur client is a leading data and cloud services provider that works with enterprise companies to build systems and infrastructures for them. As a Data Engineer, you'd be responsible for designing, building, and optimizing data pipelines, ensuring seamless data flow and analytics capabilities. This is a 6-month contract, with a possibility of extension. ...
-
Data Engineer
6 days ago
Canada IT Chapter Full timeThe Data Engineer will be responsible for leading the delivery of data on data and analytics platform. These services are at the forefront of building out data engineering practice cloud-native technologies. The Senior Data Engineer Service Provider must have experience in leading, designing, implementing, and collaborating with stakeholders to achieve the...
-
Data Engineer
2 weeks ago
Canada hireVouch Full timeOur client is a leading data and cloud services provider that works with enterprise companies to build systems and infrastructures for them.As a Data Engineer, you'd be responsible for designing, building, and optimizing data pipelines, ensuring seamless data flow and analytics capabilities.This is a 6-month contract, with a possibility of...
-
Data Engineer
22 hours ago
Canada Quantum World Technologies Inc. Full time8 year's experience in IT with a focus on data engineering and architecture. In addition to deep knowledge of PostgreSQL, the candidate should have expertise of Snowflake or ElasticSearch.In-depth knowledge of architecture, functions, and data warehousing concepts. ETL & Data Modeling: Advanced skills in ETL processes, data modeling, and data warehousing....
-
Data Engineer
4 weeks ago
Canada hireVouch Full timeOur client is a leading data and cloud services provider that works with enterprise companies to build systems and infrastructures for them. As a Data Engineer, you'd be responsible for designing, building, and optimizing data pipelines, ensuring seamless data flow and analytics capabilities. This is a 6-month contract, with a possibility of extension. ...
-
Data Engineer
2 weeks ago
Canada Sage Recruiting Full timeJob Description: Senior Data EngineerLocation: 100% Remote (Anywhere in Canada)Salary: $160,000 - $180,000 baseBenefits: Full benefits (dental, medical, health) and unlimited vacationAbout Our Client: Our client is a well-funded fintech startup that's changing the personal banking experience and redefining how people manage their finances. Our client's...
-
Data Engineer
7 days ago
Canada Sage Recruiting Full timeJob Description: Senior Data Engineer Location: 100% Remote (Anywhere in Canada) Salary: $160,000 - $180,000 base Benefits: Full benefits (dental, medical, health) and unlimited vacation About Our Client: Our client is a well-funded fintech startup that's changing the personal banking experience and redefining how people manage their finances. Our client's...
-
Data Engineer
1 week ago
Canada Sage Recruiting Full timeJob Description: Senior Data Engineer Location: 100% Remote (Anywhere in Canada) Salary: $160,000 - $180,000 base Benefits: Full benefits (dental, medical, health) and unlimited vacation About Our Client: Our client is a well-funded fintech startup that's changing the personal banking experience and redefining how people manage their finances. Our...
-
Data Engineer
7 days ago
Canada Sage Recruiting Full timeJob Description: Senior Data EngineerLocation: 100% Remote (Anywhere in Canada)Salary: $160,000 - $180,000 baseBenefits: Full benefits (dental, medical, health) and unlimited vacationAbout Our Client: Our client is a well-funded fintech startup that's changing the personal banking experience and redefining how people manage their finances. Our client's...
-
Data Engineer
17 hours ago
Canada Haptiq Full timeOverview Haptiq Technology and Solutions, a leader in uniting companies and their investors through innovative technology, is looking for a Principal Fullstack Engineer to join our dynamic team in Canada. Haptiq excels in transforming potential into tangible performance, offering a suite of meticulously designed software solutions that cater to a wide...
-
Data Engineer
22 hours ago
Canada Avanciers Inc. Full timeAvanciers is hiring GCP - Data Engineer at Vancouver, BC for one of our Fortune 500 clients. This is a Full-Time role with the client. Benefits will be covered by the Client. Job Title: GCP - Data Engineer Location: Vancouver – Onsite Duration: Full Time Role Key Responsibilities:Should be proficient in writing SQL & Python scripts to implement.Should have...
-
Data Engineer
3 weeks ago
Canada TMF Group Full timeWe never ask for payment as part of our selection process, and we always contact candidates via our corporate accounts and platforms. If you are approached for payment, this is likely to be fraudulent. Please check to see whether the role you are interested in is posted on our career website.JOB DESCRIPTION We are looking for a Data Engineer to join our...
