Data Engineer
3 weeks ago
Our client is a VC-backed high-growth startup building software to make access to knowledge and support easy and instant for the Franchising Industry. The software can look through all information stored across operating manuals, file management systems, training videos, chat groups, etc., and provide summarized answers to franchisee operational questions instantly. However, their end goal is to become the universal interface into all elements of their customer's tech stack. Enabling our customers to query their data, launch workflows and run their operations using simple natural language from one place.
As the Data Engineer, you would own the ingestion process and the challenging problem of reliably converting various data formats into a canonical and well-organized standard. Moreover, you would own our orchestration pipeline for the entire data scraping/ingestion process with an eye toward building a reliable and robust system that can scale effectively to hundreds/thousands of customer organizations.
Overview of Responsibilities:
- Own and lead implementation for ingesting scraped customer data effectively and reliably into information amenable to use in a Retrieval Augmented Generation (RAG) pipeline.
- Own and upgrade our orchestration pipeline with an eye towards rapid and reliable refreshes of customer data and clear visibility into the setup for business purposes.
- Collaborate with ML Engineers to design and implement novel techniques to surface valuable information from user documentation using LLMs, multimodal models etc.
- Play an influential role in engineering discussions by lending technical expertise and contributing to planning on other projects.
Required skills and experience:
- Prior experience working as a data engineer or backend engineer with a heavy focus on data engineering/orchestration.
- Strong experience with Python programming.
- Experience working with SQL-based databases such as PostgreSQL.
- Have set up and overseen an orchestration pipeline (preferably Airflow)
- Ability to make informed decisions trading off speed versus stability depending on business needs.
Desirable skills and experience:
- Experience working with messy customer data including PDFs, Excel sheets, scraped website information etc.
- Prior experience working with AWS and setting up Lambdas, or Elastic Container Service clusters.
- History of working on other DevOps tasks such as setting up CI/CD pipelines, setting up infrastructure as code, security, etc.
- Experience working in conjunction with information retrieval/machine learning teams.
*** Experience working in Seed or Series A level companies is a must have requirement***
-
Data Engineer
2 weeks ago
Canada KData AI Full timeCompany DescriptionKData AI is a leading firm in Data and AI Engineering in the cloud, dedicated to delivering exceptional results for businesses worldwide. We specialize in transforming data into actionable insights through innovative AI and engineering solutions. Follow us on Twitter: @KDataAI.Role DescriptionThis is a full time role for a Data Engineer....
-
Data Engineer
2 weeks ago
Canada KData AI Full timeCompany DescriptionKData AI is a leading firm in Data and AI Engineering in the cloud, dedicated to delivering exceptional results for businesses worldwide. We specialize in transforming data into actionable insights through innovative AI and engineering solutions. Follow us on Twitter: @KDataAI.Role DescriptionThis is a full time role for a Data Engineer....
-
Data Engineer
7 days ago
Canada Summer Full timeTitle: Data EngineerLocation: Remote CanadaAbout this Role: As a Data Engineer, you will play a crucial role in designing, building, and optimizing our data infrastructure. You'll be responsible for developing scalable data pipelines, implementing ETL processes, and ensuring the quality and integrity of our data. Your expertise will contribute to the success...
-
Data Engineer
5 days ago
Canada Summer Full timeTitle: Data Engineer Location: Remote Canada About this Role: As a Data Engineer, you will play a crucial role in designing, building, and optimizing our data infrastructure. You'll be responsible for developing scalable data pipelines, implementing ETL processes, and ensuring the quality and integrity of our data. Your expertise will contribute to the...
-
Data Engineer
3 weeks ago
Canada hireVouch Full timeOur client is a leading data and cloud services provider that works with enterprise companies to build systems and infrastructures for them. As a Data Engineer, you'd be responsible for designing, building, and optimizing data pipelines, ensuring seamless data flow and analytics capabilities. This is a 6-month contract, with a possibility of extension. ...
-
Data Engineer
1 week ago
Canada hireVouch Full timeOur client is a leading data and cloud services provider that works with enterprise companies to build systems and infrastructures for them. As a Data Engineer, you'd be responsible for designing, building, and optimizing data pipelines, ensuring seamless data flow and analytics capabilities. This is a 6-month contract, with a possibility of extension. ...
-
Data Engineer
1 day ago
Canada IT Chapter Full timeThe Data Engineer will be responsible for leading the delivery of data on data and analytics platform. These services are at the forefront of building out data engineering practice cloud-native technologies. The Senior Data Engineer Service Provider must have experience in leading, designing, implementing, and collaborating with stakeholders to achieve the...
-
Data Engineer
1 week ago
Canada hireVouch Full timeOur client is a leading data and cloud services provider that works with enterprise companies to build systems and infrastructures for them.As a Data Engineer, you'd be responsible for designing, building, and optimizing data pipelines, ensuring seamless data flow and analytics capabilities.This is a 6-month contract, with a possibility of...
-
Data Engineer
3 weeks ago
Canada hireVouch Full timeOur client is a leading data and cloud services provider that works with enterprise companies to build systems and infrastructures for them. As a Data Engineer, you'd be responsible for designing, building, and optimizing data pipelines, ensuring seamless data flow and analytics capabilities. This is a 6-month contract, with a possibility of extension. ...
-
Data Engineer
2 weeks ago
Canada Sage Recruiting Full timeJob Description: Senior Data EngineerLocation: 100% Remote (Anywhere in Canada)Salary: $160,000 - $180,000 baseBenefits: Full benefits (dental, medical, health) and unlimited vacationAbout Our Client: Our client is a well-funded fintech startup that's changing the personal banking experience and redefining how people manage their finances. Our client's...
-
Data Engineer
3 days ago
Canada Sage Recruiting Full timeJob Description: Senior Data Engineer Location: 100% Remote (Anywhere in Canada) Salary: $160,000 - $180,000 base Benefits: Full benefits (dental, medical, health) and unlimited vacation About Our Client: Our client is a well-funded fintech startup that's changing the personal banking experience and redefining how people manage their finances. Our...
-
Data Engineer
2 days ago
Canada Sage Recruiting Full timeJob Description: Senior Data Engineer Location: 100% Remote (Anywhere in Canada) Salary: $160,000 - $180,000 base Benefits: Full benefits (dental, medical, health) and unlimited vacation About Our Client: Our client is a well-funded fintech startup that's changing the personal banking experience and redefining how people manage their finances. Our client's...
-
Data Engineer
2 days ago
Canada Sage Recruiting Full timeJob Description: Senior Data EngineerLocation: 100% Remote (Anywhere in Canada)Salary: $160,000 - $180,000 baseBenefits: Full benefits (dental, medical, health) and unlimited vacationAbout Our Client: Our client is a well-funded fintech startup that's changing the personal banking experience and redefining how people manage their finances. Our client's...
-
Data Engineer
2 weeks ago
Canada TMF Group Full timeWe never ask for payment as part of our selection process, and we always contact candidates via our corporate accounts and platforms. If you are approached for payment, this is likely to be fraudulent. Please check to see whether the role you are interested in is posted on our career website.JOB DESCRIPTION We are looking for a Data Engineer to join our...
-
Data Engineer
4 weeks ago
Canada TMF Group Full timeWe never ask for payment as part of our selection process, and we always contact candidates via our corporate accounts and platforms. If you are approached for payment, this is likely to be fraudulent. Please check to see whether the role you are interested in is posted on our career website.JOB DESCRIPTION We are looking for a Data Engineer to join our...
-
Data Engineer
11 hours ago
Canada TMF Group Full timeWe never ask for payment as part of our selection process, and we always contact candidates via our corporate accounts and platforms. If you are approached for payment, this is likely to be fraudulent. Please check to see whether the role you are interested in is posted on our career website. JOB DESCRIPTION We are looking for a Data Engineer to join our...
-
Data Engineer
4 weeks ago
Canada TMF Group Full timeWe never ask for payment as part of our selection process, and we always contact candidates via our corporate accounts and platforms. If you are approached for payment, this is likely to be fraudulent. Please check to see whether the role you are interested in is posted on our career website. JOB DESCRIPTION We are looking for a Data Engineer to join our...
-
Data Engineer
4 weeks ago
Canada, CA TMF Group Full timeWe never ask for payment as part of our selection process, and we always contact candidates via our corporate accounts and platforms. If you are approached for payment, this is likely to be fraudulent. Please check to see whether the role you are interested in is posted on our career website.JOB DESCRIPTION We are looking for a Data Engineer to join our...
-
Data Engineer
4 weeks ago
Canada TMF Group Full timeWe never ask for payment as part of our selection process, and we always contact candidates via our corporate accounts and platforms. If you are approached for payment, this is likely to be fraudulent. Please check to see whether the role you are interested in is posted on our career website. JOB DESCRIPTION We are looking for a Data Engineer to join our...
-
Data Engineer
4 weeks ago
Canada TMF Group Full timeWe never ask for payment as part of our selection process, and we always contact candidates via our corporate accounts and platforms. If you are approached for payment, this is likely to be fraudulent. Please check to see whether the role you are interested in is posted on our career website. JOB DESCRIPTION We are looking for a Data Engineer to join our...
