Senior Data Engineer

1 week ago

Canada, CA DeepRec.ai Full time

Senior Data Engineer – ML Services and Data Pipelines

Location: Remote

Employment Type: Contract

Experience: 5+ years

3 month contract + multiple extensions possible


Our client is based in the USA but employs a remote workforce. You should be available for team meetings at 8am PT and have some cross over for team collaboration in PST/ET timezone. Happy to hire remote worldwide.


We are seeking a skilled and experienced Data Engineer to design and implement scalable and extensible Machine Learning services and data pipelines within our AWS/Kubernetes environment. In this role, you will be responsible for setting up infrastructure to support the ingestion, processing, and indexing of text-based data. You will collaborate closely with our ML and software engineering teams to build robust pipelines and enable efficient data processing and document indexing.


Key Responsibilities


  • ML Infrastructure Setup: Design, implement, and maintain scalable ML infrastructure on AWS and Kubernetes to support current and future project needs. Tools like Amazon SageMaker for model development, deployment, and monitoring, and AWS Lambda for serverless computing will be key.
  • Data Pipeline Development: Develop, deploy, and manage data pipelines that process large volumes of data for machine learning use cases, with a focus on efficient text data processing. Use AWS Glue for ETL jobs, Amazon Kinesis for real-time data streaming, and AWS Step Functions to coordinate workflows.
  • Vector Database Integration: Set up and maintain vector databases to support natural language processing (NLP) models, ensuring efficient and accurate text-based data retrieval and analysis. Amazon OpenSearch Service and Amazon DynamoDB can be leveraged for indexing and storing large volumes of vectorized data.
  • Document Indexing System: Design and implement a system for document ingestion and indexing, providing seamless access to data for downstream ML and analytics processes. Tools like Amazon S3 for scalable storage and AWS Lambda for automation and processing will play a critical role.
  • Development Pipeline Setup: Establish a development pipeline for document ingestion, collaborating with DevOps and data science teams to ensure continuous integration and deployment practices using AWS CodePipeline and AWS CodeBuild.


Required Skills & Qualifications



  • Experience: 5+ years of experience in data engineering with a focus on ML infrastructure and data pipelines.
  • Technical Expertise:
  • Strong background in AWS and Kubernetes for deploying and managing scalable ML and data solutions.
  • Proficiency in data pipeline tools and frameworks such as AWS Glue, Amazon Kinesis, AWS Step Functions, or similar.
  • Experience with text-based data processing, NLP techniques, and vector databases (e.g., Amazon OpenSearch Service, Amazon DynamoDB, or third-party vector databases like Pinecone, Weaviate, or FAISS).
  • Programming Skills: Advanced skills in Python or similar languages for data engineering and ML pipeline development.
  • Data Handling & Storage: Proficiency in data storage solutions, including Amazon S3, AWS Redshift, Amazon RDS, and AWS Data Lake.
  • Problem-Solving Abilities: Ability to troubleshoot and resolve issues within ML pipelines and data processing environments.


Preferred Qualifications


  • Hands-on experience with ML Ops frameworks and practices.
  • Familiarity with continuous integration/continuous deployment (CI/CD) for data engineering workflows using AWS CodePipeline, AWS CodeBuild, and AWS CloudFormation.

  • Senior AI Data Engineer

    4 days ago

    Canada, CA Intuitive.Cloud Full time

    About us:Intuitive.Cloud is one of the fastest-growing (INC 5000, CRN) Cloud & SDx solution and services companies supporting enterprise customers on a global scale. Intuitive is an "Engineering Company" delivering measurable value and key business outcomes.Intuitive Superpowers:- DataOps & AI/ML- Cloud Native, AppSecOps, DevSecOps- Cloud Migration &...

  • Senior Consultant

    5 months ago

    Canada, CA LevelUp Finance - An Array Company Full time

    Position: Senior Consultant - Financial & Operational Business Intelligence / Data EngineerLocation: Remote in CanadaExperience:At least 5 years of experience using SQL & Power BI in the workplace.Experience with accounting/finance, sales operations, or other data-intensive business operations is a plus.Experience in planning applications (e.g., Adaptive,...

  • Data Engineer

    1 month ago

    Canada, CA Electric Mind inc. Full time

    As Data Engineer, you will provide hands-on expertise as it relates to data management, analytics, and business intelligence. This opportunity will allow you to utilize your unique blend of technical, software and cloud engineering experience.While working with teams in an Agile environment, you will be:Designing high quality data pipelines/architectures...

  • Data Engineer

    4 weeks ago

    Canada, CA Capgemini Full time

    As a Data Engineer, you will play a pivotal role in transforming operations by ensuring high-quality data and efficient tooling. This position involves building, maintaining, and optimizing data pipelines to drive insights, improve product solutions, and support cross-functional teams.Experience:5+ years of relevant experience in software development,...

  • Sr. Azure Data Engineer

    2 days ago

    Canada, CA Samay Consulting Full time

    At Samay India, we are looking to build a Centre of Excellence for our BI team and are actively looking to hire Azure Data Engineers in both Architect and Senior level developer roles. Prior experience building pipelines and transforming data using Azure Data Factory is mandatory. Knowledge of a Cloud Datawarehouse like Azure Synapse or Snowflake is also...

  • Data Engineer

    2 days ago

    Canada, CA MSR Technology Group Full time

    Position: AWS Data EngineerLocation: Toronto, ONMandatory Skills: AWS, Glue, SQL, Redshift, LambdaJob Description:10+ years of experience in Data Engineer with AWS, Glue, Lambda, SQL, Python, DevOps, Redshift.Must have working knowledge in designing and implementing data pipelines on any of the cloud providers (AWS is preferred). Must be able to work with...

  • Data Engineer

    1 week ago

    Canada, CA The Personal Development School Full time

    Job DescriptionThe Personal Development School is an e-learning platform in the emotional wellness space. It teaches courses on personal growth, mental health, and well-being with a focus on relationships and attachment theory. We seek a skilled Data Engineer with Data Analytics and development expertise to enhance our data infrastructure and analytics...

  • Senior Scala Engineer

    1 month ago

    Canada, CA Signify Technology Full time

    Senior Scala Engineer - 12 month - No Sponsorship, No Third PartiesSummary:We are seeking a talented Senior Scala Specialist to join our dynamic team at a US-based company dedicated to building an innovative streaming platform. The ideal candidate will have extensive experience in Scala and functional programming, along with a strong understanding of AWS...

  • Data Engineer

    6 days ago

    Canada, CA technology& Full time

    Job Title: Data EngineerExperience Range: 8-14 yearsLocation: Canada (100% Remote, must be based within Canada)Job Type: ContractDuration: 6+ MonthsRate: CAD $60/hr - $65/hr (all-inclusive)Work Authorization: Open to all candidates legally authorized to work in Canada (No third-party candidates)Start Date: ASAPJob DescriptionAs a Data Engineer, you will play...

  • Cleared Data Engineer

    2 weeks ago

    Canada, CA OptiSource Ltd Full time

    OptiSource.We are professional, agile and our goal is to We are a fast growing company with existing funded contracts. We are always looking for talent in data engineering field. Currently we are teamed up with high end research groups and university to build innovative AI/ML solutions to be deployed in an enterprise. The opportunity allows building of...

  • GCP Data Engineer

    4 weeks ago

    Canada, CA Avanciers Full time

    GCP Data Engineer in Toronto- Hybrid (FTE/Contract)Job Description:10+ years of experience with Data Warehouse / Data Platforms5+ years of experience creating ELT data pipelines from scratch, working with structured, semi-structured, and unstructured data and SQL.Experience building data pipelines, and GCP Services, Dataproc, BigQuery, Cloud Spanner, Cloud...

  • Senior Full Stack Engineer

    1 month ago

    Canada, CA Space Executive Full time

    My client, an Insurtech scale-up originally headquartered in the US disrupting the Insurance market focusing on Data and AI is seeking a Senior Full Stack Engineer to join their team.This is a permanent role and will be fully remote. You can be based anywhere in Canada. The salary on on offer is market leading and you also receive stock options + a bonus...

  • Data Engineer I

    1 week ago

    Canada, CA NUMERATOR LIMITED Full time

    About the jobWe’re reinventing the market research industry. Let’s reinvent it together.At Numerator, we believe tomorrow’s success starts with today’s market intelligence. We empower the world’s leading brands and retailers with unmatched insights into consumer behavior and the influencers that drive it.Numerator is looking for a Data Engineer I...

  • Data QA Engineer

    4 weeks ago

    Canada, CA Silicontek Inc Full time

    Title: Data QA Engineer Location: Markham, Canada (Onsite, 3 days/week)Overview: We are seeking a Data QA Engineer with expertise in ETL, data warehousing, and QA automation to support enterprise-level data quality initiatives. This role requires strong analytical skills and hands-on experience in SQL, Python, and various data management tools.Key...

  • Senior LLM Engineer

    4 days ago

    Canada, CA Intuitive.Cloud Full time

    About us:Intuitive.Cloud is one of the fastest-growing (INC 5000, CRN) Cloud & SDx solution and services companies supporting enterprise customers on a global scale. Intuitive is an "Engineering Company" delivering measurable value and key business outcomes.Intuitive Superpowers:- DataOps & AI/ML- Cloud Native, AppSecOps, DevSecOps- Cloud Migration &...

  • Senior Data Scientist

    4 weeks ago

    Canada, CA Canterr, Inc. Full time

    Job DescriptionWe are looking for advanced analytical, action-oriented, and collaborative individuals that can translate business problems to actionable insights and data driven solutions to influence operational results. Successful individuals in this role excel at both qualitative and quantitative analysis while also being eager to communicate upward and...

  • AWS Data Engineer

    4 weeks ago

    Canada, CA Virtual Labs IT Consulting Full time

    6 years of data engineering experience with a proven track record of designing and implementing scalable ingestion framework.* 4 years hands on experience on the AWS platform using, Redshift, S3, DynamoDB, EMR serverless, Lambdas, Step Functions, etc.* Proficiency in ETL development and design using Python, PySpark, Big Data (Hadoop/SPARK)* Strong SQL...

  • ETL data engineer

    2 weeks ago

    Canada, CA HCLTech Full time

    Required Skills 5-8 years’ experience as Data Engineer Specializes in ETL (Extract, Transform, Load) using ETL tools DataStage (Primary), Matillion, ADF etc. Strong knowledge of relevant data query language e.g., SQL At least 3-4 years of US Healthcare Payer experience Experience in Snowflake, Netezza. Experience with Data warehousing techniques(Star...

  • Senior Clinical Data Manager

    1 month ago

    Canada, CA Warman O'Brien Full time

    Senior Clinical Data Manager | Remote | Global Pharma | Permanent | Up to $135,000 CADLocationHome-based, Remote WorkingSummaryA global CRO is looking for a Senior Clinical Data Manager to join their innovative and exciting team. You will get to work across different sponsors gaining a wealth of experience in data management activates. What’s it Like to...

  • Senior Software Engineer

    1 month ago

    Canada, CA Playwire Full time

    Playwire is a full-service digital innovation partner specializing in online advertising and proprietary technologies to elevate publishing brands. As we continue our rapid expansion, we are seeking a Senior Software Engineer (TypeScript) to join our team.This role will contribute to the development of our next-generation website revenue amplification...