AI Quality Engineer

1 week ago

Toronto, Ontario, Canada New York Technology Partners Full time

We are looking for a AI Quality Engineer with strong automation expertise and hands-on experience validating LLMs, GenAI workflows, and AI-driven applications. This role involves building automated test suites, validating AI outputs, ensuring system reliability, and contributing to the quality strategy for next-generation AI features.

Core Experience

  • 4-6 years as a Software Engineer, SDET, or Automation Engineer.
  • Strong coding skills in Python, TypeScript, or Java.
  • Hands-on experience developing automation scripts, tools, or frameworks.
  • Practical experience using LLMs, prompt engineering, and evaluating AI-generated outputs.
  • Familiarity with agentic AI systems and exposure to tools like LangGraph, AutoGen, or CrewAI.
  • Basic understanding of Model Context Protocol (MCP) or context-aware workflow automation (nice to have)

AI / ML Technologies

  • Practical experience with AI/ML frameworks:
  • LangChain, Hugging Face, GPT models, vector databases, RAG pipelines.
  • Experience with ML/DL libraries such as:
  • Scikit-learn, PyTorch, TensorFlow, Keras, Transformers, OpenCV.
  • Ability to work with embeddings, similarity search, and content evaluation metrics.
  • GenAI & AI Agent Development
  • Ability to integrate or build GenAI components, including RAG pipelines or agent-based workflows.
  • Support model evaluation tasks such as: Output quality checks Hallucination detection, Prompt validation ,Regression checks for model updates &Automation & Quality Engineering
  • Experience building automation using Python, PyTest, Selenium, Playwright, or API testing libraries.
  • Ability to design and execute automated tests for: Functional Integration ,API
  • Basic performance & reliability testing
  • Hands-on experience testing RESTful APIs and building automated API suites. Cloud, DevOps & CI/CD
  • Exposure to deploying AI or automation solutions on AWS.
  • Working knowledge of CI/CD pipelines, including: Automated testing ,Model validation steps
  • Versioning & artifact management SDLC & Collaboration

  • Engineering Manager

    1 week ago

    Toronto, Ontario, Canada Arteria AI Full time

    About Arteria AIArteria AI is an award-winning, AI-powered documentation platform that transforms documents into data, facilitating critical banking processes like trading and lending. Leading banks worldwide use Arteria to connect costly, manual documentation processes to enterprise automation, providing valuable insights to decision-makers.Founded in...

  • Quality Engineering

    1 week ago

    Toronto, Ontario, Canada Quality IT Resourcing Full time

    We are seeking a Quality Engineering / DevSecOps Consultant with deep expertise in automation, CI/CD, and secure software delivery pipelines within the Capital Markets and Trading domain. The ideal candidate will have a strong understanding of front-office trading systems, order management workflows, market data feeds, and post-trade processes, combined with...

  • Network Engineer, AI/ML Infrastructure

    7 days ago

    Toronto, Ontario, Canada Boson AI Full time

    About The Role We're seeking an experienced Network Engineer to design, build, and optimize the high-performance networking infrastructure powering our AI/ML operations in Toronto. You'll work at the cutting edge of network technology—managing InfiniBand and ultra-high-speed Ethernet fabrics that connect NVIDIA H100 and A100 GPUs, over 20PB of Ceph...

  • Senior Technical Program Manager II, Conversational AI

    2 weeks ago

    Toronto, Ontario, Canada SoundHound AI Full time

    Your Career, our Future—Together.Ready to join something big? At SoundHound AI, we bring voice, generative, and conversational AI together to transform how people interact with products and services. From voice-enabled vehicles to food ordering and customer support, our multilingual, omnichannel technology already impacts hundreds of millions worldwide.The...

  • AI Strategist

    1 week ago

    Toronto, Ontario, Canada Siena AI Full time

    Meet Siena Siena is the first intelligence layer for customer experience. We're creating an operating system of AI agents that learn, remember, and act across every customer touchpoint—from support conversations to shopping experiences to voice and social media interactions. Siena doesn't just automate support; it powers shopping agents, builds...

  • AI Strategist

    1 week ago

    Toronto, Ontario, Canada Siena AI Full time

    Meet SienaSiena is the first intelligence layer for customer experience. We're creating an operating system of AI agents that learn, remember, and act across every customer touchpoint—from support conversations to shopping experiences to voice and social media interactions.Siena doesn't just automate support; it powers shopping agents, builds persistent...

  • Machine Learning Engineer

    2 weeks ago

    Toronto, Ontario, Canada Boson AI Full time

    About Boson AI: At Boson AI, we are not just building AI solutions; we are pioneering the future of enterprise AI. Driven by a passion for cutting-edge AI research, particularly in the transformative areas of large language models and agentic systems, our mission is to tackle the most complex real-world problems for businesses and unlock significant value....

  • Machine Learning Engineer

    2 days ago

    Toronto, Ontario, Canada Boson AI Full time

    About Boson AI: At Boson AI, we are not just building AI solutions; we are pioneering the future of enterprise AI. Driven by a passion for cutting-edge AI research, particularly in the transformative areas of large language models and agentic systems, our mission is to tackle the most complex real-world problems for businesses and unlock significant value....

  • Network Engineer, AI/ML Infrastructure

    2 weeks ago

    Toronto, Ontario, Canada Boson AI Full time

    About The Role We're seeking an experienced Network Engineer to design, build, and optimize the high-performance networking infrastructure powering our AI/ML operations in Toronto. You'll work at the cutting edge of network technology—managing InfiniBand and ultra-high-speed Ethernet fabrics that connect NVIDIA H100 and A100 GPUs, over 20PB of Ceph...

  • Network Engineer, AI/ML Infrastructure

    2 weeks ago

    Toronto, Ontario, Canada Boson AI Full time US$150,000 - US$250,000

    About The RoleWe're seeking an experienced Network Engineer to design, build, and optimize the high-performance networking infrastructure powering our AI/ML operations in Toronto. You'll work at the cutting edge of network technology—managing InfiniBand and ultra-high-speed Ethernet fabrics that connect NVIDIA H100 and A100 GPUs, over 20PB of Ceph storage,...