AI Model Evaluation Engineer for Enterprise Language Generation

5 days ago


Montreal, Quebec, Canada ServiceNow Full time
About Us

At ServiceNow, we're redefining the future of work. Our Advanced Technology Group (ATG) is a customer-focused innovation team that builds intelligent software and smart user experiences using advanced technologies.

We're a group of researchers, applied scientists, engineers, and product managers with a dual mission: building and evolving our AI platform, and partnering with teams to build products and end-to-end AI-powered work experiences.

Job Overview

We're seeking an experienced AI Model Evaluation Engineer to join our team. As a key member of our research and development team, you'll play a major role in building next-generation Large Language Models (LLMs) for enterprise language generation.

You'll work closely with our cross-functional team to develop innovative solutions that power our NOW platform with AI-driven experiences for our customers.

Responsibilities
  • Apply your expertise in AI/ML to tackle real-world challenges and datasets, developing practical and scalable solutions.
  • Research, propose, and implement suitable models and techniques for LLM benchmarking.
  • Participate in data collection and synthetic data generation, creating custom benchmarks to evaluate diverse LLM capabilities.
  • Collaborate daily with our team of developers, applied research scientists, product managers, and quality engineers to deliver high-quality software.
  • Contribute to the development of tools for LLM evaluation and perform model/error analysis.
  • Develop innovative, patentable ideas to ensure our product remains competitive within the industry.
  • Potentially co-author research papers with our Research Group for publication at top AI conferences.
  • Work closely with product owners to understand detailed requirements and own code from design to delivery of high-impact solutions.
Qualifications

To succeed in this role, you'll need:

  • 4+ years of relevant experience or equivalent work experience with a Bachelor's degree, or 2 years with a Master's degree, or a PhD with no experience.
  • Solid expertise in Python programming.
  • Solid theoretical knowledge and hands-on experience with Machine Learning/Deep Learning.
  • Experience with LLM benchmarking and evaluation is essential; experience generating synthetic data is a plus.
  • The ability to read and experiment with recent research papers.
  • A high level of creativity, quick problem-solving abilities, and adaptability.
  • Effective communication skills to convey research findings to both technical and non-technical stakeholders.
Benefits

We offer a range of benefits, including:

  • A competitive estimated salary range of $160,000 - $220,000 per year.
  • A flexible, remote-friendly work environment.
  • Ongoing professional development and growth opportunities.
  • A collaborative, inclusive team culture.
  • A comprehensive benefits package.


  • Montreal, Quebec, Canada ServiceNow Full time

    Transform the Future of Work with ServiceNowWe are seeking a highly skilled Senior AI Researcher to join our Advanced Technology Group, driving innovation in Large Language Models (LLMs) for Enterprise Language Generation. As a key member of our team, you will play a pivotal role in building next-generation LLMs that power AI experiences on the NOW platform,...

  • AI Engineer

    2 weeks ago


    Montreal, Quebec, Canada Alexa Translations Full time

    About Alexa TranslationsAlexa Translations specializes in providing translation services for the legal, financial, and securities sectors. Leveraging proprietary A.I. technology and a team of highly specialized linguistic experts, we deliver unmatched speed and quality. Our machine translation engine is best-in-class and specifically trained for the...


  • Montreal, Quebec, Canada Servicenow Full time

    About the RoleAt Servicenow, we are seeking a skilled Data Scientist to join our Advanced Technology Group. The ideal candidate will have expertise in Python and Machine Learning/Deep Learning theoretical knowledge and hands-on experience.This role involves building next generation Large Language Models (LLMs) for Enterprise Language Generation that will...


  • Montreal, Quebec, Canada Reliant AI Full time

    About Reliant AIWe're pushing the boundaries of machine learning to revolutionize information analysis. Our team is dedicated to developing cutting-edge generative AI software that extracts insights from vast datasets.Job DescriptionSalary: $180,000 per yearJob Title: Senior Machine Learning Engineer - Generative AI ExpertWe're seeking an experienced senior...


  • Montreal, Quebec, Canada Reliant AI, Inc. Full time

    We're seeking a seasoned software architect to lead the development of our next-generation machine learning platform. This is an exciting opportunity to design and implement scalable, distributed systems that leverage generative AI to provide comprehensive answers.The ideal candidate will have extensive experience in architecting and delivering machine...

  • Software Engineer

    1 month ago


    Montreal, Quebec, Canada Maxa AI Full time

    About MaxaMaxa AI is revolutionizing the way businesses make sense of their ERP and business data. By automating the process of turning raw data into actionable insights, we're empowering executives and their teams to make data-driven decisions.The RoleWe're seeking a highly motivated Software Engineer with expertise in natural language processing (LLMs) to...


  • Montreal, Quebec, Canada Reliant AI, Inc. Full time

    About Reliant AI, Inc.:We're a pioneering force in machine learning software, harnessing the power of generative AI to revolutionize information analysis. Our cutting-edge algorithms examine key sources and deliver comprehensive answers for even the most intricate queries.Our vision is to unlock the transformative impact of generative AI by tackling the...

  • AI Engineering Leader

    4 weeks ago


    Montreal, Quebec, Canada 2K Full time

    Principal AI EngineerWe are seeking a highly skilled Principal AI Engineer to lead our most ambitious AI-driven projects. In this role, you will shape the future of AI at 2K by developing and deploying advanced AI models and systems. You'll work with innovative tools such as Generative AI and LLaMA APIs from Meta to push the boundaries of what's possible....

  • Data Management Lead

    1 month ago


    Montreal, Quebec, Canada 2K Full time

    Job Title: Data Management LeadJob Summary:We are seeking a highly skilled Data Management Lead to join our team. The successful candidate will be responsible for managing, leading, and motivating teams to achieve project goals. This role requires strong leadership skills, excellent communication, and the ability to work closely with project managers and...


  • Montreal, Quebec, Canada Alexa Translations Full time

    Alexa Translations is seeking a highly skilled Senior Generative AI Engineer to join our team. As a key member of our organization, you will be responsible for developing and implementing state-of-the-art algorithms using Generative AI (GenAI) and Large Language Model (LLM) technologies.About the RoleAs a Senior Generative AI Engineer, you will report...

  • AI Prompt Specialist

    1 month ago


    Montreal, Quebec, Canada AI Square Corp Full time

    AI Square Corp: AI Prompt EngineerWe are seeking a skilled AI Prompt Engineer to contribute to our team of AI consulting experts. The ideal candidate should possess a strong background in linguistics and experience working with large language models such as ChatGPT or Bard.Key Responsibilities:Design and develop text-based prompts to optimize AI tool...


  • Montreal, Quebec, Canada Cerence Full time

    Cerence, the global leader in AI for transportation, is seeking an experienced AI Engineering Leader to lead the design, development, and implementation of large language models (LLMs) for our software products.What You Will Do:Lead a team of software developers in the creation of cutting-edge LLMs that power our speech recognition and natural language...


  • Montreal, Quebec, Canada Reliant AI Full time

    About Reliant AI:We are a cutting-edge technology company that leverages generative AI to revolutionize the way we analyze information. Our mission is to empower users with comprehensive, factual answers to even the most complex queries.Our Approach:We combine our expertise in reinforcement learning and natural language processing to develop innovative...


  • Montreal, Quebec, Canada Alignerr Full time

    Alignerr is a community of subject matter experts who align AI models by creating high-quality data in their field of expertise to build the future of Generative AI. Our platform is used by Fortune 500 enterprises and leading AI teams. We are dedicated to achieving excellence and recognize the importance of bringing our talented team together.About the...

  • AI Research Scientist

    4 weeks ago


    Montreal, Quebec, Canada Ai Build Limited Full time

    Job Title: AI Research ScientistJob Summary:We are seeking a highly skilled AI Research Scientist to join our team at Ai Build Limited. As a key member of our Advanced Engineering group, you will be responsible for conducting comprehensive literature surveys, designing Python-level algorithm reference code, and planning test metrics and evaluation...


  • Montreal, Quebec, Canada Reliant AI Full time

    About Reliant AIWe're a cutting-edge AI company that's revolutionizing the way we approach information analysis. Our mission is to empower individuals and organizations to make informed decisions by harnessing the power of generative AI.Job DescriptionWe're seeking a highly skilled Senior Machine Learning Engineer to join our team. As a key member of our...


  • Montreal, Quebec, Canada Reliant AI Full time

    About Reliant AIWe're a team of scientists, builders, and entrepreneurs passionate about harnessing the power of generative AI to transform the biopharma industry. Our mission is to develop cutting-edge machine learning software that analyzes key information sources and provides comprehensive, factual answers to complex queries.We're looking for a senior...


  • Montreal, Quebec, Canada Ai Brainer Full time

    AI Consultant Position at Mirabilis.aiWe are seeking an experienced AI Consultant to join our team at Mirabilis.ai, a leading AI consulting firm specializing in AI integration and deployment.About the RoleThis is an exciting opportunity for a talented individual to work with clients to understand their business goals, challenges, and opportunities related to...


  • Montreal, Quebec, Canada Ai Build Limited Full time

    Company OverviewAi Build Limited, a leading company in AI-based low-level sensor fusion and perception software, is seeking an experienced researcher to join its Advanced Engineering group.We specialize in developing advanced driver assistance systems (ADAS), autonomous driving (AD), and parking applications utilizing AI and computer vision algorithms. Our...


  • Montreal, Quebec, Canada Servicenow Full time

    At ServiceNow, we're shaping the future of work with our innovative, AI-driven platform. As a key contributor to our Foundation Models Lab, you'll play a pivotal role in designing and implementing cutting-edge language models that power our enterprise applications.About the RoleWe're seeking an exceptional AI systems developer to join our team and drive the...