Senior Research Scientist, Model Evaluation

15 hours ago

Toronto, Ontario, Canada Cohere Full time $120,000 - $180,000 per year

Who are we?

Our mission is to scale intelligence to serve humanity. We're training and deploying frontier models for developers and enterprises who are building AI systems to power magical experiences like content generation, semantic search, RAG, and agents. We believe that our work is instrumental to the widespread adoption of AI.

We obsess over what we build. Each one of us is responsible for contributing to increasing the capabilities of our models and the value they drive for our customers. We like to work hard and move fast to do what's best for our customers.

Cohere is a team of researchers, engineers, designers, and more, who are passionate about their craft. Each person is one of the best in the world at what they do. We believe that a diverse range of perspectives is a requirement for building great products.

Join us on our mission and shape the future

Why this role?

Evaluation is critical to making progress in scaling intelligence. As models continue to become superhuman in many real-world use cases, we must continue to develop new evaluation techniques that accurately reflect what models are already capable of, as well as set the agenda for what future models should be capable of. In this role, you are responsible for creating these next-generation evaluation methods and infrastructure to measure LLM progress.

As a Senior Research Scientist, Model Evaluation, you will:

  • Create ambitious new evaluation benchmarks that push the limits of what our models can accomplish.

  • Work on highly cross-functional teams to translate model feedback into trustworthy, repeatable evaluations.

  • Conduct research to advance the state-of-the-art in LLM evaluation methods, including training LLM judges; refining LLM-based data synthesis pipelines; and improving evaluation efficiency.

  • Build scalable and reusable tools for digging into model performance.

You may be a good fit if:

  • You enjoy rapidly building prototypes that demonstrate the boundaries of what LLMs are capable of, and you have developed resources to measure those capabilities.

  • You have spent dozens of hours reviewing complex data and LLM outputs to ensure high data quality.

  • You are obsessive about rigorously measuring AI capabilities, and also about making sure your measurements actually align with the capabilities you care about.

  • You have strong software engineering skills.

If some of the above doesn't line up perfectly with your experience, we still encourage you to apply

We value and celebrate diversity and strive to create an inclusive work environment for all. We welcome applicants from all backgrounds and are committed to providing equal opportunities. Should you require any accommodations during the recruitment process, please submit an Accommodations Request Form, and we will work together to meet your needs.

Full-Time Employees at Cohere enjoy these Perks:

An open and inclusive culture and work environment

Work closely with a team on the cutting edge of AI research

Weekly lunch stipend, in-office lunches & snacks

Full health and dental benefits, including a separate budget to take care of your mental health

100% Parental Leave top-up for 6 months for employees based in Canada, the US, and the UK

Personal enrichment benefits towards arts and culture, fitness and well-being, quality time, and workspace improvement

Remote-flexible, offices in Toronto, New York, San Francisco, London and Paris, as well as a co-working stipend

6 weeks of vacation

Note: This post is co-authored by both Cohere humans and Cohere technology.

  • Senior Research Scientist, Model Evaluation

    5 days ago

    Toronto, Ontario, Canada Cohere Full time $120,000 - $180,000 per year

    Who are we?Our mission is to scale intelligence to serve humanity. We're training and deploying frontier models for developers and enterprises who are building AI systems to power magical experiences like content generation, semantic search, RAG, and agents. We believe that our work is instrumental to the widespread adoption of AI.We obsess over what we...

  • Senior Research Scientist, Model Evaluation

    14 hours ago

    Toronto, Ontario, Canada Cohere Full time $120,000 - $180,000 per year

    Who are we?Our mission is to scale intelligence to serve humanity. We're training and deploying frontier models for developers and enterprises who are building AI systems to power magical experiences like content generation, semantic search, RAG, and agents. We believe that our work is instrumental to the widespread adoption of AI.We obsess over what we...

  • Senior Research Scientist, Model Evaluation

    23 hours ago

    Toronto, Ontario, Canada Cohere Full time

    Who are we?Our mission is to scale intelligence to serve humanity. We're training and deploying frontier models for developers and enterprises who are building AI systems to power magical experiences like content generation, semantic search, RAG, and agents. We believe that our work is instrumental to the widespread adoption of AI.We obsess over what we...

  • Senior Research Engineer, Model Evaluation

    1 week ago

    Toronto, Ontario, Canada Cohere Full time $120,000 - $180,000 per year

    Who are we?Our mission is to scale intelligence to serve humanity. We're training and deploying frontier models for developers and enterprises who are building AI systems to power magical experiences like content generation, semantic search, RAG, and agents. We believe that our work is instrumental to the widespread adoption of AI.We obsess over what we...

  • Research Scientist, World Models

    1 week ago

    Toronto, Ontario, Canada DeepMind Full time $120,000 - $200,000 per year

    SnapshotJoin an ambitious team building generative world models. We believe World Models will power numerous domains, such as visual reasoning, simulation, planning for embodied agents, and real-time interactive entertainment. The team will contribute directly to building the next generation of generative world models, working directly with the Genie team...

  • Senior Applied Scientist – Research Products

    1 week ago

    Toronto, Ontario, Canada Thomson Reuters Full time $120,000 - $180,000 per year

    Senior Applied Research Scientist  Are you excited about working at the forefront of applied research in an industry setting? Thomson Reuters is seeking scientists with a passion for solving problems using state-of-the-art information retrieval, natural language processing and generative AI. About the Role:Senior Applied Scientists are experts in Machine...

  • Senior Applied Scientist

    23 hours ago

    Toronto, Ontario, Canada Thomson Reuters Full time $120,000 - $180,000 per year

    Senior Applied Scientist – Research Products Are you excited about working at the forefront of applied research in an industry setting? Thomson Reuters Labs  in Toronto is seeking scientists with a passion for solving problems using state-of-the-art information retrieval, natural language processing and generative AI. What does Thomson Reuters Labs do?...

  • Senior Applied Scientist

    15 hours ago

    Toronto, Ontario, Canada Thomson Reuters Full time $120,000 - $180,000 per year

    Senior Applied Scientist – Research Products Are you excited about working at the forefront of applied research in an industry setting? Thomson Reuters Labs  in Toronto is seeking scientists with a passion for solving problems using state-of-the-art information retrieval, natural language processing and generative AI. What does Thomson Reuters Labs do?...

  • Staff Research Scientist, Machine Learning

    2 weeks ago

    Toronto, Ontario, Canada KenkoTech Futures Full time $120,000 - $200,000 per year

    Job Title:Staff Research Scientist, ML Location:USA / Canada (On-Site / Remote) Stage:Series C (~$250M funding)PLEASE FOLLOW THE KENKOTECH PAGE AND CONNECT WITH THE JOB POSTERAbout the Opportunity:We're partnering with a trailblazing biotech harnessing artificial intelligence to transform how medicines are discovered. Their proprietary AI platform fuses...

  • Research Scientist, Social Memory

    3 days ago

    Toronto, Ontario, Canada DeepMind Full time $120,000 - $180,000 per year

    SnapshotArtificial Intelligence could be one of humanity's most useful inventions. At Google DeepMind, we're a team of scientists, engineers, machine learning experts and more, working together to advance the state of the art in artificial intelligence. We use our technologies for widespread public benefit and scientific discovery, and collaborate with...