Member of Technical Staff, Pretraining evaluations
3 weeks ago
Member of Technical Staff, Pretraining EvaluationsJoin to apply for the Member of Technical Staff, Pretraining Evaluations role at Cohere.Become part of a team building AI systems to power magical experiences across content generation, semantic search, RAG, and agents. Cohere trains and deploys frontier models for developers and enterprises, aiming to scale intelligence to serve humanity.Why This Role?As a Member of Technical Staff in the pretraining evals team, you will play a key role in helping us make modelling decisions based on experimental outcomes for our large language models (LLMs). Your primary focus will be on developing better ways to measure base model progress.This role combines expertise in statistics, data science, model evaluation and experience with base model capabilities and how to measure them. If you are interested in measuring model performance accurately as a crucial part of advancing artificial intelligence, we encourage you to apply.Note: We have offices in London, Paris, Toronto, San Francisco, and New York, but we embrace remote-friendly policies with no restrictions on location.ResponsibilitiesDeeply understand each evaluation task in our base model evaluation suite, knowing what each task measures, its strengths and limitations.Suggest and implement improvements to the evaluation suite, adding new tasks to measure unmeasured capabilities or removing redundant or low-signal tasks.Improve statistical understanding of evaluation benchmarks and increase signal-to-noise ratio of the suite.QualificationsFamiliarity with base model evaluations and their differences from post-trained models.Strong statistical skills and experience evaluating scientific experiments related to data collection and model performance.Ability to convey statistical information effectively to broad audiences using visualizations and easy-to-understand numbers.Extremely strong software engineering skills.Proficiency in programming languages such as Python and ML frameworks (PyTorch, TensorFlow, JAX).Excellent communication skills to collaborate effectively with cross-functional teams and present findings.One or more papers at top-tier venues (NeurIPS, ICML, ICLR, AIStats, MLSys, JMLR, AAAI, Nature, COLING, ACL, EMNLP).If some of the above doesn’t line up perfectly with your experience, we still encourage you to applyBenefits🤝 Open and inclusive culture.🧑💻 Work closely with a team on the cutting edge of AI research.🍽 Weekly lunch stipend, in-office lunches & snacks.🦷 Full health and dental benefits, including a separate budget for mental health.🐣 100% parental leave top-up for up to 6 months.🎨 Personal enrichment benefits toward arts and culture, fitness, well-being, quality time, and workspace improvement.🏙 Remote-flexible; offices in Toronto, New York, San Francisco, London, Paris, and co-working stipend.✈️ 6 weeks of vacation (30 working days).Seniority LevelMid-Senior levelEmployment TypeFull-timeJob FunctionEngineering and Information TechnologyReferrals increase your chances of interviewing at Cohere by 2x. #J-18808-Ljbffr
-
Toronto, Canada Cohere Full timeMember of Technical Staff, Pretraining Evaluations Join to apply for the Member of Technical Staff, Pretraining Evaluations role at Cohere . Become part of a team building AI systems to power magical experiences across content generation, semantic search, RAG, and agents. Cohere trains and deploys frontier models for developers and enterprises, aiming to...
-
Toronto, Canada The Rundown AI, Inc. Full timeWho are we? Our mission is to scale intelligence to serve humanity. We’re training and deploying frontier models for developers and enterprises who are building AI systems to power magical experiences like content generation, semantic search, RAG, and agents. We believe that our work is instrumental to the widespread adoption of AI. We obsess over what we...
-
Toronto, Canada Cohere Full timeA leading AI company in Toronto is seeking a Member of Technical Staff specializing in pretraining evaluations. This role involves developing assessment strategies for large language models and requires strong statistical and software engineering skills. You will work with cross-functional teams to make impactful modeling decisions and contribute...
-
Toronto, Canada Cohere Full timeA leading AI company in Toronto is seeking a Member of Technical Staff specializing in pretraining evaluations. This role involves developing assessment strategies for large language models and requires strong statistical and software engineering skills. You will work with cross-functional teams to make impactful modeling decisions and contribute...
-
Staff Engineer, Pretraining Evaluations
3 weeks ago
Toronto, Canada The Rundown AI, Inc. Full timeA cutting-edge AI research company is seeking a Member of Technical Staff in Toronto to enhance model evaluation processes for large language models. The role necessitates strong statistical skills and software engineering expertise in Python and frameworks like PyTorch and TensorFlow. Successful candidates will improve evaluation methodologies and...
-
Member of Technical Staff, Data Engineering
2 weeks ago
Toronto, Canada Cohere Full timeMember of Technical Staff, Data Engineering Join to apply for the Member of Technical Staff, Data Engineering role at Cohere Who are we? Our mission is to scale intelligence to serve humanity. We’re training and deploying frontier models for developers and enterprises who are building AI systems to power magical experiences like content generation,...
-
Member of Technical Staff, Pre-Training Data
4 weeks ago
Toronto, Canada Cohere Full timeMember of Technical Staff, MLE (Pre-Training Data) Join to apply for the Member of Technical Staff, MLE (Pre-Training Data) role at Cohere Continue with Google Continue with Google Member of Technical Staff, MLE (Pre-Training Data) Join to apply for the Member of Technical Staff, MLE (Pre-Training Data) role at Cohere Who are we?Our mission is to scale...
-
Member of Technical Staff, Pre-Training Data
3 weeks ago
Toronto, Canada Cohere Full timeMember of Technical Staff, MLE (Pre-Training Data)Join to apply for the Member of Technical Staff, MLE (Pre-Training Data) role at CohereContinue with Google Continue with GoogleMember of Technical Staff, MLE (Pre-Training Data)Join to apply for the Member of Technical Staff, MLE (Pre-Training Data) role at CohereWho are we?Our mission is to scale...
-
Toronto, Canada The Rundown AI, Inc. Full timeWho are we? Our mission is to scale intelligence to serve humanity. We’re training and deploying frontier models for developers and enterprises who are building AI systems to power magical experiences like content generation, semantic search, RAG, and agents. We believe that our work is instrumental to the widespread adoption of AI. We obsess over what we...
-
Member of Technical Staff, Data Engineering
3 weeks ago
Toronto, Canada Cohere Full timeMember of Technical Staff, Data Engineering Join to apply for the Member of Technical Staff, Data Engineering role at Cohere Who are we? Our mission is to scale intelligence to serve humanity. We’re training and deploying frontier models for developers and enterprises who are building AI systems to power magical experiences like content generation,...
