Trust and Safety AI Evaluator

1 day ago

Vancouver, Metro Vancouver Regional District, Canada Cohere Full-time

Use Trust & Safety judgment to evaluate model outputs for Cohere, working remotely as a part-time independent contractor in Canada or the United States. Written English is essential.

This Data Annotation Specialist role focuses on safety evaluation for Large Language Models supporting AI systems. You will recognize risks across user intents, apply detailed rubrics, and explain why outputs are safe, unsafe, high-quality, or low-quality. The work includes sensitive material and requires cultural awareness, bias sensitivity, and emotional resilience.

Key Responsibilities:

  • Assess model outputs against safety guidelines and style standards
  • Evaluate content across sensitive, ambiguous, and context-dependent scenarios
  • Write realistic prompts and adversarial examples for safety testing
  • Audit annotations and flag unclear or inconsistent guidelines
  • Deliver consistent decisions across high-volume evaluation tasks

Requirements:

  • One or more years in Trust & Safety, moderation, or related analysis
  • Strong contextual and sociocultural judgment with bias awareness
  • Emotional resilience with explicit, violent, or psychologically disturbing content
  • Excellent written English and ability to justify evaluations clearly
  • Comfort working independently with new tools remotely

Bring nuanced safety analysis, cultural judgment, and clear writing to Cohere’s AI evaluation and data quality work.