AI Media Evaluation Analyst

5 days ago

Delta, British Columbia, Canada Scale AI Full-time

Evaluate AI-generated media for Outlier remotely on your own schedule, using English communication, careful listening, and visual attention to detail.

As an Outlier Expert, you will review images, videos, and voice-model conversations for Scale AI's training platform. You will compare outputs, assess adherence, quality, and realism, and explain findings in writing. Tasks include caption labeling and audio evaluations.

Key Responsibilities:

  • Roleplay live English conversations with AI voice models
  • Rank speech for naturalness, usefulness, and clarity
  • Compare generated images and videos against prompts
  • Inspect outputs for artifacts, motion, text, and audio issues
  • Write specific evidence-based explanations in English

Requirements:

  • Fluent English with clear written communication
  • Strong attention to visual and audio details
  • Comfort discussing realistic situations naturally
  • Experience writing concise reviews or explanations
  • Ability to decide between flawed alternatives

Apply analytical judgment, conversational English, and precise feedback to help Outlier improve advanced AI systems through evaluation.