AI Coding Model Evaluation Engineer
Save this job and keep your search organized
Create a free account to save jobs, create alerts and return to this listing from your dashboard.
By continuing, you agree to our Terms & Privacy Policy.
Evaluate AI-generated code with Python or TypeScript expertise as an independent contractor supporting Turing’s model improvement work.
This experienced software engineering role focuses on reviewing coding agents across real-world repositories. You will assess correctness, robustness, maintainability, and agent behavior, then create evaluation signals that improve coding models. The work includes collaboration with researchers and engineers.
Key Responsibilities:
- Evaluate AI-generated code across real-world software repositories
- Review agent behavior, tool usage, and code changes
- Identify technical errors and recurring model failure modes
- Create rubrics and evaluation criteria for coding tasks
- Produce preference data and actionable findings
Requirements:
- Five or more years of software engineering experience
- Strong Python, TypeScript, JavaScript, Go, or production language proficiency
- Experience with substantial real-world codebases
- Strong code-review skills and technical judgment
- Experience using modern LLMs or AI coding tools
Apply software engineering judgment, code review, and written communication to improve Turing’s AI coding models.
-
Generative AI Model Evaluator
3 days ago
Toronto, ON, Canada Scale AI Full-timeEvaluate generative AI models remotely with fluent English, strong communication, and careful attention to detail. Choose flexible freelance hours that suit your schedule.Outlier seeks detail-oriented English speakers to help train voice, image, and video models. You will assess outputs across adherence, quality, and realism. The work includes live...
-
Remote Software Engineer — Train
2 days ago
Toronto, ON, Canada DataAnnotation Remote Full-time $62,480 - $88,987 ContractAn innovative company is seeking a skilled Software Engineer to help train AI chatbots. This exciting remote role allows you to work on your own schedule while contributing to the development of cutting-edge AI technology. You'll be responsible for creating diverse coding challenges, writing high-quality code snippets, and evaluating AI-generated code for...
-
English AI Model Evaluator
3 days ago
Toronto, ON, Canada Scale AI Full-timeUse fluent English and careful judgment to evaluate AI voice, image, and video models for Outlier. Work remotely as a flexible freelance contributor.Outlier seeks detail-oriented English speakers to provide human feedback for generative AI models. You will assess natural conversations, compare generated media, and explain findings clearly. The work supports...
-
AI-Focused Full-Stack Engineer for Model Evaluation
22 hours ago
Toronto, ON, Canada Turing Enterprises, Inc. Full-time €135,000 - €213,000 ContractTuring Enterprises, Inc. is seeking a Software Engineering evaluator to curate datasets for training large language models, focusing on Python backend services and ML infrastructure, with additional work in JavaScript, C/C++, Java, Rust, and Go. You will evaluate AI-generated code for efficiency and reliability, collaborate with researchers, and build...
-
English AI Model Evaluator Remote
3 days ago
Toronto, ON, Canada Scale AI Full-timeEvaluate generative AI outputs remotely using fluent English, careful listening, and strong written communication. Flexible freelance hours let you work when convenient.Outlier seeks detail-oriented English speakers to help improve voice, image, and video models. You will assess natural conversations, compare generated media, and explain findings clearly....
-
Toronto, ON, Toronto Census Division, ON; Ontario, Canada Turing Enterprises, Inc. Full-timeTuring Enterprises, Inc. is seeking a Software Engineering evaluator to curate datasets for training large language models, focusing on Python backend services and ML infrastructure, with additional work in JavaScript, C/C++, Java, Rust, and Go. You will evaluate AI-generated code for efficiency and reliability, collaborate with researchers, and build...
-
English AI Voice Model Evaluator
24 hours ago
toronto, on, Canada Scale AI Full-timeEvaluate AI voice models in fluent English as a remote freelance contributor with flexible hours and strong communication skills. Help assess naturalness, usefulness, and clarity. Outlier seeks detail-oriented English speakers to provide human feedback for generative AI voice, image, and video models. You will hold scenario-based conversations, compare...
-
Mathematical AI Model Evaluator
5 days ago
Toronto, ON, Canada DataAnnotation Remote Full-time $62,480 - $88,987 ContractApply advanced mathematical reasoning remotely with DataAnnotation while evaluating AI chatbot outputs and solving complex problems. Choose full-time or part-time work.DataAnnotation seeks mathematicians to train AI models through careful evaluation and problem solving. The contract role focuses on measuring chatbot progress, assessing logic, and improving...
-
Remote English AI Voice Model Evaluator
3 days ago
Toronto, ON, Canada Scale AI Full-timeOutlier is seeking detail-oriented, fluent English speakers to help train AI models from anywhere in Canada. You will engage in live conversations with AI voice models, rate responses, and assess image/video prompts alignment.This freelance role offers flexibility in hours and a remote setup, making it ideal for a student or professional seeking...
-
AI Voice Model Evaluator
3 days ago
Toronto, ON, Canada Scale AI Full-timeOutlier helps the world’s most innovative companies improve their AI models by providing human feedback. Are you a strong communicator with fluency in English who would like to lend your expertise to train AI models? About the opportunity: Outlier is looking for talented, detail-oriented English speakers to help train generative AI voice, image, and video...