AI Coding Model Evaluation Engineer
11 hours ago
Toronto, ON, Canada
Turing Enterprises, Inc.
Full-time
€83,000 - €138,000 Contract
Free with email or Google
Save this job and keep your search organized
Create a free account to save jobs, create alerts and return to this listing from your dashboard.
Free with email or Google
By continuing, you agree to our Terms & Privacy Policy.
Evaluate AI-generated code with Python or TypeScript expertise as an independent contractor supporting Turing’s model improvement work.
This experienced software engineering role focuses on reviewing coding agents across real-world repositories. You will assess correctness, robustness, maintainability, and agent behavior, then create evaluation signals that improve coding models. The work includes collaboration with researchers and engineers.
Key Responsibilities
- Evaluate AI-generated code across real-world software repositories
- Review agent behavior, tool usage, and code changes
- Identify technical errors and recurring model failure modes
- Create rubrics and evaluation criteria for coding tasks
- Produce preference data and actionable findings
Requirements:
- Five or more years of software engineering experience
- Strong Python, TypeScript, JavaScript, Go, or production language proficiency
- Experience with substantial real-world codebases
- Strong code-review skills and technical judgment
- Experience using modern LLMs or AI coding tools Apply software engineering judgment, code review, and written communication to improve Turing’s AI coding models.
Key Responsibilities
- Evaluate AI-generated code across real-world software repositories
- Review agent behavior, tool usage, and code changes
- Identify technical errors and recurring model failure modes
- Create rubrics and evaluation criteria for coding tasks
- Produce preference data and actionable findings
Requirements:
- Five or more years of software engineering experience
- Strong Python, TypeScript, JavaScript, Go, or production language proficiency
- Experience with substantial real-world codebases
- Strong code-review skills and technical judgment
- Experience using modern LLMs or AI coding tools Apply software engineering judgment, code review, and written communication to improve Turing’s AI coding models.