Internal Medicine AI Evaluation Specialist
Save this job and keep your search organized
Create a free account to save jobs, create alerts and return to this listing from your dashboard.
By continuing, you agree to our Terms & Privacy Policy.
This role is for one of our clients
Compensation: $130-$180 per hour
Join a leading AI research initiative focused on advancing healthcare-focused artificial intelligence. We are seeking experienced Internal Medicine physicians to contribute their clinical expertise toward training and evaluating next-generation AI models capable of sophisticated medical reasoning.
In this role, you will develop realistic clinical scenarios, evaluate AI-generated responses against evidence-based standards, and help improve how AI systems reason about diagnosis, treatment planning, and patient management within Internal Medicine.
We welcome Internal Medicine attending physicians, hospitalists, fellows, final-year residents, and other qualified Internal Medicine professionals who are passionate about applying their expertise to the future of healthcare AI.
Requirements
Key Responsibilities
Design Clinical Scenarios
- Create realistic Internal Medicine case studies based on everyday clinical practice.
- Develop prompts covering:
- Diagnostic reasoning
- Differential diagnosis
- Treatment planning
- Risk stratification
- Evidence-based guideline adherence
- Long-term patient management
Create Reference-Quality Responses
- Produce attending physician-level reference solutions that reflect current clinical best practices.
- Ensure recommendations align with established evidence-based guidelines.
Evaluate AI Performance
- Review and assess AI-generated clinical responses using structured evaluation frameworks.
- Identify reasoning gaps, inaccuracies, omissions, and opportunities for improvement.
- Provide detailed written feedback to improve AI model performance and clinical reliability.
Collaborate with Clinical Teams
- Participate in onboarding sessions and specialty calibration meetings.
- Work with multidisciplinary teams to ensure consistency, accuracy, and quality across medical evaluations.
Required Qualifications
Applicants should meet one of the following criteria:
Attending Physicians
- Board-certified in Internal Medicine.
- Current, active, and unrestricted medical license.
Resident Physicians
- Final-year Internal Medicine resident.
- Recent residency graduates must be board-eligible.
Fellows
- Board-certified or board-eligible in Internal Medicine or a related subspecialty.
- Current, active, and unrestricted medical license.
Professionals with expertise in Internal Medicine subspecialties are encouraged to apply, including:
- Cardiology
- Hematology & Oncology
- Endocrinology
- Nephrology
- Gastroenterology
- Pulmonology
- Other Internal Medicine subspecialties
Preferred Qualifications
- Strong knowledge of evidence-based medicine and clinical practice guidelines.
- Experience in clinical teaching, medical research, or quality improvement initiatives.
- Excellent written communication and analytical skills.
- Interest in artificial intelligence, clinical decision support, or healthcare technology.
Engagement Details
- Fully remote with 100% asynchronous work.
- Standard commitment of approximately 20 hours per week, with opportunities for additional hours based on p
-
Internal Medicine AI Evaluation Specialist
7 days ago
Toronto, ON, Toronto Census Division, ON; Ontario, Canada Weekday 1 Full-time €130 - €180This role is for one of our clients Compensation: $130-$180 per hour Join a leading AI research initiative focused on advancing healthcare-focused artificial intelligence. We are seeking experienced Internal Medicine physicians to contribute their clinical expertise toward training and evaluating
-
Internal Medicine AI Evaluation Specialist
7 days ago
Ottawa, ON, Canada Weekday 1 Full-timeThis role is for one of our clientsCompensation: $130-$180 per hourJoin a leading AI research initiative focused on advancing healthcare-focused artificial intelligence. We are seeking experienced Internal Medicine physicians to contribute their clinical expertise toward training and
-
Internal Medicine AI Evaluation Specialist
7 days ago
Mississauga, ON, Peel Regional District, ON; Ontario, Canada Weekday 1 Full-time €130 - €180This role is for one of our clients Compensation: $130-$180 per hour Join a leading AI research initiative focused on advancing healthcare-focused artificial intelligence. We are seeking experienced Internal Medicine physicians to contribute their clinical expertise toward training and evaluating
-
Ottawa, ON, Ottawa Census Division, ON; Ontario, Canada Weekday 1 Full-timeWeekday 1 is seeking experienced Internal Medicine physicians to contribute to training and evaluating AI models focused on healthcare. This fully remote role involves developing realistic clinical scenarios and references, evaluating AI-generated responses, and providing feedback to improve AI reasoning in diagnosis and treatment planning. The engagement...
-
Toronto, ON, Toronto Census Division, ON; Ontario, Canada Weekday 1 Full-timeWeekday 1 is seeking experienced Internal Medicine physicians to contribute to training and evaluating AI models focused on healthcare. This fully remote role involves developing realistic clinical scenarios and references, evaluating AI-generated responses, and providing feedback to improve AI reasoning in diagnosis and treatment planning. The engagement...
-
Ottawa, ON, Canada Weekday 1 Full-timeWeekday 1 is seeking experienced Internal Medicine physicians to contribute to training and evaluating AI models focused on healthcare. This fully remote role involves developing realistic clinical scenarios and references, evaluating AI-generated responses, and providing feedback to improve AI reasoning in diagnosis and treatment planning.The engagement is...
-
Mississauga, ON, Peel Regional District, ON; Ontario, Canada Weekday 1 Full-timeWeekday 1 is seeking experienced Internal Medicine physicians to contribute to training and evaluating AI models focused on healthcare. This fully remote role involves developing realistic clinical scenarios and references, evaluating AI-generated responses, and providing feedback to improve AI reasoning in diagnosis and treatment planning. The engagement...
-
Co-op/Intern AI Quality, Evaluation
3 days ago
Ottawa, Ontario, Canada Kinaxis Full-time $27 - $39 PermanentAbout KinaxisAre you looking to join an innovative, market-leading company where you can truly elevate your career? At Kinaxis we are serious about culture, we are serious about technology, we are serious about customers, and we are serious about not taking ourselves too seriously. If you are looking to be part of an incredible growth story, then we...
-
Internal Medicine Specialist
2 days ago
Ottawa, ON, Canada Canadian Armed Forces | Forces armées canadiennes Full-time $180,000 - $240,000 ContractOverview As a member of the military, Internal Medicine Specialists belong to the Medical Specialist occupation in the Canadian Armed Forces (CAF) and provide high-quality acute and critical care services, whether at Canadian Forces Health Services Clinics or while deployed on international, peacekeeping or humanitarian operations. They have the opportunity...
-
Internal Medicine Specialist
4 days ago
Ottawa, Ontario, Canada PetsApp Full-timeWe are seeking physicians in BC, Alberta, Saskatchewan, Manitoba and Ontario who are passionate about helping patients achieve remission of type 2 diabetes through lifestyle medicine.As part of our team, you will have the opportunity to work full-time or part-time (1-2 days per week) providing virtual care to our patients. Our virtual care offering is a...