Medical Expert – AI Clinical Reasoning Evaluation
About the Role
A high-impact healthcare AI initiative is seeking experienced medical professionals to support the development and evaluation of advanced clinical reasoning systems. This opportunity focuses on improving the quality, accuracy, and reliability of AI-generated healthcare outputs through expert medical oversight.
This opportunity is ideal for board-certified physicians, senior trainees, fellows, and specialty practitioners who possess strong clinical judgment and a commitment to evidence-based medicine. Candidates should be comfortable reviewing complex medical scenarios and providing structured evaluations.
The work involves creating realistic clinical cases, assessing AI-generated responses, and delivering actionable feedback that contributes to ongoing model improvement. Attention to clinical accuracy, guideline adherence, and reasoning quality is critical for success.
What You'll Do
- Design clinically realistic case prompts and specialty-specific scenarios
- Develop high-quality reference responses demonstrating expert-level clinical reasoning
- Evaluate AI-generated outputs using structured assessment frameworks
- Assess diagnostic reasoning, treatment recommendations, and risk stratification approaches
- Verify alignment with evidence-based guidelines and standards of care
- Provide detailed written feedback to support model improvement efforts
- Participate in onboarding activities and calibration sessions as required
- Maintain consistency and accuracy across assigned evaluation tasks
Requirements
- Board certification in a recognized medical specialty with an active, unrestricted medical license
- OR final-year resident physician status with board eligibility upon completion
- OR fellowship training with board-certified or board-eligible status in a primary specialty
- Strong expertise in clinical decision-making and patient care reasoning
- Ability to evaluate medical content against evidence-based standards
- Excellent written communication and analytical skills
- Ability to work independently in a fully remote, asynchronous environment
- Availability for approximately 20 hours per week, with potential for increased workload based on project demand
- Eligibility to work remotely from Canada or the United States
- Familiarity with AI-assisted workflows, clinical documentation review, or model evaluation methodologies preferred
- Experience using digital collaboration tools and structured review systems preferred
- Knowledge of emerging AI technologies and healthcare applications preferred