Emergency Medicine AI Evaluation Specialist
About the Role
A frontier AI research initiative is seeking Emergency Medicine professionals to support the development of advanced healthcare reasoning systems through high-quality clinical evaluation and scenario design. The work contributes directly to improving how AI systems interpret, analyze, and respond to acute and emergency care situations.
This opportunity is ideal for experienced Emergency Medicine physicians and senior trainees with strong clinical judgment, evidence-based reasoning skills, and the ability to evaluate nuanced medical decision-making. Candidates who are comfortable working independently in structured research environments and maintaining high standards of clinical accuracy are particularly well suited for this role.
The work involves developing realistic emergency care scenarios, reviewing AI-generated clinical responses, and producing detailed evaluations aligned with evidence-based medical standards where precision, consistency, and expert reasoning are critical for success.
What You'll Do
- Design clinically realistic Emergency Medicine prompts and case scenarios
- Develop attending-level reference responses for acute care situations
- Evaluate AI-generated clinical outputs against structured assessment criteria
- Identify inaccuracies, unsafe recommendations, and reasoning gaps in model responses
- Provide written feedback to improve AI clinical reasoning performance
- Assess triage decisions, diagnostic reasoning, and disposition planning quality
- Participate in onboarding sessions and specialty calibration activities
- Maintain consistent evaluation standards across diverse emergency care topics
- Manage remote asynchronous work independently while meeting quality expectations
Requirements
- Board-certified Emergency Medicine physicians with active unrestricted medical license preferred
- Final-year Emergency Medicine residents eligible to apply
- Dual-boarded EM/IM or EM/Peds physicians preferred
- Fellows must be board-certified or board-eligible in primary specialty
- Strong understanding of evidence-based Emergency Medicine practice
- Ability to evaluate nuanced clinical reasoning and decision-making quality
- Excellent written communication and structured feedback skills
- High attention to detail and consistent judgment standards
- Comfort working independently in remote project-based environments
- Availability for approximately 20 hours per week preferred
- Familiarity with AI evaluation workflows or clinical review processes preferred
- Ability to participate in onboarding and specialty calibration sessions
- Must be authorized to work as an independent contractor in eligible regions
- Candidates requiring H1-B or STEM OPT sponsorship are not eligible for this engagement