Back to all jobs
Healthcare

Clinical AI Evaluation Specialist – General Physician

United States – Fully Remote Part-time Platform: Mercor USD 150 / hour United States
Apply directly on the partner platform.

About the Role

A high-impact healthcare AI initiative focused on evaluating and improving advanced medical AI systems. The project involves assessing AI-generated clinical outputs across diagnosis, medical reasoning, patient safety, and care recommendations.

This opportunity is suited to licensed medical practitioners with completed residency training, active clinical practice, and experience in primary care, internal medicine, or hospital-based specialties. Candidates must demonstrate strong clinical judgment and the ability to assess medical information against established standards of care.

The work involves reviewing AI-generated medical responses, scoring clinical decisions, documenting structured feedback, and identifying safety-critical errors. Clinical accuracy, evidence-based judgment, and precise written communication are essential to successful execution.

What You'll Do

  • Evaluate AI-generated medical outputs for clinical accuracy, safety, and compliance with established medical standards.
  • Assess diagnostic conclusions and clinical reasoning against expert medical judgment.
  • Review treatment recommendations and patient care workflows for clinical appropriateness.
  • Score medical case evaluations using structured assessment criteria.
  • Document detailed feedback on clinical clarity, reasoning quality, and medical rigor.
  • Identify and flag errors, inconsistencies, and failure patterns that could compromise patient safety.
  • Collaborate asynchronously with AI research teams to improve model performance.
  • Apply current clinical knowledge to evaluate complex medical scenarios.

Requirements

  • MD or DO degree from an accredited medical school.
  • Active and unrestricted medical license in the United States.
  • Completed residency training.
  • Currently practicing medicine and actively seeing patients.
  • Minimum 3 years of post-residency professional experience in the current role strongly preferred.
  • Clinical background in one or more of the following specialties:
  • - Primary Care
  • - General Internal Medicine
  • - Family Medicine
  • - Emergency Medicine
  • - Critical Care
  • - Hospital-based medical specialties
  • Strong clinical reasoning and diagnostic evaluation skills.
  • Ability to assess medical recommendations against evidence-based standards of care.
  • Excellent written communication and structured documentation skills.
  • High attention to detail when identifying clinical inaccuracies and patient safety risks.
  • Ability to work independently in a fully remote environment.
  • Availability for at least 20 hours per week during the project period.
  • Availability to begin immediately.
  • Must be physically located in the United States.
  • Ability to work as an independent contractor.
  • Comfortable working asynchronously with AI research teams.
  • Preferred: At least 3 years of post-residency experience in an active clinical practice setting.
Application Note: By submitting your profile for this partnered position, our team can quickly review your background and reach out to present you with this specific opportunity or match you with similar AI Training projects.