Psychiatry AI Evaluation Specialist (Remote Contract)
About the Role
A leading AI initiative is seeking Psychiatry professionals to contribute to the development and evaluation of advanced healthcare reasoning systems. This opportunity focuses on improving how AI models interpret, assess, and respond to complex mental health scenarios using evidence-based clinical standards.
This opportunity is ideal for board-certified psychiatrists, Psychiatry subspecialists, final-year residents, and qualified fellows who possess strong clinical judgment and an interest in advancing responsible AI applications in healthcare.
The work involves creating realistic psychiatric case scenarios, assessing model-generated outputs, developing high-quality reference responses, and providing structured feedback. Attention to clinical accuracy, consistency, and evidence-based reasoning is critical to success in this role.
What You'll Do
- Design realistic psychiatric case studies and clinical reasoning scenarios
- Develop attending-level reference responses for mental health assessments and treatment planning
- Evaluate AI-generated outputs using structured scoring frameworks
- Assess diagnostic formulations, risk evaluations, medication management approaches, and psychotherapy-related content
- Provide detailed written feedback to support model improvement efforts
- Participate in onboarding activities and specialty calibration sessions
- Maintain adherence to evidence-based psychiatric standards throughout evaluations
Requirements
- Board-certified Psychiatrist with an active, unrestricted medical license, or equivalent eligibility status based on training level
- Final-year Psychiatry resident or qualified fellow with appropriate professional standing
- Clinical experience in General Adult Psychiatry or a recognized Psychiatry subspecialty
- Strong understanding of evidence-based psychiatric assessment and treatment practices
- Ability to evaluate complex clinical reasoning with accuracy and consistency
- Excellent written communication and analytical skills
- Comfort working independently in a remote, asynchronous environment
- Availability for approximately 20 hours per week or more, depending on project needs
- Authorized to work as an independent contractor in an eligible location
- Preferred experience reviewing, annotating, auditing, or evaluating AI-generated content
- Preferred familiarity with healthcare AI, large language models, or structured quality-review workflows
- Preferred experience using digital collaboration and documentation tools, including AI-assisted productivity platforms such as ChatGPT or similar systems