Radiology AI Evaluation Specialist
About the Role
A structured AI evaluation initiative focused on improving visual document understanding and instruction-following capabilities within the radiology domain. The opportunity involves applying clinical imaging expertise to create high-quality evaluation tasks and objective assessment frameworks.
This opportunity is ideal for radiology professionals and medical imaging specialists who can translate domain knowledge into clear, measurable AI evaluation workflows. Candidates should have strong attention to detail, clinical reasoning ability, and experience interpreting radiology-related visual information.
The work involves developing grounded benchmark tasks, defining expected outputs, and assessing AI model performance on medical imaging documents, where accuracy, clinical relevance, and consistent evaluation standards are critical.
What You'll Do
- Create complex radiology-focused evaluation tasks using visual medical documents
- Develop clear ground-truth outputs and objective scoring criteria
- Review AI-generated responses for accuracy, completeness, and clinical alignment
- Analyze model performance across visual understanding and instruction-following scenarios
- Apply radiology expertise to identify errors, limitations, and improvement opportunities
- Document evaluation methodologies and task requirements
- Provide structured feedback to improve AI system reliability
Requirements
- Professional experience or advanced training in radiology, medical imaging, healthcare, or a related clinical field
- Strong understanding of radiology workflows, imaging concepts, and clinical interpretation processes
- Ability to evaluate visual medical information with high accuracy and consistency
- Strong written communication skills for creating documentation and evaluation criteria
- Analytical mindset with attention to detail and structured problem-solving ability
- Ability to work independently in a remote project environment
- Availability for approximately 15–20 hours per week
- Access to a reliable remote work setup
- Experience with AI tools, machine learning evaluation, medical datasets, or clinical AI applications preferred
- Candidates must be eligible to work remotely from the United States or Canada
- Independent contractor availability preferred