Radiology Visual Document Understanding Specialist
About the Role
A specialized artificial intelligence evaluation initiative is seeking radiology professionals to contribute domain expertise to benchmark development and model assessment activities. The project focuses on evaluating AI capabilities in visual document understanding, medical content interpretation, and instruction-following tasks within radiology-related contexts.
This opportunity is ideal for individuals with strong radiology knowledge who can create accurate, evidence-based evaluation materials and apply expert judgment to assess technical outputs. Candidates should be comfortable translating professional expertise into structured tasks, objective evaluation criteria, and verifiable ground-truth responses.
The work involves developing benchmark datasets, designing realistic radiology-focused scenarios, creating assessment rubrics, and supporting AI model evaluation processes where precision, clinical accuracy, and methodological consistency are critical.
What You'll Do
- Develop radiology-focused evaluation tasks for AI model assessment
- Create realistic visual document understanding scenarios based on radiology concepts and workflows
- Author objective ground-truth answers and validation criteria
- Design clear scoring rubrics and evaluation frameworks
- Assess AI-generated responses for accuracy, completeness, and adherence to instructions
- Review benchmark materials to ensure clinical and technical correctness
- Identify ambiguities, inconsistencies, or quality issues within evaluation datasets
- Document reasoning and expected outcomes for assessment tasks
- Collaborate with project stakeholders to improve benchmark quality and reliability
- Contribute domain expertise to support ongoing AI evaluation and performance measurement efforts
Requirements
- Professional expertise in radiology or a closely related medical imaging field
- Strong understanding of radiology documentation, reporting standards, and clinical terminology
- Ability to evaluate medical content with high attention to detail and accuracy
- Experience creating, reviewing, or interpreting clinical documentation and imaging-related materials
- Strong analytical and critical-thinking skills
- Excellent written communication and technical documentation abilities
- Ability to develop objective evaluation criteria and structured assessment frameworks
- Comfort working independently in a remote environment
- Ability to maintain consistency across large-scale evaluation projects
- Availability for approximately 15–20 hours per week
- Eligibility to work as an independent contractor
- Location within the United States or Canada
- Preferred: Experience with dataset creation, benchmarking, quality assurance, or evaluation methodologies
- Preferred: Experience reviewing AI-generated outputs or participating in AI evaluation initiatives
- Preferred: Familiarity with visual document understanding, medical AI systems, or healthcare technology projects
- Preferred: Experience using AI-assisted tools such as ChatGPT in professional or research workflows
- Preferred: Background in academic research, clinical education, or medical content development