AI Research Generalist – Model Evaluation & Training
About the Role
A structured AI research initiative focuses on benchmarking and improving advanced language models through high-quality research, question development, answer generation, and evaluation materials. The work supports the assessment of model reasoning, information accuracy, and response quality across a range of subject areas.
This opportunity is suited to individuals with a bachelor's degree and 2–3 years of relevant professional experience, or candidates currently pursuing or recently completing postgraduate studies. Strong research skills, sound reasoning, and the ability to synthesize information from diverse sources are essential. Candidates from STEM and non-STEM disciplines are encouraged to apply if they meet the relevant experience and qualification criteria.
The work involves developing research questions, preparing evidence-based answers, evaluating AI-generated outputs, and producing structured content for model training and benchmarking. Accuracy, clarity, critical thinking, and consistent adherence to evaluation guidelines are critical to success.
What You'll Do
- Develop high-quality research questions that assess factual knowledge, reasoning, and analytical capabilities.
- Research topics using credible online sources and information from multiple relevant references.
- Gather, compare, and synthesize information into accurate, clearly structured written responses.
- Prepare answer sets, evaluation materials, and reference content for AI model training.
- Evaluate AI-generated responses for factual accuracy, relevance, completeness, and logical consistency.
- Identify errors, unsupported claims, ambiguities, and weaknesses in model-generated content.
- Apply project-specific evaluation rubrics and quality standards consistently.
- Document research findings and provide clear explanations supporting evaluation decisions.
- Manage assigned tasks independently while meeting quality expectations and project deadlines.
- Adapt to evolving research requirements, content guidelines, and model evaluation procedures.
Requirements
- A bachelor's degree and approximately 2–3 years of relevant professional experience, or current enrollment in or recent completion of a master's degree or equivalent postgraduate studies.
- Strong online research skills and the ability to assess the credibility and relevance of information sources.
- Excellent written communication, editing, and information-summarization skills.
- Demonstrated critical thinking, analytical reasoning, and attention to detail.
- Ability to communicate complex information clearly and organize findings into structured written deliverables.
- Ability to work independently in a remote, project-based environment.
- Ownership of a MacBook with an Apple Silicon M-series chip.
- Access to a compatible MacBook Pro or MacBook Air running macOS 15 or higher, subject to project-specific device requirements.
- Availability for approximately 10–20 hours per week.
- Ability to follow detailed instructions, evaluation rubrics, and research quality standards.
- Reliable internet access and the ability to complete assigned work using the required digital tools.
- Willingness to complete a short skills assessment as part of the project evaluation process.
- Preferred: An interdisciplinary academic background spanning multiple subject areas.
- Preferred: Experience in academic research, technical writing, data annotation, content evaluation, or research-based analysis.
- Preferred: Familiarity with generative AI systems, large language models, AI benchmarking, or model evaluation workflows.
- Preferred: Experience developing assessment questions, reviewing written responses, or validating factual claims.
- Engagement is on an independent-contractor basis, with workload and project duration subject to project requirements.
- Applicants requiring H-1B or STEM OPT support are not eligible for this opportunity.