Back to all jobs
General
Generalist – AI Systems Evaluation & Annotation (Remote)
United States Full-time Platform: Mercor
About the Role
A leading AI research initiative is seeking highly detail-oriented Generalists to support human data pipelines used in training and evaluating advanced AI systems. This role is best suited for individuals who are methodical, precision-driven, and committed to consistently high-quality output.
You will be working on structured evaluation, labeling, and review workflows where accuracy is critical. Attention to nuance is essential—small errors can scale quickly in these environments.
Location Requirement
🇺🇸 United States
What You'll Do
- Review, evaluate, and annotate AI-generated outputs with strong attention to detail
- Apply complex guidelines consistently across tasks
- Identify subtle inconsistencies, edge cases, and quality gaps
- Deliver clear, structured feedback to improve model performance
- Maintain high accuracy across repetitive, high-volume workflows
- Proactively flag unclear or ambiguous instructions
Requirements
- Exceptionally detail-oriented with a strong quality-first mindset
- Prior experience in roles such as content reviewer, rater, or evaluator in AI/data pipelines
- Ability to follow nuanced instructions with precision
- Strong written communication and analytical reasoning skills
- High reliability and consistency in repetitive task environments
- Comfortable working independently over extended task cycles
- Experience with large-scale AI data annotation or evaluation programs
- Background in QA, operations, trust & safety, or structured review systems
- Familiarity with multi-step, rubric-based evaluation frameworks
- Ability to provide calibrated feedback aligned with strict quality benchmarks
Application Note: By submitting your profile for this partnered position, our team can quickly review your background and reach out to present you with this specific opportunity or match you with similar AI Training projects.