Senior Data Science Evaluation Specialist
About the Role
A leading AI-focused initiative is seeking experienced data science professionals to support the evaluation and improvement of advanced AI systems. This opportunity centers on assessing the quality of data science outputs and helping establish rigorous standards for high-performance analytical work.
This opportunity is ideal for individuals with substantial industry experience in data science who can apply strong analytical judgment across a range of deliverables. Candidates should be comfortable evaluating technical work, defining quality benchmarks, and providing clear written reasoning for assessment decisions.
The work involves developing evaluation frameworks, reviewing analytical outputs, and applying consistent scoring methodologies, where attention to detail, methodological rigor, and strong communication are critical for success.
What You'll Do
- Design task-specific evaluation criteria for data science deliverables, including analyses, predictive models, dashboards, experiments, and business recommendations
- Assess completed work samples against defined evaluation standards
- Provide detailed written justifications supporting evaluation outcomes and scoring decisions
- Apply consistent, evidence-based judgment across multiple review scenarios
- Identify strengths, weaknesses, and areas for improvement within evaluated outputs
- Incorporate structured feedback and refine evaluation approaches as project requirements evolve
- Contribute to the development of high-quality benchmarks for analytical and data science work
- Collaborate within a structured review environment to maintain evaluation consistency
Requirements
- 5+ years of professional experience in data science or advanced analytics
- Strong background in business, product, operations, growth, or applied data science environments
- Expertise in experiment design, A/B testing methodologies, and statistical analysis
- Advanced proficiency in SQL and Python for data analysis and modeling
- Experience defining metrics and measuring business or product performance
- Ability to evaluate analytical outputs with consistency and strong technical reasoning
- Excellent written communication skills with the ability to explain complex decisions clearly
- Strong attention to detail and commitment to quality standards
- Comfort working in a review-based environment where evaluations are calibrated against peer assessments
- Ability to work independently in a remote contract setting
- Preferred qualifications include experience with AI evaluation initiatives, human-in-the-loop assessment programs, machine learning model review, benchmark development, or AI training projects