Senior Data Science Evaluation Specialist
About the Role
A leading AI research initiative is seeking experienced data science professionals to contribute expertise in evaluating real-world analytical work. The project focuses on defining high-quality standards for data science outputs and assessing completed work against structured evaluation frameworks.
This opportunity is ideal for individuals with strong experience in business, product, growth, or operations data science who can apply rigorous analytical judgment across a variety of data-driven scenarios. Candidates should be comfortable evaluating both technical execution and the quality of business insights derived from data.
The work involves creating assessment criteria for data science deliverables, reviewing completed analyses and models, and producing evidence-based evaluations. Success requires strong statistical reasoning, attention to detail, and the ability to communicate analytical conclusions with clarity and consistency.
What You'll Do
- Develop task-specific evaluation criteria for data science deliverables
- Assess analytical reports, predictive models, dashboards, and experiment results
- Evaluate SQL and Python-based analyses for accuracy and methodological quality
- Review A/B testing designs, experimentation frameworks, and statistical approaches
- Assess metric definitions and measurement methodologies
- Evaluate business recommendations derived from analytical findings
- Assign structured scores based on predefined evaluation standards
- Produce detailed written justifications supporting assessment outcomes
- Apply consistent and evidence-based judgment across evaluations
- Review AI-generated and human-produced work samples
- Incorporate feedback from reviewers and refine evaluation methodologies
- Contribute subject matter expertise to AI training and evaluation initiatives
Requirements
- 5+ years of professional experience in data science, analytics, or a related field
- Experience in product, growth, business, operations, or commercial data science environments
- Strong expertise in experiment design and A/B testing methodologies
- Advanced knowledge of statistical analysis and causal inference concepts
- Proficiency in SQL for data extraction, transformation, and analysis
- Proficiency in Python for analytical and data science workflows
- Experience defining, tracking, and evaluating business and product metrics
- Ability to assess analytical methodologies and validate conclusions
- Strong understanding of data visualization and dashboard design principles
- Experience communicating insights to business leaders or executive stakeholders
- Excellent written communication and documentation skills
- Strong analytical judgment and attention to detail
- Ability to work independently in a remote contract environment
- Comfortable receiving feedback and participating in calibration exercises
- Experience evaluating data quality and analytical rigor
- Availability for project-based contract work preferred
- Experience with machine learning models and predictive analytics preferred
- Prior experience in AI training, evaluation, human-data projects, or quality assessment preferred
- Familiarity with AI tools such as ChatGPT or similar technologies preferred
- Experience working within large-scale technology, product, or digital businesses preferred
- Ability to receive payments through supported international payment platforms preferred