AI Software Engineering Expert
About the Role
A structured AI evaluation initiative focused on improving advanced AI systems through realistic software engineering challenges. The work involves creating reinforcement learning environments that test an AI model's ability to solve complex engineering problems using software tools and real-world codebases.
This opportunity is ideal for experienced software engineers with strong programming, debugging, algorithms, data structures, refactoring, and performance optimization skills. Prior AI or machine learning experience is not required; practical software engineering expertise is the primary qualification.
The work involves designing reproducible engineering environments, creating deterministic verification systems, and developing reference solutions that accurately measure model performance. Technical correctness, reproducibility, maintainability, and precise evaluation are critical.
What You'll Do
- Design reinforcement learning environments for complex software engineering tasks.
- Create realistic challenges involving bug fixing, feature implementation, codebase refactoring, and performance optimization.
- Develop reproducible environments and deterministic verification mechanisms.
- Produce golden reference solutions that establish expected technical outcomes.
- Build tasks that require AI agents to discover and reason over information from software tools and external systems.
- Debug complex software issues and validate proposed implementations.
- Review environments and solutions for correctness, reliability, and maintainability.
- Optimize code and evaluation workflows for performance and scalability.
- Assess whether engineering tasks accurately measure model reasoning and implementation ability.
- Refine task specifications and verification criteria based on observed model behavior.
- Collaborate with technical contributors in a flexible, remote project environment.
Requirements
- Strong proficiency in at least one of C++, Python, Java, Go, TypeScript, or Rust.
- Deep understanding of algorithms, data structures, and performance tuning.
- Demonstrated experience debugging complex software issues.
- Strong experience implementing features in existing software codebases.
- Proven ability to refactor code while maintaining correctness and usability.
- Experience optimizing software for performance and scalability.
- Strong written and verbal communication skills.
- High attention to technical detail and ability to produce reproducible implementations.
- Ability to work effectively in collaborative, cross-functional, remote environments.
- Availability for approximately 15 hours per week with flexible scheduling.
- Ability to meet defined minimum weekly task requirements.
- Experience with large-scale or distributed codebases is preferred.
- Familiarity with modern AI or machine learning systems is preferred but not required.
- Experience with rigorous code reviews and software engineering best practices is preferred.