MCP Software Engineering Expert
About the Role
A structured AI evaluation initiative focused on testing advanced models against complex software engineering problems using Model Context Protocol (MCP) tools. The work involves building reproducible reinforcement learning environments that measure both effective tool use and practical engineering ability.
This opportunity is ideal for experienced software engineers who are strong in debugging, feature development, refactoring, algorithms, data structures, and performance optimization. Prior AI or machine learning experience is not required; strong software engineering expertise is the primary qualification.
The work involves designing realistic engineering tasks, integrating MCP-based information discovery, implementing deterministic verification, and developing reference solutions that reliably evaluate model performance. Reproducibility, technical precision, and maintainable implementation are critical.
What You'll Do
- Design reinforcement learning environments that test complex software engineering capabilities through MCP tools.
- Create realistic tasks involving bug fixes, feature implementation, codebase refactoring, and performance optimization.
- Build reproducible environments with deterministic setup and verification processes.
- Develop golden reference solutions that establish expected technical outcomes.
- Integrate MCP servers and workflows requiring agents to discover and reason over relevant information.
- Validate task environments for correctness, reliability, and reproducibility.
- Debug complex software issues and implement maintainable solutions.
- Evaluate whether tasks accurately measure both MCP tool usage and software engineering ability.
- Optimize environment and verification performance where required.
- Review and refine code, task specifications, and evaluation criteria.
- Collaborate with technical contributors in a remote, structured project environment.
Requirements
- Strong proficiency in at least one of C++, Python, Java, Go, TypeScript, or Rust.
- Strong understanding of algorithms, data structures, and software performance optimization.
- Demonstrated ability to debug complex software issues and deliver maintainable solutions.
- Practical experience implementing software features in existing codebases.
- Strong codebase refactoring skills with attention to maintainability and correctness.
- Proven ability to optimize software for performance and scalability.
- Strong written and verbal communication skills.
- High attention to technical detail and ability to produce reproducible work.
- Ability to work effectively in collaborative, remote environments.
- Availability for approximately 15 hours per week with flexible scheduling.
- Ability to complete a defined minimum volume of tasks per week.
- Experience with large-scale or distributed codebases is preferred.
- Familiarity with MCP, modern AI systems, or machine learning workflows is preferred but not required.
- Experience participating in rigorous code reviews and establishing software engineering best practices is preferred.