Software Engineer – Open-Source Development & AI Evaluation
About the Role
A specialized AI development initiative focuses on evaluating and improving software engineering capabilities through reproducible coding challenges and reinforcement learning environments. The work involves creating realistic programming tasks that assess an AI model's ability to debug software, implement features, refactor existing code, and optimize performance.
This opportunity is suited to experienced software engineers with demonstrable open-source contributions or strong evidence of programming proficiency through technical assessment. Candidates should have practical expertise in at least one major programming language, a strong understanding of algorithms and data structures, and experience working with complex codebases. Previous AI experience is not required.
The work involves designing deterministic coding challenges, implementing reference solutions, developing automated verification systems, and evaluating code quality across different programming scenarios. Reproducibility, technical correctness, robust testing, and the ability to distinguish valid solutions from incorrect implementations are critical to success.
What You'll Do
- Design realistic coding challenges that assess debugging, feature development, refactoring, and performance optimization.
- Build reproducible reinforcement learning environments for evaluating software engineering tasks.
- Develop reference implementations that demonstrate correct and maintainable solutions to programming problems.
- Implement deterministic verifiers and automated tests that accept valid solutions and reject incorrect implementations.
- Analyze and resolve software defects across diverse codebases using Python, Java, Rust, Go, C++, TypeScript, or related technologies.
- Implement new features while maintaining scalability, reliability, and compatibility with existing systems.
- Refactor legacy code to improve readability, maintainability, modularity, and execution efficiency.
- Identify performance bottlenecks and apply appropriate optimization techniques.
- Apply algorithms, data structures, and foundational computer science principles to complex programming problems.
- Create reproducible test cases that validate functional correctness, edge cases, and performance requirements.
- Review and validate code submissions for correctness, quality, and compliance with technical specifications.
- Document technical reasoning, implementation decisions, debugging strategies, and expected outcomes.
- Maintain consistent quality across submitted tasks while meeting project-specific output requirements.
Requirements
- Strong hands-on software development experience in at least one of the following: Python, Java, Rust, Go, C++, TypeScript, or a comparable programming language.
- Demonstrable open-source contributions, preferably through a GitHub profile or another verifiable portfolio of software projects.
- Ability to complete a technical assessment when existing portfolio evidence does not sufficiently demonstrate the required expertise.
- Strong understanding of algorithms, data structures, logical reasoning, and core computer science concepts.
- Practical experience debugging complex software defects and implementing reliable fixes.
- Experience developing and testing new features in existing codebases.
- Understanding of codebase refactoring, software architecture, and maintainable programming practices.
- Ability to identify performance bottlenecks and implement measurable optimizations.
- Experience with automated testing, reproducible test cases, and validation of software behavior.
- Ability to develop deterministic verification logic for programming challenges.
- Familiarity with version control systems, particularly Git and GitHub.
- Strong analytical skills and the ability to solve unfamiliar technical problems independently.
- Ability to communicate technical reasoning through clear documentation and structured explanations.
- Ability to follow detailed task specifications, acceptance criteria, and submission requirements.
- Availability to begin work promptly after onboarding and complete approximately 15 hours per task run, with actual effort varying by task.
- Ability to meet minimum weekly task submission requirements established by the project.
- Willingness to complete a technical interview and any applicable programming assessment independently, without AI assistance or help from others during those assessments.
- Preferred: Experience developing backend services, APIs, or scalable software systems.
- Preferred: Experience working with large codebases, legacy system modernization, or complex feature implementation.
- Preferred: Familiarity with reinforcement learning environments, AI coding benchmarks, automated evaluation systems, or AI model training workflows.
- Preferred: Experience designing challenging programming exercises, grading systems, or automated code evaluation pipelines.
- Preferred: A history of delivering high-quality, performant software through open-source or collaborative engineering projects.