Senior Kotlin Software Engineer (AI Evaluation)
About the Role
A structured AI evaluation initiative focused on improving the reliability and reasoning capabilities of conversational systems in software engineering contexts. The work centers on analyzing how models generate, interpret, and explain Kotlin-based solutions across a range of programming challenges and system-level tasks.
This opportunity is ideal for experienced Kotlin engineers with strong problem-solving ability and a deep understanding of modern software development practices. It suits individuals who can independently validate code, identify subtle issues, and assess the quality of technical explanations.
The work involves reviewing AI-generated code, executing and validating outputs, and providing structured feedback on correctness and clarity, where precision and consistency are critical to improving system performance
What You'll Do
- Evaluate AI-generated responses to software engineering and coding tasks
- Execute and validate Kotlin code to ensure correctness and performance
- Identify logical errors, inefficiencies, and edge case failures
- Annotate outputs with detailed and structured feedback
- Assess code readability, maintainability, and architectural soundness
- Perform fact-checking using reliable technical references
- Apply standardized evaluation frameworks and scoring criteria
- Ensure outputs align with expected engineering and conversational standards
Requirements
- 5+ years of professional experience in software engineering or related fields
- Strong expertise in Kotlin programming language
- Ability to solve medium to hard algorithmic problems independently
- Experience executing, testing, and debugging production-level code
- Strong understanding of data structures, algorithms, and system design principles
- High attention to detail in reviewing technical reasoning and outputs
- Fluent English communication skills
- Experience using LLMs in coding workflows and understanding their limitations
- Ability to follow structured evaluation frameworks and guidelines
- Bachelor’s degree or higher in Computer Science or related discipline
- Experience contributing to open-source projects with accepted contributions
- Familiarity with additional programming languages or ecosystems (preferred)
- Experience in model evaluation, RLHF, or data annotation (preferred)
- Background in competitive programming or technical assessments (preferred)
- Experience reviewing code in production environments (preferred)
- Ability to explain complex technical concepts clearly to varied audiences (preferred)