01 / THE DETAILSRole overview
Get to know the opportunity
Develop the benchmarks, datasets and research tools used to measure and improve advanced AI coding agents.
02 / THE DETAILSWhat you’ll be doing
Your day-to-day contribution
- Define coding-agent benchmarks, scoring methods, rubrics and evaluation standards.
- Create reference examples and datasets, investigate model failures and recommend improvements.
- Build tooling for large-scale experiments and review workflows, and communicate findings in technical studies.
03 / THE DETAILSWho this could suit
See where your experience could fit
Software engineers and AI researchers interested in rigorous experiments, reliable measurements and how coding models succeed or fail on real engineering tasks.
04 / THE DETAILSWhat you’ll need
Experience, skills and essential requirements
- At least three years of experience in software engineering, ML, AI research, evaluation or a related technical discipline.
- Strong programming in Python, C++ or comparable languages; experience designing or validating benchmarks, coding tasks or technical assessments; and familiarity with LLMs, coding agents, reinforcement learning or model evaluation.
- You should be able to automate workflows, analyse complex systems and explain technical findings clearly.
- Research publications, open-source work, agentic workflows and experience leading ambiguous research projects are useful additions.
You don’t have to match every requirement exactly. We welcome applicants with different backgrounds, levels of experience and transferable skills.
05 / THE DETAILSPay and working arrangements
Rates, commitment and working arrangements
Remote, full-time employment. The detailed description states a USD base salary of $200,000–$260,000 per year. The listing header instead shows $8–$9 per hour and labels the role hourly; these conflict with the full-time salary terms in the description. The salary above follows the detailed description, and the final offer should confirm the terms. Equity eligibility, possible performance bonuses, health-insurance reimbursement of up to 100%, paid time off and a matched 401(k) are listed, subject to policy. No eligible-country list is provided.
06 / THE DETAILSWhat happens next
How to move forward
Apply through Find Jobs in AI with your profile and CV. We’ll review your application and, if you’re a good fit, we’ll be in touch about the next steps. The hiring process may use AI-assisted screening and assessment alongside human review.