Find Jobs in AI All opportunities
Open for applicationsCoding & data

Coding Agent Research & Evaluation Engineer

Open role
Pay
$200,000–$260,000 / year
Location
Remote
Experience level
Experienced

A CLOSER LOOK

Explore the opportunity

01 / THE DETAILS

Role overview

Get to know the opportunity

Develop the benchmarks, datasets and research tools used to measure and improve advanced AI coding agents.

02 / THE DETAILS

What you’ll be doing

Your day-to-day contribution
  • Define coding-agent benchmarks, scoring methods, rubrics and evaluation standards.
  • Create reference examples and datasets, investigate model failures and recommend improvements.
  • Build tooling for large-scale experiments and review workflows, and communicate findings in technical studies.
03 / THE DETAILS

Who this could suit

See where your experience could fit

Software engineers and AI researchers interested in rigorous experiments, reliable measurements and how coding models succeed or fail on real engineering tasks.

04 / THE DETAILS

What you’ll need

Experience, skills and essential requirements
  • At least three years of experience in software engineering, ML, AI research, evaluation or a related technical discipline.
  • Strong programming in Python, C++ or comparable languages; experience designing or validating benchmarks, coding tasks or technical assessments; and familiarity with LLMs, coding agents, reinforcement learning or model evaluation.
  • You should be able to automate workflows, analyse complex systems and explain technical findings clearly.
  • Research publications, open-source work, agentic workflows and experience leading ambiguous research projects are useful additions.

You don’t have to match every requirement exactly. We welcome applicants with different backgrounds, levels of experience and transferable skills.

05 / THE DETAILS

Pay and working arrangements

Rates, commitment and working arrangements

Remote, full-time employment. The detailed description states a USD base salary of $200,000–$260,000 per year. The listing header instead shows $8–$9 per hour and labels the role hourly; these conflict with the full-time salary terms in the description. The salary above follows the detailed description, and the final offer should confirm the terms. Equity eligibility, possible performance bonuses, health-insurance reimbursement of up to 100%, paid time off and a matched 401(k) are listed, subject to policy. No eligible-country list is provided.

06 / THE DETAILS

What happens next

How to move forward

Apply through Find Jobs in AI with your profile and CV. We’ll review your application and, if you’re a good fit, we’ll be in touch about the next steps. The hiring process may use AI-assisted screening and assessment alongside human review.

Practice for this role

Check an AI answer against a source

Notice: “Returns are accepted within 30 days with a receipt. Opened software is excluded.” AI answer: “You can return any item within 45 days without a receipt.”

Identify the errors and write a corrected answer using the notice.

Sign in for private feedback or work through the example below.

See the rubric and example answer
  • Time limit · Correct 45 days to 30 days.
  • Receipt requirement · State that a receipt is required.
  • Exception · State that opened software is excluded.

Returns are accepted within 30 days with a receipt. Opened software is excluded.

Application checklist
Requirements from the listing

At least three years of experience in software engineering, ML, AI research, evaluation or a related technical discipline. Strong programming in Python, C++ or comparable languages; experience designing or validating benchmarks, coding tasks or technical assessments; and familiarity with LLMs, coding agents, reinforcement learning or model evaluation. You should be able to automate workflows, analyse complex systems and explain technical findings clearly. Research publications, open-source work, agentic workflows and experience leading ambiguous research projects are useful additions.

Sign in to check your saved CV.

Pay$200,000–$260,000 / year
Continue to apply