Find Jobs in AI All opportunities
Open for applicationsAI training

Domain Expert — AI Reasoning & Evaluation Tasks

20 openings
Pay
Paid per task · listed $30–$60/hr
Location
Remote · Worldwide
Experience level
Expert

A CLOSER LOOK

Explore the opportunity

01 / THE DETAILS

Role overview

Get to know the opportunity

Use specialist knowledge to design challenging tasks and assess the accuracy of advanced AI responses.

02 / THE DETAILS

What you’ll be doing

Your day-to-day contribution
  • Create questions and scenarios that test reasoning within your subject area.
  • Check model answers against authoritative primary sources.
  • Document reasoning errors and apply evaluation rubrics consistently.
03 / THE DETAILS

Who this could suit

See where your experience could fit

Experienced researchers or practitioners in history, archives, law, finance, journalism or technology history.

04 / THE DETAILS

What you’ll need

Experience, skills and essential requirements
  • Hands-on experience of AI training, annotation, RLHF or model evaluation is requested, alongside deep subject expertise and strong research skills.
  • You should explain nuanced findings clearly and follow detailed instructions.
  • Adversarial task design, graduate research or additional language skills can be useful.

You don’t have to match every requirement exactly. We welcome applicants with different backgrounds, levels of experience and transferable skills.

05 / THE DETAILS

Pay and working arrangements

Rates, commitment and working arrangements

Paid per task; the role listing gives a pay range of $30–$60 USD per hour alongside its output-based payment terms. Payment is for tasks that meet the project specifications, and the time needed for each task can vary. Minimum weekly submission requirements apply; their quantity and the payment schedule are not specified. Worldwide remote contractor work, with a flexible schedule and an expected commitment of around 15 hours per week. Project duration is not specified.

06 / THE DETAILS

What happens next

How to move forward

Apply through Find Jobs in AI with your profile and CV. We’ll review your application and, if you’re a good fit, we’ll be in touch about the next steps.

Practice for this role

Review a specialist conclusion

A quality report required checks on five samples. Four passed, and the fifth was not measured. The draft says “All five samples passed, so the batch meets the release rule.”

Explain what the evidence supports and what you would ask the team to check before release.

Sign in for private feedback or work through the example below.

See the rubric and example answer
  • Evidence · Four passed and the fifth has no recorded result.
  • Conclusion · The evidence does not establish that all five passed or that the release rule was met.
  • Follow-up · Measure and document the fifth sample before deciding.

Four samples passed. There is no result for the fifth, so the release rule has not been verified. Measure and document the fifth sample before deciding.

Application checklist
Requirements from the listing

Hands-on experience of AI training, annotation, RLHF or model evaluation is requested, alongside deep subject expertise and strong research skills. You should explain nuanced findings clearly and follow detailed instructions. Adversarial task design, graduate research or additional language skills can be useful.

Sign in to check your saved CV.

PayPaid per task · listed $30–$60/hr
Continue to apply