Senior AI Interaction Evaluator – Up to $200/hr

About the role

We are seeking a senior‑level software engineer to evaluate the quality of interactions with modern AI coding agents such as OpenAI Codex and Claude Code. This contract position focuses on assessing how well the model mimics the reasoning and judgment of an experienced developer, rather than writing production code.

Key responsibilities

• Evaluate AI‑generated coding interactions from start to finish.
• Judge whether outputs are useful, high‑level correct, and aligned with strong engineering judgment.
• Assess the quality of explanations, reasoning, and overall interaction flow.
• Distinguish between different levels of response quality and provide clear, opinionated feedback.
• Help define standards for great interactions with AI‑first development tools.

Required profile

• Staff or principal‑level engineer (or equivalent experience).
• Deep familiarity with AI‑assisted development workflows.
• Comfortable making rigorous, subjective judgments about code quality and explanations.
• Ability to evaluate code without fully executing every line.
• Willingness to give direct, opinionated feedback.

Required skills

• TypeScript or JavaScript.
• Python.
• Hands‑on experience with OpenAI Codex.
• Hands‑on experience with Claude Code.
• Experience with the Cursor AI‑first IDE or similar tools.

What we offer

• Hourly rate up to $200 for US/Canada, $150 for EU/Latam, $100 for other regions.
• 10–20 hours per week, remote work.
• Contract through early May with possible extension.
• Take‑home evaluation exercise and one behavioral interview as part of the hiring process.