Freelance Agent Evaluation Engineer needed to create challenging tasks and evaluation criteria for AI coding agents in a simulated environment, with an estimated 20 hours of effort per task and up to $21 per hour equivalent pay.

Requirements

  • Degree in Computer Science, Software Engineering, or related fields
  • 5+ years in software development, primarily Python (FastAPI, pytest, async/await, subprocess, file operations)
  • Background in full-stack development, with experience building React-based interfaces (JavaScript/TypeScript) and robust back-end systems
  • Experience writing tests (functional, integration — not just running them)
  • Docker containers, and familiarity with infrastructure tools (Postgres, Kafka, Redis)
  • CI/CD understanding (GitHub Actions as a user: triggers, labels, reading results)
  • English proficiency - B2

Benefits

  • Part-time, non-permanent projects
  • Opportunity to work on AI-related projects
  • Competitive pay ($21 per hour equivalent)
  • Flexible work schedule