••••••• is an AI-powered mental healthcare platform. The AI Receptionist Team builds and improves Voice AI Agents that interact with patients and automate parts of the patient journey.

Responsibilities:

  • Analyze production behavior of Voice AI Agents and conversational AI workflows.
  • Investigate hallucinations, inconsistencies, silent failures, and quality degradation in AI-driven conversations.
  • Analyze call transcripts, logs, generated outputs, and operational datasets to find patterns and actionable insights.
  • Design evaluation frameworks and monitoring systems to measure AI quality, reliability, business impact, and performance.
  • Measure effects on operational efficiency, business outcomes, and patient experience; detect quality shifts, anomalies, and emerging failure modes.
  • Partner with Product, Engineering, QA, and Operations teams and support decisions in ambiguous, fast-changing environments.

Requirements:

  • 5+ years of experience in Product Analytics, Data Analytics, Operational Analytics, or analytics for complex production systems.
  • Strong SQL skills and Python experience for analytical investigations and large-scale text datasets.
  • Experience with ML pipelines or AI-powered products.
  • Understanding of LLM limitations, hallucinations, evaluation challenges, prompt sensitivity, and AI failure modes.
  • Experience building monitoring systems, analytical frameworks, or operational dashboards.
  • Strong communication skills; able to translate analysis into product decisions and work effectively in ambiguous environments.
  • Fluent Russian, English B2 or higher, and availability for CET-overlapping hours.
  • Conversational AI, AI agents, LLM output evaluation, AI quality metrics, evaluation methodologies, and prompt iteration experience are a plus.

Conditions and benefits: Full-time, fully remote, long-term B2B collaboration. Competitive compensation and growth opportunities; health insurance after probation, sports and wellness compensation, personalized English lessons via Preply, 19 working days of vacation per year, 3 additional wellness days, paid sick leave for the first 5 working days, and gifts for important life events.