MEDvidi (US-based AI-powered mental healthcare platform) is hiring an AI Product Analyst to evaluate, monitor and improve Voice AI Agents that interact with patients and automate parts of the patient journey.
Responsibilities:
- Analytics for Voice AI Agents and conversational AI systems
- Investigate hallucinations, inconsistencies, silent failures and quality degradation
- Analyze call transcripts, conversation logs, AI outputs and operational datasets
- Measure AI quality, reliability and impact on patient experience and business outcomes
- Build monitoring frameworks and dashboards for AI performance
- Detect quality shifts, anomalies and emerging failure modes
- Partner with Product, Engineering, QA and Operations teams
Requirements:
- 3+ years in Product Analytics, Data Analytics or Operational Analytics
- Strong SQL skills
- Proficiency in Python for analytical investigations and working with large-scale text datasets
- Understanding of LLM limitations, hallucinations, prompt sensitivity and AI failure modes
- Experience building monitoring systems, analytical frameworks or dashboards
- Strong communication skills and ability to translate insights into product decisions
- Fluent Russian and English (B2+)
- CET-overlapping working hours
Nice to have:
- Experience with conversational AI or AI agents
- Experience evaluating LLM outputs and AI quality metrics
- Experience with ML pipelines or AI-powered products
Conditions & Benefits:
- Fully remote long-term B2B collaboration
- Health insurance, wellness compensation, paid vacation & sick leave
- Personalized English lessons via Preply
- High ownership and direct impact on product decisions