AI Operations Analyst

Reporting to Manager, Production Support & Service Reliability, monitors AI-driven workflows in production, reviews output-quality incidents, supports fallback and human-review decisions, and partners on trust-sensitive escalations. This role helps ensure AI-enabled workflows remain reliable, supportable, and safe in live finance-oriented environments.

Key Responsibilities

AI Workflow Monitoring

• Monitor AI-driven workflows in production for abnormal behavior, trust issues, degraded outputs, and recurring exception patterns.
• Review signals related to output quality, fallback use, low-confidence patterns, and workflow anomalies.
• Escalate issues that could affect live user trust, business quality, or finance-sensitive outcomes.

Incident Investigation

• Support investigation of output-quality incidents, orchestration failures, prompt or workflow regressions, and tool-call problems in production.
• Help distinguish true software defects from model behavior issues, data-quality problems, or process misunderstandings.
• Partner closely with Engineering, QA, and business stakeholders on judgment-heavy incidents.

Operational Controls

• Support human-review patterns, fallback routes, and safe-operating responses for AI-enabled features.
• Help define and apply production review practices for higher-risk workflows.
• Document evidence and issue patterns in ways that support repeatability and better release decisions.

Continuous Improvement

• Identify recurring AI operations issues and recommend improvements to supportability, monitoring, and quality feedback loops.
• Feed production learnings back into QA, Product, and Engineering.
• Help improve operational trust in AI-enabled solutions over time.

Requirements

Required Qualifications

• 4+ years of experience in production support, AI operations, business process operations, analytics, QA, or a similar role.
• Strong analytical skills and comfort evaluating nuanced workflow or output issues.
• Ability to work across engineering and business teams in ambiguous situations.
• Experience with AI-enabled workflows, operational analytics, business process validation, or support for workflow-heavy systems.
• Strong written communication and documentation discipline.

·Bachelor's degree preferred.

Key Traits for Success

• Thoughtful, precise, and calm under pressure.
• Strong at spotting subtle trust or quality issues others may miss.
• Comfortable balancing evidence, judgment, and business impact.
• Focused on safe operations and clear escalation.

Benefits

Base salary plus performance-based bonus.

Actual compensation packages are determined by evaluating a wide array of factors unique to each candidate, including but not limited to skill set, years and depth of experience, education, certifications, cost of labor, and internal equity.