Note: The job is a remote job and is open to candidates in USA. Accordion is a financial consulting firm that leverages data, technology, and AI to drive value for clients. The AI Operations Associate is responsible for monitoring AI-driven workflows, investigating incidents, and ensuring operational controls are in place for reliable and safe AI-enabled processes.
Responsibilities
- Monitor AI-driven workflows in production for abnormal behavior, trust issues, degraded outputs
- Review signals related to output quality, fallback use, low-confidence patterns
- Escalate issues affecting live user trust, business quality, or finance-sensitive outcomes
- Track adoption rates, usage frequency, and tool traction
- Investigate output-quality incidents, orchestration failures, prompt/workflow regressions
- Distinguish true software defects from model behavior, data-quality, or process issues
- Partner with Engineering, QA, and business stakeholders on judgment-heavy incidents
- Support human-review patterns, fallback routes, and safe-operating responses
- Define and apply production review practices for higher-risk workflows
- Document evidence and issue patterns for repeatability and release decisions
- Cover vendor-deployed solutions with same rigor as internally built workflows: managing vendor SLAs, update cycles, and incident patterns
- Identify recurring issues and recommend improvements to monitoring and quality feedback loops
- Feed production learnings back into QA, Product, and Engineering
- Feed structured, reportable data into the measurement platform including performance metrics, incident patterns, adoption signals for portfolio-level visibility
- Act as product owner for AI usage and adoption dashboards
Skills
- 4+ years experience in production support, AI operations, business process ops, analytics, or QA
- Strong analytical skills evaluating nuanced workflow/output issues with focus on safe operations
- High attention to detail; proven experience spotting subtle trust or quality issues
- Ability to work across engineering and business teams in ambiguous situations
- Comfort balancing evidence, judgment, and business impact under pressure
- Experience with AI-enabled workflows, operational analytics, or workflow-heavy systems
- Strong written communication and documentation discipline
- Python proficiency required
- Experience utilizing APIs
- Bachelor's degree preferred
Company Overview