Session
Evals and Benchmarks
Data & Retrieval Systems (RAG, Search)

33 Failure Modes in RAG Systems: What Breaks in Production, Stage by Stage

Speaker

Anupama Garani

VP, AI Engineering, PIMCO

Abstract

RAG systems fail in many ways. One metric at one stage cannot find them. This talk gives a full map: 33 failure modes across 7 pipeline stages: ingestion, representation, retrieval, generation, evaluation, deployment, and agentic orchestration. The map comes from a structured review of 48 sources, with a formal definition, a visible sign, and an evidence grade (Strong / Moderate / Limited) for each mode. It was published at TrustNLP 2026 (ACL). One finding matters most for agents in production: research studies retrieval and generation well, but it studies representation, evaluation, and agentic orchestration far less. 12 modes have no dedicated peer-reviewed empirical evidence, and all 8 agentic modes are in that set. You will learn to find each failure by stage, test for it before users do, and fix the stage that broke, not only the final answer.

From the same track

Summit (2 days).

MLOps World | GenAI Summit 2026 is a two days of case studies, workshops, and expo on taking AI/ML and agentic systems into production – at the Etter-Harbin Alumni photo – full-bleed hero or browse files.

Share