AI Interview DropIntermediate TierScenario+25 XP on read
Your RAG system returns confident but wrong answers. The LLM is fine in isolation. How do you debug the pipeline?
#RAG#LLM#Evaluation
Core Summary
In RAG, generation quality is capped by retrieval quality — if the right chunk never reaches the context window, no prompt can save you. Debug by measuring retrieval and generation separately, because they fail for completely different reasons and only one of them is usually broken.
Hints
Hint 1: Separate retrieval failure from generation failure first
Hint 2: Check whether the correct chunk is even in the index
Hint 3: Chunking strategy causes more failures than embedding model choice
Reported in interviews at OpenAI, Anthropic, Perplexity