The Attribution Blind Spot: Detecting When Language Models Rely on Memory Rather Than Retrieved Context
A new arXiv paper (2605.26778v1) exposes a critical flaw in retrieval-augmented generation (RAG): when retrieved documents overlap with pretraining data, models can generate faithful-looking text from parametric memory alone, making output-level checks insufficient. The authors call this the 'attribution blind spot.' To address it, they propose Computational Reality Monitoring (CRM), inspired by cognitive science's reality monitoring framework. CRM compares internal representations with and without context to detect membership-conditioned representational divergence that output-level monitors miss. CRM does not certify individual generation sources but identifies whether pretraining exposure leaves a measurable internal trajectory signature, establishing a necessary substrate for source attribution. The method was tested across nine model variants, demonstrating that internal trajectory analysis can reveal reliance on memory even when output appears context-governed. This work highlights a fundamental limitation in current RAG evaluation and offers a new approach for verifying grounding in high-stakes applications.
Developers cannot trust RAG outputs without internal verification, as models may rely on memory instead of retrieved context.