arXiv cs.AIWednesday · May 27, 2026FREE

TADDLE: A Tool-Augmented Agent for Detecting Deficient LLM-Generated Peer Reviews

TADDLE, introduced in a new arXiv paper, addresses the challenge of detecting defects in LLM-generated peer reviews, which are increasingly common at major venues. The system decomposes detection into four specialized analysis tools: Verify, Correct, Complete, and Transform, orchestrated by an agent. An integrator synthesizes outputs into binary and multi-label classifications via two-stage semi-supervised learning. The benchmark comprises 1,800 reviews on 50 ICLR 2025 papers, multi-label-annotated by 18 domain experts against a taxonomy of six defect categories (plus a non-deficient label). Experiments show TADDLE performs strongly on both binary detection and multi-label classification. This work provides the first expert-annotated benchmark for this task, enabling systematic evaluation of review quality.