Residual Drift Dominates Contradiction in Multi-Turn Constraint Reasoning
A recent arXiv paper, "Residual Drift Dominates Contradiction in Multi-Turn Constraint Reasoning," investigates failure modes in multi-turn reasoning systems. Contrary to expectations that systems fail via logical contradiction, the research demonstrates that the dominant failure type is "satisfiable drift." In this mode, the system's internal state remains consistent, but the generated answer subtly violates previously established commitments. To analyze these failures, researchers developed DRIFT-Bench, a solver-instrumented benchmark comprising 816 test problems across three distinct constraint domains. The study evaluated four methods on DRIFT-Bench using four open-weight models ranging from 8B to 120B parameters. MUS-Repair, a technique that provides minimal unsatisfiable subsets back to the generator, consistently showed the strongest performance, improving by 1.8 to 15.0 percentage points over the best non-MUS baseline. A key finding was the nature of residual errors after structured feedback: models rarely contradicted themselves, with contradiction dropping to near zero. Instead, 98-100% of residual errors across all settings were attributed to satisfiable drift. This highlights that models tend to "forget" prior commitments while maintaining internal consistency. The authors conclude that reliable multi-turn systems must incorporate separate validation mechanisms to ensure that returned answers adhere to the system's maintained state. The code for DRIFT-Bench is publicly available on GitHub.
Developers building multi-turn AI systems must implement explicit validation steps to prevent models from silently violating prior commitments.