Beyond Final Answers: Auditing Trajectory-Level Hallucinations in Multi-Agent Industrial Workflows
Researchers introduced Trajel, a dataset and evaluation framework for auditing trajectory-level hallucinations in multi-agent industrial workflows. Unlike existing benchmarks that evaluate only final outputs, Trajel captures failures in intermediate Thought-Action-Observation steps. The framework uses a five-type hallucination taxonomy (factual, referential, logical, procedural, scope-based) over expert-annotated agent traces from AssetOpsBench. Key findings: nearly half of hallucinated trajectories involve multiple hallucination types simultaneously; automated detectors with high binary accuracy still misclassify the subtlest types; trajectory-aware detection significantly outperforms standard post-hoc verification. The work highlights that existing benchmarks miss common failure modes, making taxonomy-grounded evaluation necessary for safer agentic deployment. The dataset and framework are publicly available on arXiv.
Developers building multi-agent systems need trajectory-level hallucination auditing for safer deployment.