The Deterministic Horizon: Impossibility Results as Design Specifications for Trustworthy AI Systems
A paper on arXiv (cs.AI) titled 'The Deterministic Horizon: Impossibility Results as Design Specifications for Trustworthy AI Systems' proves that transformer architectures have an accuracy ceiling determined solely by architecture—specifically, layer count and embedding width—that cannot be surpassed by any amount of training data, adapter rank, sample size, or loss function. This 'Deterministic Horizon' was measured across twelve transformer architectures, ranging from 19 to 31 layers. Fine-tuning on optimal-length traces recovers less than four percentage points. The mechanism is a capacity invariant of the residual stream, and an information-theoretic conversion yields super-exponential accuracy decay past the horizon. The paper also provides an unconditional circuit-complexity lower bound for modular exponentiation against constant-depth prime-modulus circuits. These impossibility results are framed as design specifications for trustworthy AI systems, turning fundamental limits from curiosities into engineering rules.
Developers must consider architectural limits when designing models, as performance cannot be improved beyond a fixed horizon.