Mind Your Tone: Does Tone Alter LLM Performance?
A new study on arXiv (2605.29027) investigates how tonal variations in prompts affect LLM accuracy on multiple-choice questions. Using two datasets—a 50-question set with five tone variants and a 570-question MMLU subset spanning 57 subjects with seven tone variants—the researchers tested four cost-efficient models: ChatGPT-4o, ChatGPT-5-nano, Gemini 2.5 Flash, and Gemini 2.5 Flash Lite. Results show tonal effects are systematic but highly model-dependent: some models exhibited small, statistically significant shifts, while others showed large accuracy swings across tones. Subject-level differences in tone sensitivity were also identified. The authors propose a routing framework to explain how tones may attune internal reasoning modes. The study warns users against assuming tone-robust reliability in LLM deployments, highlighting the need for careful prompt engineering.
Developers must test prompts across tones to avoid accuracy swings in production.