paper
arXiv cs.AI
October 31st, 2026 at 4:00 AM

ROM: Real-time Overthinking Mitigation via Streaming Detection and Intervention

arXiv:2603.22016v4 Announce Type: replace-cross Abstract: Large reasoning models (LRMs) often suffer from overthinking, continuing with redundant verification and repeated attempts even after reaching a correct solution. This wastes computation and can overturn an initially correct answer. We show that the transition from productive to redundant reasoning is decodable from late-layer hidden states. These representations separate the two phases around first-correct-solution boundaries beyond what position and lexical controls explain, and distinguish correct from incorrect solution endings. Building on this finding, we propose ROM (Real-time Overthinking Mitigation), which uses a lightweight streaming detector (0.07--0.25\% of backbone parameters) to monitor a frozen LRM and intervene at well-formed reasoning boundaries without answer extraction or probe decoding. Counterfactual Self-Correction (CSC) augments detector training with wrong$\rightarrow$correct trajectories to preserve useful self-correction. Heads are trained separately for each backbone using shared MATH500 supervision and transfer zero-shot to four unseen benchmarks. Evaluated against ten baselines across five backbones from three families and five benchmarks, ROM$_{\text{CSC}}$ reduces response length by 28--77\% (45\% on average), with significant reductions in all 25 settings and accuracy matching or exceeding vanilla decoding in 22. Measurements on two backbones further show 20--70\% lower end-to-end latency with less than 0.5\% detector overhead per decoding step. Code is available at https://github.com/SaFo-Lab/ROM.

#ai
#product
#open_source

Get stories like this every Monday

The week's most important AI news in one email. Free, unsubscribe anytime.

Score: 2.94

Engagement proxy: 0

Canonical link: https://arxiv.org/abs/2603.22016