paper
arXiv cs.CV
November 5th, 2026 at 5:00 AM

Beyond Scene Shortcuts: Action-Centric Video-Text Alignment for Fine-Grained Action Understanding

arXiv:2611.06302v1 Announce Type: new Abstract: Fine-grained action understanding requires vision-language models to ground what an agent does, how it acts, and why it acts in spatio-temporal evidence. However, richer supervision can backfire: fine-tuned models may generate detailed action descriptions grounded in static scenes and salient objects rather than motion, a failure we call Environment-Grounded Action Hallucination (EGAH). To mitigate EGAH, we propose Staged Attention Rectification and Reinforcement (STARR), an action-centric video-text alignment framework that breaks scene-action shortcuts through Action-Environment Dissociation and reinforces action-focused grounding with an Attention Consistency Reward within Group Relative Policy Optimization. We further construct FineActionR&R, combining temporally localized captions with hierarchical and temporal-causal reasoning data. Built on Qwen2.5-VL-7B, FineActionVLM-R1 improves average MotionBench accuracy by 3.01 percentage points over its base model and achieves 91.50% on HAU-MCQ-Bench. Qualitative comparisons further indicate more faithful fine-grained action descriptions with fewer environment-induced hallucinations.

#ai

Get stories like this every Monday

The week's most important AI news in one email. Free, unsubscribe anytime.

Score: 2.82

Engagement proxy: 0

Canonical link: https://arxiv.org/abs/2611.06302