Not All Rollouts are Useful: Down-Sampling Rollouts in LLM Reinforcement Learning

ArXi:2504.13818v4 Announce Type: replace-cross Reinforcement learning with verifiable rewards (RLVR) has emerged as the leading approach for enhancing reasoning capabilities in large language models. However, it faces a fundamental compute and memory asymmetry: rollout generation is embarrassingly parallel and memory-light, whereas policy updates are communication-heavy and memory-intensive. To address this, we