Probe with Participation Trophies: Random-Reward RL as a Probe of LLM Capability
Yu Mao, Lei Yu, Zining Zhu, Yusheng Zheng, Haohang Li, Freda Shi, Yutong Yin, Zhaoran Wang and Jingcheng Niu.
arXiv preprint
TL;DR
Spurious-reward RL probes a model’s reachability: what further training can attain from its current state. Rather than attributing random-reward gains to specific RL mechanisms or data contamination, we connect the spurious-reward paradox to reachability and propose random-reward RL as a probing tool. Across OLMo checkpoints, two models with the same 3.5% baseline reach 8.5% and 55% under correct-answer rewards, and training splits into three phases: early pre-training with minimal RL gains, later pre-training where correct rewards work, and mid-training where even random rewards bring substantial improvements. Because the reward carries no information, this sidesteps the long-standing probing question of whether success reflects what the model knows or what the probe learns.
How to Cite
@article{mao2026participation,
title = {Probe with Participation Trophies: Random-Reward {RL} as a Probe of {LLM} Capability},
author = {Mao, Yu and Yu, Lei and Zhu, Zining and Zheng, Yusheng and Li, Haohang and Shi, Freda and Yin, Yutong and Wang, Zhaoran and Niu, Jingcheng},
journal = {arXiv preprint arXiv:2610.01066},
year = {2026},
}