Much of LLMs' success today is due to test-time scaling. We train on a proxy task of challenging math problems, where it is impossible to sample successes within the training token budget. PPM…

PPM enables training on extremely challenging math problems. Standard GRPO cannot learn, as it is near-impossible to sample successes. Surprisingly, training at 4K tokens outperforms training at 8K,…