How GRPO Trains Small Language Models with Verifiable Rewards

Group Relative Policy Optimization (GRPO) is a reinforcement learning technique for training language models using verifiable outcomes rather than supervised solutions. The method generates a group of responses to a single prompt, scores each one, and uses the difference between an attempt's reward and the group's average reward to guide model updates. This approach eliminates the need for a separate critic model, reducing memory requirements for training. The design of the reward function is critical, as it defines success and determines whether the model genuinely learns to solve a problem or simply learns to exploit the scoring rules.