ML//training//post-training//GRPO

GRPO (Group Relative Policy Optimization) is DeepSeek's alternative to PPO for RL fine-tuning: it scores multiple completions relative to each other within a group, so no separate RM is needed.

GRPO (Group Relative Policy Optimization) is DeepSeek's alternative to PPO for RL fine-tuning: it scores multiple completions relative to each other within a group, so no separate RM is needed.

It is simpler, more stable and cheaper than PPO, because the group-relative score replaces PPO's learned value function, and it was used to train DeepSeek R1's reasoning capabilities. It belongs to the approach of RL without process reward models that challenged OpenAI's PRM methods.