gspo-group-sequence-policy-optimization--a1fe0b54·2 events·first seen Aliases: GSPO (Group Sequence Policy Optimization), Group Sequence Policy Optimization
Researchers present a systematic study of reward function design for reinforcement learning applied to LLM-based BPMN process model generation, training Llama 3.1 8B and Qwen 2.5 14B across 48 configurations using Group Sequence Policy Optimization. Key findings: RL substantially improves syntactic and pragmatic quality while preserving semantic fidelity, equal reward weighting outperforms targeted weighting, and reward design effects interact with model architecture in non-trivial ways. The paper argues reward composition is as consequential as the decision to apply RL at all, with implications for any multi-dimensional structured generation task.
Qwen researchers introduce Group Sequence Policy Optimization (GSPO), a new RL algorithm designed to address severe training instability and model collapse observed in existing methods like GRPO during extended training runs. The core motivation is enabling stable RL scaling for language models to improve reasoning and problem-solving capabilities with increased compute. The paper targets a known bottleneck in post-training pipelines where instability prevents further performance gains.