technique

N-GRPO

techniqueactiveprovisionaln-grpo-426da1d7·1 events·first seen 7d ago

Aliases: N-GRPO

Co-occurring entities

DeepSeek-R1-Distill-Qwen Semantic Neighbor Mixing GRPO (Group Relative Policy Optimization)

More like this (12)

GRPO AdvGRPO IH-GRPO GRPO (Group Relative Policy Optimization)Latent-Anchored GRPO GRU GSPO (Group Sequence Policy Optimization)GRASP NDCG TRPO GPIC SPPO

Recent events (1)

4arXiv · cs.CL·7d ago·source ↗

N-GRPO: Semantic Neighbor Mixing for Improved Policy Optimization in LLM Reasoning

A new arXiv preprint introduces N-GRPO, an exploration strategy for the GRPO reinforcement learning framework that improves solution diversity during rollout by mixing embeddings of anchor tokens with their nearest semantic neighbors rather than using token-level sampling or random noise. The method is evaluated on DeepSeek-R1-Distill-Qwen models of various sizes and shows consistent improvements on math reasoning benchmarks plus out-of-distribution generalization. The work targets a known limitation in RLHF-style training: redundant rollout trajectories that reduce effective learning signal.

Evaluation and Benchmarking Alignment and RLHF N-GRPO DeepSeek-R1-Distill-Qwen Semantic Neighbor Mixing +1 more