GSPO

Group Sequence Policy Optimization

Chujie Zheng et al. · arXiv 2025

Read paper · arXiv:2507.18071v1

Construct length-normalized sequence importance ratios and clip the policy objective at the response level.

What you will implement

Implement sequence ratios and the GSPO loss; test masking, clipping regions, and gradients.

• 11. GSPO Sequence Ratios and Clipping

  Implement the sequence importance ratio and clipped policy loss.

  Two functions; group advantages, sampling and distributed RL are outside this module.