Reinforcement Learning

Practice Reinforcement Learning in Python with research-based exercises, implementation guides, and tests on PaperLeet.

• 1. GAE

  Estimate advantages from exponentially weighted TD residuals, with lambda controlling the bias–variance tradeoff.

  ICLR 2016 · 1 exercises
• 2. PPO

  Clip probability ratios in the policy objective to limit the incentive for excessively large updates.

  arXiv 2017 · 1 exercises
• 4. DeepSeekMath · GRPO

  Construct relative advantages from a group of responses to the same prompt and update the policy with the GRPO objective.

  arXiv 2024 · 1 exercises
• 5. Dr. GRPO

  Analyze length and difficulty biases in GRPO and adjust advantage scaling and loss normalization.

  arXiv 2025 · 1 exercises
• 10. GSPO

  Construct length-normalized sequence importance ratios and clip the policy objective at the response level.

  arXiv 2025 · 1 exercises
• 12. DAPO

  Improve policy optimization with informative-group filtering, asymmetric clipping, and token-level aggregation.

  arXiv 2025 · 1 exercises
• 25. GLM-5

  Extract head-wise orthogonalized updates and asynchronous-RL importance corrections from the GLM-5 technical report.

  arXiv 2026 · 2 exercises
• 28. SDPO

  Turn environment feedback and successful attempts into token-level self-distillation.

  ICML 2026 · 2 exercises
• 34. PKPO

  Transform rewards to optimize a pass@k policy-gradient objective.

  NeurIPS 2025 Spotlight · 1 exercises
• 38. DiffusionNFT

  Construct positive and negative implicit velocity fields from feedback to optimize diffusion models.

  ICLR 2026 Oral · 1 exercises