Practice AI research algorithms
LeetCode for AI Researchers. Implement algorithms from research papers in Python and test your understanding.
- 1. GAE
Estimate advantages from exponentially weighted TD residuals, with lambda controlling the bias–variance tradeoff.
ICLR 2016 · 1 exercises - 2. PPO
Clip probability ratios in the policy objective to limit the incentive for excessively large updates.
arXiv 2017 · 1 exercises - 3. DPO
Turn the implicit reward margin of preference pairs into an objective that directly optimizes a language model.
NeurIPS 2023 · 1 exercises - 4. DeepSeekMath · GRPO
Construct relative advantages from a group of responses to the same prompt and update the policy with the GRPO objective.
arXiv 2024 · 1 exercises - 5. Dr. GRPO
Analyze length and difficulty biases in GRPO and adjust advantage scaling and loss normalization.
arXiv 2025 · 1 exercises - 6. DDPM
Build a generative model from a forward noising process and a learned reverse denoising process.
NeurIPS 2020 · 2 exercises - 7. Progressive Distillation
Distill two deterministic teacher steps into one student step, progressively reducing the sampling budget.
ICLR 2022 · 2 exercises - 8. DDIM
Use a non-Markovian sampling update to skip timesteps, with eta controlling stochasticity.
ICLR 2021 · 1 exercises - 9. Flow Matching
Learn a continuous generative flow by regressing the velocity field of conditional probability paths.
ICLR 2023 · 1 exercises - 10. GSPO
Construct length-normalized sequence importance ratios and clip the policy objective at the response level.
arXiv 2025 · 1 exercises - 11. GKD · On-policy Distillation
Query the teacher on student-generated prefixes to reduce the mismatch between training and generation.
ICLR 2024 · 1 exercises - 12. DAPO
Improve policy optimization with informative-group filtering, asymmetric clipping, and token-level aggregation.
arXiv 2025 · 1 exercises - 13. LoRA
Freeze pretrained weights and learn an update through a low-rank matrix product.
ICLR 2022 · 1 exercises - 14. QLoRA
Train low-rank adapters on a quantized frozen base to reduce fine-tuning memory.
NeurIPS 2023 · 1 exercises - 15. MeanFlow
Learn an interval flow using the identity between instantaneous and average velocities for one-step generation.
arXiv 2025 · 1 exercises - 16. JiT
Predict clean data directly and explicitly convert between prediction and loss spaces.
arXiv 2025 · 1 exercises - 17. Drifting Models
Construct a drifting field that attracts real samples and repels generated samples, then train a generator against a stopped-gradient target.
arXiv 2026 · 1 exercises - 18. DMD
Use the score difference between real and generated distributions to construct a distribution-matching generator update.
CVPR 2024 · 1 exercises - 19. DMD2
Remove DMD's paired-regression requirement using faster fake-score updates and adversarial auxiliary training.
arXiv 2024 · 1 exercises - 20. Consistency Models
Map noisy states along the same generative trajectory to a consistent clean endpoint.
ICML 2023 · 1 exercises