Post-training

Practice Post-training in Python with research-based exercises, implementation guides, and tests on PaperLeet.

• 13. LoRA

  Freeze pretrained weights and learn an update through a low-rank matrix product.

  ICLR 2022 · 1 exercises
• 14. QLoRA

  Train low-rank adapters on a quantized frozen base to reduce fine-tuning memory.

  NeurIPS 2023 · 1 exercises
• 3. DPO

  Turn the implicit reward margin of preference pairs into an objective that directly optimizes a language model.

  NeurIPS 2023 · 1 exercises
• 11. GKD · On-policy Distillation

  Query the teacher on student-generated prefixes to reduce the mismatch between training and generation.

  ICLR 2024 · 1 exercises
• 27. OPSD

  Use a privileged-context teacher from the same model to supervise student rollouts.

  arXiv 2026 · 2 exercises
• 28. SDPO

  Turn environment feedback and successful attempts into token-level self-distillation.

  ICML 2026 · 2 exercises
• 33. DistiLLM-2

  Use distinct distillation directions on teacher- and student-generated responses.

  ICML 2025 Oral · 1 exercises