GLM-5

GLM-5: from Vibe Coding to Agentic Engineering

GLM-5 Team et al. · arXiv 2026

Read paper · arXiv:2602.15763v2

Extract head-wise orthogonalized updates and asynchronous-RL importance corrections from the GLM-5 technical report.

What you will implement

Implement Muon Split's per-head orthogonalization and a double-sided importance-sampling policy loss in two independent CPU exercises.

• 31. Muon Split: Head-wise Orthogonalization

  Split an attention projection matrix by output head, compute each polar factor, and merge.

  Implement only head-wise orthogonalization, using an exact-SVD teaching model; not the full Muon optimizer or GLM training.

• 32. GLM-5: Double-sided Off-policy Correction

  Compute double-sided masked importance weights and a detached-weight token-level policy-gradient objective.

  Two numerical functions with the report’s strict boundaries and an explicit stop-gradient convention; no asynchronous scheduler or coding agent.