SVDQuant

SVDQuant: Absorbing Outliers by Low-Rank Components for 4-Bit Diffusion Models

Muyang Li et al. · ICLR 2025 Spotlight

Read paper · arXiv:2411.05007v1

Absorb difficult-to-quantize components in a high-precision low-rank branch and quantize the residual.

What you will implement

Smoothing, SVD low-rank decomposition, and a 4-bit residual forward pass.

• 46. SVDQuant Smoothing, Low-rank and Residual Quantization

  Implement smoothing, truncated SVD and a quantized residual two-branch forward.

  Given smoothing factors and an INT fake quantizer; one SVD, no calibration search, FP4, iterative refinement or GPU kernels.