VAR

Visual Autoregressive Modeling: Scalable Image Generation via Next-Scale Prediction

Keyu Tian et al. · NeurIPS 2024

Read paper · arXiv:2404.02905

Generate visual tokens from coarse to fine, predicting tokens within each scale in parallel.

What you will implement

Next-scale conditioning and attention masks, plus multiscale residual quantization and reconstruction.

• 50. VAR Next-Scale Prediction

  Build lower-scale teacher inputs and a scale-causal visibility matrix.

  Frozen tokenizer helpers are provided; implement teacher inputs and masking, without a full Transformer.

• 51. VAR Multiscale Residual Quantization

  Quantize residual features from coarse to fine and accumulate full-resolution reconstructions.

  Given features, a shared codebook and scale projections, implement frozen tokenizer encoding and decoding; VQ-VAE training is outside scope.