BatchNorm
Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
Sergey Ioffe, Christian Szegedy · ICML 2015
Distinguish population variance for training normalization from unbiased running estimates, then implement frozen-statistics inference.
What you will implement
Complete a pure function returning output and next statistics; affine training and inference-independence experiments are supplied, without reproducing ImageNet results.
- 98. BatchNorm: Batch Moments, Moving Estimates and Inference
Distinguish population variance for training normalization from unbiased running estimates, then implement frozen-statistics inference.
Complete a pure function returning output and next statistics; affine training and inference-independence experiments are supplied, without reproducing ImageNet results.