Group Normalization

lastmod 2022-03-21created 2019-02-252 edits

Let's take a look at the paper Group Normalization, published by Yuxin Wu and Kaiming He in March 2018.

This paper introduces a normalization technique that works well as a replacement for batch normalization when the batch size is extremely small. It seems to be a useful technique for models like Faster RCNN, where the batch size during training has no choice but to be extremely small.

Image

Documentation

Group Normalization

https://arxiv.org/pdf/1803.08494.pdf

See also

• Deep learning
• Deep learning:Performance Tuning

Favorite site

• Group Normalization for Mask R-CNN (Detectron, Mask R-CNN)