Shake-Shake regularization
lastmod 2018-12-26created 2018-12-262 edits
About
In image recognition, data augmentation techniques that increase the amount of data have a large effect on performance. It is generally known that in deep learning, the more data the better. If you have only a small amount of data, the easiest thing to do is to acquire more data. Realistically, however, this takes time and manpower, and even when these are available, it is often difficult to acquire additional data. In such cases, artificially increasing the amount of data can give the effect of having acquired more data.
Anyone who studies or actually uses deep learning has inevitably come across augmentation techniques. In the introduction, I will briefly introduce commonly used data augmentation techniques.
Data augmentation generally means increasing one image into multiple images, and the technique differs depending on how the original image is transformed.
The Shake-Shake regularization paper is related to the data augmentation techniques described above. The difference is that ordinary augmentation targets the input image, whereas the Shake-shake technique, to quote the paper, claims to apply augmentation to "Internal Representations". The figure below explains this well.
To put it very simply, you can understand it as augmenting not at the input image stage but at the internal representation, i.e., Feature map, stage; more precisely, it augments the Gradient during training.
Documentation
- Shake-Shake regularization
- https://arxiv.org/abs/1705.07485
-
1705.07485.pdf
See also
Favorite site
- [Recommended] Shake-Shake Regularization Review