Xavier

lastmod 2026-11-10created 2017-09-013 edits

Xavier (Xavier initialization, Glorot initialization) is a method for setting the initial values of neural network weights. It scales the variance of the weight distribution according to the number of input and output nodes of each layer, so that signals and gradients do not vanish or explode as they pass through the layers. It works well with tanh and sigmoid activation functions, and is included as a default initialization option in frameworks such as Caffe.

See also

• Deep learning
• Caffe

Favorite site

• An Explanation of Xavier Initialization ¹

References