Caffe

Fills a Blob with values where is set inversely proportional to number of incoming nodes, outgoing nodes, or their average. More...
#include <filler.hpp>
Public Member Functions  
XavierFiller (const FillerParameter ¶m)  
virtual void  Fill (Blob< Dtype > *blob) 
Public Member Functions inherited from caffe::Filler< Dtype >  
Filler (const FillerParameter ¶m)  
Additional Inherited Members  
Protected Attributes inherited from caffe::Filler< Dtype >  
FillerParameter  filler_param_ 
Fills a Blob with values where is set inversely proportional to number of incoming nodes, outgoing nodes, or their average.
A Filler based on the paper [Bengio and Glorot 2010]: Understanding the difficulty of training deep feedforward neuralnetworks.
It fills the incoming matrix by randomly sampling uniform data from [scale, scale] where scale = sqrt(3 / n) where n is the fan_in, fan_out, or their average, depending on the variance_norm option. You should make sure the input blob has shape (num, a, b, c) where a * b * c = fan_in and num * b * c = fan_out. Note that this is currently not the case for inner product layers.
TODO(dox): make notation in above comment consistent with rest & use LaTeX.