Caffe

Fills a Blob with values where is set inversely proportional to number of incoming nodes, outgoing nodes, or their average. More...
#include <filler.hpp>
Public Member Functions  
MSRAFiller (const FillerParameter ¶m)  
virtual void  Fill (Blob< Dtype > *blob) 
Public Member Functions inherited from caffe::Filler< Dtype >  
Filler (const FillerParameter ¶m)  
Additional Inherited Members  
Protected Attributes inherited from caffe::Filler< Dtype >  
FillerParameter  filler_param_ 
Fills a Blob with values where is set inversely proportional to number of incoming nodes, outgoing nodes, or their average.
A Filler based on the paper [He, Zhang, Ren and Sun 2015]: Specifically accounts for ReLU nonlinearities.
Aside: for another perspective on the scaling factor, see the derivation of [Saxe, McClelland, and Ganguli 2013 (v3)].
It fills the incoming matrix by randomly sampling Gaussian data with std = sqrt(2 / n) where n is the fan_in, fan_out, or their average, depending on the variance_norm option. You should make sure the input blob has shape (num, a, b, c) where a * b * c = fan_in and num * b * c = fan_out. Note that this is currently not the case for inner product layers.