Caffe
Public Member Functions | List of all members
caffe::MSRAFiller< Dtype > Class Template Reference

Fills a Blob with values $ x \sim N(0, \sigma^2) $ where $ \sigma^2 $ is set inversely proportional to number of incoming nodes, outgoing nodes, or their average. More...

#include <filler.hpp>

Inheritance diagram for caffe::MSRAFiller< Dtype >:
caffe::Filler< Dtype >

Public Member Functions

 MSRAFiller (const FillerParameter &param)
 
virtual void Fill (Blob< Dtype > *blob)
 
- Public Member Functions inherited from caffe::Filler< Dtype >
 Filler (const FillerParameter &param)
 

Additional Inherited Members

- Protected Attributes inherited from caffe::Filler< Dtype >
FillerParameter filler_param_
 

Detailed Description

template<typename Dtype>
class caffe::MSRAFiller< Dtype >

Fills a Blob with values $ x \sim N(0, \sigma^2) $ where $ \sigma^2 $ is set inversely proportional to number of incoming nodes, outgoing nodes, or their average.

A Filler based on the paper [He, Zhang, Ren and Sun 2015]: Specifically accounts for ReLU nonlinearities.

Aside: for another perspective on the scaling factor, see the derivation of [Saxe, McClelland, and Ganguli 2013 (v3)].

It fills the incoming matrix by randomly sampling Gaussian data with std = sqrt(2 / n) where n is the fan_in, fan_out, or their average, depending on the variance_norm option. You should make sure the input blob has shape (num, a, b, c) where a * b * c = fan_in and num * b * c = fan_out. Note that this is currently not the case for inner product layers.


The documentation for this class was generated from the following file: