Fills a Blob with values $x \sim N(0, \sigma^2)$ where $\sigma^2$ is set inversely proportional to number of incoming nodes, outgoing nodes, or their average. More...

#include <filler.hpp>

Inheritance diagram for caffe::MSRAFiller< Dtype >:

Public Member Functions
	MSRAFiller (const FillerParameter &param)

virtual void	Fill (Blob< Dtype > *blob)

Public Member Functions inherited from caffe::Filler< Dtype >
	Filler (const FillerParameter &param)

Additional Inherited Members
Protected Attributes inherited from caffe::Filler< Dtype >
FillerParameter	filler_param_

Detailed Description

template<typename Dtype>
class caffe::MSRAFiller< Dtype >

Fills a Blob with values $x \sim N(0, \sigma^2)$ where $\sigma^2$ is set inversely proportional to number of incoming nodes, outgoing nodes, or their average.

A Filler based on the paper [He, Zhang, Ren and Sun 2015]: Specifically accounts for ReLU nonlinearities.

Aside: for another perspective on the scaling factor, see the derivation of [Saxe, McClelland, and Ganguli 2013 (v3)].

It fills the incoming matrix by randomly sampling Gaussian data with std = sqrt(2 / n) where n is the fan_in, fan_out, or their average, depending on the variance_norm option. You should make sure the input blob has shape (num, a, b, c) where a * b * c = fan_in and num * b * c = fan_out. Note that this is currently not the case for inner product layers.

The documentation for this class was generated from the following file:

include/caffe/filler.hpp

Public Member Functions

Additional Inherited Members

Detailed Description

template<typename Dtype> class caffe::MSRAFiller< Dtype >

template<typename Dtype>
class caffe::MSRAFiller< Dtype >