Skip to content

Xavier filler and inner product parameters #1575

@denizyuret

Description

@denizyuret

XavierFiller in filler.hpp takes a blob and fills it with U[-scale, scale] where scale = sqrt(Dtype(3) / fan_in). fan_in is calculated as blob->count() / blob->num(). Now if this is a parameter blob (I am assuming it is, since we are passing it to a filler), blob->num() is always 1 (in fact the GaussianFiller checks for this condition) so we are in effect scaling all weights to sqrt(3/count) instead of sqrt(3/fan_in). Isn't the fan_in simply blob->width()?

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions