BatchNormalization¶

class BatchNormalization(eps: float = 1e-08)¶

Normalizes the data and applies linear transformation.

Gamma and beta are reprisented with weights and biases foe ease of implementation with optimizers.

input = (input - mean) / variance output = input * weights + bias

Example:

batch = BatchNormalization()
net = NeuralNet([
        ...
        batch,
        ...
    ])

__init__(eps: float = 1e-08) → None¶

Initializes the object.

Parameters: eps (float, optional) – prevents division by zero, defaults to 1e-8

Methods

`__init__`([eps])	Initializes the object.
`backward`(grad)	Used to calculate the gradients of weights and biases.
`build`(input_size)	Called by the model, before its training step.
`forward`(inputs[, training])	Called each time the data passes throughout the nework.
`get_params`()	Returns params of layer.

backward(grad: numpy.ndarray) → numpy.ndarray¶

Used to calculate the gradients of weights and biases.

Parameters: grad (Tensor) – gradient from previous layer or loss function.
Returns: Gradient
Return type: Tensor

build(input_size: int) → None¶

Called by the model, before its training step.

It sets the input size and initializes the weights.

forward(inputs: numpy.ndarray, training: bool = False) → numpy.ndarray¶

Called each time the data passes throughout the nework.

Parameters

inputs (Tensor) – output from the previous layer
training (bool, optional) – set to true during training, and is false when network predicts

Returns

processed input data

Return type

Tensor

get_params() → Dict¶

Returns params of layer.

BlueBird