Dropout¶

class Dropout(output_size: int, droput_rate: float)¶

Dropout ignores some of the inputs and scales other ones. Usefull to escape overfitting.

Inherits Linear layer, and changes only constructor and forward method.

Backward is the same as in Linear.

Example:

dropout = Dropout(50, dropout_rate=0.02)
net = NeuralNet([
        ...
        dropout
        ...
    ])

__init__(output_size: int, droput_rate: float) → None¶

Initializes the object.

Parameters

Methods

`__init__`(output_size, droput_rate)	Initializes the object.
`backward`(grad)	Used to calculate the gradients of weights and biases.
`build`(input_size)	Called by the model, before its training step.
`forward`(inputs[, training])	Called each time the data passes throughout the nework.
`get_params`()	Returns params of layer.

backward(grad: numpy.ndarray) → numpy.ndarray¶

Used to calculate the gradients of weights and biases.

Parameters: grad (Tensor) – gradient from previous layer or loss function.
Returns: Gradient
Return type: Tensor

build(input_size: int) → None¶

Called by the model, before its training step.

It sets the input size and initializes the weights.

forward(inputs: numpy.ndarray, training: bool = False) → numpy.ndarray¶

Called each time the data passes throughout the nework.

It only disables inputs during training, otherwise it acts like regular Linear layer.

Parameters

inputs (Tensor) – output from the previous layer
training (bool, optional) – set to true during training, and is false when network predicts

Returns

processed input data

Return type

Tensor

get_params() → Dict¶

Returns params of layer.

BlueBird