LeakyRelu

class LeakyRelu(alpha: float = 0)

Bases: bluebird.activations.activation.Activation

Relu activation function as object.

Inherits all of its atributes from base Activation class.

Only functions are specified, which you can see in previous page.

It is important to note that leaky relu activation works only with small variances, so weights should be initializes with a weight initializes that does that.

Example:

leaky_relu = LeakyRelu()
net = NeuralNet([
        ...
        leaky_relu,
        ...
    ])

__init__(alpha: float = 0)

Methods

__init__([alpha])
backward(grad)	Used to calculate the gradients of weights and biases.
build(input_size)	Used to finalize building layers.
forward(inputs[, training])	Called each time the data passes throughout the nework.
get_params()	Returns params of layer.

backward(grad: numpy.ndarray) → numpy.ndarray

Used to calculate the gradients of weights and biases.

Parameters

grad (Tensor) – gradient from previous layer or loss function.

Returns

f_prime(grad), applies deravtion of the activation function to the gradient

Return type

Tensor

build(input_size) → None

Used to finalize building layers.

Important to note, you should set the input_size for the layer in here. Does not apply to the activation functions, you don’t need implement it in them.

Parameters

input_size (int) – output size from previous layer

Raises

NotImplementedError –

forward(inputs: numpy.ndarray, training: bool = False) → numpy.ndarray

Called each time the data passes throughout the nework.

Parameters

• inputs (Tensor) – output from the previous layer

• training (bool, optional) – set to true during training, and is false when network predicts

Returns

f(inputs), applies the activation function to the data

Return type

Tensor

get_params() → Dict

Returns params of layer.

Returns

all params of layer

Return type

dict