Softmax

class Softmax

Bases: bluebird.activations.activation.Activation

Softmax activation function as object.

Inherits all of its atributes from base Activation class.

Only functions are specified, which you can see in previous page.

Example:

softmax = Softmax()
net = NeuralNet([
        ...
        softmax,
        ...
    ])

__init__()

Initializes the object.

Function and it’s derivation are of the type: F = Callable[[Tensor], Tensor]

Parameters

• f (F[Callable]) – activation function

• f_prime (F[Callable]) – derivation of activation function

Methods

__init__()	Initializes the object.
backward(grad)	Used to calculate the gradients of weights and biases.
build(input_size)	Used to finalize building layers.
forward(inputs[, training])	Called each time the data passes throughout the nework.
get_params()	Returns params of layer.

backward(grad: numpy.ndarray) → numpy.ndarray

Used to calculate the gradients of weights and biases.

Parameters

grad (Tensor) – gradient from previous layer or loss function.

Returns

f_prime(grad), applies deravtion of the activation function to the gradient

Return type

Tensor

build(input_size) → None

Used to finalize building layers.

Important to note, you should set the input_size for the layer in here. Does not apply to the activation functions, you don’t need implement it in them.

Parameters

input_size (int) – output size from previous layer

Raises

NotImplementedError –

forward(inputs: numpy.ndarray, training: bool = False) → numpy.ndarray

Called each time the data passes throughout the nework.

Parameters

• inputs (Tensor) – output from the previous layer

• training (bool, optional) – set to true during training, and is false when network predicts

Returns

f(inputs), applies the activation function to the data

Return type

Tensor

get_params() → Dict

Returns params of layer.

Returns

all params of layer

Return type

dict