Activation

class Activation(f: Callable[[numpy.ndarray], numpy.ndarray], f_prime: Callable[[numpy.ndarray], numpy.ndarray], *argv)

Bases: bluebird.layers.layer.Layer

Default activation, that all other activations inherit

Example:

class CustomActivation(Activation):
     def __init__(self, f: F, f_prime: F) -> None:
         super().__init__(f, f_prime)
         # where f is activation function and f_prime is its derivation
         ...

     # forward and backward methods should not be touched (unless you know what you are doing)

__init__(f: Callable[[numpy.ndarray], numpy.ndarray], f_prime: Callable[[numpy.ndarray], numpy.ndarray], *argv) → None

Initializes the object.

Function and it’s derivation are of the type: F = Callable[[Tensor], Tensor]

Parameters

• f (F[Callable]) – activation function

• f_prime (F[Callable]) – derivation of activation function

Methods

__init__(f, f_prime, *argv)	Initializes the object.
backward(grad)	Used to calculate the gradients of weights and biases.
build(input_size)	Used to finalize building layers.
forward(inputs[, training])	Called each time the data passes throughout the nework.
get_params()	Returns params of layer.

backward(grad: numpy.ndarray) → numpy.ndarray

Used to calculate the gradients of weights and biases.

Parameters

grad (Tensor) – gradient from previous layer or loss function.

Returns

f_prime(grad), applies deravtion of the activation function to the gradient

Return type

Tensor

build(input_size) → None

Used to finalize building layers.

Important to note, you should set the input_size for the layer in here. Does not apply to the activation functions, you don’t need implement it in them.

Parameters

input_size (int) – output size from previous layer

Raises

NotImplementedError –

forward(inputs: numpy.ndarray, training: bool = False) → numpy.ndarray

Called each time the data passes throughout the nework.

Parameters

• inputs (Tensor) – output from the previous layer

• training (bool, optional) – set to true during training, and is false when network predicts

Returns

f(inputs), applies the activation function to the data

Return type

Tensor

get_params() → Dict

Returns params of layer.

Returns

all params of layer

Return type

dict