Layer

class Layer

Bases: object

Default layer, that all other layers and activations inherit.

Example:

class CustomLayer(Layer):
     def build(self, input_size) -> None:
         super().__init__()
         ... Make sure you do initialize input_size(it's inherited from previous layer)

     def forward(self, inputs: Tensor, training: bool = False) -> Tensor:
         ... when data is passing forward

     def backward(self, grad: Tensor) -> Tensor:
         ... Make shoure you calculate gradients for weights and biases if needed
         ... (self.grads['w'] and self.grads['b'])

__init__() → None

Initialize self. See help(type(self)) for accurate signature.

Methods

__init__()	Initialize self.
backward(grad)	Used to calculate the gradients of weights and biases.
build(input_size)	Used to finalize building layers.
forward(inputs[, training])	Called each time the data passes throughout the nework.
get_params()	Returns params of layer.

backward(grad: numpy.ndarray) → numpy.ndarray

Used to calculate the gradients of weights and biases.

Parameters

grad (Tensor) – gradient from previous layer or loss function.

Raises

NotImplementedError –

build(input_size) → None

Used to finalize building layers.

Important to note, you should set the input_size for the layer in here. Does not apply to the activation functions, you don’t need implement it in them.

Parameters

input_size (int) – output size from previous layer

Raises

NotImplementedError –

forward(inputs: numpy.ndarray, training: bool = False) → numpy.ndarray

Called each time the data passes throughout the nework.

Parameters

• inputs (Tensor) – output from the previous layer

• training (bool, optional) – set to true during training, and is false when network predicts

Raises

NotImplementedError –

get_params() → Dict

Returns params of layer.

Returns

all params of layer

Return type

dict