Adam¶

class Adam(lr: float = 0.001, b1: float = 0.9, b2: float = 0.999, t: int = 0, epsilon: float = 1e-08)¶

Adam optimizer.

Example:

optim = Adam(lr=0.005)
net.build(optimizer=optim)

__init__(lr: float = 0.001, b1: float = 0.9, b2: float = 0.999, t: int = 0, epsilon: float = 1e-08) → None¶

Initializes the object.

Parameters

lr (float, optional) – learning rate, defaults to 0.001
b1 (float, optional) – used to decay the running average of the gradient, defaults to 0.9
b2 (float, optional) – used to decay the running average of the squared gradient, defaults to 0.999
t (int, otpional) – time step, best to leave at zero, defaults to 0
epsilon (float, optional) – small value to scape the division by zero, best to leave it alone, defaults to 1e-8

Methods

`__init__`([lr, b1, b2, t, epsilon])	Initializes the object.
`build`(net)	Called before training, optimizer needs the model to be able to iterate over params.
`step`()	Traning step.

build(net: NeuralNet) → None¶

Called before training, optimizer needs the model to be able to iterate over params.

This function is called douring build in your model.

step() → None¶

Traning step.

This function is run during each of your training steps, it updates the model

BlueBird