RMSProp
RMSProp, matching torch.optim.RMSprop.
class RMSProp : Optimizer— python/vkml/optim.py:202
Divides the gradient by a running root-mean-square of recent gradients.
The running average starts at zero, as torch's does, so the first step is (1−α)·g² rather than g². That difference persists for many steps through the exponential average, so it is not a detail — a reimplementation that initialises from the first gradient diverges from torch for a long time.
Construction¶
__init__¶
def __init__self, params, lr: float=0.01, alpha: float=0.99, eps: float=1e-08, weight_decay: float=0.0, momentum: float=0.0, centered: bool=False— python/vkml/optim.py:210