LR Schedulers

class cellarium.ml.lr_schedulers.LinearLR(optimizer: Optimizer, num_warmup_steps: int, num_training_steps: int, last_epoch: int = -1)[source]

Bases: LambdaLR

Learning rate scheduler with a learning rate that decreases linearly from the initial lr set in the optimizer to 0, after a warmup period during which it increases linearly from 0 to the initial lr set in the optimizer.

Parameters:

  • optimizer (Optimizer) – The optimizer for which to schedule the learning rate.

  • num_warmup_steps (int) – The number of steps for the warmup phase.

  • num_training_steps (int) – The total number of training steps.

  • last_epoch (int) – The index of the last epoch when resuming training.