LearningRateDecayMethod

public enum LearningRateDecayMethod

Learning rate decay method

  • Each epoch update following: lr = lr_initial - decay * t where t is current epoch..

    Declaration

    Swift

    case Step
    Show on GitHub

  • Each epoch udpate following: lr = lr_initial * e^(-decay * t) where t is current epoch.

    Declaration

    Swift

    case Exponential
    Show on GitHub

  • Each epoch udpate following: lr = lr_initial /(1 + decay * t) where t is current epoch.

    Declaration

    Swift

    case Inverse
    Show on GitHub

  • Calculate decayed lr for current epoch.

    Declaration

    Swift

    func decayLR(initialLR: Float, decay: Float, epoch: Int) -> Float

    Parameters

    initialLR

    initial lr
    decay

    decay hyperparameter

    Return Value

    decayed lr

    Show on GitHub