ML//neural network//optimizer

An optimizer decides how the weights are updated once the gradients have been computed. Three choices are common:

An optimizer decides how the weights are updated once the gradients have been computed. Three choices are common:

SGDMultiplies the gradient by the learning rate and subtracts it from the weight; simple but slow

MomentumAccumulates past gradients, like a ball rolling downhill