ML//neural network//backpropagation

The algorithm that makes neural nets learn — chain rule applied backwards through the graph.

The algorithm that makes neural nets learn — chain rule applied backwards through the graph.

Compute the gradient of the loss function with respect to every weight, then nudge each one.

Forward pass computes the output, backward pass computes all gradients in one sweep.

In standard fine-tuning and DPO: touches ALL parameters (attention + MLP). Nothing frozen by default.

In LoRA: gradients only flow through the small adapter matrices — base weights are frozen, backprop skips them.