Math

The study of structure, quantity, space, and change — the language in which all formal sciences are expressed.

The study of structure, quantity, space, and change — the language in which all formal sciences are expressed.

In ML: linear algebra (matrix multiplications in every layer), calculus (gradients and backpropagation), probability theory (distributions, sampling, Bayesian reasoning), and optimization (loss landscapes, gradient descent)

Compositionality — the principle that complex functions are built by composing simpler ones — is both a mathematical concept and the operational mechanism of transformers and chain-of-thought reasoning.

Statistics vs machine learning: statistics asks "what does the data tell us?" (inference). ML asks "what can we predict from the data?" (generalization). The boundary is blurry — regularization, cross-validation, and bias-variance tradeoffs live in both.

The unreasonable effectiveness of mathematics in ML: architectures designed from mathematical principles (attention as soft dictionary lookup, residual connections as Euler integration, normalization as variance stabilization) consistently outperform architectures designed by intuition alone.