Math

The study of structure, quantity, space, and change, the language in which all formal sciences are expressed.

The study of structure, quantity, space, and change, the language in which all formal sciences are expressed.

In ML: linear algebra (matrix multiplications in every layer), calculus (gradients and backpropagation), probability theory (distributions, sampling, Bayesian reasoning), and optimization (loss landscapes, gradient descent)

Compositionality (the principle that complex functions are built by composing simpler ones) is both a mathematical concept and the operational mechanism of transformers and chain-of-thought reasoning.

Statistics vs machine learning: statistics asks "what does the data tell us?" (inference). ML asks "what can we predict from the data?" (generalization). The boundary is blurry: regularization, cross-validation, and bias-variance tradeoffs live in both.

The unreasonable effectiveness of mathematics in ML: architectures designed from mathematical principles (attention as soft dictionary lookup, residual connections as Euler integration, normalization as variance stabilization) consistently outperform architectures designed by intuition alone.