ML//RAG//TF-IDF

TF-IDF (term frequency, inverse document frequency) is the classic document ranking from 1972. It scores a term in a document as the product \(\mathrm{TF}\times\mathrm{IDF}\), where TF is how often the term appears in that document and IDF how rare the term is across all documents.

TF-IDF (term frequency, inverse document frequency) is the classic document ranking from 1972. It scores a term in a document as the product TF×IDF\mathrm{TF}\times\mathrm{IDF}TF×IDF, where TF is how often the term appears in that document and IDF how rare the term is across all documents.

It is still a strong baseline for keyword search, and part of the boring IR stack (TF-IDF, BM25, inverted indices) that production RAG still depends on. BM25 is TF-IDF with a nonlinear saturation of term frequency and a normalization by document length.