ML//neural network//embedding//cosine similarity

Cosine similarity is a similarity measure between two vectors that depends only on the angle between them, and it is the standard way to rank documents against a query in semantic search. It is \(\cos\theta = \frac{A \cdot B}{\lVert A \rVert \, \lVert B \rVert}\). It ranges over \([-1, 1]\): 1 means the same direction, 0 orthogonal, \(-1\) opposite.

Cosine similarity is a similarity measure between two vectors that depends only on the angle between them, and it is the standard way to rank documents against a query in semantic search. It is cos⁡θ=A⋅B∥A∥ ∥B∥\cos\theta = \frac{A \cdot B}{\lVert A \rVert , \lVert B \rVert}cosθ=∥A∥∥B∥A⋅B​. It ranges over [−1,1][-1, 1][−1,1]: 1 means the same direction, 0 orthogonal, −1-1−1 opposite.

It is the standard metric for semantic search in RAG: the query and the documents are encoded as vectors, and the documents with the highest cosine similarity win.

Cosine is preferred to the dot product because it normalizes for magnitude: a long document vector should not score higher just because it has more words.

After contrastive learning fine-tuning, the query Capital of France? and the passage Paris is the capital reach a high cosine similarity.

It is the backbone of vector databases and of sentence transformer search pipelines.