ML//transformer//attention//Q, K, V

Q, K and V are the three roles every token plays simultaneously in attention.

Q, K and V are the three roles every token plays simultaneously in attention.

Query QQQThe question what am I looking for?, which each token broadcasts as a 128-dimensional vector through WQW_QWQ​

Key KKKThe answer to what do I have to offer?, through which each token advertises its relevance via WKW_KWK​

Value VVVThe answer to what information do I carry?, the actual content that flows if attention fires, via WVW_VWV​

The dot product Q⋅KQ \cdot KQ⋅K measures the alignment between what one token seeks and what another offers, and gives the attention scores. VVV is not simply the original embedding, because W_V learns to extract only the subspace relevant for its head: bank attending to river should receive river-related features, not all of river. Each head has its own QQQ, KKK and VVV, which means different learned projections and different questions asked.