ML//transformer//decoder

A decoder is the GPT-style architecture: self-attention with causal masking, in which each token sees only the tokens before it.

A decoder is the GPT-style architecture: self-attention with causal masking, in which each token sees only the tokens before it.

GPT is decoder-only, with no encoder and no cross-attention. When it translates, it learned translation as a pattern during pre-training, without a dedicated encoder.

In encoder-decoder models (T5, translation) the decoder has both causal self-attention and cross-attention to the encoder output. In that cross-attention QQQ comes from the target language (what is being generated) and KKK and VVV from the source (the encoder output), starting from [BOS] as the cold start.

Masking enables the KV cache: previous tokens are closed, and their representations do not change when a new token arrives.