ML//transformer//encoder-decoder

The encoder-decoder is the original Transformer architecture: an encoder processes the input bidirectionally and a decoder generates the output autoregressively.

The encoder-decoder is the original Transformer architecture: an encoder processes the input bidirectionally and a decoder generates the output autoregressively.

The encoder uses self-attention without masking (every token sees every other); the decoder uses self-attention with causal masking plus cross-attention to the encoder.

In translation, the encoder processes El gato duerme with full bidirectional understanding, and the decoder generates The cat sleeps autoregressively, attending to the encoder through cross-attention at each step.

T5 kept this structure, GPT dropped the encoder (decoder-only) and BERT dropped the decoder (encoder-only). That split between generation and understanding defined the field.