ML//RNN//sequence-to-sequence

Sequence-to-sequence is a neural network architecture that turns one sequence into another, such as a sentence into its translation, with two recurrent networks: an encoder RNN compresses the input into a fixed-size vector, and a decoder RNN generates the output from it. It was the architecture behind early Google Translate.

Sequence-to-sequence is a neural network architecture that turns one sequence into another, such as a sentence into its translation, with two recurrent networks: an encoder RNN compresses the input into a fixed-size vector, and a decoder RNN generates the output from it. It was the architecture behind early Google Translate.

Its bottleneck is that vector, since the entire input has to be squeezed into it, and attention was invented to fix exactly that. The transformer encoder-decoder then replaced it, keeping the encode-then-decode structure and dropping recurrence for parallel attention.