ML//RNN//GRU

2026-03-05

A GRU (Gated Recurrent Unit) is a simplified LSTM that merges the forget and input gates into a single update gate and combines the cell state and the hidden state into one vector. It has fewer parameters, trains faster and performs comparably, which makes it the usual first recurrent model to try on a sensor sequence when a gated unit is needed at all.

Two gates instead of LSTM's three: the reset gate controls how much of the previous hidden state to forget, the update gate controls how much of the new candidate state to adopt. The simplification makes GRU cheaper per timestep.

Like LSTM, GRU was designed to solve the vanishing gradient problem in RNNs. The update gate blends old and new state by interpolation instead of rewriting it, so gradients can flow through time without shrinking, enabling the model to learn long-range dependencies.

On most tasks GRU and LSTM produce similar results. The choice usually comes down to the compute budget (GRU is about 15% faster per step) and the dataset size (LSTM's extra gate helps on very large datasets with long sequences).

Both were superseded by Transformers for most NLP tasks: attention handles long-range dependencies more naturally than gating through sequential steps. GRU/LSTM survive in low-latency applications (edge devices, real-time signals, virtual sensors in process plants) where Transformer overhead is prohibitive, since a recurrent step costs the same small amount however long the history.

GRU (2014, Cho et al.) appeared shortly after LSTM's revival and quickly became the default lightweight RNN. The competition between GRU and LSTM motivated the search for even simpler sequence models, which eventually led to SSMs and Mamba.