ML//neural network//activation function//GELU

- Gaussian Error Linear Unit, smooth approximation to ReLU

Gaussian Error Linear Unit, smooth approximation to ReLU

x * Φ(x) where Φ is the Gaussian CDF.

Used in Transformers, BERT, GPT. Slightly better than ReLU for language tasks.