ML//prompt tuning

- Train a small set of continuous "soft prompt" vectors prepended to the input, keep the model frozen.

Train a small set of continuous "soft prompt" vectors prepended to the input, keep the model frozen.

Prefix-tuning: soft prompts at every layer. Prompt tuning (Lester et al.): only at the input embedding layer.

Much cheaper than fine-tuning — only the prompt vectors are trainable (a few thousand parameters)

A learned prompt embedding can outperform hand-written prompts, especially on classification tasks.