ML//model//GPT//GPT-3//few-shot learning

Few-shot learning is a way of adapting a language model to a task by placing a few examples in the prompt, from which the model generalizes to new inputs. It is the explicit form of in-context learning, with deliberate examples where the general ability works from any pattern in the context. There is no gradient update and no fine-tuning: the model learned to learn during pre-training.

Few-shot learning is a way of adapting a language model to a task by placing a few examples in the prompt, from which the model generalizes to new inputs. It is the explicit form of in-context learning, with deliberate examples where the general ability works from any pattern in the context. There is no gradient update and no fine-tuning: the model learned to learn during pre-training.

It did not work reliably below about 10B parameters; scale was the key. The attention heads that implement pattern matching do the heavy lifting.