ML//model//GPT//GPT-3//zero-shot learning

Zero-shot learning is the ability of a model to solve a task from an instruction alone, with no examples. A prompt such as `Translate this to French: Hello`, with zero demonstrations, is zero-shot. The model relies entirely on its pre-training knowledge to understand the task format, where few-shot learning adds examples and teaches by showing. Both are modes of in-context learning, and zero-shot is its extreme case: the context is the task description alone, with no exemplars.

Zero-shot learning is the ability of a model to solve a task from an instruction alone, with no examples. A prompt such as Translate this to French: Hello, with zero demonstrations, is zero-shot. The model relies entirely on its pre-training knowledge to understand the task format, where few-shot learning adds examples and teaches by showing. Both are modes of in-context learning, and zero-shot is its extreme case: the context is the task description alone, with no exemplars.

GPT-3 revealed the capability hierarchy: zero-shot below few-shot, and few-shot below fine-tuned. But the gap between zero-shot and few-shot is often smaller than expected, especially for well-understood tasks.

It works because during pre-training on billions of tokens, the model encountered countless examples of translation, summarization, Q&A in natural format. Zero-shot just activates those learned patterns via the right prompt.

CLIP is zero-shot by design: it maps images and text into the same latent space, so it can classify any image by checking cosine similarity against arbitrary text labels it has never been trained on.

It is the practical bridge to prompt engineering: since zero-shot performance depends entirely on how the instruction is phrased, prompt engineering exists to close the gap between zero-shot and few-shot without needing examples.

Emergent behavior often manifests as zero-shot capabilities: models below a certain scale can't do a task at all (even with examples), then suddenly larger models do it with zero examples.