Learning from a Whisper: Few-Shot and Zero-Shot Learning

How LLMs generalize from a tiny number of examples in the prompt.

Team · · 7 min read

Before Large Language Models, if you wanted an AI to perform a task (like translating French to English), you had to "train" it on millions of pairs of French-English sentences.

In 2020, OpenAI showed that this wasn't always necessary. They introduced In-Context Learning.

The Scale of Generalization

  1. Zero-Shot Learning: You give the model a task with no examples. "Translate 'Bonjour' to English."
  2. One-Shot Learning: You give the model one example. "Translate 'Chat' -> 'Cat'. Now translate 'Chien' ->"
  3. Few-Shot Learning: You give the model a small set of examples (usually 3 to 10).

Why Is This So Impressive?

In classical machine learning, a model cannot learn a new task without updating its weights. But with LLMs, the weights don't change.

The model is "learning" the task purely within its Attention Window. It is using its massive pre-trained knowledge of language to "infer" what the user wants based on the patterns in the prompt.

The Implications for Developers

Few-shot learning is the foundation of "Prompt Engineering." It means you don't need to be a data scientist to build an AI application. You just need to be a good "teacher." If you can provide five clear examples of what you want, a powerful LLM can usually take it from there.

It’s the shift from "Programming by Code" to "Programming by Example."