Few-shot Learning

Few-shot learning is the practice of including a small number of solved examples (typically 1–10) in the prompt to a large language model so it can perform a new task by analogy. The model isn't retrained — it learns the pattern in context during a single inference call. Because few-shot is free, fast, and reversible, it's almost always the first thing to try when off-the-shelf prompts underperform.

The capability emerged unexpectedly with GPT-3 in 2020. OpenAI's paper "Language Models are Few-Shot Learners" demonstrated that a sufficiently large LLM could match or beat purpose-trained models on dozens of NLP benchmarks given just a handful of in-prompt examples — no fine-tuning required. The discovery reshaped the field: most application developers now reach for prompts before they reach for training.

Few-shot vs zero-shot vs one-shot

The terminology is precise:

• Zero-shot — Just instructions, no examples. ("Translate this to French: '...'.")
• One-shot — One example of the task. ("English: hello. French: bonjour. Translate: 'goodbye'.")
• Few-shot — Multiple examples (typically 2–10). ("English: hello. French: bonjour. English: cat. French: chat. English: dog. French: ___")
• Many-shot — 10+ examples, only practical with long-context models. Recent research shows continued gains up to hundreds of examples for some tasks.

The right number is task-dependent. A 2025 Anthropic study found accuracy gains plateau around 8–32 examples for most classification tasks but continue rising past 100 examples for complex reasoning.

How few-shot learning works

LLMs are pretrained to predict the next token given the prior context. When you put examples in the prompt, the model effectively learns the pattern from those examples and applies it to the next input — a phenomenon called in-context learning. The model's weights don't change; the "learning" is implicit in how it conditions its predictions on the example pattern.

Best practices:

• Use diverse examples — Cover different styles, edge cases, lengths.
• Match format exactly — The model copies the format of your examples; inconsistency hurts.
• Order matters — Place hardest examples last; the model weights recent context more.
• Combine with chain-of-thought — Few-shot examples that include reasoning steps train the model to reason on the new input too.

Examples of few-shot learning in action

1. GitHub Copilot — Uses surrounding code as implicit few-shot context for completions.
2. Customer support classifiers — 5 example tickets per category beat zero-shot classification by 20+ points without training.
3. Notion AI writing styles — Few-shot examples of the user's prior writing condition the model to match voice.
4. PostKit brand voice — Brand profile examples become runtime few-shot context.
5. LangChain SQL agents — Few-shot example queries dramatically improve text-to-SQL accuracy.

How PostKit uses few-shot learning

PostKit uses few-shot learning in two places.

One: brand voice transfer. When a user defines a brand profile, they can paste 2–5 examples of past content they're proud of. Those examples become few-shot context in every generation prompt — the model literally sees how this brand sounds before writing new posts. This is far more effective than abstract voice descriptors ("witty, warm, confident") which models interpret inconsistently.

Two: platform-specific hooks. Each generation prompt includes 8–12 hand-curated examples of high-performing hooks for the target platform (TikTok 3-second hooks, X opening tweets, LinkedIn first-line scrollstoppers). These examples are versioned and updated as platform algorithms shift — a much faster feedback loop than fine-tuning would allow.

The combination — generic platform examples (for format) + brand-specific examples (for voice) — produces output that feels both platform-native and brand-consistent. Without few-shot, you'd need extensive prompt engineering to describe what those examples make obvious.

Frequently asked questions

Why is it called "learning" if no training happens? "In-context learning" is a loose term. The model isn't updating weights — it's pattern-matching from prompt to output. Researchers debate whether this constitutes genuine learning, but the functional effect (better performance from examples) is real and reproducible.

What's the difference between few-shot learning and RAG? Few-shot examples are typically static (curated by the developer) and demonstrate format/style. RAG retrieves dynamic content per query (knowledge, facts). They're complementary: a RAG system can include few-shot examples as a base prompt.

How many examples should I use? Start with 3–5; A/B-test up to ~20. Diminishing returns are typical past 8 for simple tasks but real gains continue past 100 for complex reasoning.

Does few-shot work on smaller models? Yes, but less well. Capability scales with model size — large models benefit more from few-shot. Small models often need fine-tuning to hit similar quality.

What is "in-context learning"? The umbrella term for any task-relevant information (instructions, examples, retrieved documents) included in the prompt at inference time. Few-shot is one form of in-context learning.

Can few-shot examples hallucinate too? Yes. If your examples themselves contain errors, the model will copy them. Curate examples carefully — they're effectively your specification.

When should I switch from few-shot to fine-tuning? When prompts grow so long that latency or cost becomes painful, or when accuracy plateaus despite many examples. Fine-tuning bakes the examples into the model so they don't have to be sent every request.

Related terms

• Prompt engineering
• Fine-tuning
• LLM (Large Language Model)
• RAG (Retrieval-Augmented Generation)
• Generative AI
• Hallucination (AI)

Sources

• Brown et al. — Language Models are Few-Shot Learners (OpenAI, 2020)
• Anthropic — Many-Shot In-Context Learning (2024)
• Stanford CS324 — In-Context Learning lecture notes