Prompting AI Guide

English

Română

English

Română

Appearance

Zero-Shot Prompting ​

Overview ​

Large language models (LLMs) today, such as GPT-3.5 Turbo, GPT-4, and Claude 3, are tuned to follow instructions and are trained on large amounts of data. Large-scale training makes these models capable of performing some tasks in a "zero-shot" manner. Zero-shot prompting means that the prompt used to interact with the model won't contain examples or demonstrations. The zero-shot prompt directly instructs the model to perform a task without any additional examples to steer it.

Basic Example ​

We tried a few zero-shot examples in the previous section. Here is one of the examples (ie., text classification) we used:

Prompt:

Classify the text into neutral, negative or positive. 
Text: I think the vacation is okay.
Sentiment:

Output:

Neutral

Note that in the prompt above we didn't provide the model with any examples of text alongside their classifications, the LLM already understands "sentiment" -- that's the zero-shot capabilities at work.

How Zero-Shot Learning Works ​

Instruction tuning has been shown to improve zero-shot learning Wei et al. (2022). Instruction tuning is essentially the concept of finetuning models on datasets described via instructions. Furthermore, RLHF (reinforcement learning from human feedback) has been adopted to scale instruction tuning wherein the model is aligned to better fit human preferences. This recent development powers models like ChatGPT. We will discuss all these approaches and methods in upcoming sections.

When to Use Zero-Shot ​

When zero-shot doesn't work, it's recommended to provide demonstrations or examples in the prompt which leads to few-shot prompting. In the next section, we demonstrate few-shot prompting.

Key Benefits ​

  • No Examples Required: Works without providing demonstrations
  • Immediate Use: Can be applied to new tasks instantly
  • Instruction Following: Models understand and follow direct instructions
  • Efficient: Saves time and tokens compared to few-shot approaches
  • General Purpose: Applicable across various domains and tasks

Applications ​

  • Text Classification: Sentiment analysis, topic categorization
  • Translation: Language-to-language conversion
  • Summarization: Creating concise versions of longer texts
  • Question Answering: Responding to queries without examples
  • Code Generation: Creating code from natural language descriptions

Limitations ​

  • Task Complexity: May struggle with complex reasoning tasks
  • Performance Variability: Results can be inconsistent across different tasks
  • Domain Knowledge: Limited to what was learned during training
  • Instruction Clarity: Requires very clear and specific instructions

Best Practices ​

  1. Be Specific: Use clear, unambiguous instructions
  2. Define Output Format: Specify the expected response structure
  3. Set Constraints: Define any limitations or requirements
  4. Test Iteratively: Refine prompts based on performance
  5. Consider Context: Provide relevant background when necessary

References ​

  • Wei et al. (2022) - Finetuned Language Models Are Zero-Shot Learners