What Makes Good In-Context Examples for GPT-3?

Jiachang Liu; Dinghan Shen; Yizhe Zhang; Bill Dolan; Lawrence Carin; Weizhu Chen

doi:10.18653/v1/2022.deelio-1.10

January 1, 2022Open Access

What Makes Good In-Context Examples for GPT-3?

View Full Paper

Ask AI

Bookmark

Share

Key Points

Investigate what characteristics make demonstration examples effective for in-context few-shot learning in GPT-3.
Evaluated GPT-3 performance across multiple natural language processing benchmarks using varied demonstration selection strategies.
Employed dense sentence embedding retrievers (such as RoBERTa) to identify k-nearest neighbor examples for each test query.
Compared semantic-similarity-based retrieval against random demonstration sampling across classification and generation tasks.
Demonstrations semantically closer to the input query in embedding space consistently yielded higher downstream task accuracy than randomly chosen examples.
Retrieval-based in-context selection mitigated the high variance and sensitivity typically observed in standard few-shot prompting.
Performance gains persisted across multiple task formats, highlighting the importance of input-specific context conditioning.

Ask about this paper

Answers are grounded in this paper and cite their sources.

Implication

Empirical evaluation demonstrates that semantically similar prompt examples improve GPT-3 performance across diverse NLP tasks, indicating the value of retrieval-based selection.

Abstract

No abstract yet. The journal hasn't published one we can show.

Discussion

No takes yet. Share an insight, caveat, or question.

Cite This Study

Liu et al. (2022) studied this question.

synapsesocial.com/papers/69d73cefaa68b335b4f30987 https://doi.org/10.18653/v1/2022.deelio-1.10

View Full Paper

Ask AI

Bookmark

Share

What Makes Good In-Context Examples for GPT-3?

Key Points

Ask about this paper

Implication

Abstract

Discussion

Cite This Study

What Makes Good In-Context Examples for GPT-3?

Key Points

Ask about this paper

Implication

Abstract

Discussion

Cite This Study

Keep reading

Keep reading