What question did this study set out to answer?

The aim is to address the challenges of justifying prompt selection in large language models.

April 17, 2026Open Access

Justify Your Prompts!

Key Points

The aim is to address the challenges of justifying prompt selection in large language models.
Discussed various strategies authors can use to motivate their prompt choices.
Analyzed the variability of outputs based on different prompts.
Highlighted the importance of establishing a justification framework for prompts.
Identified that LLM outputs can vary significantly between different prompts.
Emphasized the need for authors to consider the rationale behind their prompt selections.

Abstract

Abstract When you use a large language model (LLM) in your research, you often need to formulate a prompt to elicit some relevant output from the LLM. This step is challenging since (1) LLMs are known to be brittle and their results may vary drastically between different prompts; and (2) for any given task, there are infinitely many possible prompts. Thus we end up with the following problem: if you cannot try out infinitely many prompts, how do you justify your selected prompt? This paper discusses different ways in which authors may motivate their choices, and urges authors to consider how their prompting strategy might be justified.

Justify Your Prompts!

Key Points

Abstract

Cite This Study

Also Consider

Also Consider