This is the authors' abstract. We don't add key points for this paper.
Recent advancements in scaling large language models (LLMs) have enhanced various natural language processing (NLP) tasks. However, open-source moderately sized models, such as BERT, are still needed because of the high computational cost and concerns regarding data privacy from the LLMs, especially in the biomedical area. Prompt-based fine-tuning of BERT has demonstrated good performance in a few-shot setting. However, the prompt selection can result in substantial differences in final accuracy. This study introduces a simple yet effective approach that leverages LLMs, such as GPT-4 Turbo, to automatically generate and optimize task-specific prompts for BERT. Our approach includes two steps: automatic prompt generation and optimization. Initially, we design a framework to generate prompts for LLMs to infer a task-specific candidate prompt set. Subsequently, we employ a dialog with a chatbot to optimize the prompt iteratively. We conduct extensive evaluations and analyses on three different types of biomedical benchmarks. Our method demonstrates superior 5-shot learning performance, outperforming manual prompts by a substantial margin in low-resource settings, achieving up to a 7% absolute accuracy improvement. These results highlight that our method is a task-agnostic approach to utilizing LLMs and automatically enhancing performance on relatively small open-source models with limited resources and human effort.
Answers are grounded in this paper and cite their sources.
No takes yet. Share an insight, caveat, or question.
Shi et al. (2024) studied this question.
Synapse has enriched 8 closely related papers on similar clinical questions. Consider them for comparative context: