Dynamic Few-shot Learning for Computational Social Science

Large Language Models (LLMs) have revolutionized Natural Language Processing (NLP) and have recently demonstrated remarkable potential in social science research through their capacity to efficiently perform a wide range of text-as-data tasks. However, current applied social science research which relies on these models tend to use closed-source models and zero-shot learning. This study contrasts the performance of state-of-the-art closed-source LLMs with their open-source counterparts across 10 multi-class classification tasks critical to social science research, such as emotion detection and political speech classification. In addition, we introduce dynamic few-shot learning, an enhancement to static few-shot learning, that leverages semantic similarity to dynamically choose contextually relevant examples from a pre-defined corpus for improved prompt engineering. Our research indicates that dynamic few-shot learning significantly boosts classification performance, especially for much smaller models. Our findings help expand the methodological toolbox for social scientists and suggest more transparent, ethical, and efficient ways of leveraging the power of LLMs in social science research.