Computational Narratology with Transformer Word Embeddings

This paper presents a new method that uses contextual embeddings derived from sentence transformers to analyze narrative structure.We embed chunked text sequences to create narrative plot trajectory curves.This enables us to then apply commonly used unsupervised word embedding techniques, such as semantic axes projection, to narrative sequence analysis at the scale of entire books.Our aim is to prototype several simple examples of the kinds of narratology techniques this method might enable.We fit a pretrained sentence transformer model to books, represented as embeddings of chunked sequences, from nearly 5% of the Project Gutenberg English Books Corpus filtered to works in the fiction category (totaling 2965 texts).Texts date from the 1500s through the early 1900s, with the majority of texts concentrated in the nineteenth century.This period and dataset offer an opportunity to investigate the rise and development of the early novel form.Our analysis measures narrative trajectories along seven binary oppositions -class, gender, morality, collectivity/individual, nature/artifice, order/disorder, emotion/reason and segments texts by decade, genre, and dialogue gender.We find trends broadly relevant to hypotheses from postclassical and feminist narratological theory and also present novel aggregate findings on narrative presentation and story shape. Plain Language SummaryThis paper presents a method for recovering and comparing story trajectory curves.It draws on word embeddings, a technique for representing words as points in a geometric space, where similarities in word meaning are denoted by similarities in geometric position.Contemporary embedding models, called contextual embeddings, can represent full sequences according to proximity in meaning and differentiate between distinct meanings and uses of the same words (polysemy).We use contextual sentence models to represent 2963 novels from Project Gutenberg fiction, concentrated primarily on the nineteenth century but ranging from the 1500s to the early 1900s.To do this, we divide or 'chunk' each book into 128-word units, represent these chunks using contextual sentence embeddings, and structure them into sequences.We then apply a technique regularly used in the literature to construct 'axes' representing binary oppositions as continua (in our case gender, class, morality, collectivity/individual, nature/artifice, order/disorder, and emotion/reason) and measure the relative position of each text chunk along these 'axes'.This allows us to trace representational shifts along binary oppositions over the course of a book's narrative arc.We compute average sequences across all books and segment them by decade, genre, and dialogue speaker gender.Our analysis finds clear trends for each covariate, many of which confirm ideas from narratological theory.While word embeddings have been used in historical language analysis, e.g. to trace how words change over time periods, their use in representing books as chunked sequences is less common and builds upon work in book plot arc modeling with techniques like sentiment analysis.Our method is statistically robust and provides a proof-of-concept for how to analyze texts individually and in aggregate as sequences of embeddings.