oolong: An R package for validating automated content analysis tools

While the validation of statistical properties of topics models is well established, the substantive meaning of categories uncovered is often less clear and their interpretation reliant on "intuition" or "eyeballing."As Chang et al. (2009, p. 1) put it: "qualitative evaluation of the latent space" or figuratively, reading tea leaves.The story for dictionary-based methods is not better.Researchers usually assume these dictionaries have built-in validity and use them directly in their research.However, multiple validation studies (Boukes et al., 2020;González-Bailón & Paltoglou, 2015;Ribeiro et al., 2016) demonstrate these dictionaries have very limited criterion validity.Oolong provides a set of tools to objectively judge substantive interpretability to applied users in disciplines such as political science and communication science.It allows standardized content based testing of topic models as well as dictionary-based methods with clear numeric indicators of semantic validity.Oolong makes it easy to generate standard validation tests suggested by Chang et al. (2009) and Song et al. (2020).