Something to Do with Paying Attention: A Review of Transformer-based Deep Neural Networks for Text Classification in Digital Humanities and New Testament Studies

Researchers in the field of New Testament and Religious Studies were engaged in using digital techniques from the origins of what is now known as the field of Digital Humanities (DH). Even so, New Testament researchers have not kept abreast of the tools and techniques coming out of DH research. In particular, the latest abilities of transformer-based deep neural networks (TB-DNNs) such as BERT have yet to be comprehensively applied for text classification purposes of New Testament texts. To remedy this lacuna, we offer an exploration of recent TB-DNN usage in DH for text classification to highlight its potential for NTS. On the way, we review some of the previous text classification work done in DH and NTS. Finally, we discuss some of the barriers to implementing TB-DNN models. It is hoped that this article will stimulate NTS researchers to consider TB-DNN models in their text classification work.