Masked Language Modeling for Predicting Missing Words in Damaged Ancient Greek Texts

The ancient Greek texts are valuable for understanding and learning about the history, culture, and nuances of ancient Greek life.The texts come in many forms, including papyri, fragments of pottery, etc. Due to the nature of these materials and degradation over time, some of these texts are missing words, and even entire phrases.This makes it difficult for historians to interpret the texts.The data for this project was collected from the Perseus Collection and the 1KGreek collection, which contains 250,000 unique sentences of ancient Greek literature.The dataset was preprocessed using the import classical language toolkit (CLTK) and sentences were normalized for better encodings.After the encoding was done all our data was split by sentences and then they were fed into a Distil Bert masked language model.The word piece tokenizer for this model was trained using a vocabulary list of 35,000 words.By using the DistilBert transformer model we were able to train a masked language model based on words to achieve a Hit@5 of 34 percent, Hit@10 of 35 percent, Hit@100 of 36 percent, and a perplexity of 1.04.This model can be a valuable aid for the historians' workflow in deciphering damaged ancient texts.