Knowledge-guided Aspect-based Summarization

Contextualized pre-trained models, such as BERT 1 and BART 2, have shown great potential in various NLP tasks, pushing the state-of-the-art results to a new level. Although studies have shown that those pre-trained models have captured different kinds of knowledge due to the massive corpus they have been trained on 3, injecting task-specific external knowledge often shows further improvement 4. Here we choose aspect-based abstractive summarization as a case study and explore two different ways to inject external knowledge into BART. One is through a knowledge graph, the other is through human-defined sequence-level scores. Experiment results show that both methods can get an improvement over vanilla BART.