FactPEGASUS: Factuality-Aware Pre-training and Fine-tuning for Abstractive Summarization

We present FACTPEGASUS, an abstractive summarization model that addresses the problem of factuality during pre-training and finetuning: (1) We augment the sentence selection strategy of PEGASUS's (Zhang et al., 2020) pre-training objective to create pseudosummaries that are both important and factual;