Empirically designing and evaluating a new revision-based model for summary generation

We present a system for summarizing quantitative data in natural language, focusing on the use of a corpus of basketball game summaries, drawn from on-line news services, to empirically shape the system design and to evaluate our approach. Our initial corpus analysis revealed characteristics of textual summaries that challenge the capabilities of current language generation systems. In order to meet these challenges, we developed a revision-based model for summary generation and implemented it in our prototype system streak. A second, detailed corpus analysis was used to identify and encode the revision rules of the system. Finally, we carried out a quantitative evaluation, using several test corpora, to measure the robustness of the new revision-based model. Our results show that our new model improves both coverage and extensibility of the traditional language generation model.