Towards a more consistent and comprehensive evaluation of anaphora resolution algorithms and systems

T he article argues that recall and precision are imperfect as measures for robust anaphor a resolution algorithms, and proposes instead a success rate for anaphora resolution algorithm s and for anaphora resolution systems separately.T he article also proposes a package of evaluation measures and tasks for anaphor a resolution: it is believed that these newly added tasks which have been carried out on Mitkov's (1998) knowledge-poor approach, provide a better, more comprehensive picture of the performance of anaphora resolution algorithm s or systems.Finally, the ongoing work on the development of a consistent evaluation environment for anaphora resolution is outlined.The last few years have seen the emergence of a number of new projects on anaphora resolution, due to its importance in key NLP applications such as natural language interfaces, machine translation, automatic abstracting, and information extraction.In particular, the recent search for practical robust, corpus-based approaches has produced promising solutions (Baldwin, 1997;