Overview of the fourth message understanding evaluation and conference

The Fourth Message Understanding Conference (MUC-4) is the latest in a series of conferences that concern the evaluation of natural language processing (NLP) systems. These conferences have reported on progress being made both in the development of systems capable of analyzing relatively short English texts and in the definition of a rigorous performance evaluation methodology. MUC-4 was preceded by a period of intensive system development by each of the participating organizations and blind testing using materials prepared by NRaD and SAIC that are described in this paper, other papers in this volume, and the MUC-3 proceedings 1.