Towards a new research agenda for multimodal enterprise document understanding: What are we missing?

The field of multimodal document understanding has produced a suite of models that have achieved stellar performance across several tasks, even coming close to human performance on certain benchmarks.Nevertheless, the application of these models to real-world enterprise datasets remains constrained by a number of limitations.In this position paper, we discuss these limitations in the context of three key aspects of research: dataset curation, model development, and evaluation on downstream tasks.By analyzing 14 datasets and 7 SotA models, we identify major gaps in their utility in the context of a real-world scenario.We demonstrate how each limitation impedes the widespread use of SotA models in enterprise settings, and present a set of research challenges that are motivated by these limitations.Lastly, we propose a research agenda that is aimed at driving the field towards higher impact in enterprise applications.