ParseFormer: A Multi-Task Framework for Chinese Scientific PDF Parsing

Accurate parsing of scientific PDFs is essential for effective knowledge dissemination and utilization, supporting applications such as citation analysis and retrieval-augmented systems. However, existing solutions focus predominantly on English documents and struggle with Chinese academic journal PDFs due to their unique features, such as bilingual titles/abstracts and page concatenation. There is also a lack of publicly available datasets for full-text parsing of Chinese academic journals. To bridge this gap, we first constructed a specialized Chinese scientific PDF dataset comprising 2613 annotated papers, across 14 disciplines, featuring 16 fine-grained text function labels tailored to Chinese scientific journal characteristics. Leveraging this dataset, we propose ParseFormer, a novel multi-task learning model based on the LayoutLMv3 architecture, which simultaneously performs four key tasks: reading order detection (ROD), text function identification (TFI), paragraph boundary identification (PBI), and citation extraction (CE). To predict the reading order more efficiently, we propose a regression head trained with the List-MLE function to replace the common pairwise ordering method. Experiments demonstrate the effectiveness of the proposed approach, achieving F1-scores above 0.92 across all 16 main TFI labels, 0.98 for PBI, 0.82 for CE, and an ARD of 0.039 for ROD. This framework achieves a good balance between efficiency and accuracy, facilitating conversion of PDFs into structured JSON or Markdown formats. We hope this work can serve as a useful resource for Chinese scientific PDF mining.