Enhancing LLM generalization to high-difficulty reading comprehension through fine-tuning with intermediate-complexity data

To rigorously evaluate the advanced reasoning capabilities of large language models (LLMs) in machine reading comprehension (MRC) within professional English linguistic domains, we introduce a novel high-difficulty benchmark, Advanced English Reading Comprehension (AERC). This dataset is domain-specific, constructed from expert-level English examinations (TEM8, CATTI). Comparative experiments show that 12 state-of-the-art LLMs perform, on average, 8.61% worse on AERC than on RACE, confirming the substantially higher complexity of the proposed benchmark. To further examine cross-task generalization in MRC, we fine-tune six LLMs on RACE using Low-Rank Adaptation (LoRA). The results indicate consistent performance improvements on AERC, although increasing the size of the training set does not necessarily yield the best transfer outcomes. Notably, Qwen2.5-14B-Instruct, when fine-tuned on a 60% subset of RACE, attains an accuracy of 86.53% on AERC, exceeding that of GPT-4o (83.87%) despite its considerably smaller parameter scale. Overall, the findings demonstrate that lower-difficulty training data can enhance generalization to more challenging MRC tasks in English. They also highlight the potential of well-fine-tuned open-source models to outperform leading commercial LLMs in cognitively demanding reading comprehension scenarios.