Reinforced Visual Interaction Fusion Radiology Report Generation

Abstract The explosion in the number of more complex types of chest X-rays and CT scans in recent years has placed a significant workload on physicians, particularly in radiology departments, to interpret and produce radiology reports. There is therefore a need for more efficient generation of medical reports. In this paper, we propose the Reinforced Visual Interaction Fusion (RVIF) radiology report generation model, which adopts a novel and effective visual interaction fusion module, which is more conducive to extracting fused visual features of radiology images with clinical diagnostic significance and performing subsequent correlation. Sexual analysis and processing. In addition, a reinforcement learning step from image captioning to this task is introduced to further enhance the aligned diagnosis effect brought by the visual interactive fusion module to generate accurate and highly credible radiology reports. Quantitative experiments and visualization results prove that our model performs well on two public medical report generation datasets, IU X-Ray, and MIMIC-CXR, surpassing some SOTA methods. Compared with the SOTA model COMG+RL in 2024, the BLEU@1, 2, and 3 of the NLG metrics increased by 3.9%, 2.8%, and 0.5% respectively, METEOR increased by 2.2%, the precision P of the CE index increased by 0.4%, and the recall rate R increased by 1.5%, F1-score increased by 1.8%. Source code in https://github.com/200084/RVIF-Radiology-Report-Generation.