CNN-Enabled Generation of Comprehensive Reports from Chest and Orthopaedic Radiographs

In the realm of medical imaging, report generation poses challenges for both inexperienced and experienced physicians, prompting the exploration of automated solutions. Our research addresses this concern by proposing a novel approach to the generation of radiology reports for chest and orthopaedic radiographs, integrating advanced neural network models. Building upon the insights from existing literature, we leverage Convolutional Neural Networks (CNN) model VGG19 for the chest radiograph and ResNet50 for the orthopaedic radiograph. To synthesize coherent and detailed reports, a Bi-directional Long Short-Term Memory (Bi-LSTM) based text generator is employed, accounting for the multifaceted nature of medical imaging information. Our methodology encompasses a multitask learning framework, facilitating joint prediction of tags and paragraph generation. Experiment with publicly accessible datasets to illustrate the efficacy of our methodology and the system's ability to generate detailed reports. The chest Radiograph model, trained with VGG19, excels in discerning intricate patterns, while the orthopaedic radiograph model, utilizing the same ResNet50, demonstrates expertise in recognizing skeletal fractures.