Convolutional neural network for classification of two-dimensional array images generated from clinical information may support diagnosis of rheumatoid arthritis

This research aimed to study the application of deep learning to the diagnosis of rheumatoid arthritis (RA). Definite criteria or direct markers for diagnosing RA are lacking. Rheumatologists diagnose RA according to an integrated assessment based on scientific evidence and clinical experience. Our novel idea was to convert various clinical information from patients into simple two-dimensional images and then use them to fine-tune a convolutional neural network (CNN) to classify RA or nonRA. We semi-quantitatively converted each type of clinical information to four coloured square images and arranged them as one image for each patient. One rheumatologist modified each patient's clinical information to increase learning data. In total, 1037 images (252 RA, 785 nonRA) were used to fine-tune a pretrained CNN with transfer learning. For clinical data (10 RA, 40 nonRA), which were independent of the learning data and were used as testing data, we compared the classification ability of the fine-tuned CNN with that of three expert rheumatologists. Our simple system could potentially support RA diagnosis and therefore might be useful for screening RA in both specialised hospitals and general clinics. This study paves the way to enabling deep learning in the diagnosis of RA.