A Hybrid Multi-Task Deep Learning Framework For Brain Tumor Classification And Segmentation Using ResNet-U-Net Architecture

Accurate identification and definition of brain tu-mors is a key element in effective diagnosis and treatment planning which at the same time is very time consuming and subject to variable results with manual analysis of medical images. We present a hybrid modality multi task deep learning framework for at the same time classifying and segmenting brain tumors which uses MRI and CT scans. We have put together a ResNet-50 encoder with a U-Net based decoder which enables joint learning of spatial and semantic features in a single architecture. Also we have used a dual branch design which at the same time produces pixel level tumor segmentation and at the same time determines tumor types which include Glioma, Meningioma, Pituitary, and Normal cases. To solve for the issues of different imaging modalities we put forth a unified preprocessing pipeline which allows the model to learn modality invariant features. We report we see a classification accuracy of 97.40% and a Dice similarity coefficient of 0.843 with our put forth framework also at the same time reporting that we are able to perform efficient real time inference on CPU based systems. Also we see that multi task learning does in fact improve diagnostic performance which in turn shows off the put forth system's value as a practical tool for clinical decision support.