Transfer Learning for Image Classification: Exploring Neural and Attention Networks using Pre-Trained Models

An accurate diagnosis is a critical challenge in oncology. Errors remain an issue, as inaccuracies contribute to avoidable harm, increased costs and delays in diagnoses. Over the past decade, several deep learning solutions have offered promising solutions in improving cancer diagnosis with some having transitioned from research to clinical deployment. This project explored the effectiveness of transfer learning using two modern architectures: ResNet50, a Convolutional Neural Network, and DeiT, a transformer-based attention model. The project was conducted in two stages: First a binary classification was performed using four types of cancer to establish a baseline, Second, a multi-class classification was performed using one of the previous cancer types to test greater recognition and generalisation. Both models are initially trained on ImageNet, fine-tuned on the dataset and evaluated using accuracy and loss, f1-score and confusion matrices. Results show that both models performed comparably across all tasks, only experiencing minor fluctuations typical of training variance. Despite architectural differences, both CNN and transformer-based models are capable of producing reliable results in cancer image classification.