Multi-Dataset Lung and Colon Cancer Histopathology Classification Using Hybrid CNN-Vision Transformer

Lung and colon cancers remain among the major causes of cancer-related mortality worldwide, highlighting the need for diagnostic systems that are not only accurate and robust but also interpretable to support clinical decision-making. Histopathological image analysis forms the clinical gold standard for diagnosis; however, manual evaluation is a time-consuming process associated with inter-observer variability. While recent deep learning (DL) -based approaches have achieved high accuracy of classification, many existing models rely on single-architecture designs, generalize poorly across datasets, and are not transparent enough to provide insight into model decision-making. This paper presents a hybrid DL framework, where Convolutional Neural Networks (CNNs) and Vision Transformers (ViTs) are integrated to jointly capture fine-grained local morphological features with long-range global tissue context from histopathology images. Further, an adaptive feature fusion mechanism is adopted to fuse these complementary representations learned from the CNN and transformer branches. To enhance transparency, a post-hoc, model-agnostic Explainable Artificial Intelligence (XAI) module built upon Local Interpretable Model-agnostic Explanations (LIME) is integrated to generate interpretable visual explanations of each prediction. The model is trained and validated on the LC25000 lung and colon cancer histopathology dataset, while it reaches a classification accuracy of 100%. On LC25000, the proposed framework reaches 100% test-set accuracy with precision, recall, and F1 all at 100%. This compares favorably with the principal CNN baselines (VGG16 at 97.8%, ResNet-50 at 98.9%, DenseNet201 at 98.2%, and EfficientNetB7 at 98.7%), with a Vision Transformer baseline at 99.2%, and with the most accurate prior CNN-based pipeline reported in the recent literature for the same task (99.60% accuracy). The framework additionally yields explanation-stability scores of 0.90, 0.83, and 0.74 IoU under low, medium, and high input perturbation.