CD-Net: Histopathology Representation Learning Using Context-Detail Transformer Network

Extracting rich phenotype information, such as cell density and arrangement, from whole slide histology images (WSIs), requires analysis of large fields of view, i.e views providing more contextual information. This can be achieved through analyzing the digital slides at lower resolution. A potential drawback is missing out on details present at a higher resolution. To jointly leverage complementary information from multiple resolutions, we present a novel transformer based Context-Detail Network (CD-Net). CD-Net exploits the WSI pyramidal structure through co-training of proposed Context and Detail Modules, which operate on inputs from multiple resolutions. The residual connections between the modules enable the joint training paradigm while learning self-supervised representation for WSIs. The efficacy of CD-Net is demonstrated in classifying Lung Adenocarcinoma from Squamous cell carcinoma (N=1042 WSIs).