3021 Background: Tumor of unknown primary origin remains a critical unmet need in oncology, as histologic and anatomic diagnoses are essential for ensuring effective and timely therapies. We previously developed the Tempus Tumor Origin (TO) Laboratory Developed Test (LDT), an RNA sequencing-based machine learning algorithm that discriminates between clinically relevant cancer subtypes. However, RNA-sequencing data is not always available, with a failure rate that can be > 20% in the literature. In this study, we develop and assess the accuracy of a hematoxylin and eosin (H&E)-based machine learning algorithm to predict tumor origin. We also explore combining predictions from H&E and RNA models to improve accuracy further. Methods: We curated a cohort of 93,770 H&E whole slide images (WSIs) of tumor biopsies and surgical resections across 67 cancer subtypes from 73,634 patients. The cohort was split into 60/20/20 for training, validation, and testing, stratified by cancer subtypes and clinical covariates. We extracted tile embeddings from WSIs using a foundation model, H-optimus-0, and trained an attention-based multiple instance learning model to predict cancer subtype labels derived from clinically abstracted documents. Using a subset of 54,878 samples that also have predictions from the RNA-based Tempus TO algorithm, we trained and evaluated a multilayer perceptron multimodal model that takes the prediction scores from the H&E model and the RNA model as input. Results: On 7,092 test set samples that had both modalities available, the H&E, RNA, and multimodal models achieved top-1 accuracies of 0.85, 0.91, and 0.92 and top-3 accuracies of 0.94, 0.97, and 0.97, respectively. Among subtypes with at least 30 samples in the test set, the multimodal model statistically significantly outperformed the RNA model on five subtypes and underperformed on two. In addition, the H&E model had a top-1 accuracy of 0.77 and a top-3 accuracy of 0.90 on 3,028 samples in the test set that failed RNA sequencing due to quantity insufficient (QNS). Table 1 presents H&E model performance on the RNA QNS samples by diagnosis subtypes. Conclusions: We developed a high-performing H&E-based algorithm to predict tumor origin. With shorter turnaround time and wider availability, the H&E model can potentially deliver accurate tumor origin predictions faster and, critically, in cases where RNA-seq is unsuccessful or unavailable. Furthermore, leveraging both H&E and RNA in a multimodal model may improve the performance of future tumor origin algorithms. H&E model performance on RNA QNS test set samples in the five most prevalent diagnosis subtypes and other subtypes combined. Diagnosis subtype N Top-1 accuracy (%) Top-3 accuracy (%) Lung adenocarcinoma 497 84 94 Prostatic adenocarcinoma 387 92 95 Breast carcinoma 348 83 95 Colorectal adenocarcinoma 327 85 93 Pancreatic adenocarcinoma 272 78 94 Other 1,197 64 83
Hu et al. (Wed,) studied this question.