Deep Radiomics in 18F-FDG-PET/CT for Survival Prediction in Multiple Myeloma: Novel Embeddings Using A Foundational Models Memory

Purpose: To test whether internal memory states from a medical foundational segmentation model can serve as compact, mask-aware embeddings for predicting progression-free survival (PFS) in multiple myeloma (MM) from whole body [18F]FDG PET/CT. Methods: We analyzed 227 newly diagnosed MM patients with PET/CT and clinical data. For two regions of interest (spine dilated and full skeleton), we prompted MedSAM2 slice-wise using mask derived bounding boxes and cached the final spatio temporal memory tensor per modality. We compared two downsampling strategies to obtain per study embeddings: channel/memory averaging with a small CNN head, and a squeeze and excitation style depth-attention pooling. PET and CT embeddings were combined by late fusion (concatenation or scalar-gated mixing) and passed to a DeepSurv head. We evaluated image-only and multimodal (image+clinical) models with stratified 5-fold cross-validation. The primary endpoint was Harrell s c-index (mean +- SE across folds). Results: Image-only models using the averaging downsampler achieved up to 0.659 +- 0.015 c-index (PET, spine-dilated), comparable to baseline radiomics results. Multimodal models improved discrimination to 0.710 +- 0.032 (CT, spine dilated), with similar performance for other PET/CT+clinical variants (0.703 to 0.710), improving clinical-only baselines by 6.5%. Averaging consistently outperformed depth-attention; concatenation and gated fusion performed comparably. PET outperformed CT within the same mask in image-only settings. Conclusion: Mask aware embeddings extracted from a foundational segmentation model provide effective, data-efficient imaging biomarkers for MM PFS and, when fused with routine clinical covariates, significantly improve risk stratification over clinical only or radiomics baselines. This offers a practical path to prognostic modeling on small medical cohorts without feature design.