Context-Aware Transformer GAN for Direct Generation of Attenuation and Scatter Corrected PET Data

We present a context-aware generative deep learning framework to produce photon attenuation and scatter corrected (ASC) PET images directly from non-attenuation and non-scatter corrected (NASC) images. We trained conditional generative adversarial networks (cGAN) on either single-modality (NASC) or multi-modality (NASC+MRI) input data to map NASC images to pixel-wise continuously valued ASC PET images. We designed and evaluated four cGAN models including Pix2Pix, attention-guided cGAN (AG-Pix2Pix), vision transformer cGAN (ViT-GAN), and shifted window transformer cGAN (Swin-GAN). Retrospective 18F-fluorodeoxyglucose (18F-FDG) full-body PET images from 33 subjects were collected and analyzed. Notably, as a particular strength of this work, each patient in the study underwent both a PET/CT scan and a multisequence PET/MRI scan on the same day giving us a gold standard from the former as we investigate ASC for the latter. Quantitative analysis, evaluating image quality using peak signal-to-noise ratio (PSNR), multi-scale structural similarity index (MS-SSIM), normalized mean squared error (NRMSE), and mean absolute error (MAE) metrics, showed no significant impact of input type on PSNR (p=0.95), MS-SSIM (p=0.083), NRMSE (p=0.72), or MAE (p=0.70). For multi-modal input data, Swin-GAN outperformed Pix2Pix (p=0.023) and AG-Pix2Pix (p<0.001), but not ViT-GAN (p=0.154) in PSNR. Swin-GAN achieved significantly higher MS-SSIM than ViT-GAN (p=0.007) and AG-Pix2Pix (p=0.002). Multi-modal Swin-GAN demonstrated reduced NRMSE and MAE compared to ViT-GAN (p=0.023 and 0.031, respectively) and AG-Pix2Pix (both p<0.001), with marginal improvement over Pix2Pix (p<0.064). The cGAN models, in particular Swin-GAN, consistently generated reliable and accurate ASC PET images, whether using multi-modal or single-modal input data. The findings indicate this methodology can be used to generate ASC data from standalone PET scanners or integrated PET/MRI systems, without relying on transmission scan-based attenuation maps.