A new exploration: characterization of the differentiation trajectory of prostate cancer cells

Abstract Background: Prostate cancer is one of the most common malignancies in men, and in-depth study of its gene expression patterns is essential to understand the formation and progression of prostate cancer. Although the heterogeneity of prostate cancer cells has been explored by single-cell transcriptomics, the different differentiation states from normal epithelial cells might lead to confusion about heterogeneous tumor characteristics. In this study, we analyzed the heterogeneity of prostate cancer tumor subsets in detail using single-cell data from the GEO database by means of cell annotation and enrichment analysis, with a special focus on matching the differentiation status of normal epithelial cells. We found that there are unique or shared tumor signatures among these subpopulations, providing important clues for insight into the development of prostate cancer. Patients and methods: We searched the GEO public database (GSE193337) for prostate cancer single-cell data and conducted rigorous data quality control. The cells were annotated using Single R and Azimuth tools, and malignant epithelial cells were screened for subsequent heterogeneous clustering. Using an anchor-site integration approach, we identified normal epithelial cells that matched each tumor subset at the same TSNE neighbor plot location as a control group for subsequent studies. Pseudo-temporal trajectory analysis, functional enrichment analysis, cell communication analysis, and transcription factor regulatory network analysis were performed on the obtained tumor heterogeneous subsets. We further conducted immune infiltration analysis and pan-cancer analysis of transcription factors with aberrant transcriptional activity. Results: We found that prostate cancer cells exhibited enrichment of the androgen receptor pathway in the early stages of differentiation (malignant2, 3 subsets). All three subpopulations showed down-regulation of the enrichment of P53 and the apoptotic pathway, which might be associated with apoptotic evasion. Cell communication analysis showed that malignant2 and 3 subsets showed more active extracellular matrix signaling communication and higher levels of MHC-related molecular expression compared to normal epithelial cells matched to their respective subpopulations. All three tumor subsets expressed higher levels of EGFR, ERBB2, interferon receptor, MIF, and cell adhesion-related signals. Through transcription factor regulatory network analysis, we observed that the transcriptional activity of YY1, NKX3-1 and EHF in these subpopulations was higher than that of normal epithelial cells at the same differentiation stage, especially YY1 might act as an upstream regulator of MIF signaling pathway. ATF3 is a key upstream transcriptional regulator of differentially expressed genes in the P53 and apoptotic pathways. Immune infiltration analysis showed that the expression of YY1, EHF, NKX3-1 and ATF3 was significantly associated with the infiltration of immune cells in prostate cancer. Pan-cancer analysis showed that YY1 and NKX3-1 were significantly overexpressed in prostate cancer, while ATF3 was significantly underexpressed. Among them, the hazard ratio of YY1 in overall survival of prostate cancer was 11.9 (P<0.05), and the risk of disease-free survival and progression-free survival of ATF3 in prostate cancer was 0.791 and 0.88 (P<0.05), respectively. Conclusion: Through a detailed analysis of prostate cancer tumor subsets, particularly those matching the differentiation status of normal epithelial cells, we have identified unique or shared tumor characteristics among them. Enrichment analysis has unveiled key pathways associated with the three tumor subsets, offering valuable insights into the development of prostate cancer. The results of immune infiltration and pan-cancer analysis underscore the significance of YY1 and ATF3 in prostate cancer, correlating their abnormal expression with patient survival. This opens up new avenues for future research, holding the promise of providing more precise strategies for the personalized treatment of prostate cancer.