Reconstructing delivered dose in real time: a beam physics-embedded, language-model-driven approach

OBJECTIVE: To develop a fast and accurate physics-informed framework for real-time three-dimensional dose reconstruction directly from machine delivery beams for adaptive radiotherapy and intra-fraction dose monitoring. APPROACH: We propose a two-stage beam physics-driven dose reconstruction framework that integrates machine photon beam characteristics into fluence-to-dose estimation. In Stage 1, beam-related physical characteristics, including beam profiles and percent depth dose, are extracted from the institutional beam database using a large language model (LLM) assistant tuned on beam commissioning guidelines and encoded as beam-specific priors. In Stage 2, beam-wise fluence-to-dose reconstruction is performed in beam-eye-view space using a multimodal neural network that incorporates BEV-aligned CT images, optimal fluence, and the LLM-derived beam priors. Training uses a composite physics-aware loss, and performance is evaluated using two-dimensional and three-dimensional gamma analysis. MAIN RESULTS: The study included 180 lung radiotherapy patients for model development and internal evaluation, and an external prostate cohort of 5 patients for cross-site testing. The predicted dose distributions showed strong agreement with reference dose maps, with residual errors mainly confined to low-dose regions and within ±3 Gy relative to a maximum dose of approximately 75 Gy. The proposed framework achieved a mean 3D gamma passing rate (3%, 2 mm/10%) of 0.975 ± 0.056, outperforming 3D U-Net (0.931 ± 0.113), Dose-Net (0.939 ± 0.120), and CLIP-UNet (0.945 ± 0.087) (p < 0.05). The corresponding mean 2D gamma passing rate was 0.962 ± 0.073. External testing achieved a gamma passing rate of 0.953 ± 0.088. The model required approximately 6.3 seconds to reconstruct a 9-field treatment plan. SIGNIFICANCE: By incorporating LLM-encoded beam physics into a multimodal dose reconstruction framework, this method improves dosimetric accuracy over existing learning-based approaches while maintaining rapid inference, supporting its potential for adaptive radiotherapy and real-time three-dimensional in vivo dosimetry.