Predicting Clinical Management Decisions from Referral Information in Child and Adolescent Psychiatry Consultations: A Temporal Transportability Study

Background/Objectives: Child and adolescent psychiatry consultation-liaison services address heterogeneous management demands, while referral pathways and service organization may change over time. We examined whether information available before psychiatric assessment could predict six subsequent documented clinical management decisions and whether predictive performance remained transportable to a later service period. Methods: This retrospective prediction study included 656 unique patients aged 1–18 years. Six management domains coded from psychiatric consultation-response documentation were modeled using referral-note text, age, sex, and clinically harmonized referring department. L2-regularized logistic regression models incorporated term frequency–inverse document frequency text features. Internal performance was assessed using grouped repeated cross-validation. Temporal transportability was evaluated by applying models developed on 2013–2019 records (n = 447) to the 2022–2025 cohort (n = 209) without retraining or recalibration. The area under the precision–recall curve (AUPRC) was the primary metric. Results: Full-cohort cross-validated AUPRC ranged from 0.282 to 0.721. Temporal AUPRC ranged from 0.136 for psychosocial management to 0.663 for medication management decision; corresponding values were 0.282 for additional assessment, 0.586 for follow-up/reassessment, 0.415 for safety/disposition, and 0.575 for liaison/medical integration. Temporal discrimination was particularly weak for psychosocial management (AUROC = 0.532) and remained limited for additional assessment (AUROC = 0.587). Referring-department composition and several outcome prevalences changed substantially between periods. Although referral-note text contributed within the fitted models, its incremental temporal value over structured predictors was not established after false discovery rate correction; confidence intervals remained compatible with modest benefit or harm. Conclusions: Pre-assessment information contained management-domain-specific predictive signal for subsequent documented clinical management decisions, but transportability varied across clinical targets and service periods. These findings should be interpreted as evidence about temporal transportability rather than clinical implementation readiness. The results do not support automated clinical decision-making. Multicenter, preferably prospective, validation and evaluation of clinical utility and workflow impact are required before implementation.