Representation geometry and task performance in vision-language modeling for computed tomography enterography

PurposeComputed tomography (CT) enterography is central to inflammatory bowel disease evaluation, but optimal representation choices for vision–language learning on this volumetric modality are unclear. We establish baseline performance and quantify how aggregation, slice encoding, and retrieval context affect classification, retrieval, and report generation.ApproachWe fine-tuned vision–language models on abdominal CT enterography with parameter-efficient adapters and compared (i) mean versus attention pooling of slice embeddings, (ii) multiwindow red–green–blue Hounsfield-unit encoding versus multiplanar sampling for added spatial coverage, and (iii) report generation with versus without retrieval-augmented generation (RAG). A three-teacher pseudolabel framework enabled evaluation without expert annotations. Classification and retrieval comparisons used three training seeds.ResultsMean and attention pooling achieved the same mean three-class disease classification accuracy (55.7%; standard deviations of 2.0 and 2.4 percentage points) at the largest low-rank adaptation capacity. Neither consistently outperformed the other at matched capacity with central sampling. Sixteen slices at 20% to 80% had higher classification accuracy than 8 slices, 32 slices, and 10% to 90% sampling in all three seeds for both pooling methods. The tested multiplanar configurations had lower classification accuracy. For report generation, fine-tuning without retrieval performed at prevalence-matched chance for within-1 report-label agreement (70.4% versus 71.0% random). RAG improved within-1 report-label agreement by 7 to 14 points over chance and reduced ordinal mean absolute error from 0.98 to 0.80–0.89.ConclusionsSlice sampling and tissue-contrast–focused encoding affect performance in CT enterography vision–language systems, and RAG improved report-label agreement over the fine-tuned baseline. These results provide initial baselines and practical guidance for multimodal learning in volumetric CT.