Deep learning-driven literary evaluation: fusing visualised representations and text embeddings

Visualisations and their captions are central to data analysis, yet most models still process images and text separately and lose important cross-modal cues.To address this limitation, this paper proposes a fusion framework that jointly represents visualised elements and descriptive language within a shared transformer architecture.First, charts are decomposed into typed visual elements and encoded structurally; then, captions and analytic comments are contextualised and softly grounded to these elements; finally, a gated fusion module and multimodal transformer layers integrate both streams for retrieval, question answering and summary generation.Experiments on three figure-caption benchmarks show that the proposed model outperforms strong visual-only, text-only and dual-encoder baselines, raising top-10 accuracy by up to 2.6 percentage points and BLEU-4 scores by up to 2.8 points while reducing variance across runs.