NARIA: Information-Theoretic Narrative Alignment via Scene Graph Decomposition, Optimal Transport, and Psychometric Calibration for Long-Text-to-Video Evaluation

NARIA technical report / preprint. NarrativeBench is introduced to address a critical gap in video generation evaluation: existing benchmarks such as VBench, EvalCrafter, and LoCoT2V-Bench target short prompts and brief clips, leaving long-form narrative alignment unassessed. The benchmark comprises 1024 narrative scripts across 8 domainsmanga/anime adaptation, film screenplay, documentary narration, educational tutorial, advertisement storyboard, music video sequence, news report, and social media storyeach containing 128 scripts organized as ordered scene sequences with structured annotations for characters, actions, emotions, camera directives, and transitions. Inputs are 5002000 word scripts specifying multi-scene stories; outputs are 30120 second videos preserving the intended story arc. A multi-dimensional framework decomposes narrative fidelity into 10 dimensions including scene decomposition accuracy, character identity consistency, temporal ordering correctness, and cross-scene continuity. The framework introduces Narrative Information Content via entropy over scene-action-character tuples, normalized text-video alignment through mutual information, and a transport-based Narrative Fidelity Index (NFI) using the 1-Wasserstein distance between script-induced and video-induced scene distributions. Item response theory (IRT) is employed for script difficulty calibration. Controlled simulations demonstrate that the proposed metrics exhibit stronger monotonicity with narrative corruption and higher correlation with ground-truth narrative fidelity than CLIP-style similarity, Frechet Video Distance (FVD), and aggregate short-clip scores. Benchmark reliability stabilizes at 1024 scripts. The central claim is methodological: long-text-guided video generation should be evaluated as structured narrative alignment rather than short-clip quality scoring applied at larger scale. Existing OSF archival DOI: 10.17605/OSF.IO/KYJUE; Existing OSF archival page: https://osf.io/kyjue/. Files include the technical report PDF and the LaTeX source tarball when available.