Beyond Semantic Drift: Testing Role-Aware Semantic Metrics for Scientific Abstract Organization

The internal organization of scientfic abstracts remains difficult to quantify, and the contribution of rhetorical-role anchoring relative to simpler representations is unclear. This study evaluated role-aware semantic metrics alongside positional, lexical, and surface-level alternatives. The framework was pilot-tested on 100 PubMed periodontics abstracts: 50 systematic reviews/meta-analyses and 50 clinical studies. Sentence-level rhetorical roles were assigned using an LLM, and role-aware similarity metrics were derived from sentence-transformer embeddings. Two principal demonstration metrics produced interpretable patterns: within-Results similarity was higher in clinical-study abstracts, indicating greater Results compactness, whereas Conclusion–Results alignment was higher in review/meta-analysis abstracts, indicating stronger semantic linkage between synthesized findings and conclusions. These patterns were stable across two general-domain embedding models and were not explained by sentence count. However, positional and lexical baselines recovered part of the same review–clinical signal, and opening-frame return was not clearly separable from positional structure after a symmetric centroid-based comparison. The present analyses therefore do not establish general discriminative superiority or incremental predictive utility for rhetorical-role features. Their contribution is interpretive: rhetorical roles connect semantic-similarity patterns to explicit communicative functions. Role-aware semantic metrics should be used as descriptive tools for rhetorical–semantic analysis rather than as automated quality scores or stand-alone genre classifiers.