Peer review depends on independent expert judgment, but high reviewer disagreement can delay editorial decisions, expose inconsistent criteria, and increase the burden on area chairs and editors. This paper presents Reviewer Phrase Agreement Profiling (RPAP), a chronology-scoped framework for estimating when two reviewers are likely to agree before reviews for a target submission are written. RPAP represents each reviewer using prior publications, prior public reviews when available, citation-grounded phrase histories, scientific-document embeddings, and lightweight review-role metadata. The framework explicitly separates reviewer--reviewer affinity from reviewer--submission affinity, blocks information dated after the assignment cutoff, and stores inspectable evidence packs rather than opaque similarity scores alone. A controlled evaluation over open peer-review records compares lexical baselines, citation-informed embeddings, phrase-profile models, and a calibrated RPAP ensemble. The results show that phrase profiles improve agreement-class recall and calibration when paired with reviewer--submission affinity, while open-review data sparsity remains the central limitation. The paper argues that reviewer agreement forecasting should be used as a triage and assignment-audit signal, not as an automated substitute for editorial judgment.
Mazumder et al. (Fri,) studied this question.