A Region-Based Pilot Evaluation of Rupture and Repair from a Minimal Coherence Signal (TIE–Dialog)

This preprint presents a transparent, region-based pilot evaluation of rupture and repair detection from a minimal conversational coherence signal. We evaluate TIE–Dialog, a computational framework that represents conversational coherence as a time-indexed signal C(t), and test whether rupture- and repair-like temporal regions extracted directly from C(t) overlap with human judgments beyond chance under uncertainty-aware validation. Five annotators labeled rupture (valley) and repair-like (peak) points across six short English dialogues. To account for temporal uncertainty in turn-level labeling, point annotations were converted into fixed windows [t−2, t+2] and aggregated into consensus regions (K = 3/5 annotators). Model regions were extracted using pre-specified, dialogue-relative criteria: repair-like regions were defined as turns where C(t) ≥ Q₀.₈₅(C), while rupture-like regions were defined as turns where the first difference dC(t) = C(t) − C(t−1) fell below Q₀.₁₅(dC). Model–human overlap was quantified using mask-based Intersection-over-Union (IoU) and overlap-F1, with permutation baselines based on random regions matched in segment count and length. Human–human agreement was moderate under the region protocol, confirming that rupture and repair are reliably perceived but temporally diffuse. At zero lag, model–consensus overlap was conservative (high precision, low recall) and did not robustly exceed chance in this small dataset. Lag scans frequently favored negative best lags; however, when evaluated using a lag-scan permutation baseline (max-over-lags), the apparent anticipatory alignment did not robustly exceed chance, indicating that precursor interpretations remain exploratory in the present pilot. The study contributes (i) a region-based validation protocol for temporally uncertain dialogue events, (ii) a fully reproducible replication package with transparency exports, and (iii) a cautionary demonstration that lag scanning requires lag-controlled baselines to avoid selection artifacts. Together, these results motivate larger-scale evaluation with richer event definitions and stronger controls, while supporting coherence dynamics as a promising quantitative substrate for modeling conversational stability, breakdown, and recovery as measurable temporal structure.