Anomalies in multivariate time series often arise from temporal context and cross-channel coordination rather than isolated outliers. We present Pi-Transformer (Prior-Informed Transformer), a transformer with two attention pathways: data-driven series attention and a smoothly evolving prior attention that encodes temporal invariants such as scale-related self-similarity and phase synchrony. The prior provides an amplitude-insensitive temporal reference that calibrates reconstruction error. During training, we pair a reconstruction objective with a divergence term that encourages agreement between the two attentions while keeping them meaningfully distinct. The prior is regularised to evolve smoothly and is lightly distilled towards dataset-level statistics. At inference, the model combines an alignment-weighted reconstruction signal (Energy) with a mismatch signal that highlights timing and phase disruptions, and fuses them into a single score for detection. Across five benchmarks (SMD, MSL, SMAP, SWaT, and PSM), Pi-Transformer achieves state-of-the-art or highly competitive F1, with particular strength on timing and phase-breaking anomalies. Case analyses show complementary behaviour of the two streams and interpretable detections around regime changes. Embedding prior attention into transformer scoring yields a calibrated and robust approach to anomaly detection in complex multivariate systems. • Prior-informed dual attention : Pi-Transformer couples data-driven series attention with a prior attention built from phase synchrony and scale invariance. • Two-stream, interpretable scoring : Inference combines a prior-aligned reconstruction energy stream with a mismatch stream that emphasises timing and phase disruptions, fused point-wise. • Complementary detection behaviour : The energy and mismatch streams provide sensitivity to both amplitude or shape deviations and to timing or phase-breaking anomalies. • Broad benchmark evidence with ablations : Results on five multivariate benchmarks (SMD, MSL, SMAP, SWaT, PSM) are consistently competitive in F1, and ablations quantify the contribution of the prior components and design choices.
Maleki et al. (Sun,) studied this question.