Standard dot-product attention has two structural problems on directed graphs. First, symmetric scoringassigns equal scores to A→B and B→A, so it cannot represent causal direction. Second, softmax normalization forcesconfidence to sum to 1.0 regardless of whether the input makes sense, preventing the model from expressinguncertainty. We introduce Asymmetric Magnitude Attention (AMA), which decouples these two problems intoindependent mechanisms: (a) an antisymmetric directional score S asym = (QKT − KQT )/√dk that encodes causaldirection in the sign of the score, and (b) a sigmoid magnitude gate M ij = σ(Wmag(Qi; Kj)) that can collapse towardzero on structurally ambiguous inputs. Contrary to our initial hypothesis, ablations reveal these components areorthogonal: the gate achieves stronger paradox separation without the asymmetric score when a temporal signal isavailable (MagSep 0.386 vs 0.279), while the asymmetric score independently improves directional accuracy by 3.3to 3.9pp regardless of gate configuration. On synthetic directed acyclic graphs, the gate achieves paradox separation(MagSep) of 0.890 with 94.1% retention on unseen graph distributions. On the ogbn-arxiv directed citation network,AMA reaches 88.6% balanced causal direction accuracy versus 85.3% for standard attention (+3.3pp). AMA is thefirst framework to make this decomposition explicit and empirically measurable.
Amarnath R (Sun,) studied this question.