Answers are grounded in this paper and cite their sources.
Insider threats remain particularly challenging to detect because malicious activity can be carried out by authorised users with legitimate credentials and established access to organisational systems. Conventional population- and peer-based anomaly detection approaches may compound this challenge by treating legitimate differences in occupational behaviour as suspicious. This paper formalises and evaluates Counterfactual Routine Deviation (CRD), a personalised behavioural analytics framework that assesses observed user activity against expectations derived from the individual’s own historical behaviour. By modelling deviations from established personal routines, while retaining peer-relative deviation for comparative evaluation, CRD examines whether personalised behavioural reference points provide more informative signals than population-relative comparisons. CRD was evaluated on the CMU SEI CERT Insider Threat Dataset Release 6.2 using 559,516 user-day observations from 4000 users. Only 140 user-days were labelled within insider-risk windows (0.025% prevalence), creating a stringent base-rate setting. Behavioural features captured routine, content, destination and organisational-change signals, including authentication and device activity, file and web behaviour, removable-media use and short behavioural sequences. Performance was assessed using AUROC with bootstrap confidence intervals, average precision, precision at fixed alert budgets, scenario-stratified analysis, complete-user grouped validation, and a forward temporal holdout. Under the primary evaluation framework, the strongest discriminator was a logistic combination of CRD components, which achieved an AUROC of 0.755 (95% bootstrap CI 0.725–0.788). The best direct within-user score, all_features_within_user_crd, achieved an AUROC of 0.571, compared with 0.370 for its peer-relative counterpart. Routine within-user CRD similarly exceeded routine peer CRD (0.538 versus 0.312), while the content-based difference was smaller (0.543 versus 0.511). The unweighted combined CRD score was non-discriminative (AUROC 0.479), demonstrating that deviation magnitude alone was insufficient and that the weighting of behavioural signals materially affected performance. Operational performance remained constrained by extreme class imbalance. The logistic score achieved an average precision of 0.000801 and Precision@100 of 0.01 despite its moderate ROC discrimination. Under complete-user grouped evaluation, the logistic combination achieved an AUROC of 0.434, indicating poor and unstable discrimination for previously unseen users, while the forward temporal holdout achieved an AUROC of 0.678, indicating that some discriminatory ability was retained on chronologically later observations. Scenario-level performance was also heterogeneous: Scenario 4 accounted for 135 of 140 positive user-days (96.4%) and therefore dominated the aggregate results. These findings provide qualified support for the construction of personalised behavioural references. Within-user deviation generally contained more discriminatory information than corresponding peer-relative measures, but this advantage did not translate into high operational precision or reliable unseen-user generalisation. CRD should therefore be interpreted as a personalised behavioural ranking and research framework rather than a deployment-ready detector. Its principal contribution is the formalisation and empirical comparison of within-user and peer-relative counterfactual behavioural deviation, while further validation on independent datasets and real enterprise telemetry remains necessary.
No takes yet. Share an insight, caveat, or question.
Bowie et al. (2026) studied this question.
Synapse has enriched 10 related papers. Consider them for comparative context: