Answers are grounded in this paper and cite their sources.
A synthetic fraud benchmark is easy to build and hard to interpret: a high score may measure the generator rather than the model. We study this on FraudShield-EAC, a seeded, agent-based benchmark of East African digital payments (six channels, five countries, eight fraud scenarios) built to develop a fraud detection system. The benchmark is easy in a specific way: on the test period one engineered feature, a short-to-long-window velocity ratio, reaches an AUC of 0.879 on its own, so we report ranking metrics as margins over it. A calibrated XGBoost and LightGBM ensemble trained on all 792,162 training rows reaches an AUC of 0.975 (margin +0.096), but its recall at the specified flag threshold, 0.764, misses the target of 0.880, and more training rows did not close the gap in any run. At a fixed 30,000 rows, which rows were drawn moved recall at that threshold over a range of 0.107, against 0.032 for training on all rows; on AUC, volume mattered more. On a diagnostic model, ablations measure redundancy rather than importance (four disjoint feature groups each exceed AUC 0.80 alone), and leave-one-country-out and a novel variant held out in time are both null. The one variant specified in advance without burst structure, a reversal scam, is caught at 9.1% [4.2%, 18.4%] at a 1% false-positive budget, against 94.4% for the base scenarios. We distil the audit into a checklist for synthetic benchmarks. No fraud parameter is sourced, so every result describes the generator, not East African payments.
No takes yet. Share an insight, caveat, or question.
Marius Bayizere (2026) studied this question.
Synapse has enriched 8 closely related papers on similar clinical questions. Consider them for comparative context: