An Information-Theoretic Framework for Evaluating Class Imbalance and Rare-Attack Observability in IoT Security Benchmarks

IoT intrusion-detection studies often treat a benchmark as adequate once it is large, recent, and widely used.That assumption can fail.This study audits five public IoT and IIoT security benchmarks-CICIoT2023, Edge-IIoTset, ToN-IoT, BoT-IoT, and IoT-23-from their published class populations and introduces no synthetic classifier results; the audit combines Shannon entropy, Hill diversity, Simpson concentration, Gini inequality, rank-frequency tail fitting, rare-class observation probability, Wilson uncertainty, and a constrained power-tempered sampling model.Across the five corpora, record counts range from 461,043 to 73,370,443, normalized entropy from 0.421 to 0.600, and effective Shannon diversity from 2.20 to 3.98 classes.Size is a poor proxy for label adequacy.IoT-23 is the clearest case: its 6.05 million records include a singleton class, so a random 1% sample has only about a 1% probability of observing that label, while the worst-case Wilson half-width for a class-conditioned proportion is 0.445.We then define q i (τ) = p τ i / ∑ k p τ k and prove that its entropy is non-increasing in τ; at a 1% training budget with a target of 25 expected observations in the rarest class, the minimum-flattening exponent is 1.000 for CICIoT2023, 0.855 for Edge-IIoTset, 0.807 for ToN-IoT, 0.954 for BoT-IoT, and 0.450 for IoT-23.The last case needs an approximately 2,500-fold uplift of the singleton label.Such an uplift changes training exposure to one observation; it cannot create independent evidence.The resulting audit protocol therefore separates training rebalancing from evaluation validity and gives quantitative checks that can be applied before IoT benchmark results are interpreted as class-wide security evidence.