Shannon Was Right, But We Can Be Smarter: Adaptive Context-Aware Compression for IoT Time Series Data

We present ALEC (Adaptive Lazy Evolving Compression), a novel compression codec specifically designed for IoT sensor data. While Shannon's theoretical limits on compression remain inviolable, ALEC approaches these limits by exploiting three key properties of real-world time series: temporal predictability, contextual redundancy, and heterogeneous criticality. Our experimental results demonstrate that on variable sensor data, ALEC achieves 10–22x compression depending on context maturity, significantly outperforming general-purpose codecs (gzip: 5–8x on the same data). With preloaded context from historical data, compression ratios reach 80–95% on representative IoT datasets while maintaining sub-millisecond encoding latency. Key features: - Delta encoding with varint for minimal bit usage- Evolving shared context between encoder and decoder- Priority classification (P1-P5) for critical alerts- Preload system for optimal compression from byte one- Written in Rust, ~4KB encoder footprint Source code: https://github.com/zeekmartin/alec-codecWebsite: https://alec-codec.com