Beyond i.i.d.: Non-IID Thinking, Informatics, and Learning

In science, technology, engineering, and their applications, a ubiquitous assumption is independent and identically distributed (i.i.d. or IID). IID simplifies the intricate realities and complexities of the real world for their approximate quantifiability and tractability and asymptotic problem-solving. It, however, has also induced significant limitations and gaps in their knowledge, capability, and capacity. Reality-level problem-solving has to go beyond i.i.d. and adopt a broad-reaching non-IID thinking approach, i.e., exploring the comprehensive non-IIDness—heterogeneity and interaction (couplings and entanglement)—of an underlying problem and its behaviors, environment, data, and problem-solving system. Accordingly, this article discusses this foundational, cross-disciplinary, cross-domain, and cross-theory and -practice problem-going beyond i.i.d. and proceeding with non-IID. Concepts, challenges, and prospects of non-IIDness, non-IID thinking, informatics, and learning are discussed for developing reality-level AI, data science, machine learning, and intelligent systems by identifying, quantifying, and synthesizing heterogeneities, interactions, complexities, and intelligence in complex systems and data.