This talk reviews the history of supercomputing systems with a focus on the "balance" between computation and memory access and on the interaction of technology, architecture, and market forces in driving the evolution of these systems. Since the “Attack of the Killer Micros” began over 30 years ago, the combination of Moore’s Law and Dennard Scaling have led to astounding increases in the computational capabilities of microprocessors, while the technology behind memory subsystems has not enjoyed comparable performance improvements. The increasing imbalance between computation and memory access costs has led to stunningly complex processor implementations with increasing design and fabrication costs as well as increasingly opaque and confounding performance characteristics. A review of new technologies (such as HBM) shows that the difficulties of delivering increased memory bandwidth are not alleviated unless the underlying computer architectures are changed in fundamental ways. The combination of technology trends and economic factors suggest that system balances will continue to shift in the same directions -- favoring workloads with increasingly high compute intensity and increasing available concurrency.
John D. McCalpin (Tue,) studied this question.