Answers are grounded in this paper and cite their sources.
Large Language Models (LLMs) serve as critical foundations for a wide range of generative AI applications, which demand substantial memory and storage. The advent of monolithic 3D stackable DRAM, offering cost-effective vertical scaling akin to 3D NAND, presents a promising solution to this challenge. This article conducts a comparative analysis of a variety of 3D DRAM technologies with capacitor or capacitor-less storage mechanisms, focusing on vertical bitline (VBL) and vertical wordline (VWL) configurations. Key performance indicators, including access energy, latency, and storage density, are evaluated at the bank-level considering the 3D array core and peripheral circuits under the array. Moreover, a 3D DRAM-centric LLM accelerator is proposed and benchmarked against the off-the-shelf GPU platforms with high-bandwidth-memory (HBM) with die stacking of 2D DRAM dies. The benchmark results underscore the potential of 3D DRAM technologies in enhancing the efficiency and scalability of LLMs, heralding a new era of large-scale AI model training and inference.
No takes yet. Share an insight, caveat, or question.
Hsu et al. (2025) studied this question.
Synapse has enriched 10 related papers. Consider them for comparative context: