DRackSim: Simulating CXL-enabled Large-Scale Disaggregated Memory Systems

Memory disaggregation has emerged as an alternative to traditional server architecture in data centers that targets improved memory utilization and higher scalability.Disaggregated memory systems involve multiple independent compute and memory pools that are expected to get dedicated hardware support for memory expansion through high-speed cache-coherent interconnects such as CXL.However, CXL memory access is slower compared to local memory, which can significantly impact the system's performance, and requires a series of optimizations to make suitable use of expandable memory.Research in disaggregated systems generates the need for a simulator to evaluate new designs in the practical configurations of hardware disaggregated memory systems.This paper introduces DRackSim, a simulation infrastructure to model scalable hardware disaggregated memory systems.It models multiple compute nodes, memory pools, and a common interconnect for coherent memory access.An application-level simulation approach simulates an out-of-order x86 multi-core processor with a multi-level cache hierarchy at compute nodes.DRackSim uses a queue-based simulation to model the network interface at the end nodes and the central switch, which simulates remote memory access at both cache block and page granularity.DRackSim also models a centralized memory manager to manage address space in the memory pools.We integrate community-accepted DRAMSim2 to perform memory simulation at local and remote memory locations by using multiple DRAMSim2 instances.Finally, an incremental approach is followed to validate the core and cache subsystems of DRackSim against Gem5.Further, we model various use-case scenarios for disaggregated memory systems and evaluate their performance over various HPC and cloud benchmarks.We rigorously evaluate DRackSim for a wide range of configurations that simulate real-world deployment models and their impact on system performance.