xHeap : Transparent Hugepage Optimizations forMemory Offloading

Increasing dataset sizes require larger application heaps. With recent technology limitations in DRAM scaling, modern datacenters resort to using the virtual memory mechanisms, i.e., the swapper (swap), to transparently extend application heaps over fast, block storage, such as NVMe SSDs or compressed DRAM. At the same time, larger heaps incur increased TLB miss and page fault handling CPU costs, necessitating the use of hugepages for their reduction, now viable given the advent of lower latency and higher throughput storage devices. However, the use of Transparent Hugepages (THP) combined with memory offloading in Linux currently has significant limitations. Hugepage utilization behaviour is overly aggressive in committing memory, and too coarse-grained for memory offloading purposes. Additionally, hugepage promotions lack the responsiveness and concurrency necessary to provide timely hugepage benefits to offloaded applications. To address these issues, we design xHeap, an alternative file-backed mmio path for the Linux kernel, which (a) decouples the use of memory for device-backed heaps from other functions in the kernel, (b) transparently supports regular pages and hugepages at fine granularity, and (c) provides concurrent, asynchronous promotions to adapt to runtime application behaviour. We implement xHeap as a loadable module in Linux to minimize kernel modifications, maintaining complete interoperability with the kernel memory management subsystem, and evaluate it with a variety of workloads. We find that xHeap reduces dTLB miss cycles by up to 15×, and improves performance by up to 76%, when offloading 15-25% of application memory, compared to swap with THP.