Synapse
⌘+K
Synapse
PulseExploreClubsResearchersJournals
Instagram
HomeClubsExplore
July 15, 2026ACM SIGOPS Operating Systems Review

Elastic Memory Remapping for Multi-tenant LLM Serving

View Full Paper
Ask AI
Bookmark
Share

Authors

RLRuihao LiSPShagnik PalVPVineeth Narayan Pullu

Discussion

Loading...

Member takes

Overview

Randomized trial demonstrates improved throughput and reduced latency in multi-tenant environments, indicating enhanced GPU memory usage.

Key Points

  • This research aims to optimize GPU memory usage during LLM inference in multi-tenant settings by introducing Oneiros, a dynamic memory remapping engine.
  • Implemented Oneiros in vLLM on modern GH200 systems.
  • Reclaimed memory allocated to inactive models for active workloads.
  • Avoided synchronization overheads by utilizing unidirectional parameter transfer.
  • Reduced tail latency by up to 99.3%.
  • Improved throughput by up to 86.7% compared to vLLM.
  • Outperformed KV-cache swapping approaches with significantly higher throughput.

Cite This Study

Li et al. (2026) studied this question.

synapsesocial.com/papers/6a5723fd88b21df8754807cdhttps://doi.org/10.1145/3830422.3830427
View Full Paper
Ask AI
Bookmark
Share

Also Consider

Synapse has enriched 5 closely related papers on similar clinical questions. Consider them for comparative context:

  1. 1MIRAGE: KV Cache Optimization through Parameter Remapping for Multi-tenant LLM Serving2025
  2. 2Accelerating LLM Inference via Dynamic KV Cache Placement in Heterogeneous Memory System2025
  3. 3KVD <scp>rive:</scp> A Holistic Multi-Tier KV Cache Management System for Long-Context LLM Inference2026
  4. 4AdaptiveKV: Accelerating KV Cache Offloading with a Bandwidth-Adaptive Memory Allocation Mechanism2026
  5. 5eLLM: Elastic Memory Management Framework for Efficient LLM Serving2025