RFC R-State — Resumable State Transfer Format for Heterogeneous LLM Pipeline Architectures

Key Points

• The aim is to introduce R-State, a serialization format that supports smooth state transfer in language model architectures.
• Proposed a new structured packet format for state transfer.
• Designed to maintain internal state without detokenization and re-tokenization.
• Targets heterogeneous multi-model inference workflows.
• Demonstrates reduced latency in transferring model states.
• Preserves intermediate activations and KV-cache effectively.
• Facilitates modular workflows among different language models.