Abstract New data centers today demand more scalable and efficient solutions with the rapid advances in deep learning, distributed systems, and high-concurrency data processing. Despite the significant improvements in these areas, existing systems continue to suffer from latency, congestion control, and resource allocation, especially during high workloads and network failures. Two new frameworks for optimizing RDMA-based data flow in remote settings – ConVer and DynoFlow – are introduced in this research. ConVer employs user-space path management and multipath transfer to achieve low latency and high throughput, whereas DynoFlow provides a software-defined, modular approach that efficiently handles network failures and varying traffic loads. Per experimental results, ConVer significantly improves throughput and achieves a latency of 1.3 m s (99th percentile). DynoFlow fares better in coping with network failures and optimizing route utilization. We solve significant problems in modern data centers by demonstrating through extensive experimental evaluation that the frameworks significantly enhance throughput and resilience in distributed systems.
Chen et al. (Fri,) studied this question.