OMNeT++ based on simulation of InfiniBand interconnection networks

The objective of this study is to construct a simulation platform based on OMNeT++ to validate the potential performance bottlenecks in large-scale compute system networks. Networks composed of thousands of interconnected nodes may encounter unforeseen issues. During the validation process in a real environment, hardware fluctuations can lead to inconsistent results between actual and expected outcomes. Additionally, some servers may be leased and continuously running, which makes it impossible to perform performance tests on them and identify potential performance bottlenecks. Therefore, a simulation platform is needed to simulate and discover these underlying issues. To fulfill the aforementioned requirements, a conversion tool was developed initially to enable the simulation platform to recognize the network's topology. By integrating with OpenSM, the simulation platform acquired routing capabilities identical to those of a real network. Furthermore, the parameters of the simulation platform were adjusted to match the hardware capabilities of real network devices, successfully simulated the delay, bandwidth in point-to-point communication and bandwidth under congested communication scenarios. Finally, the simulation included the collective communication latency of the network. By comparing the simulation results with the data collected from the real environment, it was found that the error between the simulated and actual results was within 10%, thus validating the accuracy of the simulation results. The collective communication results obtained from the simulation were analyzed to identify potential network performance bottlenecks that may arise when running large-scale jobs in a real system.