Demystifying NVIDIA GPU Internals to Enable Reliable GPU Management

As GPU-dependent artificial intelligence and ma-chine learning workloads increasingly come to embedded, safety-critical systems-such as self-driving cars-real-time predictabil-ity for GPU-using tasks becomes essential. This paper identifies flaws in three different real-time GPU management approaches that are largely the result of incomplete information about NVIDIA GPU internals. Details concerning this missing information are elucidated via experiments. Based on this information, key rules of GPU scheduling are identified and shown necessary for safe GPU management.