A Survey of Caching Techniques for General Purpose Graphics Processing Units

General Purpose Graphics Processing Units (GP-GPUs) are emerging as a very powerful approach in the era of multi-cores. These processors typically employ multiple cores and use thousands of threads, they exploit the data parallelism inherent in GP-GPU applications. There has been an increasing interest in L1 data caches in GPUs. With the emergence of GP-GPU applications, L1 data caches have become performance critical. Typically the L1 data cache is private to each core or a symmetric multiprocessor. In this paper we comprehensively review the literature on using L1 data cache for GP-GPU applications. One of the earliest work[1] on exploiting locality explores throttling the number of threads thereby making the L1 data cache more efficient. There have been numerous follow-up works on making the L1 Data cache more efficient[2], [3], [4], [5], [6]. One of the most recent works[7] explores a L1 data cache that us shared between the cores. They in their study observe tremendous reuse of data across the private L1 data caches of each core. They propose a architecture that shares the private L1 data cache in a manner such that the reuse is minimized. In this paper we qualitatively compare all these techniques and discuss the advantages and disadvantages of each of these techniques and provide a comprehensive qualtiative comparison.