Predicting data cache misses in non-numeric applications through correlation profiling

Software-based latency tolerance techniques offer the potential for bridging the ever-increasing speed gap between the memory subsystem and today's high-performance processors. However, to fully exploit the benefit of these techniques, one must be careful to apply them only to the dynamic references that are likely to suffer cache misses---otherwise the runtime overheads can potentially offset any gains. In this paper, we focus on isolating dynamic miss instances in non-numeric applications, which is a difficult but important problem. Although compilers cannot statically analyze data locality in non-numeric applications, one viable approach is to use profiling information to measure the actual miss behavior. Unfortunately, the state-of-theart in cache miss profiling (which we call summary profiling) is inadequate for references with intermediate miss ratios---it either misses opportunities to hide latency, or else inserts overhead that is unnecessary. To overcome this problem, we propo...