Dynamic Warp Subdivision for Integrated Branch and Memory Divergence Tolerance: Extended ResultsReport
SIMD organizations amortize the area and power of fetch, decode, and issue logic across multiple processing units in or- der to maximize throughput for a given area and power bud- get. However, throughput is reduced when a set of threads operating in lockstep (a warp) are stalled due to long latency memory accesses. The resulting idle cycles are extremely costly. Multi-threading can hide latencies by interleaving the execution of multiple warps, but deep multi-threading using many warps dramatically increases the cost of the register files (multi-threading depth x SIMD width), and cache con- tention can make performance worse. Instead, intra-warp latency hiding should first be exploited. This allows threads that are ready but stalled by SIMD restrictions to use these idle cycles and reduces the need for multi-threading among warps. This paper introduces dynamic warp subdivision (DWS), which allows a single warp to occupy more than one slot in the scheduler without requiring extra register file space. Independent scheduling entities allow divergent branch paths to interleave their execution, and allow threads that hit to run ahead. The result is improved latency hiding and memory level parallelism (MLP). We evaluate the tech- nique on a coherent cache hierarchy with private L1 caches and a shared L2 cache. With an area overhead of less than 1%, experiments with eight data-parallel benchmarks show our technique improves performance on average by 1.7X.
All rights reserved (no additional license for public reuse)
Meng, Jiayuan, David Tarjan, and Kevin Skadron. "Dynamic Warp Subdivision for Integrated Branch and Memory Divergence Tolerance: Extended Results." University of Virginia Dept. of Computer Science Tech Report (2010).
University of Virginia, Department of Computer Science