automatically exploiting cross-invocation parallelism using runtime ...

the graph stands for an iteration in a certain loop invocation (e.g., block 1.5 is iteration 5in the first invocation of loop L1). Typically, threads do not reach barriers at the same timefor a variety of reasons. For instance, each thread may be assigned different number ofiterations and the execution time of each iteration may vary. All threads are forced to stallat barriers after each parallel invocation, losing potential parallelism. Figure 1.4(b) showsa parallel execution plan after naïvely removing barriers. Without barriers, iterations frombefore and after a barrier may overlap, resulting in better performance.A few automatic parallelization techniques exploit cross-invocation parallelism [22,50, 72, 76]. Cross-invocation parallelization requires techniques for respecting crossinvocationdependences without resorting to coarse-grained barrier synchronization. Sometechniques [22, 76] respect dependences by combining several small loops into a singlelarger loop. This approach side-steps the problem of exploiting cross-invocation parallelismby converting it into cross-iteration parallelism. Other approaches [50, 72] carefullypartition the iteration space in each loop invocation so that cross-invocation dependencesare never split between threads. However, both techniques rely on static analyses. Consequently,they cannot adapt to the dependence patterns manifested by particular inputsat runtime. Many statically detected dependences may only manifest under certain inputconditions. For many programs, these dependences rarely manifest given the most commonprogram inputs. By adapting to the dependence patterns of specific inputs at runtime,programs can exploit additional cross-invocation parallelism to achieve greater scalability.6