automatically exploiting cross-invocation parallelism using runtime ...

ferent loop. The cross-iteration dependence derived from malloc() to itself prohibitsDOALL parallelization. DOANY, instead, synchronizes these malloc() function callsusing locks. Locks guarantee only one thread can execute malloc() each time (as shownin Figure 2.3(b)). Locks enforces atomic execution but do not guarantee a specific executionorder of those malloc() calls. As a result, DOANY requires the protected operationsto be commutative. Figure 2.3(c) demonstrates another loop whose cross-iterationdependences are caused by irregular accesses to array elements (through an index array).DOANY fails to parallelize that loop since the execution order of update() matters. Inthis case, LOCALWRITE parallelization technique works if it can find a partition of theshared memory space which guarantees that each thread only accesses and updates thememory partition owned by itself. For this loop example, LOCALWRITE partitions theNode array into two sections and assigns each section to one of the worker threads. Eachworker thread executes all of the iterations, but before it executes statement 3, it checkswhether that node falls within its own memory partition. If it does not, the worker threadsimply skips statement 3 and starts executing the next iteration. LOCALWRITE’s performancegain is often limited by the redundant computation among threads (statements 1and 2 in this example). Meanwhile, a partition of the memory space is not always availableat compile time since the dependence patterns may be determined at runtime by specificinputs. Overall, these three parallelization techniques can only handle limited types ofcross-iteration dependences. They require the static analysis to prove that no other parallelizationprohibiting dependences exist.In contrast to these three techniques, DOACROSS and DSWP parallelization can handleany type of dependences. For example, consider the loop shown in Figure 2.4(a). Figure2.4(b) shows the program dependence graph (PDG) corresponding to the code. In thePDG, edges that participate in dependence cycles are shown as dashed lines. Since thestatements on lines 3 and 6 and the statement on line 5 each form a dependence cycle,each iteration is dependent on the previous one. The limitations of DOALL, DOANY15