automatically exploiting cross-invocation parallelism using runtime ...

List of Figures1.1 Scientists spend large amounts of time waiting for their program to generateresults. Among the 114 interviewed researchers from 20 differentdepartments in Princeton University, almost half of them had to wait days,weeks or even months for their simulation programs to finish. . . . . . . . . 31.2 Types of parallelism exploited in scientific research programs: one third ofthe interviewed researchers do not use any parallelism in their programs;others mainly use job parallelism or borrow already parallelized programs. 51.3 Example of parallelizing a program with barriers . . . . . . . . . . . . . . 71.4 Comparison between executions with and without barriers. A block withlabel x.y represents the y th iteration in the x th loop invocation. . . . . . . . 71.5 Contribution of this thesis work . . . . . . . . . . . . . . . . . . . . . . . . 92.1 Sequential Code with Two Loops . . . . . . . . . . . . . . . . . . . . . . . 132.2 Performance sensitivity due to memory analysis on a shared-memory machine. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 132.3 Intra-invocation parallelization techniques which rely on static analysis(X.Y refers to the Y th statement in the X th iteration of the loop): (a)DOALL concurrently executes iterations among threads and no inter-threadsynchronization is necessary; (b) DOANY applies locks to guarantee atomicexecution of function malloc; (c) LOCALWRITE goes through eachnode and each worker thread only updates the node belonging to itself. . . 14vii