Automatic parallelization and optimization for irregular ... - IEEE Xplore

for linear subscripts are completely disabled), because mostof practical irregular scientific applications have this kindof loops.Generally, in distributed memory compilation, loop iterationsare partitioned to processors according to the ownercomputes rule [1]. This rule specifies that, on a singlestatementloop, each iteration will be executed by the processorwhich owns the left hand side array reference of theassignment for that iteration.However, owner computes rule is often not best suitedfor irregular codes. This is because use of indirection inaccessing left hand side array makes it difficult to partitionthe loop iterations according to the owner computers rule.Therefore, in CHAOS library, Ponnusamy et al. [13, 14]proposed a heuristic method for irregular loop partitioningcalled almost owner computes rule, in which an iteration isexecuted on the processor that is the owner of the largestnumber of distributed array references in the iteration.Some HPF compilers employ this scheme by usingEXECUTE-ON-HOME clause [15]. However, when weparallelize a fluid dynamics solver ZEUS-2D code by usingalmost owner computes rule, we find that the almostowner computes rule is not optimal manner in minimizingcommunication cost — either communication steps or elementsto be communicated. Another drawback is that it isnot straightforward to choose optimal owner if several processorsown the same number of array references.We propose a more efficient computes rule for irregularloop partition [8]. This approach partitions iterations on aparticular processor such that executing the iteration on thatprocessor ensures• the communication steps is minimum, and• the total number of data to be communicated is minimumIn our approach, neither owner computes rule nor almostowner computes rule is used in parallel execution ofa loop iteration for irregular computation. A communicationcost reduction computes rule, called least communicationcomputes rule, is proposed. For a given irregularloop, we first investigate for all processors P k , 0 ≤k ≤ m (m is the number of processors) in which two setsFanIn(P k ) and FanOut(P k ) for each processor P k aredefined. FanIn(P k ) is a set of processors which have tosend data to processor P k before the iteration is executed,and FanOut(P k ) is a set of processors which have to senddata from processor P k after the iteration is executed. Accordingto these knowledge we partition the loop iterationto a processor on which the minimal communication is ensuredwhen executing that iteration. Then, after all iterationsare partitioned into various processors. Please refer to[8] for details.3 Global-local Array Transformation and IndexArray RemappingThere are two approaches to generating SPMD irregularcodes after loop partitioning. One is receiving required datain an iteration every time before the iteration is executed,and send the changed values (which are resident in otherprocessors originally) to other processors after the iterationis executed. Another is gather all remote data from otherprocessors for all iterations executed on this processor andscatter all changed remote values to other processors afterall iterations are executed. Because message aggregation isa main communication optimization means, obviously thelater one is better for communication performance. In orderto perform communication aggregation, this section discussesredistribution of indirection arrays.3.1 Index Array RedistributionAfter loop partitioning analysis, all iterations havebeen assigned to various processors: iter(P 0 ) ={i 0,1 ,i 0,2 ,...,i 0,α0 },..., iter(P m−1 )={i m−1,1 ,i m−1,2 ,...,i m−1,αm−1 }. If an iteration i r is partitioned to processorP k , the index array elements ix(i r ),iy(i r ), ···maynot be certainly resident in P k . Therefore, we needto redistribute all index arrays so that for iter(P k ) ={i k,1 ,i k,2 ,...,i k,αk } and every index array ix, elementsix(i k,1 ),...,ix(i k,αk ) are local accessible.As mentioned above, The source BLOCK partitionscheme for processor P k is src iter(P k )={⌈ g m ⌉∗k +1..(⌈ g ⌉ +1)∗ k}, where g is the number of iterations ofmthe loop. Then in a redistribution, the elements of an indexarray need to communicate from P k to Pk ′ can be indicatedby src iter(P k ) ∩ iter(Pk ′ ). Back to the Example1, same as Example 3, let the size of data array and indexarrays be 12, after loop partitioning analysis, we can obtainiter(P 0 )={1, 5, 8, 9, 10}, iter(P 1 )={2, 3, 4}, anditer(P 2 )={6, 7, 11, 12}. The index array elements to beredistributed are shown in Figure 2.3.2 Scheduling in Redistribution ProcedureA redistribution routine can be divided into two part:subscript computation and interprocessor communication.If there is no communication scheduling in a redistributionroutine, communication contention may occur, which increasesthe communication waiting time. Clearly in eachcommunication step, there are some processors sendingmessages to the same destination processor. This leads tonode contention. Node contention will result in overall performancedegradation.[5, 6] The scheduling can avoid thiscontention.Proceedings of the 18th International Parallel and Distributed Processing Symposium (IPDPS’04)0-7695-2132-0/04/$17.00 (C) 2004 IEEE