ing st<strong>and</strong>ard communication libraries <strong>and</strong> making no assumption on the topologyof the processor network <strong>and</strong> memory access system.Minimizing the communication was the major goal in the implementation ofthe parallel plane wave code in CPMD. Therefore, the algorithms had to be adaptedto the distributed data model chosen. The most important decisions concern thedata distribution of the largest arrays in the calculation. These arrays are the onesholding information on the wavefunctions. Three distribution strategies can beenvisaged <strong>and</strong> were used before 90,137,687,688,117 .First, the data are distributed over the b<strong>and</strong>s 687 . Each processor holds allexpansion coefficients of an electronic b<strong>and</strong> locally. Several problems arise withthis choice. The number of b<strong>and</strong>s is usually of the same magnitude as the numberof processors. This leads to a severe load-balancing problem that can only beavoided for certain magic numbers, namely if the number of b<strong>and</strong>s is a multipleof the number of cpu’s. Furthermore this approach requires to perform threedimensionalFourier transforms locally. The memory requirements for the Fouriertransform only increase linearly with system size, but their prefactor is very big <strong>and</strong>a distribution of these arrays is desirable. In addition, all parts of the program thatdo not contain loops over the number of b<strong>and</strong>s have to be parallelized using anotherscheme, leading to additional communication <strong>and</strong> synchronization overhead.Second, the data is distributed over the Fourier space components <strong>and</strong> thereal space grid is also distributed 90,137,117 . This scheme allows for a straightforward parallelization of all parts of the program that involve loops over the Fouriercomponents or the real space grid. Only a few routines are not covered by thisscheme. The disadvantage is that all three-dimensional Fourier transforms requirecommunication.Third, it is possible to use a combination of the above two schemes 688 . Thisleads to the most complicated scheme, as only a careful arrangement of algorithmsavoids the disadvantages of the other schemes while still keeping their advantages.Additionally, it is possible to distribute the loop over k–points. As most calculationonly use a limited number of k–points or even only the Γ–point, this methodis of limited use. However, combining the distribution of the k-points with one ofthe other method mentioned above might result in a very efficient approach.TheCPMD program is parallelized using the distribution in Fourier <strong>and</strong> real space.The data distribution is held fixed during a calculation, i.e. static load balancingis used. In all parts of the program where the distribution of the plane waves doesnot apply, an additional parallelization over the number of atoms or b<strong>and</strong>s is used.However, the data structures involved are replicated on all processors.A special situation exists for the case of path integral calculations (see Sect. 4.4),where an inherent parallelization over the Trotter slices is present. The problem is”embarrassingly parallel” in this variable <strong>and</strong> perfect parallelism can be observed onall types of computers, even on clusters of workstations or supercomputers (”meta–computing”). In practice the parallelization over the Trotter slices will be combinedwith one of the schemes mentioned above, allowing for good results even on massivelyparallel machines with several hundred processors.83
3.9.2 CPMD Program: Data StructuresIn addition to the variables used in the serial version, local copies have to be defined.These local variables will be indexed by a superscript indicating the processornumber. The total number of processors is P. Each processor has a certain numberof plane waves, atoms, electronic b<strong>and</strong>s <strong>and</strong> real space grid points assigned.N p atN pN p bN p PWN p DNx p , N y , N zN p =Nx p N y N znumber of atoms on processor pnumber of projectors on processor pnumber of electronic b<strong>and</strong>s or states on processor pnumber of plane-waves on processor pnumber of plane-waves for densities <strong>and</strong> potentials on processor pnumber of grid points in x, y, <strong>and</strong> z direction on processor ptotal number of grid points on processor pThe real space grid is only distributed over the x coordinates. This decision isrelated to the performance of the Fourier transform that will be discussed in moredetail in the following sections. The distribution algorithm for atoms, projectors<strong>and</strong> b<strong>and</strong>s just divides the total number of these quantities in equal junks based ontheir arbitrary numbering. The algorithms that use these parallelization schemesdo not play a major role in the overall performance of the program (at least for thesystems accessible with the computers available today) <strong>and</strong> small imperfections inload balancing can be ignored.Data structures that are replicated on all processors:r(3,N at ) nuclear positionsv(3,N at ) nuclear velocitiesf(3,N at ) nuclear forcesfnl(N p ,N b ) overlap of projectors <strong>and</strong> b<strong>and</strong>ssmat(N b ,N b ) overlap matrices between b<strong>and</strong>s.Data structures that are distributed over all processors:g(3,N p PW ) plane–wave indicesipg(3,N p PW ) mapping of G–vectors (positive part)img(3,N p PW ) mapping of G–vectors (negative part)rhog(N p PW ) densities (n, n c, n tot ) in Fourier–spacevpot(N p PW ) potentials (V loc, V xc , V H ) in Fourier–spacen(Nx p,N y,N z ) densities (n, n c , n tot ) in real–spacev(Nx p,N y,N z ) potentials (V loc , V xc , V H ) in real–spacevps(N p D ) local pseudopotentialrpc(N p D ) core chargespro(N p PW ) projectors of non-local pseudopotentialeigr(N p D ,N at) structure factorsdfnl(N p ,N p b,3) derivative of fnlcr(N p PW ,N b) b<strong>and</strong>s in Fourier spacecv(N p PW ,N b) velocity of b<strong>and</strong>s in Fourier spacecf(N p PW ,N b) forces of b<strong>and</strong>s in Fourier space.Several different goals should be achieved in the distribution of the plane waves84
- Page 1 and 2:
John von Neumann Institute for Comp
- Page 4:
2500Number200015001000CP PRL 1985AI
- Page 7 and 8:
The goal of this section is to deri
- Page 9:
¡the Newtonian equation of motion
- Page 13 and 14:
Ehrenfest molecular dynamics is cer
- Page 15 and 16:
the Car-Parrinello approach 108 , s
- Page 17:
According to the Car-Parrinello equ
- Page 20 and 21:
Figure 4. (a) Comparison of the x-c
- Page 22 and 23:
Up to this point the entire discuss
- Page 24 and 25:
parameters are those used to repres
- Page 26 and 27:
in terms of a linear combination of
- Page 28 and 29:
structure calculations, see e.g. Re
- Page 30 and 31:
“unbound electrons” dissolved i
- Page 32 and 33:
Table 1. Timings in cpu seconds and
- Page 34 and 35: stressed that the energy conservati
- Page 36 and 37: see e.g. the discussion following E
- Page 38 and 39: from a set of one-particle spin orb
- Page 40 and 41: is used, which represents exactly a
- Page 42 and 43: 2.8.3 Generalized Plane WavesAn ext
- Page 44 and 45: disposable parameters that can be o
- Page 46 and 47: The index i runs over all states an
- Page 48 and 49: f(G) are related by three-dimension
- Page 50 and 51: where j l are spherical Bessel func
- Page 52 and 53: andE self = ∑ I1√2πZ 2 IR c I.
- Page 54 and 55: ¢££¤¤¢¢¢n tot (G)inv FTn to
- Page 56 and 57: correlation energyΩ ∑E xc = ε x
- Page 58 and 59: 3.4 Total Energy, Gradients, and St
- Page 60 and 61: 3.4.3 Gradient for Nuclear Position
- Page 62 and 63: The local part of the pseudopotenti
- Page 64 and 65: ¢¢¢¢¢i = 1 . . .N b¢c i (G)¢
- Page 66 and 67: ¢¢¢¢¢c i (G)123g, E self∆V I
- Page 68 and 69: where G c is a free parameter that
- Page 70 and 71: and a matrix form of the Gram-Schmi
- Page 72 and 73: The two sets of equations are coupl
- Page 74 and 75: introducing different masses for di
- Page 76 and 77: The Lagrange multiplier have to be
- Page 78 and 79: Table 3. Relative size of character
- Page 80 and 81: of the G vectors, and only real ope
- Page 82 and 83: CALL SGEMM(’T’,’N’,M,N b ,2
- Page 86 and 87: over processors. All processors sho
- Page 88 and 89: ab = 2 * sdot(2 * N p D ,A(1),1,B(1
- Page 90 and 91: ENDCALL ParallelFFT3D("INV",scr1,sc
- Page 92 and 93: are the improved load-balancing for
- Page 94 and 95: situations where• it is necessary
- Page 96 and 97: espectively), but completely analog
- Page 98 and 99: 4.2.3 Imposing Pressure: BarostatsK
- Page 100 and 101: in the previous section. The isobar
- Page 102 and 103: 4.3.2 Many Excited States: Free Ene
- Page 104 and 105: down the generalization of the Helm
- Page 106 and 107: free energy functional discussed in
- Page 108 and 109: Figure 16. Four patterns of spin de
- Page 110 and 111: and electrons r = {r i } can be wri
- Page 112 and 113: The effective classical partition f
- Page 114 and 115: up e.g. in Refs. 132,37,596,597,428
- Page 116 and 117: The eigenvalues of A when multiplie
- Page 118 and 119: frequency of the electronic degrees
- Page 120 and 121: 5.2 Solids, Polymers, and Materials
- Page 122 and 123: the penetration of the oxidation la
- Page 124 and 125: in terms of their electronic struct
- Page 126 and 127: culations on very accurate global p
- Page 128 and 129: AcknowledgmentsOur knowledge on ab
- Page 130 and 131: 57. M. Bernasconi, G. L. Chiarotti,
- Page 132 and 133: Superiore di Studi Avanzati (SISSA)
- Page 134 and 135:
175. E. Ermakova, J. Solca, H. Hube
- Page 136 and 137:
244. H. Goldstein, Klassische Mecha
- Page 138 and 139:
313. T. Ikeda, M. Sprik, K. Terakur
- Page 140 and 141:
384. N. A. Marks, D. R. McKenzie, B
- Page 142 and 143:
442. S. Nosé and M. L. Klein, Mol.
- Page 144 and 145:
502. L. M. Ramaniah, M. Bernasconi,
- Page 146 and 147:
562. F. Shimojo, K. Hoshino, and Y.
- Page 148 and 149:
620. A. Tongraar, K. R. Liedl, and
- Page 150:
682. R. M. Wentzcovitch, Phys. Rev.