Dynamical HISQ - University of Glasgow
Dynamical HISQ - University of Glasgow Dynamical HISQ - University of Glasgow
Dynamical HISQ Craig McNeile c.mcneile@physics.gla.ac.uk University of Glasgow Dynamical HISQ – p. 1
- Page 2 and 3: Overview of dynamical HISQ I report
- Page 4 and 5: Who is doing what? Kit and Eduardo
- Page 6 and 7: HISQ and the MILC code Plaquette te
- Page 8 and 9: RHMC improvements There are a tool
- Page 10 and 11: What is level3? The famous software
- Page 12 and 13: Is level 3 useful? Machine Library
- Page 14 and 15: What did HPQCD ask for ?? Postdoc a
- Page 16 and 17: The basic idea Based on RG transfor
- Page 18: Blocking on 1-loop plaquette F(1x1)
<strong>Dynamical</strong> <strong>HISQ</strong><br />
Craig McNeile<br />
c.mcneile@physics.gla.ac.uk<br />
<strong>University</strong> <strong>of</strong> <strong>Glasgow</strong><br />
<strong>Dynamical</strong> <strong>HISQ</strong> – p. 1
Overview <strong>of</strong> dynamical <strong>HISQ</strong><br />
I report on the various efforts to generate gauge<br />
configurations with <strong>HISQ</strong> sea quarks.<br />
Unfortunately running dynamical simulations<br />
involves code optimization and efficiency.<br />
Sorry!<br />
No results. Sorry! Sorry!<br />
I will try be positive and pretend that the UK will<br />
get computer time for this or (anything). (If time,<br />
end with Iwasaki action).<br />
<strong>Dynamical</strong> <strong>HISQ</strong> – p. 2
Why do dynamical <strong>HISQ</strong>?<br />
From science case for computer time in UK.<br />
Improved scaling from more improved <strong>HISQ</strong> action and<br />
updated radiative corrections to gauge action.<br />
Run simulations with non-degenerate u,d,s and c<br />
quarks.<br />
Include electromagnetism effects. (Check MILC’s<br />
result for m u /m d from 2+1 calculations).<br />
Study Random Matrix Theory (RMT) with <strong>HISQ</strong> given<br />
that the unquenched Asqtad RMT project was<br />
unsuccessful.<br />
<strong>Dynamical</strong> <strong>HISQ</strong> – p. 3
Who is doing what?<br />
Kit and Eduardo’s work discussed later. Who is<br />
writing code.<br />
Person where project<br />
Doug Toussaint Arizona <strong>HISQ</strong> into MILC code<br />
Alexei Bazavov Arizona <strong>HISQ</strong> into MILC code<br />
Carleton DeTar Utah level 3, <strong>HISQ</strong><br />
Tommy Burch Utah level 3, <strong>HISQ</strong><br />
Alan Gray EPCC, UK optimization level 3<br />
<strong>Dynamical</strong> <strong>HISQ</strong> – p. 4
<strong>HISQ</strong> optimization<br />
Ron is a PI for a s<strong>of</strong>tware position in EPCC The grant was<br />
for 1/2 a person (Alan Gray) for 18 months. Alan started at<br />
20 % level in October.<br />
Alan has updated Kit’s code so that it can write configs<br />
in NERSC format. Also more Fortran90.<br />
Alan is studying implementation <strong>of</strong> Asqtad/<strong>HISQ</strong><br />
forces.<br />
Plan for Alan to optimize the level 3 code for<br />
Cambridge machine, and maybe Bluegene/P.<br />
The grant proposal also discussed running and testing.<br />
<strong>Dynamical</strong> <strong>HISQ</strong> – p. 5
<strong>HISQ</strong> and the MILC code<br />
Plaquette test <strong>of</strong> MILC code versus Kit’s code.<br />
Tested Alexei’s MILC inverter against<br />
Eduardo’s on 4 3 8 lattice.<br />
Doug Toussaint used <strong>HISQ</strong> inverter on 1/8 fm<br />
configs by computing the pion spectrum on<br />
an ensemble (m l /m s 0.010/0.050). Good<br />
agreement with HPQCD results.<br />
MILC will write new optimized versions <strong>of</strong> the<br />
<strong>HISQ</strong> routines, as more runs are done.<br />
<strong>Dynamical</strong> <strong>HISQ</strong> – p. 6
RHMC<br />
detM α =<br />
∫<br />
Dφ † Dφe −φ† r 2 (M)φ<br />
where r(x) = x −α/2 (rational approximation).<br />
RHMC is an exact algorithm, also allows larger δt than<br />
R algorithm ( δt ∼ 2/3m l ) so is faster (Kennedy/Clark<br />
claim 7 times better).<br />
MILC are now using the exact RHMC/RHMD algorithm<br />
for generating configurations (talk at ILDG11).<br />
<strong>Dynamical</strong> <strong>HISQ</strong> – p. 7
RHMC improvements<br />
There are a tool kits <strong>of</strong> faster algorithms<br />
Higher order integrators (Omelyan ). In MILC code.<br />
n-th root trick. detM =| detM 1/n | n =<br />
∏ nj=1<br />
∫<br />
Dφ † Dφe −φ† j M −1/n φ j<br />
Mass preconditioning<br />
Chris Richards (Liverpool) used the above algorithms<br />
(implemented by Clark) to get a 30% to 50% speed up with<br />
Asqtad, but needed ∼ 4 months <strong>of</strong> tuning (Lots <strong>of</strong><br />
parameters and trade<strong>of</strong>fs).<br />
<strong>Dynamical</strong> <strong>HISQ</strong> – p. 8
MILC plans<br />
Email from Doug Toussaint. Not started production<br />
running, but deciding this month what they want to do with<br />
<strong>HISQ</strong> (SCIDAC deadline).<br />
Alexei and Doug seeing some issues.<br />
Large jumps in the force when a first-level<br />
smeared matrix is small or when the<br />
phase <strong>of</strong> a unitarized matrix jumps.<br />
Note/email to follow.<br />
Mass dependent Naik term means additional<br />
pseudo-fermion fields are needed (Doug).<br />
<strong>Dynamical</strong> <strong>HISQ</strong> – p. 9
What is level3?<br />
The famous s<strong>of</strong>tware diagram from USQCD.<br />
The level3 library contains the inverter and force<br />
terms optimized for specific platforms. QOPQDP<br />
is the level3 library largely written by DeTar and<br />
Osborn.<br />
<strong>Dynamical</strong> <strong>HISQ</strong> – p. 10
Level3 in practise<br />
Level3 code inside chroma Asqtad inverter.<br />
convert_chroma_to_qdp(in,q_source) ;<br />
convert_chroma_to_qdp(out,psi) ;<br />
qopout = QOP_create_V_from_qdp(out);<br />
qopin = QOP_create_V_from_qdp(in);<br />
QOP_asqtad_invert(&info, fla, &inv_arg, &<br />
mass, qopout,qopin);<br />
QOP_extract_V_to_qdp(out,qopout) ;<br />
convert_qdp_to_chroma(psi,out) ;<br />
<strong>Dynamical</strong> <strong>HISQ</strong> – p. 11
Is level 3 useful?<br />
Machine Library iters Time s<br />
QCDOC Pure chroma 736 46.9<br />
QCDOC single prec. level 3 2932 6.0<br />
QCDOC double prec. level 3 742 2.6<br />
Opteron Pure chroma 1851 8893<br />
Opteron double prec. level 3 1862 4008<br />
Table 1: Performance <strong>of</strong> Asqtad<br />
<strong>Dynamical</strong> <strong>HISQ</strong> – p. 12
Pr<strong>of</strong>iling (Woloshyn and Wong)<br />
100<br />
(a)<br />
cost <strong>of</strong> different components per trajectory<br />
80<br />
percentage<br />
60<br />
40<br />
20<br />
0<br />
8 4 12 4 16 4 8 4 12 4 16 4<br />
ASQTAD<br />
<strong>HISQ</strong><br />
matrix<br />
inversion f (0) f (1) f (2)<br />
f (3) gauge comp. <strong>of</strong><br />
force<br />
eff. links<br />
others<br />
On an Opteron Kit’s code generated a trajectory for a 16 4<br />
lattice in 4 hours. Target <strong>of</strong> 3000 trajectories will take 1.4<br />
years. <strong>Dynamical</strong> <strong>HISQ</strong> – p. 13
What did HPQCD ask for ??<br />
Postdoc and running the MILC code on<br />
Time on Cambridge cluster<br />
Time on Bluegene/P at Darsebury lab in Cheshire.<br />
One rack has been delivered. ETMC twisted mass<br />
HMC code 15% efficiency (after a day’s work by<br />
Carsten Urbach).<br />
3 year project costing £1,000,000.<br />
Amazon.com <strong>of</strong>fer cloud computing. Two cores for 1 hour<br />
with 7 Gbytes <strong>of</strong> memory costs $ 0.4 (20p). (The<br />
Cambridge machine costs 6p per core hour). Any spare<br />
book-tokens could come in useful.....<br />
<strong>Dynamical</strong> <strong>HISQ</strong> – p. 14
Introduction to Iwasaki action<br />
Perhaps 50% <strong>of</strong> lattice QCD computer cycles (JLQCD,<br />
RBC, CP-PACS) use the Iwasaki (square plus<br />
rectangle) action.<br />
A classic book is one that everyone knows about, but<br />
no one has read.<br />
How to do α s n f a 2 corrections (rather than to just annoy<br />
people).<br />
Iwasaki’s paper (1983) “Renormalization Group Analysis <strong>of</strong><br />
Lattice theories and improved lattice action II” not<br />
published (available in PDF on spires) The derivation is<br />
perturbative <strong>Dynamical</strong> <strong>HISQ</strong> – p. 15
The basic idea<br />
Based on RG transformation work by Wilson<br />
e −βS′ (Φ) =<br />
∫<br />
dφe −β(T(Φ,φ)+S(φ))<br />
Fixed point solution has no a m (for any m) or one loop<br />
corrections (section 10.4 <strong>of</strong> DeGrand-DeTar book).<br />
Iwasaki’s block spin transformation on gauge potential.<br />
A (n)<br />
µ (x ′ ) = 1 8<br />
∑<br />
µ (x)<br />
x∈x ′ A (n−1)<br />
<strong>Dynamical</strong> <strong>HISQ</strong> – p. 16
The calculation<br />
Iwasaki looked at effect <strong>of</strong> blocking (n-th level) on one loop<br />
expression for Wilson loops<br />
F (n) (IxJ) =<br />
W (n) (C) = 1 − g 2N2 − 1<br />
N<br />
F (n) (C)<br />
∫<br />
12 12(k)( sin1/2Ik(n) 1<br />
sin 1/2k (n)<br />
D (n)<br />
1<br />
sin1/2Jk (n)<br />
2<br />
sin 1/2k (n)<br />
2<br />
where D 12,12 related to gluon propagator and H(k) is<br />
blocking factor. Also obtained result for ∞ blocking<br />
) 2 H (n) (k)<br />
<strong>Dynamical</strong> <strong>HISQ</strong> – p. 17
Blocking on 1-loop plaquette F(1x1)<br />
Block Wilson T. Symanzik Iwasaki<br />
0 0.5 0.36626 0.21027<br />
1 0.28810 0.25076 0.18826<br />
2 0.21623 0.20599 0.18431<br />
3 0.19446 0.19173<br />
4 0.18865 0.18794<br />
∞ 0.18649 0.18649 0.18649<br />
Iwasaki gauge action wins! (Tadpole Symanzik??)<br />
<strong>Dynamical</strong> <strong>HISQ</strong> – p. 18