PRACE hardware, software and services - Prace Training Portal
PRACE hardware, software and services - Prace Training Portal
PRACE hardware, software and services - Prace Training Portal
You also want an ePaper? Increase the reach of your titles
YUMPU automatically turns print PDFs into web optimized ePapers that Google loves.
<strong>PRACE</strong> <strong>hardware</strong>, <strong>software</strong> <strong>and</strong> <strong>services</strong><br />
David Henty, EPCC, d.henty@epcc.ed.ac.uk
Why?<br />
• Weather, Climatology, Earth Science<br />
– degree of warming, scenarios for our future climate.<br />
– underst<strong>and</strong> <strong>and</strong> predict ocean properties <strong>and</strong> variations<br />
– weather <strong>and</strong> flood events<br />
• Astrophysics, Elementary particle physics, Plasma physics<br />
– systems, structures which span a large range of different length <strong>and</strong> time scales<br />
– quantum field theories like QCD, ITER<br />
• Material Science, Chemistry, Nanoscience<br />
– underst<strong>and</strong>ing complex materials, complex chemistry, nanoscience<br />
– the determination of electronic <strong>and</strong> transport properties<br />
• Life Science<br />
– system biology, chromatin dynamics, large scale protein dynamics, protein<br />
association <strong>and</strong> aggregation, supramolecular systems, medicine<br />
• Engineering<br />
– complex helicopter simulation, biomedical flows,<br />
gas turbines <strong>and</strong> internal combustion engines,<br />
forest fires, green aircraft,<br />
– virtual power plant<br />
2
Supercomputing Drives Science through Simulation<br />
Environment<br />
Weather/ Climatology<br />
Pollution / Ozone Hole<br />
Ageing Society<br />
Medicine<br />
Biology<br />
Materials/ Inf. Tech<br />
Spintronics<br />
Nano-science<br />
Energy<br />
Plasma Physics<br />
Fuel Cells<br />
3
Sum of Performance per Country (TOP500)<br />
4
Rationale<br />
• Europe must maintain its high st<strong>and</strong>ards in<br />
computational science <strong>and</strong> engineering<br />
• Europe has to guarantee independent access to HPCsystems<br />
of the highest performance class for all<br />
computational scientists in its member states<br />
• Scientific Excellence requires peer review on European<br />
scale to foster best ideas <strong>and</strong> groups<br />
• User requirements as to variety of architectures<br />
requires coordinated procurement<br />
• EU <strong>and</strong> national governments have to establish robust<br />
<strong>and</strong> persistent funding scheme<br />
5
HPC on ESFRI Roadmap 2006<br />
First comprehensive definition of<br />
RIs at European level<br />
RIs are major pillars of the<br />
European Research Area<br />
A European HPC service<br />
• strategic competitiveness<br />
• attractiveness for researchers<br />
• access based on excellence<br />
• supporting industrial<br />
development<br />
6
capability<br />
The ESFRI Vision for a European HPC service<br />
• European HPC-facilities at the top of<br />
an HPC provisioning pyramid<br />
– Tier-0: 3-6 European Centres for Petaflop<br />
– Tier-0: ? European Centres for Exaflop<br />
– Tier-1: National Centres<br />
– Tier-2: Regional/University Centres<br />
• Creation of a European HPC<br />
ecosystem<br />
– Scientific <strong>and</strong> industrial user communities<br />
– HPC service providers on all tiers<br />
– Grid Infrastructures<br />
– The European HPC hard- <strong>and</strong> <strong>software</strong> industry<br />
Tier-0<br />
Tier-1<br />
Tier-2<br />
<strong>PRACE</strong><br />
DEISA/<strong>PRACE</strong><br />
# of systems<br />
7
<strong>PRACE</strong> in Europe<br />
8
<strong>PRACE</strong> Timeline<br />
HPCEUR<br />
HET<br />
<strong>PRACE</strong> MoU <strong>PRACE</strong> Preparatory<br />
2004 2005 2006 2007 2008<br />
EU-Grant: INFSO-RI-211528, 10 Mio. €<br />
Phase<br />
23.4. 2010<br />
<strong>PRACE</strong> Operation<br />
<strong>PRACE</strong> Implementation Phase (1IP, 2IP)<br />
2009 2010 2011 2012 2013<br />
<strong>PRACE</strong> (AISBL), a legal entity<br />
with (current) seat location in Brussels<br />
9
Purpose of Workshop<br />
• Introduce you to DECI-7 process<br />
• Get you logged on to Tier-1 machines<br />
• Make sure you can compile <strong>and</strong> run simple codes<br />
• Inform you of the applications support available<br />
• Get you started on your own codes<br />
10
Timetable<br />
13:30 - 13:45 Welcome <strong>and</strong> Introduction to SARA<br />
13:45 - 14:30 <strong>PRACE</strong> <strong>hardware</strong>, <strong>software</strong> <strong>and</strong> <strong>services</strong><br />
14:30 - 15:30 Use of Certificates, Using gsi-ssh <strong>and</strong> gridftp<br />
15:30 - 16:00 Coffee Break<br />
16:00 - 17:30 H<strong>and</strong>s on session (practical examples)<br />
09:30 - 10:15 <strong>PRACE</strong> support for the DECI projects<br />
10:15 - 10:30 Remote Visualization<br />
10:30 - 11:00 H<strong>and</strong>s on Sessions (users’ own application codes)<br />
11:00 - 11:30 Coffee Break<br />
11:30 - 12:30 H<strong>and</strong>s on Sessions (users’ own application codes)<br />
12:30 - 13:30 Lunch<br />
13:30 - 15:30 One-to-one sessions (optional)<br />
11
Access to <strong>PRACE</strong> resources<br />
• Regular calls for proposals<br />
– see http://www.prace-ri.eu/hpc-access<br />
• Successful projects<br />
– allocated a maximum number of CPU hours<br />
– given access for a limited period of time<br />
• Linked calls – can apply for Tier-0 or Tier-1<br />
– Tier-1 access is via DECI (continuation of DEISA scheme)<br />
– active projects (you!) are DECI-7<br />
– call already open for DECI-8 starting May 2012<br />
12
Tier-0 Systems<br />
• IBM Blue Gene/P “JUGENE” (GCS@Jülich, Germany)<br />
• Bull Bullx cluster “CURIE” (GENCI@CEA, France)<br />
• Cray XE6 “HERMIT” (GCS@HLRS, Germany)<br />
• SuperMUC (GCS@LRZ, Germany)<br />
• MareNostrum (BSC, Spain)<br />
• FERMI (CINECA, Italy)<br />
13
Tier-1 Systems: specialist<br />
• Cray XT4/5/6 <strong>and</strong> Cray XE6<br />
– EPCC (UK)<br />
– KTH (Sweden)<br />
– CSC (Finl<strong>and</strong>)<br />
• IBM Blue Gene/P<br />
– IDRIS (France)<br />
– RZG (Germany)<br />
– NCSA (Bulgaria)<br />
• IBM Power 6<br />
– RZG (Germany)<br />
– SARA (The Netherl<strong>and</strong>s)<br />
– CINECA (Italy)<br />
14
Tier-1 Systems: clusters<br />
– FZJ (Germany, Bull Nehalem cluster)<br />
– LRZ (Germany, Xeon cluster)<br />
– HLRS (Germany, NEC Nehalem cluster plus GP/GPU cluster)<br />
– CINES (France, SGI ICE 8200)<br />
– BSC (Spain, IBM PowerPC)<br />
– CINECA (Italy, Westmere plus GP/GPU cluster)<br />
– PSNC (Pol<strong>and</strong>, Bullx plus GP/GPU cluster <strong>and</strong> HP cluster)<br />
– ICHEC (Irel<strong>and</strong>, SGI ICE 8200).<br />
15
DECI Terminology<br />
• Every project has<br />
– a single HOME site<br />
– one or more EXECUTION sites<br />
• Home site<br />
– main point of contact<br />
– you will have a named person<br />
– responsible for login accounts etc.<br />
• Execution sites<br />
– where you run your jobs<br />
16
Accounts<br />
• HOME site<br />
– must apply for an account here<br />
– you supply a certificate from your national Certificate Authority<br />
– automatically propagated to execution site(s) …<br />
– … may be additional info needed based on local arrangements<br />
– e.g. signing up to codes of conduct<br />
• EXECUTION sites<br />
– same user name as home site<br />
– recommended access is via gsissh …<br />
– …. some sites may support ssh<br />
17
Security Infrastructure<br />
• St<strong>and</strong>ard public/private key setup<br />
• private key: only owner knows<br />
• public key: known to everyone<br />
• one encrypts, other decrypts: authentication <strong>and</strong> privacy<br />
figure by Borja Sotomayor ,<br />
http://gdp.globus.org/gt4-<br />
tutorial/multiplehtml/ch09s03.html<br />
from J. Schopf, Globus Alliance
Certificates<br />
• Similar to passport or driver’s licence<br />
• All <strong>PRACE</strong> users have an X509 Certificate<br />
• certified by their national certificate authority<br />
• have special temporary certificates for training sessions<br />
• Enables secure authentication<br />
•figure by Rachana<br />
Ananthakrishnan (from<br />
J. Schopf, Globus<br />
Alliance)
Managing certificates<br />
• Certificate must be installed on your local machine<br />
• protected by a private password/passphrase<br />
• List of user certificates is held by <strong>PRACE</strong><br />
• central LDAP database<br />
• Lightweight Directory Access Protocol<br />
• <strong>PRACE</strong> sites synchronise with the LDAP at regular intervals<br />
• Ask your home site if you have problems!
CPU accounted in “st<strong>and</strong>ard core-hours”<br />
• e.g. conversion factors for DEISA were:<br />
CPU<br />
normalisation<br />
AMD Opt@2.2 1.4<br />
BGP@0.85 0.33<br />
I2 DC@1.6 1.1<br />
Intel Harpertown@2.5 1.4<br />
Intel Harpertown@3 1.6<br />
Intel Nehalem@2.8 2.8<br />
Intel Nehalem@2.93 3<br />
Intel Westmere EP@2.67 2.7<br />
Intel Westmere EX@2.4 2.6<br />
Intel Westmere EX@2.67 2.7<br />
NEC SX8 6<br />
NEC SX9 36<br />
P4+@1.5 0.88<br />
P6@4.7 3<br />
PPC@2.3 0.8<br />
X2 4<br />
XE6 12C@2.1 1.25<br />
XEON X5560@2.80 2.8<br />
XEON X5570@2.93 3<br />
XT5 CSCS 1.2<br />
XT5 DC@2.3 1.4<br />
XT5 DC@2.7 1.4<br />
XT6 1.25<br />
21
Current status<br />
• Tier-1 systems new to <strong>PRACE</strong><br />
– DECI-7 is the first <strong>PRACE</strong> DECI<br />
– integration may not be complete for all sites<br />
• <strong>Training</strong> accounts<br />
– we provide temporary accounts for today<br />
– pr1utrXX (XX = 11, 12, …, 35)<br />
– notional home site is EPCC (HECToR)<br />
– temporary certificate from ECMWF CA<br />
– execution sites are at SARA, CSC <strong>and</strong> CINECA<br />
– chosen to span a range of architectures<br />
22
DECI-7 Statistics<br />
• 54 applications<br />
– for 200 million st<strong>and</strong>ard core-hours<br />
• 35 successful projects<br />
– allocated 90 million st<strong>and</strong>ard core-hours<br />
• Start date: 1 st November 2011<br />
• End date: 31 st October 2012<br />
– you MUST use your CPU allocation in this period<br />
• Final reports must be submitted<br />
– due within 3 months of project completion<br />
23
Useful resources<br />
• Documentation<br />
– currently provided via DEISA site<br />
http://www.deisa.eu/usersupport/user-documentation<br />
– migrating to http://www.prace-ri.eu<br />
• Reporting problems<br />
– email support@prace-ri.eu<br />
– Trouble Ticket System will be opened up to users in the future<br />
• Monitoring CPU usage<br />
– done via DART tool<br />
– home site can advise on installing this<br />
24
www.unicore.eu<br />
• Run a graphical client on local machine<br />
• uniform interface to different HPC systems<br />
• support for data transfer, workflows etc.<br />
25
Meal tonight@8pm: Brasserie Harkema<br />
26
Change language