GSC Sentinel-2 PDGS STBD - emits - ESA

More documents

Recommendations

Info

GSC Sentinel-2 PDGS STBDIssue 1 Revision 2 (draft) - 25.07.2010GMES-GSEG-EOPG-TN-09-0031page 19 of 60These values have been used for the estimation of the needed processing resources withinthe operational system budget analysis.Production workflows on average datastripFront-End Processing 0.72Estimated Time (h)w/o parallelisationup to the Level-0 (consolidated) 1.67Level-0 (consolidated) -> Level-1B without refinement(including 1A and 1B compression)Level-0 (consolidated) -> Level-1B with refinement(including 1A and 1B compression)Level-0 (consolidated)-> Level-1C without refinement(including 1A, 1B and 1C compression)Level-0 (consolidated)-> Level-1C with refinement(including 1A, 1B and 1C compression)Full bulk-Processing: Level-0 -> Level-1C with refinement(including 1A, 1B and 1C compression)35.045.191.2101.2102.2Table 4: Benchmark processing times for selected production workflows (excludingoptional processing steps).3.3 Processing Parallelisation AnalysisTo meet the product timeliness requirements (cf. section 2.2), the PDGS processing systemwill implement parallelisation mechanisms such that end-to-end performance can be scaledwith the processing hardware. While some specific steps of the processing algorithm willhave sequential prerequisites preventing parallelisation, most of the others will allow thealgorithm to proceed independently in parallel over multiplied hardware units.This section reviews the parallelisation opportunities inherent to the PDGS Level-0 andLevel-1 processing flows which shall lead, when adequately exploited, to a fully scalableend-to-end processing system.3.3.1 PARALLELISATION APPROACHAssuming a large number of available hardware elements, the overall time required forprocessing will be minimised by maximising the number of IPF components run in parallelonto separate elements (e.g. one CPU core).ESA UNCLASSIFIED – For Official Use© ESAThe copyright of this document is the property of ESA. It is supplied in confidence and shall not be reproduced, copied orcommunicated to any third party without written permission from ESA.
GSC Sentinel-2 PDGS STBDIssue 1 Revision 2 (draft) - 25.07.2010GMES-GSEG-EOPG-TN-09-0031page 20 of 60This implies that the IPF components are bounded out of a logical decomposition of theprocessing which maximises the parallelisation opportunities of every individual componentwithin the workflow.This first-level parallelisation will hereafter be referred to as “outer parallelisation” allowingseveral instances of the same IPF component (or sequence of IPF components) to runindependently and in parallel onto distinct hardware (e.g. each one processing a differentportion of the input data). This implies the definition of independent “processing-units” (e.g.by fragmenting the input-data), and to allocate each processing unit to an IPF component orsequence of components as illustrated on Figure 5.Figure 5: IPF Outer ParallelisationIn this example, the data to be processed is divided into several independent processingunits that go through a given processing flow in parallel, step by step, each unit-processinggenerating at its term distinct product items. In this mode of operation, it is up to thesupervision subsystem (DPC) to split the input data stream, and to trigger and manageindependent processing instances in parallel on each fragment.In complement, a second level of parallelisation may be applied to some specific IPFcomponents with “inner-parallelisation” techniques. In this case, the IPF component will itselftake advantage of multi-core/multi-CPU target hardware to implement its own workflow (e.g.using MPI programming techniques) as illustrated on Figure 6.ESA UNCLASSIFIED – For Official Use© ESAThe copyright of this document is the property of ESA. It is supplied in confidence and shall not be reproduced, copied orcommunicated to any third party without written permission from ESA.
Page 2 and 3: GSC Sentinel-2 PDGS STBDIssue 1 Rev
Page 13 and 14: - (A 1 - D 1 ) ≤ 100 minutes, i.e
Page 23: GSC Sentinel-2 PDGS STBDIssue 1 Rev
Page 27 and 28: 3.3.2 ANALYSIS OF PARALLELISATION O
Page 59: GSC Sentinel-2 PDGS STBDIssue 1 Rev

GSC Sentinel-2 PDGS STBD - emits - ESA

You also want an ePaper? Increase the reach of your titles

Delete template?

Save as template?