The ESA Earth Observation Payload Data Ground Segment ...

More documents

Recommendations

Info

EO payload data processing needs EO satellites dump payload data at typical rates between few Mbpsto more than 300Mbps (more than 500Mbps in the near future andmuch more in future with Ka-band) Big volume of archive data for reprocessing (L0), often in the orderof hundreds TB per reprocessing campaign Requirement is to speed more and more reprocessing time <strong>Data</strong> segmentation is normally the single satellite dump, i the orderof ~1-10 GB, much more in the future Additional need to auxiliary data for processing whose size is also issome cases not negligible (GB) Algorithms varies from very fast (few minutes per single segment)to “very slow” (many hours, up to 12) Processing algorithms are often not well parallelized/ parallelizable difficult to get advantage of high parallel processinginfrastructure like multi-core CPUs and GPUs Need anyway to implement a robust and performing infrastructureto parallelize multiple segments processing I/O main bottleneck
MMFI data processing infrastructurePFM (Processing Facility Management)Essentially a specialized job schedulerMain function is the abstraction of the processingsystems (IPF – Instrument Processing Facility) to thehigher level elements of the MMFI (ordering, archive,etc.)Implements our Standard IPF InterfaceSelection of best auxiliary data for processing(several selection policies available, expandable byimplementing new modules)Powerful subscription mechanism to LI, based on OQL(Object Query Language), to be notified of dataappearance in the archive (systematic processing,reprocessing, circulation, etc.)Recurrent auxiliary data caching on processing nodesto improve performanceOptimization of CPU resources, i.e. multi-(core)-CPUs
Page 1 and 2: The ESA Earth ObservationPayload Da
Page 3 and 4: Rationale for PDGS harmonization (1
Page 5 and 6: Ground Segment DecompositionSpecifi
Page 8 and 9: MultiMission Facility Infrastructur
Page 10 and 11: MMFI Main Functionality‣ Data Ing
Page 12 and 13: MMFI ArchitectureCentral Infrastruc
Page 14 and 15: MMFI Elements All MMFI Elements are
Page 16 and 17: MM PDGS benefits Lower cost & risks
Page 20 and 21: Distributed Processing Each MMFI ce
Page 22 and 23: DPCS The Distributed Processing Cap
Page 24 and 25: DPCS Solution Search Workload Manag
Page 26 and 27: DPC Structured Processing BusMMFI A
Page 28 and 29: DPC Conclusion Grid technology Rati
Page 30 and 31: G-POD ObjectivesProvide a “user-s
Page 32 and 33: G-POD History and Status 2002-2004:
Page 34 and 35: Improved Data AccessMM PDGS Future

The ESA Earth Observation Payload Data Ground Segment ...

You also want an ePaper? Increase the reach of your titles

Delete template?

Save as template?