Advances in the NetCDF Data Model, Format, and Software - Unidata
Advances in the NetCDF Data Model, Format, and Software - Unidata Advances in the NetCDF Data Model, Format, and Software - Unidata
Status of netCDF-4 adoption: Logjam appears to be broken • NetCDF-4 enhanced model support in language APIs: C, Java (read only), C++ (beta), Fortran • Partial support for netCDF-4 enhanced model also in NCO, NCL, Panoply, Python API, … • NetCDF-4 classic model support in analysis and visualization apps: IDL, GrADS, CDAT, MATLAB, IDV, NCO, NCL, CDO, PyNGL, ncview, Panoply, Ferret, OGC WMS and WCS clients • Data providers using netCDF-4 classic model format for transparent compression and chunking: groups in NASA, NOAA, GFDL, COLA • CMIP5 decided to continue using classic model and classic format (no compression) due to time accessing compressed data on server
Concluding Remarks • Data providers may begin to use compression/chunking with confidence that most users and software can read it transparently, after relinking with netCDF-4 • Developers may adapt software to netCDF-4 format by relinking • Developers may adapt software to enhanced data model incrementally, with examples that such adaptation is practical • Upgrading software to make use of higher-level abstractions of netCDF-4 enhanced data model has significant benefits – Data providers can use more natural representation of complex data semantics – More natural conventions become possible – End users can access other types of data through netCDF APIs • As we keep pushing common tasks into libraries, scientists can focus on doing science instead of data management
- Page 1 and 2: Advances in the NetCDF Data Model,
- Page 3 and 4: NetCDF: more than a format • Data
- Page 5 and 6: The netCDF “classic” data model
- Page 7 and 8: NetCDF classic format Strengths Lim
- Page 9 and 10: NetCDF enhanced data model Strength
- Page 11 and 12: Recent advances Standards Refactori
- Page 13 and 14: NetCDF-Java/Common Data Model archi
- Page 15 and 16: Performance improvements • Refact
- Page 17 and 18: Experience developing nccopy utilit
- Page 19: Last year: game of “chicken”; w
- Page 23 and 24: Extra Slides
- Page 25 and 26: Groups • Like directories in a fi
- Page 27 and 28: Compound types Uses include: • Re
- Page 29 and 30: Guidance for developers • Add sup
Conclud<strong>in</strong>g Remarks<br />
• <strong>Data</strong> providers may beg<strong>in</strong> to use compression/chunk<strong>in</strong>g with<br />
confidence that most users <strong>and</strong> software can read it<br />
transparently, after rel<strong>in</strong>k<strong>in</strong>g with netCDF-4<br />
• Developers may adapt software to netCDF-4 format by rel<strong>in</strong>k<strong>in</strong>g<br />
• Developers may adapt software to enhanced data model<br />
<strong>in</strong>crementally, with examples that such adaptation is practical<br />
• Upgrad<strong>in</strong>g software to make use of higher-level abstractions of<br />
netCDF-4 enhanced data model has significant benefits<br />
– <strong>Data</strong> providers can use more natural representation of complex data<br />
semantics<br />
– More natural conventions become possible<br />
– End users can access o<strong>the</strong>r types of data through netCDF APIs<br />
• As we keep push<strong>in</strong>g common tasks <strong>in</strong>to libraries, scientists can<br />
focus on do<strong>in</strong>g science <strong>in</strong>stead of data management
Change language