Raster datasets - Northeast Arc Users Group

Goals• Overview of Raster data• Raster storage• Personal GDB• SDE GDB• Raster datasets• Raster catalogs• Raster attributes

The Many Uses of Raster Data• Geographic data:– Represents a geographic extent (i.e. elevation, aerial data, orthorectifiedphotos, satellite imagery, scanned maps/docs)

The Many Uses of Raster Data• Attributes of features:– Associated with a geographic object, typically an attribute tovector data (i.e. picture of a pipe joint, real-estate, fire hydrant,telephone pole)

The Many Uses of Raster DataArt / Graphics / Pictures:– Not geographic data (i.e. Logo, symbol, legendgraphic, picture/photograph)

Raster Data• Storage– Smallest unit is a pixel– Pixels are arranged incolumns and rows– Each pixel has one numericvalue– Each pixel is independent– A raster must have at leastone bandPixel1000 rows1000 columns• Data types– Discrete data indicates a quality of a variable– Continuous data indicates a quantity of a variable

Pyramids• Reduced resolution copies of original raster• Stored alongside raster• Improves display performance• Adds about 33% more storage• Pyramids are not usedduring analysis• Pyramids are used for printingwhen necessary

Resampling Differences

Pixel Resampling Methods• Used in both pyramid creation and display resampling• Interpolates pixel values from original raster valuesForDiscreteData Nearest neighbor• Value of nearest pixel• Does not create new values• Fastest overallForContinuousData Bilinear interpolation• Weighted average of four nearest pixels• Faster than cubic convolution• Smooths most imagery Cubic convolution• Sixteen nearest pixels• Usually most pleasing result• Sharpens most imagery

More Raster Properties• Pixel type and depth• Compression type• Statistics– Min, Max, Mean, Std. Deviation• Default Rendering• Raster format• Spatial Location– Georeferencing– Spatial Reference• Geographic Coordinate System - GCS• Projected Coordinate System – PCS

Pixel Type and Depth• Pixel Type– Signed or unsigned• Pixel Depth– 1 bit (0s and 1s)– 2 bit (0, 1, 2, 3)– 4 bit (0 to 15)– 8 bit• Unsigned (0 to 255)• Signed (-128(to 127)– 16 bit (-128,(128,…, , 0,…, , 127)• Unsigned (0 to 65535)• Signed (-32768(to 32767)– 32 bit (-128,(128,…, , 0,…, , 127)• Unsigned (0 to 4294967296)• Signed (-2147483648(to 2147483647)– Floating Point• Values with decimals– i.e. 3457.4067

Compression• None/Raw– No compression utilized• Lossless– All original values maintained in result– Compression ratios (2:1)• Lossy• RLE/RLC, LZ77, LZW, PackBits, , CCITT G2/G4,– NOTE: User chooses how much loss– Original values are changed by algorithm• JPEG (5:1 to 15:1)– “Visually lossless”• Wavelets– JPEG2000, MrSID, ECW (Commonly 20:1, but up to 50:1)– Note: Loss is more obvious in higher ratios

Raster Format Support inArcGIS 9.1READ/WRITE• GRID and GRID Stacks• ERDAS Imagine (*.imgimg)• TIFF/GeoTIFF(*.tiftif) ) 4GB• Rasters in the Geodatabase– ArcSDE and Personal GeodatabaseREAD• MrSID (Gen2 and Gen3)• JPEG• JPEG2000• BMP, GIF, PNG• BIL, BIP, BSQ• ERDAS 7.4 LAN, GIS,RAW• ER-MapperERS• Intergraph CIT/COT(Military Formats)• DTED (Level 1 and 2)• ADRG• CADRG• CIB• NITF (2.0 & 2.1)• DIGEST (ASRP/USRP)

Raster Formats in ArcGIS(continued)• Downloadable Formats– ECW from ERMapper– ENVI from RSI-ENVI– PIX from PCI– “Creating a Customized Raster Format DLL”• Download new 9 version fromhttp://edn.esri.com

Raster Rendering/Display• Rendering Choices– Stretched• Stretches values along a color ramp– Default: Raster has more than 25 unique values– Unique Values• Uses random colors for individual values– Default: Raster has fewer than 25 unique values– Colormap• Uses pre-chosen colors for individual values– Default: Colormap is present (internal/external)– RGB Composite• Displays each band through a different color (Red, Green,and Blue)– Default: Raster has 3 or more bands– Classified• Places ranges of pixels into separate categories– i.e. 1 to 100 = Red, 101 to 175 = Blue, etc…– ** Not a default renderer: : User can choose this

Spatial Location of Raster Data• Georeferencing– Moving your raster into the right location– May involve polynomial warping• Spatial Reference– The name of your specific coordinate system• Geographic Coordinate System - GCS• Projected Coordinate System – PCS– Data can then be reprojected

Georeferencing Tools

Demonstration ofBasic Raster Functionality/Properties• All in core ArcGIS 9.1– No extensions needed

• Raster datasetStoring raster data in thegeodatabase– Any valid raster format that organizes datainto bands• Raster catalog– A collection of raster datasets• Attribute of type raster– An attribute to a feature that is of type raster

Raster Dataset• Any valid raster format that organizes data intobands (e.g., MrSID, GRID, Tiff, IMG, etc.)• Raster datasets can come from individual files• Raster datasets can be mosaicked– Mosaicked raster dataset will have the same properties(e.g., spatial reference, data type, cell size)3712a.bsq3712b.bsqUC2003.IMAGE_37123712c.bsq3712d.bsq

Raster Catalog• A collection ofraster datasets• Raster datasetsdefined in a tableformat– One record foreach rasterdataset

Storing raster data in a personalgeodatabase• Raster data seemingly stored in thegeodatabase• All data stored externally from geodatabase– Managed by the geodatabase– Not managed by the geodatabase• Advantages– Organization through common location– Raster catalogs provide a selection environment

Managed GDB Raster• Managed by thegeodatabase– Stored in folder atsame directory level aspersonal geodatabase– Raster data isconverted to IMGformat– Data over 2GB will useIMG and IGE files forstorage• Raster datasets mustbe managed• Raster catalogs canbe managedC:\StudentUSA.mdbnew_york_cataloglos_angelesUSA.idbc1r_1.imgc2r_1.imgRaster dataconvertedinto IMGfiles

Unmanaged GDB Raster• Not managed by thegeodatabase– Raster data is referencedthrough NAME field– Path name is used to finda raster file• Only raster catalogs canbe unmanaged• No recreation of data• Fast to createC:\F:\StudentImagesCanada.mdbvancouver_catalogF:\Images\Van1.sidF:\Images\Van2.sidF:\Images\Van3.sidCanada.idbVan1.sid

Personal GDB storageconsiderationsDescription:Managed• Data converted into IMG andstored in a folderUnmanaged• Files are referenced by theGDB (no data conversion)Raster types:• Raster datasets• Raster catalogs• Raster attributes• Raster Catalogs• Raster AttributesAdvantages:• GDB maintains andadministers the IMG files• Data stored in a commonformatDisadvantages: • Data duplication• Slower to create as data isconverted to IMG format• No data duplication• Fast to create• User needs to maintain andadminister the raster files• Can not create rasterdatasets

Rasters in theEnterprise GeodatabaseClient ApplicationTCP/IP NetworkArcSDE… are just like other ArcSDE feature classes… are just like other file-based rastersRDBMS

Retrieving Raster Data from the ArcSDEGeodatabaseImage name, desiredextent, and screenresolution are passedto databaseMyImageClientSmallest number ofcompressed tiles arereturned to the client, wherethey are uncompressedDBMS

ArcSDE storage comparison• Raster data stored in ArcSDE since 8.1• Converts rasters to ArcSDE raster format in theDBMS• Stored in multiple tables• Tiled (behind the scenes)• Managed by the geodatabase• ArcSDE offers– High performance– Support for large data– Centralized geodatabase management– Stores all rasters in same format

ArcSDE storageSupportingSupportingrasterrastertablestables• RASTER (IMAGE): Storespixel values and rasterproperties through a series oftables• SHAPE: Stores footprintsthrough a series of geometrytables• OBJECTID: Unique identifierfor vector footprintsRASTER.USA_CATALOGOBJECTID NAME RASTER SHAPE1 NEW_YORK 1 12 CHICAGO 2 23 LOS ANGELES 3 3GeometryGeometrytablestables

Raster Dataset• Any valid raster format that organizes data intobands (e.g., MrSID, GRID, Tiff, IMG, etc.)• Raster datasets can be mosaicked– Mosaicked raster dataset will have the same properties(e.g., spatial reference, data type, cell size)3712a.bsq3712b.bsqUC2003.IMAGE_37123712c.bsq3712d.bsq

• Copy RasterGeoprocessing tools forraster datasets– Creates raster dataset of existingraster– Converts to GRID, TIFF, IMG, orGDB format– Loads raster datasets into a rastercatalog• Create Raster Dataset– Creates an empty raster dataset

• MosaicGeoprocessing tools forraster datasets– Mosaics raster datasets into anexisting raster dataset– Overlapping pixel options: first, last,blend, mean, max, min• Mosaic To New Raster– Mosaics raster datasets into a newraster dataset– Same overlapping pixel options

Managing raster datasets inArcCatalog• View rasterproperties• Assign spatialreference• Buildpyramids• Calculatestatistics

Raster Catalog• A collection ofraster datasets• Raster datasetsdefined in a tableformat– One record foreach rasterdataset

Raster Catalog• Each raster dataset keepsits own properties• Can mosaic rasters withineach raster dataset• Footprint for each rasterdatasetFootprint stored asvector data

Querying with Footprints• Footprints used for querying– Select By Attribute– Select By Location

Geoprocessing Tools for Rastercatalogs• Copy Raster Catalog Items– Makes a copy of a raster catalog• Create Raster Catalog– Creates an empty raster catalog– Load data using• Copy Raster tool• Catalog’s s context menu• Delete Raster Catalog Items– Deletes datasets from the catalog– The catalog schema is not deleted

Managing raster catalogs inArcCatalog• New windows to manage and view rastercatalogsStandard tabsData viewQuery window

Mosaicked raster datasets vs.Raster CatalogsMosaic• Single rasterdataset formultiple rasters• All properties mustbe the same:• Spatial reference• Data type• Colormap• Set of pyramidsRaster Catalog• A collection ofindependent rasterdatasets• Properties canbe different

Mosaicked raster datasets vs.Raster CatalogsMosaic• Quicker, onlyreads one setof properties• Lose overlappingpixelsRaster Catalog• Slower, mustread each set ofproperties• Overlap preserved

Rendering raster data in ArcMap• Each raster dataset in a raster catalog canuse different rendering properties– Add multiple renderers– ArcMap will choose best amongavailable options• Statistics– Current extent– Individual rasterdataset– Custom

Raster Rendering/Display• Rendering Choices– Stretched• Stretches values along a color ramp– Default: Raster has more than 25 unique values– Unique Values• Uses random colors for individual values– Default: Raster has fewer than 25 unique values– Colormap• Uses pre-chosen colors for individual values– Default: Colormap is present (internal/external)– RGB Composite• Displays each band through a different color (Red, Green,and Blue)– Default: Raster has 3 or more bands– Classified• Places ranges of pixels into separate categories– i.e. 1 to 100 = Red, 101 to 175 = Blue, etc…– ** Not a default renderer: : User can choose this

Raster Attributes• New Raster column type• One raster column per table– Multiple rasters supported through related tablesRaster Field• In personal GDB, can be managed or not-managed• In ArcSDE all rasters converted to ArcSDE rasterformat

Creating and using raster attributes• Load raster duringArcMap edit session

Creating and using raster attributes• Use Identify tool toview raster attributes

Conclusions• Raster storage is possible in Personal andSDE GDBs• Many options:– Raster datasets– Raster catalogs– Mosaic into raster dataset– Raster as an attribute field• Advantages:– Raster Catalogs, Footprints to query by, singlestorage solution for vector & raster