// Parallel Orthorectification of DigitalGlobe Imagery on ABoVE

Overview

High-resolution DigitalGlobe imagery is available on the ABoVE Science Cloud (ASC) for approved ABoVE Science Team members. Access may be requested here. Most of the DigitalGlobe imagery on the ASC is Basic (1B) Imagery, as defined by DigitalGlobe:

"Basic (1B) Imagery is the least processed of the Core Imagery product series and is corrected for radiometric distortions, internal sensor geometry, optical distortions, and sensor distortions. Basic Imagery is neither geo-referenced nor mapped to a cartographic projection. Basic Imagery is provided with sensor models and is intended for sophisticated photogrammetric processing such as orthorectification. Basic Imagery is a scene-based product."

~DigitalGlobe Core Imagery Product Guide

Depending on use case, Basic (1B) Imagery may require orthorectification in order to be used for location analysis. This guide describes the use of the Polar Geospatial Center's "Imagery Utils" to orthorectify such data.

An ABoVE Science Cloud Webinar was held on this topic on 2/17/17. Find the webinar here, and a link to the slides here.

Prerequisites:

1. Copy Polar Geospatial Center's orthorectification tools to your home directory:
   • Get the download link for the latest stable version of the code by visiting the latest release Github page and copying the "source code (zip)" link.
   • After logging into the ASC, users are automatically placed in their home directory (/home/userid). Run these commands from /home: wget -O source.zip <copied url>
unzip source.zip
   • Now run ls. There should be a folder called imagery_utils-<version> where <version> is the latest imagery_utils release.


2. Make an output folder where orthorectified images will be stored (e.g. in a /nobackup or /home directory): mkdir /adapt/nobackup/people/userid/output


3. Create a text file with the locations of the NGA imagery to be orthorectified. (See Appendix 2 for locating NGA imagery on the ASC). Note: each image location should be placed on a separate line, for example:
   • root_location/NGA/WV02/1B/2010/204/WV02_10300100061FE500_X1BS_052402487010_01/WV02_20100723220557_10300100061FE500_10JUL23220557-M1BS-052402487010_01_P004.ntf
   • root_location/NGA/WV02/1B/2010/204/WV02_10300100061FE500_X1BS_052402487010_01/WV02_20100723220557_10300100061FE500_10JUL23220557-M1BS-052402487010_01_P005.ntf
   • Note: root file locations on the ASC are not shared publicly. A version of this file can be found in the ASC at root_location/NGA_footprints/Documentation/. If you are unsure of the location, contact [email protected] or [email protected] for assistance.


4. Determine the EPSG code to use to project your imagery. These EPSG codes can be found online: epsg.io or http://spatialreference.org/ref/epsg/.


5. Have a DEM available for use in the orthorectification process. On the ASC there is a DEM folder available with multiple DEM datasets. The two most commonly used by PGC are:
   • Alaska: National Elevation Dataset:
     root_location/dem/alaskaned_mosaic_wgs84.tif or
     root_location/dem/AKNED_ABoVE_tiles
   • Canada: Canadian Digital Elevation Model (CDEM):
     root_location/dem/CDEM_ABoVE_Tiles
   • Note: These two datasets are available on the ASC using the ABoVE grid.

Batch processing of NGA imagery:

1. Log into a Linux VM (e.g. ssh above101)
2. Start by loading the latest GDAL environment: module load gdal
3. Call orthorectification python script with desired parameters (see Appendix 1 for parameter options):python ~/imagery_utils-<version>/pgc_ortho.py <image_paths_textfile.txt> <output_folder> -p 4326 -d /explore/nobackup/projects/dem/alaskaned_mosaic_wgs84.tif -t Byte -c rf --parallel-processes 2

Suggestions for script usage on ADAPT

Most users require one of four image processing parameter profiles:

1. Basic visual analysis over ice (-t Byte -c rf): This profile is optimized to provide imagery for visual analysis over bright surfaces like ice and snow. It outputs a smaller, 8-bit image instead of the entire 11 bit radiometric range, and it balances the image colors.
2. Basic visual analysis over non-ice (-t Byte -c mr): This profile is optimized to provide imagery for visual analysis over darker surfaces like rock and vegetation. It outputs a smaller 8-bit image instead of the full 11 bit radiometric range and balances the image colors, plus it brightens up the low end of the visual spectrum to provide good contrast in dark areas.
3. Terrain correction only (-t UInt16 -c ns): For users who wish to use the scripts only for terrain correction, this set of parameters leaves the image DN values as is and only applies terrain correction according to the DEM supplied.
4. Top-of-atmosphere reflectance or radiance (-t Float32 -c rf or -c rd): This set of parameters corrects the DN values to top-of-atmosphere reflectance (-c rf) or radiance (-c rd) and stores the result in a floating point raster. The resulting rasters take up more space on user filesystems but are more useful for some quantitative analyses.
5. Height—wgs84 vertical heights are needed for DEM inputs

Because the ADAPT system is a collection of VMs, as opposed to an HPC system with schedulers, the imagery_utils tool is not applicable here. Users can ignore the –pbs, --qsubscript, and -l options. Instead, they should experiment with the parallel-processes option to process several images in parallel.

If high I/O loads and low CPU usage is experienced, even when the –parallel-processes option is near the number of cores on the VM, consider using a working directory that is a local scratch space so that any processing is not limited by network connectivity (--wd option).

Error Assessment

The RPC sensor model has well understood error margins, +-4m LE90 and CE90. However, the sensor model error assessment assumes a perfect elevation model. The greatest source error in images will come from the uncertainty of the DEM used to correct for terrain distortions, especially with high off-nadir images. Often, the error is less than 30m, sometimes less than 5m, and, with a poor DEM in a mountainous area in Antarctica, it can be as high as 500m.

Appendix 1: Input Parameters of pgc_ortho.py

Appendix 2: Locating DG Imagery on the ASC

1. In root_location/NGA_footprints, users can locate the most recent DigitalGlobe (DG) footprints files showing footprint locations of all the DG data accessible on the ASC.
   • In root_location/NGA_footprints, the first level of subfolders are organized by the date of the footprints files such that MM_DD_YYYY defines the month, day, and year of the footprints files.
   • Within root_location/NGA_footprints /MM_DD_YYYY/, individual footprint files are organized by sensor, product level, and year: SSNN_PP_YYYY. Where SS is the sensor type, NN is the sensor number, PP is the product level (such as 1B) and YYYY is the year of acquisition.
   • Sensor Types:
     • GE – GeoEye
     • IK – Ikonos
     • QB – Quick Bird
     • WV – World View
2. Users can download footprint files and view them on their local machine or open them in the ASC using: QGIS (Linux side of the ASC) or ArcGIS (Windows side of the ASC)
3. The filepath location of each footprint is stored in the "s_filepath" attribute of the table