Reading HSI Data Files

The standard means of opening and accessing a hyperspectral image file with SPy is via the image function, which returns an instance of a SpyFile object.

The SpyFile Interface

SpyFile is the base class for creating objects to read hyperspectral data files. When a SpyFile object is created, it provides an interface to read data from a corresponding file. When an image is opened, the actual object returned will be a subclass of SpyFile (BipFile, BilFile, or BsqFile) corresponding to the interleave of the data within the image file.

Let’s open our sample image.

In [1]: from spectral import *

In [2]: img = open_image('92AV3C.lan')

In [3]: img.__class__
Out[3]: spectral.io.bilfile.BilFile

In [4]: print(img)
	Data Source:   '/home/thomas/spectral_data/92AV3C.lan'
	# Rows:            145
	# Samples:         145
	# Bands:           220
	Interleave:        BIL
	Quantization:  16 bits
	Data format:     int16

The image was not located in the working directory but it was still opened because it was in a directory specified by the SPECTRAL_DATA environment variable. Because the image pixel data are interleaved by line, the image function returned a BilFile instance.

Since hyperspectral image files can be quite large, only metadata are read from the file when the SpyFile object is first created. Image data values are only read when specifically requested via SpyFile methods. The SpyFile class provides a subscript operator that behaves much like the numpy array subscript operator. The SpyFile object is subscripted as an MxNxB array where M is the number of rows in the image, N is the number of columns, and B is thenumber of bands.

In [5]: img.shape
Out[5]: (145, 145, 220)

In [6]: pixel = img[50,100]

In [7]: pixel.shape
Out[7]: (220,)

In [8]: band6 = img[:,:,5]

In [9]: band6.shape
Out[9]: (145, 145, 1)

The image data values were not read from the file until the subscript operator calls were performed. Note that since Python indices start at 0, img[50,100] refers to the pixel at 51st row and 101st column of the image. Similarly, img[:,:,5] refers to all the rows and columns for the 6th band of the image.

SpyFile subclass instances returned for particular image files will also provide the following methods:

Method

Description

read_band

Reads a single band into an MxN array

read_bands

Reads multiple bands into an MxNxC array

read_pixel

Reads a single pixel into a length B array

read_subregion

Reads multiple bands from a rectangular sub-region of the image

read_subimage

Reads specified rows, columns, and bands

SpyFile objects have a bands member, which is an instance of a BandInfo object that contains optional information about the images spectral bands.

Loading Entire Images

It is important to note that image data are read by a SpyFile object on demand and the data are not cached. Each time the SpyFile subscript operator or one of the SpyFile read methods are called, data are read from the corresponding image data file, regardless of whether the same data have been previously read. This is done to avoid consuming too much memory when working with very large image files. It also improves performance when performing operations that only require reading a small portion of the data in a large image (e.g., reading RGB bands to display the image). The downside of reading data on demand and not caching the data is that there can be a significant run time penalty when running algorithms that require access to all of the data. Performance will be even worse if the algorithm requires iterative access to the data.

To improve performance of spectral algorithms, it is preferable to load the entire image into memory using the load method, which returns an ImageArray object. ImageArray provides the full numpy.ndarray interface, as well as the SpyFile interface.

In [1]: arr = img.load()

In [2]: arr.__class__
Out[2]: spectral.image.ImageArray

In [3]: print(arr.info())
	# Rows:            145
	# Samples:         145
	# Bands:           220
	Data format:   float32

In [4]: arr.shape
Out[4]: (145, 145, 220)

Because SPy is primarily designed for processing in the spectral domain, spectral.ImageArray objects in memory will always have data interleaved by pixel, regardless of the interleave of the source image data file. In other words, the numpy.ndarray shape will be (numRows, numCols, numBands). ImageArray objects always contain 32-bit floats.

Note

Before calling the load method, it is important to consider the amount of memory that will be consumed by the resulting ImageArray object. Since spectral.ImageArray uses 32-bit floating point values, the amount of memory consumed will be approximately 4 * numRows * numCols * numBands bytes.

NumPy memmap Interface

As an alternative to loading an entire image into memory, a somewhat slower (but more memory efficient) way to access image data is to use a numpy memmap object, as returned by the open_memmap method of SpyFile objects. memmap objects can also be used to write date to an image file.

File Formats Supported

ENVI Headers

ENVI 1 is a popular commercial software package for processing and analyzing geospatial imagery. SPy can read images that have associated ENVI header files and can read & write spectral libraries with ENVI headers. ENVI files are opened automatically by the SPy image function but images can also be opened explicitly as ENVI files. It may be necessary to open an ENVI file explicitly if the data file is in a separate directory from the header or if the data file has an unusual file extension that SPy can not identify.

In [5]: import spectral.io.envi as envi

In [6]: img = envi.open('cup95eff.int.hdr', 'cup95eff.int')

In [7]: import spectral.io.envi as envi

In [8]: lib = envi.open('spectra.hdr')

In [9]: lib.names[:5]
Out[9]: 
['construction asphalt',
 'construction  concrete',
 'red smooth-faced brick',
 'weathered red brick',
 'bare red brick']

See also

Functions for writing image data to files:

create_image:

Creates a new image file with allocated storage on disk.

save_image:

Saves an existing image or ndarray to a file with an ENVI header.

AVIRIS

SPy supports data files generated by the Airborne Visible/Infrared Imaging Spectrometer (AVIRIS) 2. AVIRIS files are automatically recognized by the open_image function; however, spectral band calibration files are not automatically recognized; therefore you may want to open the image as an AVIRIS file explicitly and specify the cal file.

In [10]: img = aviris.open('f970619t01p02_r02_sc01.a.rfl', 'f970619t01p02_r02.a.spc')

You can also load the band calibration file separately (this may be necessary if the band calibration file is in AVIRIS format but the image is not).

In [11]: img = open_image('92AV3C.lan')

In [12]: img.bands = aviris.read_aviris_bands('92AV3C.spc')

ERDAS/Lan

The ERDAS/Lan file format is automatically recognized by image. It is unlikely that a file would need to be opened explicitly as a Lan file but it can be done as follows.

In [13]: import spectral.io.erdas as erdas

In [14]: img = erdas.open('92AV3C.lan')
1

ENVI is a registered trademark of Exelis Visual Information Solutions.

2

http://aviris.jpl.nasa.gov/