Compression and Prioritization

Every camera on Spirit and Opportunity produces a 1024x1024 image at 12 bits per pixel, and the average returned rate was 1.13 bits per pixel [1]. That ratio, roughly ten to one, is what makes a Mars imaging mission possible on a link that returns about 56 Mb/sol. What is compressed, how hard, and in what order it is sent are three separate decisions.

The compressors

MER carries three: ICER for lossy and progressive compression, a modified LOCO for lossless, and LZO for engineering data [7]. Sojourner’s Mars Pathfinder used a Rice compressor and JPEG [2].

ICER is wavelet-based and progressive: the compressed stream is organized so that reconstructions of successively higher quality can be produced as more of it arrives [1]. It partitions the image into independent error-containment segments so that a lost packet damages only its segment, which mattered because a fair number of MER packets were lost. Its parameters are the wavelet filter, the number of decomposition stages, the number of error-containment segments, the byte quota and a minimum loss parameter.

LOCO exists alongside it because although ICER can compress losslessly, LOCO’s simple predictive approach is several times faster at similar effectiveness [1]. On six Mars Pathfinder test images at 12 bits per pixel, lossless rates averaged 8.78 bits/pixel for ICER, 8.84 for LOCO, 9.10 for JPEG 2000 and 9.36 for the CCSDS Rice compressor; the Rice implementation actually used by Pathfinder was worse still, ranging from 9.38 to 10.61 bits/pixel across the five single-frame images [2]. On the same images companded to 8 bits, ICER and LOCO both averaged 5.02 bits/pixel against 5.22 for JPEG 2000 and 5.52 for CCSDS Rice.

What was actually selected in flight

As of 7 February 2004 [1]:

Compressor	Image type	Images	Combined area, Mpixel	Volume, MB	Average rate, bits/pixel
ICER	regular	5132	2867.9	387.3	1.13
ICER	thumbnail	6075	24.9	3.8	1.27
LOCO	regular, lossless	489	124.5	92.7	6.24
LOCO	reference pixels	1114	36.5	9.2	2.12
none	regular	4	4.2	6.0	12.00

Source: [1], Table 1. Four images in the entire set were returned uncompressed. Lossless LOCO at 6.24 bits/pixel costs about five and a half times the volume of the ICER default, so it was reserved for cases where fidelity was the point [1].

Parameter selection varied by camera and by purpose [1]. Panoramic camera full frames on Spirit used 3 or 4 decomposition stages, wavelet filters A or E, 1 to 20 segments, and averaged 1.12 bits/pixel over 1014 images. Navigation camera full frames used 12 to 20 segments and averaged 0.97 bits/pixel over 235 images, the lowest rate of any full frame category, because navigation images are not science products [1]. Microscopic Imager frames took 24 to 32 segments and averaged 1.34 bits/pixel on Spirit and 2.36 on Opportunity. Hazard camera full frames used 16 to 32 segments, the highest segment counts, because they are the images a lost packet can least afford to damage [1].

Two mechanisms reduce volume before compression runs. Images are companded from 12 to 8 bits per pixel to improve fidelity in dark regions, which is worthwhile because the sensor noise is signal dependent [1]. Every acquired image also produces a 64x64 thumbnail by averaging 16x16 blocks, ICER-compressed to about 650 bytes, which is about 0.005 bits per pixel of the original; thumbnails preview full images that may not be sent for some time. Beyond that, MER sends subframes of interest at full quality with the whole frame at much lower quality, performs pixel averaging, and sends row sums, column sums or histograms alone.

What it cost on the flight processor

Timing for the MER implementation of ICER was measured under VxWorks on a 20 MHz RAD6000 identical to the flight units [2]. Three results hold regardless of the absolute scale: compression is faster when fewer compressed bytes are produced, the time at a bit rate of zero is essentially the cost of the wavelet decomposition alone, and the choice of wavelet filter and the number of decomposition stages have little effect on speed. The published figure gives compression time against output rate for a 1024x1024 test image with one error-containment segment, but its vertical axis is labeled in milliseconds per pixel, which cannot be literal at this image size, so the absolute value is not usable from the open document [2].

The byte quota does not give exact control over output size. ICER checks the quota only between compression of segments of subband bit planes, and does not account for the backlog still held in the segments’ interleaved entropy coders, so the output almost always exceeds the quota when the quality goal is not reached [2]. The overshoot depends on the segment count through two competing effects: more segments mean a larger total coder backlog, but also more frequent quota checks, so overshoot is minimized at a moderate segment count. MER controlled quality with the byte quota alone, setting the minimum loss parameter to zero, and transmitted the entire bitstream rather than truncating to the exact target, because truncation produces small quality variations between segments [1].

Standards

CCSDS publishes two relevant Blue Books: Image Data Compression, CCSDS 122.0-B-2, and Lossless Data Compression, CCSDS 121.0-B-3, the latter being the Rice-based algorithm that appears in the comparison above [5][6]. ICER-3D extends the ICER approach to hyperspectral imagery by compressing across the spectral dimension as well as the spatial ones [3]. No Mars surface vehicle has flown the CCSDS image compression standard; the flight compressors are mission-specific.

How downlink priority is set

Priority on MER is a property of the data product and it is set and revised from the ground. Each communication window command carries the durations for the real-time and recorded data priority tables along with start time, duration, antenna, rates and hardware configuration, and the window executes inside the communications behavior part of the flight software rather than through the sequence engine [4]. Data products carry a priority in their metadata file alongside collection time [8].

Onboard data organization places the highest priority playback data, fault and warning event reports and spacecraft health reports, earliest in a direct-to-Earth pass [4]. That ordering interacts with receiver lockup: lockup takes about 1 minute for high gain antenna rates but varies from pass to pass, so the project must plan a lockup allowance before the valuable playback starts. Too short and sent data is lost; too long and less data fits. The comm window lockup parameter was changed from 1 minute to 3 minutes for downlink rates of 3160 bit/s and higher, and 3 minutes consumes 10 percent of the return capacity of a 30 minute window; it was reduced to 2 minutes for some windows in the later extended missions [4].

Reprioritization is a routine operations activity. UHF reports were generated for every Odyssey pass at low priority in the extended mission by attaching a report sequence to the start of window preparation; to actually return one, data management issued a command raising its priority. Reports were reprioritized every seventh sol so telecom could spot-check relay performance, could be raised individually for interesting passes, and were marked for automatic deletion after 7 sols [4].

Priority also serves fault diagnosis. During the Spirit sol 18 flash anomaly a high priority communication window overrode the 10 bit/s fault default to give 40 or 300 bit/s on the low gain antenna, and the repeating event reports that came down at that rate are what first pointed at the flash memory [4].