Data handling

Onboard storage, file systems, compression, prioritization and the downlink budget that decides which measurements a mission actually keeps.

The problem

A surface vehicle generates more data than it can send. The return link is set by antenna geometry, relay orbiter geometry and buffer allocations, and the cameras and instruments are sized for what the vehicle can sense, not for what the link carries. On the Mars Exploration Rovers, relay through orbiters carried 89 percent of the returned data by the end of the 90-sol primary mission, at an average of about 56 Mb per sol per rover through Odyssey [3]. Data handling is the set of decisions that fit the first quantity into the second: what is stored and where, how hard it is compressed, which products leave first, and how the link itself is scheduled. The flight computer adds a second constraint: the RAD6000 class processors that flew on MER and MSL run at 20 MIPS, so any method has to fit in minutes of processor time and a small memory [21].

What has flown

MER, 2004. Image compression was the main tool, with ICER and LOCO used from the first sols [1]. ICER, a progressive wavelet compressor with segment-based error containment, averaged 1.13 bits per pixel over 5132 images in the first month, against 6.24 bits per pixel for modified LOCO lossless over 489 images [1]. The design rationale and the pre-flight benchmark on Mars Pathfinder images are in [2]. The rovers also sent a 64 by 64 thumbnail of every image at about 650 bytes [1]. Odyssey’s allocation was 120 Mb per rover [3].

Storage faults, MER and MSL. Spirit was effectively lost for three sols after a defect in the file system library, in which deleted-file entries remained in a RAM-resident structure and exhausted the memory needed to mount the volume at boot [4][25]. Curiosity’s A-side computer was recovered in a flight software build that moved the file system from NAND flash into half of the NOR flash, cutting usable volume by about 99 percent, from 2 GB to 64 MB [5]. Mars 2020 inherited and selectively rewrote the MSL data management and file system infrastructure [6]. Perseverance keeps four locations for critical files, none synchronized automatically, so every parameter change must be applied to the backup by ground command [7].

MSL, 2012 onward. Mastcam and MARDI returned most images as lossy JPEG with a lossless option [12]. AEGIS autonomous target selection has been in routine operations since sol 1343, at a cost of a few minutes of plan time [18]. On the ground side, assessing whether the previous drive executed nominally went from a manual task to one that is more than 80 percent automatic, with 13 of 16 reported aspects automatable [19].

Perseverance, 2021 onward. Relay planning precomputes data volume for every overflight at every allowable rate and duration, then filters against pass count, volume and latency limits [29]. LDPC coding was commissioned on the Mars Relay Network in 2021: four MAVEN demonstration overflights returned 70 percent more data than convolutional coding predicts, and one overflight returned 2340.2 Mbits [29]. The Rapid Traverse Campaign planned 94.8 percent of 5,063.4 m autonomously, which moved the saved plan time into driving [24].

EO-1, 2003 onward. The Autonomous Sciencecraft Experiment ran onboard science analysis, planning and execution on a flight computer with tight memory and processor limits [20]. It is the main flown precedent for onboard data selection outside the rovers.

What is being tried now

Standard compression. The CCSDS Blue Books standardize an image compressor and a Rice-based lossless compressor [8][9]. ICER-3D extends the wavelet approach to hyperspectral cubes and has been shown as a ground analysis, not as flight software [11]. Status: the standards are published; ICER-3D is ground only.

Onboard prioritization by content. OASIS proposed ranking navigation images for downlink by analyzing science content on board [14], with rock-based ranking by target signature, novelty or representative sampling [15] and caption-based ranking against scientist text queries [16]; SCOTI is the captioning version, tested in a simulator [16]. Status: presentations, ground tests and simulation. A recent framework, SYNOPSIS, is a C library that runs science-autonomy algorithms, scores each observation for science utility and diversity, and ranks products by a maximum marginal relevance rule. It was integrated into cFS and F Prime flight software and exercised in a simulator on archived, already-downlinked MSL data; tests on flight-like processors are listed as future work [27]. Autonomous detection of microscopy features and summarization reached average compression ratios of 1888 and 2247 on a flight-like processor in analog testing [17].

Faster processors for perception-driven selection. The AEGIS rock-detection front end takes 10 to 15 minutes on the 20 MIPS RAD6000, and a 64-core ground-test port reduced it to 2.9 s on one core [21]. Status: ground test.

Command-to-product tracing. Ground tooling associates each downlinked product with the uplink command that produced it, using an identifier embedded in product metadata; this was demonstrated in a 2001 field test, not on a flown vehicle [13].

Delay-tolerant file transfer. CFDP defines acknowledged and unacknowledged file delivery over long-delay links [10]. Delay-tolerant networking was simulated for Mars relay scenarios [22] and certified in ground test for the International Space Station [23]. Neither has been reported as a surface-rover data path.

Relay network growth. A notional next-generation Mars relay network of three orbiters at about 6000 km with directional X-band proximity links is a planning proposal with no link budget [28]. The Mars Sample Return campaign concept uses a relay allocation tool for three co-located assets near Jezero, modeled under assumed inputs [30]. The International Mars Ice Mapper study assesses relay and direct-to-Earth communications trades as a concept, with computed link budgets only [31].

The numbers

Quantity	Value	Source
ICER mean rate, MER first month	1.13 bits/pixel over 5132 images	[1]
Modified LOCO lossless mean rate	6.24 bits/pixel over 489 images	[1]
ICER lossless vs LOCO, six Pancam frames	4.50 vs 4.53 bits/pixel	[1]
MER thumbnail size	about 650 bytes	[1]
MER relay share of returned data	89 percent at sol 90	[3]
MER Odyssey allocation	120 Mb per rover	[3]
Curiosity file system after recovery	64 MB, down from 2 GB	[5]
Relay network return, late 2015	561 Mb per day over 446 overflights in three months	[26]
LDPC gain on MAVEN overflights	70 percent over convolutional prediction	[29]
Single Perseverance-MAVEN overflight	2340.2 Mbits	[29]
AEGIS rock detection, RAD6000 vs 64-core port	10 to 15 minutes vs 2.9 s	[21]
Rapid Traverse autonomy	94.8 percent of 5,063.4 m	[24]

Interactions with fault protection

On a solar-powered rover the protection backbone is the autonomous shutdown and wakeup cycle rather than a safe mode [25]. File system faults in [4] and [5] show how a storage defect propagates into the reset behavior that cycle depends on.

What is not established

Compression statistics for MER cover the first month of operations on early-mission terrain and reflect operator-set byte quotas, not image content [1]. The ICER benchmark rests on six Mars Pathfinder images, and the authors note that PSNR is a poor proxy for the scientists’ ranking of filters [2]. The flight rate figures therefore do not transfer to other cameras or terrain. Relay volumes are specific to the Odyssey and Global Surveyor orbits and do not generalize to other constellations [3]. The relay snapshot is read from a single figure [26], and the LDPC gain is measured against predicted, not paired, convolutional passes [29]. Most onboard prioritization methods have not flown: the caption, novelty and signature rankings are presentation, simulation or ground-test results [15][16], the microscopy summarization was tested in analog conditions [17], and SYNOPSIS reports no onboard runtime, memory or power figures [27]. The relay concept papers contain assumptions and computed budgets, not measurements [28][30][31]. The corpus holds no flown comparison of ground-directed and onboard prioritization on the same downlink budget, and no source cited here covers storage reliability under radiation beyond the specific anomalies above.