Onboard Science Autonomy

Onboard science autonomy is the selection of what to measure, or what to keep, by software running on the vehicle rather than by an operator on the ground. It exists because the observation opportunity and the uplink cycle do not coincide: a rover that finishes a drive after the last uplink of the sol is standing in front of unmeasured rock until the next plan arrives, and an orbiter that images a target does not know whether the image is worth its downlink allocation [3][1].

Autonomous Sciencecraft Experiment, EO-1

ASE flew on Earth Observing 1, launched 21 November 2000 into a 705 km sun-synchronous orbit at 98.7 degrees inclination with 16 day repeat tracks [1]. Each scene from the Advanced Land Imager, Hyperion and Atmospheric Corrector produced over 20 Gbit onto the solid state recorder; Hyperion resolves 220 bands from 0.4 to 2.5 micrometers at 30 m ground sample over a 7.5 x 42 km footprint.

The autonomy software is three layers [1]:

Layer	Component	Time scale
Mission planning	CASPER, Continuous Activity Scheduling Planning Execution and Replanning	tens of minutes per planning cycle
Execution	SCL, Spacecraft Command Language, plus an independent fault protection layer	several seconds
Flight software	EO-1 flight software, low level control	below SCL

The closed loop is: CASPER plans periodic Hyperion imaging of a target list; onboard science algorithms analyze the image; a detection triggers downlink of that image, and a non-detection instead posts a goal to acquire the next highest priority target, which CASPER merges into the running plan [1]. Conflict repair is explicit: given a projected solid state recorder memory conflict, CASPER enumerates repair methods, selects one, and picks an activity to move or delete from those that subscribe to the contended resource [2]. The classifier set covers thermal anomaly detection from infrared spectral peaks, cloud detection from thresholds on six bands, flood scene classification from spectral ratios, change detection, trainable feature detection and Gabor filter anomaly detection [1]. Trigger-based acquisition reduces downlinked data volume.

What it cost on a Mongoose M5

EO-1 carries two Mongoose M5 processors, one for command and data handling and one inside the Wideband Advanced Recorder Processor. Each runs at 12 MHz for about 8 MIPS with 256 MB of RAM under VxWorks, and ASE runs on the WARP M5 so that it cannot disturb the main spacecraft processor [1]. Against a contemporary desktop at 2000 to 3000 MIPS, 5 to 20 MIPS was the typical spacecraft figure.

The allocation ASE actually received was narrower than the part [2]:

Resource	EO-1 total	Allocated to ASE
Processor	8 MIPS Mongoose V, MIPS R3000	4 MIPS, shared between SCL, CASPER and science processing
RAM	256 MB	40 MB for CASPER code and heap, of which a 32 MB heap; 32 MB was already the original flight software
Filesystem	none in the original design	8 MB ramdisk, 1 MB of it for CASPER log files
Uplink	2 kbit/s to 2 Mbit/s, 10 to 15 minute contacts	CASPER image uplinked compressed

Getting CASPER inside that envelope was an exercise in subtraction, against a planning cycle whose duration is on the order of tens of minutes [1]. The image began at 12 MB against 8 MB of code and heap space on the testbed; removing software components took about 3 MB off, moving to a reduced C++ standard library implementation took about another 3 MB, and addressing code duplication brought the final image to 5 MB [2]. For uplink, a 9 MB image was compressed to 1.5 MB and the existing flight software was patched to decompress it into memory, because contacts are short and infrequent.

The planning horizon was cut for the same reason. A single science observation expands to over 78 activities in the CASPER model, so a week of EO-1 operations, over 100 science observations and 50 ground contacts, would be over 7800 activities and exhaust the 32 MB heap; scheduling cost also grows with goal count through goal interaction [2]. The answer was a hierarchical horizon: abstract planning across the week enforcing constraints such as one observation per orbit, detailed planning only across the next 24 hours, with past observations dropped out of the window automatically.

ASE completed over 5000 data collection events against an original success criterion of 1000 [1].

AEGIS, MER, MSL and Mars 2020

AEGIS, Autonomous Exploration for Gathering Increased Science, selects instrument targets from an onboard image. It first flew on Opportunity in 2010 for Pancam target selection, then on Curiosity for ChemCam in two roles, autonomous target selection in NavCam images and autonomous pointing refinement in RMI images, then on Perseverance for SuperCam [4][5].

The pipeline is: acquire a NavCam stereo source image; run the Rockster segmentation algorithm, which is edge detection followed by flood fill and morphological operations to close contours; evaluate each candidate on properties including pixel count, a stereo 3D size estimate and the semi-major and semi-minor axes of a fitted ellipse; filter and rank under an operator-supplied scene profile; check the chosen pointing against a sun-safety exclusion zone and against an onboard model of rover articulation for collision; place a visual target tracking frame on the target; and command the spectrometer raster [4][5]. Two safety checks exist because ChemCam’s 15 cm primary mirror can focus sunlight into the instrument and damage it, and because a laser shot at the rover body would trip a collision fault; the collision check projects the current position of all rover hardware into the NavCam image and masks those pixels [3]. To the end of the period reported, AEGIS on Curiosity had never selected a target on the rover body.

What it cost on the Rover Compute Element

Run duration on the Curiosity flight RCE scales with the number of targets found, because each target needs its own stereo computation and feature extraction; through sol 2830 AEGIS visually identified a target set in under 500 s in the usual case, on the spacecraft’s RAD750 [4]. Point stereo alone costs on the order of a second per target plus a few seconds of setup, and a source image can yield up to 255 targets, so stereo dominates the long runs. The same 133 MHz RAD750 takes 47 s for a visual odometry step [8].

Measured performance

Vehicle	Instrument	First operational use	Flight record
Opportunity	Pancam	2010	first flight demonstration of autonomous target selection [4]
Curiosity	ChemCam	sol 1343, planned 13 May 2016	54 runs selecting 59 targets between sols 1343 and 1663, of which 52 were post-drive NavCam runs and 2 were RMI pointing refinements [3]
Perseverance	SuperCam	sol 449, 24 May 2022, expanded capability from sol 698	170 targets selected by 25 September 2024 [5]

No AEGIS run on Curiosity failed to find targets or produced a poor quality observation: in every case the system detected feasible targets in the source image, ranked them, selected the commanded number, and obtained the ChemCam measurement [3]. After the sol 1400 parameter update, every LIBS point landed on outcrop for 44 of the 47 targets measured; modeled blind targeting of the same scenes would have managed that for only 10. The three residual cases are partial: two measurements fell on loose float rock rather than bedrock in fields of view holding little or no outcrop, and in one case three of nine LIBS points fell on sand covering the outcrop while the other six struck the outcrop itself. AEGIS was used on 53 percent of plans containing a drive, 36 percent of single-sol plans and 64 percent of multi-sol plans [3]. Target selection takes 150 to 450 seconds on the rover’s RAD750 flight computer, scaling nearly linearly with the number of targets found.

Pointing refinement recovers observations rather than creating them, and had been used only twice by sol 1663. On sols 1424 and 1463 the ground-selected pointing missed a bright vein; AEGIS recognized the offset and corrected it, saving the observation [3]. The mast points to about 2 mrad, the RMI field of view is 19 mrad across and the 3x3 LIBS grid is spaced at 1 mrad, so the residual error matters for features that small [3]. A second attempt in a later plan would otherwise have been required.

Throughput at instrument level changed at rollout. ChemCam laser shot rate rose from about 255 shots per sol before the sol 1343 AEGIS rollout to about 311 shots per sol after it [4]. Target class distribution over 281 AEGIS runs since rollout differs from the notional blind-target distribution over 215 cases, with the dominant class taking 75 to 77 percent of AEGIS runs against 46 to 47 percent notionally blind.

On Perseverance the measure is share of the science plan. Over the 83 sol drive campaign from sol 1092 to 1175, covering 2.44 km from Bunsen Peak to Bright Angel, the team prepared 48 uplink plans and used AEGIS in 45 of them; AEGIS supplied half or more of the LIBS-VISIR targets in 35 plans and all of them in 12; of 112 targets measured in the period, 56 were human-selected and 56 were AEGIS-selected [5].

Failure modes

AEGIS fails by declining to act. If every candidate is filtered out by the scene profile, or falls outside the SuperCam laser range, or carries a poor stereo range estimate, no coordinate frame is placed and the observation is not attempted [5]. Curiosity has the same residual: runs that found no targets are limited to a mast fault that pointed the source image at the sky, an image centered on the rover’s own arm, and one entry into a new geological unit where nothing in the scene met the science suitability settings [4]. The documented worst case is geometric rather than algorithmic: an autonomous drive ended with the source image pointing west on a slope that tilted the camera upward enough to put the sun in the NavCam field of view, which left few targets found and all of them filtered for poor range confidence, and the fix was an operational constraint on source image azimuth for the rest of that campaign [5].

Scene profile choice moves the failure mode rather than removing it. The Perseverance default favors light-toned outcrop because in-place rock is usually lighter than the surrounding regolith, which means that in scenes with sparse, small, dark or shadowed float rocks it will select light-toned regolith that the Rockster edge finder has enclosed in a trace; rougher regolith texture makes this more likely because it yields more enclosable edges [5].

Pointing precision is the standing physical limit on the whole capability. Motor backlash, motor control error, resolver accuracy, thermal effects and rover settling or shifting give a targeting uncertainty of about 6 mrad, reducible to about 2 mrad by commanded compensation, against LIBS rasters whose points are spaced 1 to 2 mrad apart and which span 2 to 20 mrad [4]. Mars 2020 closes the loop with vision and resolver feedback, designed for 1 mrad, and models backlash disturbance to guide the actuator slew along a path where backlash is controlled.

For a planner, the failure mode is state divergence. On EO-1 the spacecraft consumed more solid state recorder memory blocks than planned when data collection ran several seconds long, so the telemetered free block count disagreed with CASPER’s model, which is the class of defect that onboard planner telemetry exists to expose: iteration counters, elapsed time per repair, stack margin, and pre-repair and post-repair conflict counts [2].

Scheduling as the adjacent problem

The Mars 2020 onboard scheduler addresses the cost of the conservatism that ground scheduling requires. Activities are sequenced with conservative durations so that overruns almost never cancel a dependent activity, targeted at under one plan in 100; a study of Mars Science Laboratory campaigns found submaster sequences finishing on average more than 20 percent early, which leaves the rover awake and idle until the next fixed start time [6]. The onboard scheduler is priority-first and does not backtrack: for each activity it finds valid temporal intervals, accommodates required preheat and maintenance heating, and generates a wake and sleep schedule, and it will not modify an already scheduled activity. On the ground side, the Copilot scheduler has generated and scheduled Perseverance’s sleep and heating activities alongside the user-supplied ones since the February 2021 landing, sharing its core algorithms with the onboard planner, with the explainable scheduling tool Crosscheck over it so that operators can see how the schedule was constructed [7]. The power a heating activity draws depends on temperature and temperature varies through the sol, so the energy cost of an activity and its heaters is not constant across the plan horizon; state of charge rises only when the CPU is off and nothing else is drawing power [7].