Simulation Fidelity for Software

A simulator used to verify flight software is not asked to be physically correct. It is asked to produce, at the software’s interfaces, a signal the software cannot distinguish from the flight signal in the respects the software depends on. Where it fails at that, the failure is specific and mostly documented: force, imagery and time [1][2][3]. The terramechanics and rendering fidelity questions belong to simulation and digital twins. This page covers what those gaps do to the software under test.

Fidelity is chosen per interface, not per simulator

MSL’s workstation simulator models every hardware behavior the flight software can observe, as finite state machines, integrating differential equations where the dynamics demand it [3]. The deliberate exception is content: for several science instruments it produces data that is correct in format and not in content, because the flight software depends on the format and not on the content. The rule generalizes, and it bounds what these venues can find: nothing downstream of a value the simulator did not make right [1][3].

Mars 2020’s Surface System Development Environment makes the choice explicit by keeping one interface to the flight software and three interchangeable backends behind it: a deterministic abstraction layer for force and motion, a terrain simulator with slip modeling, visual odometry, pose tracking and rendered imagery, and a motor server driving real testbed hardware over TCP/IP [2], the physical venues being the scarce resource the environment exists to conserve. The same test can be raised or lowered in fidelity without being rewritten, which is what makes the fidelity a per-run choice.

Contact force is modeled as a plane and a margin

The force interfaces are where the divergence is largest and best documented.

SSim does not model the physics of a drill bit in rock. It uses force feedback averaged from past flight data with a margin allocation, and bounds terrain contact forces by an assumed hardness with margin [1]. For the operational purpose, checking that a commanded sequence will complete, that is sufficient; for predicting the force a specific rock will produce, it is not, and the paper says so.

The Mars 2020 abstraction layer computes a simulated force as a function of the end-effector’s penetration into a parameterized plane, then converts that force into the strain gauge voltages the flight software actually reads [2]. Contact is a plane and a penetration depth, and the flight software sees plausible gauge counts. Docking simulation in the surface simulator used for operations does the same thing, modeling the dock as a rigid flat plane and populating the reaction forces into simulated strain gauge channels; it does not capture the post and cone contact physics, and is described as effective for finding commanding and sequencing errors rather than for predicting loads.

The consequence for verification is a clean split. Sequencing errors, state errors and command ordering errors are found in simulation. Contact dynamics, limit cycling and force sensor behavior in contact are found on hardware: thermal testing of the Perseverance arm found a limit-cycling condition during docking that required an algorithm change, which no planar contact model would have produced [8]. Docking was also exercised more than 1200 times on a hardware testbed before flight.

Sensor models exist to keep the algorithms fed

The terrain simulator renders camera imagery for visual odometry, stereo ranging and map generation, and applies artificial visual texture to the terrain to guarantee the image processing has features to find [2]. The simulated world is deliberately easier to see than the real one, because the purpose is to exercise the pipeline rather than to test whether it finds features at all. Feature starvation over bland terrain, which is the dominant flight failure mode of visual odometry, is therefore not something this venue can find.

The slip model behind the imagery is described as approximated rather than as terramechanics, and the wheel drop calculation as approximated from a moving window of poses [2]. Both are adequate for the flight software’s decision logic and inadequate as predictions of vehicle behavior.

Time is the fidelity that gets traded first

Every software-only venue in the record buys throughput by abandoning real time. The MSL workstation simulator runs 2 to 10 times faster than real time depending on scenario complexity, and the trade is stated plainly: minimizing idle time and running fast costs real-time fidelity [3]. The Mars 2020 build runs 6 times faster than real time [2]. SSim executes rover behaviors up to 1000 times faster than the flight compute element [1] and does not reproduce the real-time constraints or the determinism of the flight software; it is single-threaded for repeatability where the flight software is not. The terrain simulator runs at 8 Hz of simulated time [2].

Two timing gaps follow. The first is bus response: the simulated PCI bus cannot respond on the microsecond timescale real hardware does, and its responses have to be coordinated on the simulator’s time slices [2]. The second is concurrency: a single-threaded deterministic simulator cannot produce the interleavings a preemptive multi-tasking flight software will produce on the vehicle. The sol 516 image loss is that class of defect: a background image compression task lost CPU to higher-priority communication tasks during a communication pass, and the automatic camera power off triggered before the readout completed, losing the data [7].

The workstation venues are not fidelity-free on timing, though. MSL’s flight software internal testing describes the workstation simulator running at 512 Hz, a higher rate than the real hardware testbeds, over a VxWorks simulator [4], and the avionics simulator maintains a deterministic event-driven scheduler with multiple rate groups: 1 kHz dynamics, 200 Hz bus events, a 64 Hz timing interrupt and 400 Hz IMU [3]. The scheduling structure is reproduced; the latency is not.

What no simulator injects

Real hardware failures are not injected into hardware-in-the-loop venues, for the obvious reason that damaging the testbed is expensive [2]. Fault injection is a software venue capability instead: the MSL workstation simulator injects faults either at the functional level or by setting individual bits, and supports pause, resume and single step around the injection [3]. So faults are exercised in the venue with the least physical fidelity, which puts the burden on the fault model being right [3][6].

The onboard planner verification states the limit that follows from all of this: the problem space the planner will meet in flight is unbounded and had to be constrained to a finite set of testable units, and neither unknown deviations from predicted activity execution nor the behavioral uncertainty of flexible execution could be covered [5]. Its testable scenarios were drawn from hundreds of sols of real planning rather than generated, which is a statement about what the team trusted to be representative, and the same campaign leaned on the workstation simulator for volume and on hardware venues only where thermal fidelity was required.

The fidelity that matters is not always physical

The defect study across seven spacecraft found that outdated or missing procedures were involved in one fifth of 199 critical post-launch anomalies, and that requirements changes were dominated by new requirements for rare events and for compensating hardware degradation rather than by corrections to wrong requirements [6]. Neither class is reachable by raising the fidelity of a contact model. The same study’s 311 MER developmental problem reports were useful in a way no simulator intends: the reports where testers believed the software was wrong when it was correct predicted which features would confuse operators, and drove training and documentation before launch [6].

Summary of the known divergences

Interface	What the simulator does	What it therefore cannot find
Contact force	penetration into a parameterized plane converted to strain gauge voltages [2]; historical averages with margin [1]	contact dynamics, limit cycling, load prediction [8]
Camera imagery	rendered with artificial texture added to guarantee features [2]	feature starvation and the visual odometry failures that follow from it [1]
Slip and wheel drop	approximated models, not terramechanics [2]	mobility performance, sinkage, embedding
Concurrency	single-threaded deterministic execution [1]	priority inversion under load, as on sol 516 [7]
Timing	2 to 1000 times real time, deterministic single-threaded execution [1][3]	interrupt latency, race conditions, priority inversion under load
Bus response	coordinated on simulator time slices [2]	microsecond-scale hardware response behavior
Instrument data	correct format, sometimes not correct content [3]	anything downstream of instrument data values
Hardware faults	injected in software only, functionally or bitwise [3]	failure modes the fault model does not contain [6]