Human interaction

Operating a machine across a light-time delay: teleoperation, predictive display, supervisory control, and the ground tooling that plans a sol.

The problem

A planetary robot cannot be driven in real time. Round trip light time to a planetary surface is at least 6 minutes and up to 42 minutes in non-optimal geometry [11], and a bilateral control loop that closes through the link inherits that delay and can destabilize [6][7]. The Earth-Moon loop averages 2.6 s and the minimum Earth-Mars loop is 188 s [2]. With a delay in the loop, operators fall back to open-loop moves followed by a wait, and the extra completion time is proportional to the number of waits [1]. The problem predates space: the 1978 review of undersea teleoperators treats supervisory control, where the operator specifies goals and the machine closes the fast loop, as the response to a delay too long for direct force reflection [1].

The subsystem therefore has two jobs: keep a human useful when the loop delay approaches or exceeds the dynamics being controlled, and give a small ground team the software to plan, rehearse and uplink a day of commands inside a fixed shift.

What has flown

Lunokhod-1 (1970). Operated from the ground with five go commands and one stop, over narrow-band television at one frame per 3 to 20 s, by a five-person crew [5].

ROTEX (Spacelab D-2, 1993). Predictive display and task-level programming: the operator worked on a simulated workspace updated from real images [8].

ETS-VII (1999). The arm closed a bilateral loop through a 6 to 7 s round trip, and kinesthetic force feedback improved task performance at that delay [6]. GETEX drove the same arm from Germany with a predictive display [7].

MER ground tooling (2004 onward). RSVP simulated a sequence with the same flight software that runs on the rover, and the command load grew from Pathfinder’s roughly 100 commands per sol to more than 1000 for MER [13]. Persistence and data distribution were moved off site to serve a distributed team [17].

ISS crew-controlled surface telerobotics. Three Expedition 36 astronauts drove the NASA Ames K10 rover in an analog yard over three sessions. Rover utilization was 65 to 80 percent, Bedford workload ratings were 1 to 3, and situation awareness answers were correct 93, 80 and 81 percent of the time in sessions 1 to 3, with 1 to 2 s command latency through Ku-band that the authors state is not representative of a lunar far-side relay [23]. Three subjects preclude statistics [23].

KONTUR-2 (2015 onward). A two degree of freedom joystick on the ISS was coupled to ground robots under Time Domain Passivity Control over measured link delay, jitter and packet loss [9][10]. The METERON experiments flew supervised autonomy with a humanoid on the ground, reaching a 20 to 30 ms round trip on direct S-band line of sight at the price of an 8 minute window per pass [11].

Perseverance (2021 onward). COCPIT, a web application with a Redis transaction store and websocket synchronization, replaced MSLICE and builds on the Playbook tool from NASA Ames [14].

What is being tried now

Supervised autonomy interfaces. The operator issues goals and the remote side closes the fast loop; Ferrell and Sheridan formulated this in 1967 [2], and the ISS-to-ground work above is its flown test [11]. Flown in analog form only: operation of a ground robot from orbit.

Low-latency telerobotics from lunar orbit. A Gateway-based concept would have a crew command a far side rover at 0.3 s round trip [21]. In a ground experiment with injected latency, mean time to discovery rose at 2.6 s versus 0.3 s delay, and 5 frames/s was the video threshold; the text gives a 150 percent penalty and the figure caption 50 percent [21]. Ground tested only, with an unstated operator count.

Learned delay compensation. A 2024 simulation trains a deep reinforcement learning controller for a free-floating manipulator under constant and random delay [12]. Simulation only; it has not flown.

Gesture and wearable command. The JPL BioSleeve decodes up to 16 hand gestures at 10 Hz from 500 ms windows of forearm EMG and maps them to four autonomy levels from mimicry to stored procedures [22]. It is a laboratory proof of principle with no accuracy figure published [22].

Interface efficiency in field operation. A classification of ATHLETE drive command terminations in a 2010 lava flow campaign found that heading recommand and jitter caused more than half, so the traverse rate stayed rarely above 1.5 km/h against a 5 km/h design speed [24]. The cause was the Wii Nunchuk joystick scheme, and the remedies are proposals [24].

Ground tooling and planning. Cloud workflow services moved MER tactical operations and MSL downlink image processing off site [17][19]. Ground-side automated planning prototypes exist but have not flown [20].

The numbers

Quantity	Value	Source
Earth-Moon loop delay	2.6 s	[2]
Earth-Mars minimum loop delay	188 s	[2]
Mars round trip light time	6 to 42 min	[11]
ETS-VII round trip	6 to 7 s	[6]
ISS direct S-band round trip	20 to 30 ms	[11]
ISS Ku-band K10 command latency	1 to 2 s	[23]
Lunokhod-1 television	1 frame per 3 to 20 s	[5]
MER commands per sol	more than 1000	[13]
MSL restricted sols	41 percent of the mission	[15]
3-sol plan versus 1-sol plan	15 percent less computer time, 11 percent less energy	[15]

The binding constraint on rovers is the tactical timeline. MSL’s productivity study, drawn from interviews and three campaigns, ranks ground in the loop for target selection and drive planning as the largest single factor [15]. Robotic arm activities were planned on about half of MSL’s first 200 sols [16]. MER used a one-sol tactical timeline with one science team leader, which Cassini’s multi-investigator planning does not share [18].

Early field work set the pattern. An MIT simulation in 1967 compared rate control under long delay with supervisory control [3]. An MSFC mountaintop test drove a vehicle 6 km away: a 3.7 km sortie took 5 hours, and at 8 s of delay self-paced driving produced about 25 percent more course departures than stop-and-go driving [4].

What is not established

No flown system compensates delay by learning a model of it online, and no experiment in this set reports a controller adapting to a link that degrades mid-session [12][9]. Telepresence from lunar orbit is not flown; the evidence is ISS-to-ground analog work at latencies below the far-side relay budget [23][21]. The MSL productivity study attributes the losses mostly to process rather than tooling, and the pre-landing and post-landing figures for the later Perseverance tools are not reported in the held sources [15][14]. The ATHLETE analysis could not characterize behavior under communication delay because remote operation was rare in the campaign [24].