Skip to content
Space Robotics

Rollin' Justin

Program pages DLR: Rollin' Justin

Rollin' Justin servicing a Smart Payload Unit in the Solex analogue terrain during the METERON SUPVIS Justin campaign, wearing the experiment patch on its chest. The two seven-degree-of-freedom lightweight arms terminate in twelve-degree-of-freedom hands, the right one open to show the finger drives; the head carries the stereo cameras, and the four wheel units of the omnidirectional platform are in the extended footprint. This is the ground robot that ISS crew commanded through a tablet DLR. CC BY 3.0 [33].

Overview

Section titled “Overview”

Rollin’ Justin is a wheeled humanoid built by the DLR Institute of Robotics and Mechatronics. It is the ground element of two ISS telerobotics experiments that between them tested the two ends of the command spectrum: KONTUR-2, in which cosmonauts flew a force-feedback joystick and drove ground robots as avatars, and METERON SUPVIS Justin, in which astronauts supervised the robot as a co-worker through a tablet. The two campaigns together covered teleoperation of robots on Earth from orbit across four years, 2015 to 2018 [2].

The upper body was an existing two-arm humanoid; the 2009 addition of a mobile platform made it Rollin’ Justin [1]. The upper body carries 43 degrees of freedom and the platform adds 10 to 12.

Specifications

Section titled “Specifications”
ParameterValue
Total mass199 kg
Upper body massabout 50 kg
Platform massabout 150 kg
Upper body degrees of freedom43
Arm span3000 mm
Lift capacity20 kg
Torso travel600 mm forward, 300 mm back
Battery48 V nominal Li-polymer, 40 Ah
Average power drawabout 600 W
Enduranceabout 3 h
Top speed1.5 m/s
Upper body control rate1 kHz

Source: [1].

The DLR program page states figures that differ from the platform design paper: 51 degrees of freedom for the whole system, distributed as 8 in the platform, 7 in each arm, 12 in each hand, 2 in the neck and 3 in the torso; a standing height of 1.91 m; a working range from floor level to 2.7 m; a maximum speed of 2 m/s; and an operating time exceeding 60 minutes [33]. Nominal load capacity of 20 kg agrees between the two [1], [33].

Mission profile

Section titled “Mission profile”
ParameterValueSource
Roleground element of ISS telerobotics experiments, commanded from orbit
KONTUR-2cosmonauts driving ground robots through a force-feedback joystick
METERON SUPVIS Justinastronauts supervising the robot through a tablet
Campaign span2015 to 2018
First public presentation2008[33]
Current roleresearch platform in service robotics[33]

Rows with no marker are from [2].

Mobility

Section titled “Mobility”

The platform is a four-wheel omnidirectional base with a variable footprint. Each wheel unit carries an independent brushless DC hub motor rated at 30 N m driving a steel rim with a 5 mm solid rubber tire of 216 mm diameter, giving 1.5 m/s (5.4 km/h) [1]. Steering is a Harmonic Drive FHAC-mini brushless DC motor and 100:1 harmonic gearbox reaching 360 deg/s and 28 N m, run by an Elmo Whistle digital servo-drive with a Renishaw RE22S absolute encoder reading the gearbox output. Rollin’ Justin passes steps up to 40 mm and climbs ramps to 17 degrees [1].

The footprint changes because manipulation and locomotion want different geometry. Extended, the wheelbase is 985 by 815 mm, which with a center of gravity about 260 mm above the ground is enough to prevent tipping during an emergency stop at full speed with the upper body bent forward; retracted it is 685 by 515 mm, which lets the platform go anywhere a wheelchair can go [1]. Overall dimensions run from 1220 by 1052 mm to 812 by 644 mm, with height varying between 658 and 728 mm on the suspension and 693 mm in the neutral state. The extension mechanism is passive: a parallel linkage moves each wheel horizontally without vertical displacement, actuated by the wheel motors themselves rather than by dedicated actuators, and held by a toothed-rack lock driven by a two-position servo. Each leg carries a spring damper allowing 35 mm of deflection about equilibrium so that four wheels stay in contact on uneven ground and doorsteps are crossed gently [1]. For SUPVIS Justin this platform is what let the upper body navigate the Solex ground environment and carry tools between work sites [2].

Torques on the last torso axis range from 248 N m static to 1800 N m under load, which is what set the stability requirement on the base [1].

Power and energy

Section titled “Power and energy”

The battery is a 48 V nominal lithium-polymer pack of 13 cells in series at 3.7 V each, 40 Ah capacity, with a maximum discharge current of 200 A and a maximum charge current of 80 A [1]. Terminal voltage swings between 40 and 54 V with state of charge. Against an average whole-robot draw of about 600 W the pack runs for about three hours, and recharges in one hour at maximum charge current [1]. A balancing and safety board monitors and corrects voltage, current and temperature per cell, and the pack sits behind a stainless protective cover because of the energy density.

Six separately fused supply lines distribute 48 V, three rails of 24 V, 12 V and 5 V, with current measured per line and displayed on a status panel; the supply circuit handles 3.7 kW continuous [1]. Because the platform runs untethered a wireless emergency stop is required: a radio and infrared receiver drives a current loop carrying an OK signal to the motor controllers and supplies, with the transmitter sending on both 433 MHz and 868 MHz and over infrared so that a single disturbed path does not defeat it, and a receiver timeout of 200 ms. Onboard battery endurance is what set the session length available for ground work during the ISS campaigns [2]. The design budget allocated 1 kW to actuation plus at least two computers, with 15 to 50 W per hand, 50 to 150 W per arm and 100 to 250 W for torso and head [1].

Thermal

Section titled “Thermal”

No thermal control system is published for Rollin’ Justin itself; several fans cool the platform interior [1]. The flight hardware of KONTUR-2, the joystick, is the part with a documented thermal design. It is passively cooled to two constraints: the housing must never exceed 40 C, because the cosmonaut touches it, and the electronics must not overheat [3]. Heat generating parts are therefore placed at maximum distance from touchable outer surfaces and thermally tied only to the bottom plate, with space-qualified gap pads and graphite foils conducting heat into the mechanical structure and suppressing hot spots, optimized against finite-element models. The model was built in ANSYS 14.0 against an ISS ambient of 28 C or below, complete insulation between joystick and ISS structure, heat flows carried with 10 percent margin, an electroplated chromium housing at emissivity 0.1 and a black anodized aluminum adapter plate at 0.82, and convection from the 0.05 m/s ISS supply airflow near the unit [9]. Dissipation by state is about 1.9 W from the DC/DC converter and about 5.0 W from the microcontroller module in every state, with the motor control modules adding about 3.5 W in idle and a load-dependent figure in operation.

Because real motor load depends on the task and on how the cosmonaut handles the joystick, a software temperature control system reads nine calibrated sensors at 0.1 Hz, intervenes automatically, and gates the operator’s state changes on a temperature criterion, dropping to standby when the unit is too hot [3]. An OpHalf state permanently halves the commanded handle force to extend running time before the housing limit is reached, against a required operating time of 30 minutes.

Compute and avionics

Section titled “Compute and avionics”

Four MiniITX standard PCs with dual-core processors sit in the platform for control, communication and peripheral I/O, linked by Gigabit Ethernet through an eight-port switch, with a second eight-port switch for the video and PMD cameras and a Wireless-LAN bridge as the uplink to external control [1]. About 50 liters of chassis volume holds the battery, power supply, computers and infrastructure.

Bus choice follows the component rather than the other way round, because the platform was built from off-the-shelf parts selected on size and function [1]. The hub motor wheels run CAN 2.0B at a 16 ms command rate; the Whistle steering servo drives run CANopen at 4 ms; absolute encoders for steering angle and leg extension report over SSI; the arms, torso and head run the SERCOS real-time bus. The upper body control loop runs at 1 kHz [1].

The KONTUR-2 joystick avionics are a separate design point, built to ISS rules. Safety-critical parts use highly reliable components while computing and motor electronics are military, industrial or automotive grade [3]. The microcontroller module is a commercial part with a processor and an Ethernet switch, sized for the force feedback controller, the operational state machine, error handling, I/O, temperature supervision and communication with the motor modules and the Portable Control Computer. Motor control modules speak EtherCAT to it. Brushless DC motors were chosen both for ripple-free constant force and to avoid carbon brush wear polluting cabin air. Input power is 23 to 29 V so the nominal 28 V Russian segment supply connects directly, behind a power switch, EMC filter and two-stage overcurrent protection whose current limiter also soft-starts to bound inrush [3].

Software runs VxWorks 6.9 with two boot paths against radiation-corrupted memory: bootloader and kernel come from the first SD card partition by default with an alternative pair on flash, selected on CRC32 checksums held in EEPROM [3]. The application is startable from SD card, FTP folder or flash in that order, verified by Simple File Verification and revalidated every minute [3], [8]. Above three kernel-space drivers sits the HIROSCO component framework, which manages access over ECSS PUS and exposes a TCP/IP server for monitoring and control.

Autonomy

Section titled “Autonomy”

SUPVIS Justin treats the robot as a co-worker: the robot’s own intelligence plans and executes low-level tasks while the astronaut supervises, issuing high-level commands and checking status and progress [2]. Whole-body control gives compliant interaction with the environment, and reasoning and planning are implemented as Action Templates. For each known object a template holds the properties and preconditions of every action available on that object together with the geometric procedure for carrying it out, so joining templates lets the robot plan and execute a complex task from a single astronaut command.

Because models of known objects can be loaded in advance, the robot’s computational load is eased and the task set is bounded, which is what makes the approach suited to a planetary surface habitat where the assets were sent from Earth and are therefore known [2]. The architecture is built so that communication delay and jitter arising from distance and disturbance do not gate the task, because the astronaut commands at task level rather than at joint level [4].

The command list a template set generates can be large. A Mission Control Center on Earth prunes it context-specifically with symbolic and geometric filters, steering the astronaut toward task completion without taking the command away [2].

Delay is the reason the two arrangements exist. Ferrell’s 1964 experiments showed that operators facing a transmission delay adopt a move-and-wait strategy unprompted, so completion time grows linearly with delay at a slope set by the number of pauses for feedback [12]. A direct haptic loop is therefore only practical at delays far below that regime, which the 20 to 30 ms S-band path met and the roughly 820 ms Ku-band path did not [2]. Supervisory commanding is the response at the longer delay. The earlier DLR ROKVISS experiment, a two-joint torque-controlled manipulator on the ISS Russian Service Module since February 2005, had already flown telepresence and automatic modes; KONTUR-2 reused it as one of its ground robots [13].

KONTUR-2 is the opposite arrangement. The human closes the loop directly, and the robot carries no task-level autonomy at all; the design problem is stability under delay rather than planning. Time delay, jitter and data loss are handled by the Time Domain Passivity Control Approach [6]. The DLR side used a four-channel bilateral architecture with both joystick and ROKVISS impedance controlled, position and velocity sent one way and computed and measured forces the other; RTC used a dual-channel architecture with its own controllers for Surikat.

Communications

Section titled “Communications”

The two experiments used different links [2].

KONTUR-2SUPVIS Justin
Formatpoint-to-pointvia TDRS
BandS-bandKu-band
Uplink256 kbit/s10 Mbit/s maximum
Downlink4 Mbit/s10 Mbit/s maximum
Round trip20 to 30 msabout 820 ms
Availability10 minutes per 90 minute orbit during direct fly-overcontinuous link theoretically possible

Source: [2].

KONTUR-2 reused the S-band antenna designed for the ROKVISS experiment [6]. The SUPVIS Justin path runs ISS to TDRS to a US ground station and onward to Europe over the internet, and it is the gateway count and the distances that produce the larger delay; the compensation is that sessions can run for hours rather than for a fly-over [2]. The link budget assumed at proposal stage was more modest: 2 Mbit/s between the Columbus Control Center and the ISS at 2 to 10 seconds round trip, before the as-flown Ku-band path through TDRS replaced it [11]. An HD camera on the ISS let the ground team watch the astronaut, with voice on the ISS voice loop.

Ground training for KONTUR-2 ran over the internet between Star City and Oberpfaffenhofen at a mean delay of 65 ms, higher than the space link itself [6]. RTC sessions were connected to the ISS by paths other than S-band alone and saw increased delays, high packet loss and higher jitter, and the system still worked.

Payload and instruments

Section titled “Payload and instruments”

Sensing on the platform is arranged for autonomy rather than for science. Two color video cameras, one facing front and one facing left, provide visual odometry from optical flow, giving a redundant odometry solution with reduced error [1]. Four photonic mixer device cameras look front, left, right and back, returning a depth value per pixel for a 3D view of the surroundings that resolves walls, doors, obstacles and moving objects such as human legs. Every wheel carries forward and backward bumper switches as a last resort, since the wheels are the outermost parts, while the area between the legs is covered by the PMD cameras. A force sensor in each leg reads 0 to 1000 N, so weight on each wheel gives the center of gravity, and an inertial unit sits in the head [1].

For SUPVIS Justin the robot carried an assortment of electronic and mechanical tools [2].

The KONTUR-2 flight hardware is a two degree of freedom force feedback joystick, built around a cardan joint with a brushless DC motor and cable capstan reducer per axis [3]. It produces up to 15 N with a range of plus or minus 20 degrees on both axes, and the handle is inclined 15 degrees forward along its long axis to sit inside the wrist’s radial and ulnar abduction limits. A deadman button at the handle means the motors only apply force while the handle is gripped, disabled in software only for calibration and functional test; seven LEDs on the top face report operational state and error conditions. Driver-level safety adds a Cartesian force limit, temperature supervision, a power-on handle position calibration, a position controller for autonomous handle motion and an alive check [3], [8].

Crew-safety engineering is visible throughout. All power cables are double insulated in materials meeting ISS offgassing and flammability requirements, connectors have metallic housings, the unit is fully enclosed in a metal housing bonded to ISS structural ground with all internal parts galvanically isolated from case ground so that an accidental short to case cannot produce harmful current, and every board is conformally coated [3]. A protection cap guards the handle in transport and non-operation, qualified by impact tests at 556 N over 0.3 to 1.5 s, and in operation it mounts to the rear of the housing as an armrest. The transport bag is foam under flame-retardant Nomex, covering shock and random and sinusoidal vibration from 20 to 2000 Hz on all three axes, and measured to cut peak accelerations by a factor of about 24 [3].

Modes of operation

Section titled “Modes of operation”

The joystick control loop samples at 1000 Hz, standard for haptics and long enough to carry the real-time controllers; the intrinsic delay from a torque command to its effect on the handle is 1 ms, and with the encoder resolution this gives a maximum stable stiffness of 1.57 N m/rad [3].

KONTUR-2 ran ten-minute sessions through 2015 and 2016 [2]. Five cosmonauts took part; three ran on-board studies against a local simulation and two commanded ground robots. Oleg Kononenko and Sergey Volkov performed the ROKVISS tasks [6]. Four ground robots were flown against: ROKVISS, a two degree of freedom torque-sensing arm used for peg-in-slot and contact work; Surikat, an RTC kinematically redundant manipulator with a stylus and light targets and no torque sensors, so force feedback was model-based; Yula, a small RTC mobile robot needing position-speed rather than position control; and Space Justin. Tasks covered tracing physical geometric profiles, following light patterns on an LED grid, grasping and manipulating an inflated beach ball, and shaking hands with a person on the ground [2].

SUPVIS Justin ran three ISS-to-ground sessions of four hours between 2017 and 2018 with five astronauts, each session carrying more complex tasks than the last [2]. The first session, on 25 August 2017, validated system usability with data readout, system reboot, survey and navigation and involved three astronauts [5], [2]. The second, on 2 March 2018, added dexterous manipulation including solar panel readjustment and dust wiping, with NASA astronaut Scott Tingle commanding the robot to clean a solar panel [5]. The third added component retrieval and mechanical assembly for a full end-to-end installation [2]. Six experiment protocols were designed for the five crew members and every protocol run in all three sessions was completed, some under tight time constraints from ISS crew scheduling.

Ground operations

Section titled “Ground operations”

The ground environment is Solex, the Simulated SOLar Farm EXperimental setup at DLR, built with the components expected in a future Martian habitat and extended through the experiment life cycle; the final configuration has solar panels on smart payload units, antenna receivers and a lander unit [2], [7].

The astronaut interface is a tablet. A high-resolution feed from the robot’s camera lets the operator see through the robot’s eyes, known objects are overlaid with their 3D models, and the Action Template commands are offered by point and click [2]. The interface was revised between the first and second sessions on the basis of crew performance and feedback [5]. Command options are presented on a dynamically updated interface that offers the currently feasible actions and returns execution feedback from the robot, across a catalog of survey, navigation and repair tasks [4].

Technologies developed

Section titled “Technologies developed”

The two campaigns produced a comparison rather than a single result. KONTUR-2 showed that a multi-degree-of-freedom force reflection joystick can be paired with robots of different formats and complexities, delivering crisp force reflection thanks to the 20 to 30 ms delay and the controller’s delay handling [2]. The handshake was carried out with both the cosmonaut and the person on the ground reporting they felt the forces the other exerted, which was the first time a multi-DOF haptic input device was paired with a multi-DOF robot as an avatar to give a dexterous sense of physical contact between two people. It also demonstrated task-specific mapping of a two degree of freedom input device onto a seven degree of freedom arm.

The cost is operator load. Astronauts described the avatar modality as mentally and physically tiring, and it can only be run for 20 to 30 minutes before performance degrades, which was not a constraint for KONTUR-2 because each session gave only about ten minutes anyway [2].

SUPVIS Justin produced the opposite profile. The stated aim was to reduce the astronaut’s workload by delegating tasks to the robot, against the high mental and sometimes physical workload of direct teleoperation [4]. Astronauts consistently rated the supervised autonomy system easy to use and not a heavy mental workload, and all of them rated the concept capable of letting one crew member manage a team of multiple robots [2]. Crew with prior ground training completed the same task noticeably faster; one crew member with no prior knowledge was recruited on the spot and completed the protocol after interactive training from another crew member; speed and confidence rose visibly across the three sessions.

The conclusion drawn from the pair is a division by task rather than a winner. The avatar suits exploration, unknown environments and unexpected situations, where human intelligence and dexterity carry the command down to a low level. The co-worker with supervised autonomy suits large-scale tasks over known objects, where ease of use and low mental workload matter more than immediacy [2]. Both rest on the same finding: an operator in orbit above the surface has a low enough delay for either method, while keeping the crew out of the surface environment.

The METERON program continued past both. Analog-1 returned to haptic control of a surface rover from the ISS, taking the touch-based modality that KONTUR-2 established into a rover task [10].

Section titled “Related approaches”

Justin sits among four families of ground-to-orbit and crew-assist robotics, and only some of them have flown.

Direct teleoperation from the ground through delay was flown first on the Japanese ETS-VII arm. In 1999 the arm was teleoperated from the ground under bilateral force coupling with a 6 to 7 s round trip, most of which the authors attribute to ground data relay rather than the orbital path [24]. DLR ran its own GETEX experiments on the same arm in April 1999 using the MARCO task-level tele-sensor-programming approach, which is closer to SUPVIS Justin’s task-level commanding than to the KONTUR-2 joystick [26]. The ETS-VII arm kept its tip positioning accuracy through 18 months and more than 100 experiment days [25], and the Handbook of Robotics chapter on space robotics tabulates it with the other orbital manipulators [22]. These flights bound the delay regime in which force feedback was used: seconds for ETS-VII, 20 to 30 ms for KONTUR-2.

Supervised autonomy as a control methodology predates SUPVIS Justin. JPL framed a ground-built, uplinked command sequence as supervised autonomy and applied it to a dual-arm telerobot testbed, with local-site sequence control stated as feasible up to round-trip delays of about 10 s [16]. That demonstration was laboratory-only, in 1 g. JPL’s later SURROGATE framework moved the same idea to whole-body manipulation on a two-armed torso robot on a tracked base, where the operator supplies intent and regions of interest and the system plans the motion; its three demonstrations are not a trial series and report no success rate [17]. Neither has left a laboratory.

Humanoid crew assistants flew in a different form. NASA’s Robonaut 1 reached 43 degrees of freedom with two seven-degree-of-freedom arms, matching the Justin upper body count, under the premise of fitting existing human interfaces [18]. Robonaut 2 arrived on the ISS in February 2011 and was operated as a fixed-base upper body, which makes it the one humanoid in this comparison that has flown, and the one without a mobile base [23]. Rollin’ Justin is the reverse: a mobile base on the ground, commanded from orbit. JPL proposed humanoids as lunar surface partners in a roadmap whose milestones ran from 2023 to 2050 [19], and its only hardware at the time was a 50 cm HOAP-2 laboratory robot [20]. Those milestones are projections, not results.

Earlier handling of the operator interface appears in the Shuttle arm work. The translational hand controller for the Shuttle manipulator was evaluated by operators in a real-time simulation, with a maximum combined handle force of 10 lb [28]. The Robotic Refueling Mission on the ISS, run from the ground over eleven years, gives the closest flown analogue to SUPVIS Justin’s ground-commanded servicing tasks, and its lessons concern real-time replanning and operations-day length rather than the robot [27].

Surveys place these efforts within the field. A NASA-commissioned 2003 survey projected surface rover capability and command cycles by respondent consensus, not measurement [21]. Inside the station, a 2018 pre-launch comparison of the four ISS free flyers (SPHERES, Int-Ball, CIMON and Astrobee) argues that SPHERES fell short as an operations tool because its ultrasonic-beacon positioning confined it to a two meter cube and it needed manual CO2 and battery swaps each session [32]. That is a crew-assist robot in the cabin, the opposite arrangement to Justin commanded from orbit. A tutorial review of space manipulators covers ISS arms and servicing demonstrators [30], and the NASA ISAM state of play catalogs robotic manipulation and servicing entries, dividing manipulators at an 8 m reach [31]. For the Earth-Mars case, where one-way light delay reaches 24 minutes, a 2024 position paper proposed an offline astronaut assistant built on a language model with retrieval, which is a proposal and not a flown system [29]. The METERON work covers delays between these cases: 20 to 30 ms and about 820 ms [2], well below the Mars regime, so SUPVIS Justin’s task-level commanding is a precondition for that regime and not a demonstration of it. Roadmap milestones from the humanoid vision papers [19], [20] are likewise unmet.

Status and what is not established

Section titled “Status and what is not established”

Flown: the KONTUR-2 joystick on the ISS with ground robots in 2015, and three SUPVIS Justin sessions in 2017 and 2018 [2]. Not flown: Rollin’ Justin itself has not left Earth. Every command from orbit went to a robot in a terrestrial analogue environment, so no result covers vacuum, dust, partial gravity or surface thermal conditions; the DLR page lists no space-environment qualification of the robot [33]. The METERON operations environment that carries ISS-to-ground robot services was described in a pre-flight design paper that reports no latency or success-rate data [14]. Humanoid crew-partner roles on a lunar or Mars surface remain a roadmap whose milestones were the authors’ projections under a program since abandoned [15].

The performance evidence is limited. KONTUR-2 accuracy in microgravity is reported qualitatively, with no tabulated tracking or force error [6]. Delay figures are ranges without distributions. The SUPVIS Justin conclusions rest on five astronauts in three sessions, and no common metric compares the avatar and co-worker modes. The platform specifications in [1] and the program page [33] disagree on degrees of freedom, speed and endurance, and the program page gives no test conditions. The joystick thermal case is closed by control logic: idle steady state of 46.4 C exceeds the 40 C housing limit [9], and no on-orbit temperatures are reported. The related-approach sources in the preceding section are laboratory, survey or proposal documents, and none reports a measurement made on Rollin’ Justin.

References

Section titled “References”

References

  1. Fuchs, M., Borst, C., Robuffo Giordano, P., Baumann, A., Kraemer, E., Langwald, J., Gruber, R., Seitz, N., Plank, G., Kunze, K., Burger, R., Schmidt, F., Wimboeck, T. and Hirzinger, G. (2009). Rollin' Justin: Design Considerations and Realization of a Mobile Platform for a Humanoid Upper Body . IEEE International Conference on Robotics and Automation. Source
    BibTeX
    @inproceedings{fuchs2009rollin,
  title = {Rollin' Justin: Design Considerations and Realization of a Mobile Platform for a Humanoid Upper Body},
  author = {Fuchs, M. and Borst, Ch. and Robuffo Giordano, P. and Baumann, A. and Kraemer, E. and Langwald, J. and Gruber, Robin and Seitz, Nikolaus and Plank, Georg and Kunze, K. and Burger, R. and Schmidt, F. and Wimboeck, T. and Hirzinger, G.},
  booktitle = {IEEE International Conference on Robotics and Automation},
  pages = {4131-4137},
  year = {2009},
  doi = {10.1109/robot.2009.5152464},
  abstract = {Research on humanoid robots for use in servicing tasks, e.g. fetching and delivery, attracts steadily more interest. With Rollin' Justin a mobile robotic system and research platform is presented that allows the implementation and demonstration of sophisticated control algorithms and dexterous manipulation. Important problems of service robotics such as mobile manipulation and strategies for using the increased workspace and redundancy in manipulation task can be studied in detail. This paper gives an overview of the design considerations for a mobile platform and their realizations to transform the formerly table-mounted humanoid upper body system Justin into Rollin' Justin, a fully self-sustaining mobile research platform.}
}
  2. Lii, N. Y., Riecke, C., Leidner, D., Schätzle, S., Schmaus, P., Weber, B., Krueger, T., Stelzer, M., Wedler, A. and Grunwald, G. (2018). The Robot as an Avatar or Co-worker? An Investigation of the Different Teleoperation Modalities through the KONTUR-2 and METERON SUPVIS Justin Space Telerobotic Missions . International Astronautical Congress, IAC-18-B3.6-A5.3.5x47302. Source
    BibTeX
    @inproceedings{lii2018robot,
  title = {The Robot as an Avatar or Co-worker? An Investigation of the Different Teleoperation Modalities through the KONTUR-2 and METERON SUPVIS Justin Space Telerobotic Missions},
  author = {Lii, Neal Y. and Riecke, Cornelia and Leidner, Daniel and Schätzle, Simon and Schmaus, Peter and Weber, Bernhard and Krueger, Thomas and Stelzer, Martin and Wedler, Armin and Grunwald, Gerhard},
  booktitle = {International Astronautical Congress},
  number = {IAC-18-B3.6-A5.3.5x47302},
  year = {2018},
  url = {https://elib.dlr.de/133023/}
}
  3. Riecke, C., Artigas, J., Balachandran, R., Bayer, R., Beyer, A., Brunner, B., Buchner, H., Gumpert, T., Gruber, R., Hacker, F., Landzettel, K., Plank, G., Schätzle, S., Sedlmayr, H.-J., Seitz, N., Steinmetz, B.-M., Stelzer, M., Vogel, J., Weber, B., Willberg, B. and Albu-Schäffer, A. (2016). KONTUR-2 Mission: The DLR Force Feedback Joystick for Space Telemanipulation from the ISS . International Symposium on Artificial Intelligence, Robotics and Automation in Space. Source
    BibTeX
    @inproceedings{riecke2016kontur,
  title = {KONTUR-2 Mission: The DLR Force Feedback Joystick for Space Telemanipulation from the ISS},
  author = {Riecke, Cornelia and Artigas, Jordi and Balachandran, Ribin and Bayer, Ralph and Beyer, Alexander and Brunner, Bernhard and Buchner, Hans and Gumpert, Thomas and Gruber, Robin and Hacker, Franz and Landzettel, Klaus and Plank, Georg and Schätzle, Simon and Sedlmayr, Hans-Jürgen and Seitz, Nikolaus and Steinmetz, Bernhard-Michael and Stelzer, Martin and Vogel, Jörg and Weber, Bernhard and Willberg, Bertram and Albu-Schäffer, Alin},
  booktitle = {International Symposium on Artificial Intelligence, Robotics and Automation in Space},
  year = {2016},
  url = {https://elib.dlr.de/105449/}
}
  4. Schmaus, P., Leidner, D., Bayer, R., Pleintinger, B., Krüger, T. and Lii, N. Y. (2019). Continued Advances in Supervised Autonomy User Interface Design for METERON SUPVIS Justin . IEEE Aerospace Conference. Source
    BibTeX
    @inproceedings{schmaus2019continued,
  title = {Continued Advances in Supervised Autonomy User Interface Design for METERON SUPVIS Justin},
  author = {Schmaus, Peter and Leidner, Daniel and Bayer, Ralph and Pleintinger, Benedikt and Krüger, Thomas and Lii, Neal Y.},
  booktitle = {IEEE Aerospace Conference},
  pages = {1-11},
  address = {Big Sky, Montana},
  year = {2019},
  doi = {10.1109/aero.2019.8741885},
  abstract = {The exploration of the universe remains a challenging endeavor, constantly pushing the limits of technology. Of special interest is the investigation of the other planets of our solar system such as Mars, which has been examined by various tele-operated and (semi-) autonomous satellites and landers. But an important milestone that is needed for a deeper understanding of the planet is still missing: A crewed landing. In order to send humans to such a remote location, an infrastructure for the landing crew including an energy supply, a habitat, and a return vehicle needs to be provided on the surface of the planet. The construction and maintenance of these structures is envisioned to be done by semiautonomous robots that are commanded from orbiting spacecrafts. The teleoperation of such ground-based robots poses high demands on the capabilities of the system including robot autonomy, orbiter-robot communication, and human-robot interface design. The METERON SVPVIS Justin space telerobotics experiment suite has been initiated by the German Aerospace Center (DLR) together with the European Space Agency (ESA) to investigate the requirements for such a system and evaluate an approach. During the experiment sessions, astronauts onboard the International Space Station (ISS) command DLR's humanoid service robot Rollin' Justin on Earth to execute complex surveillance, service, and repair tasks in a simulated Martian solar farm. The robot uses its local intelligence to support the astronaut operator upon task completion allowing a simple intuitive command interface and lowering the requirements on the communication link. This work gives an overview of the developed robotic system, communication link, and tablet computer user interface (UI). In particular the tight coupling between the autonomy system of the robot and the UI, that allows the intuitive robot commanding including action parameterization, is described in detail. The first space-ground experiment sessions of METERON SUPVIS Justin were conducted in August 2017, and March 2018 by four astronauts in total. During the first session, three astronauts demonstrated the operational readiness of our system by commanding Rollin' Justin to perform surveillance and inspection tasks. The astronauts were even able to successfully command the robot in scenarios, which were not trained prior to their spaceflight. This was possible, because our astronaut-robot collaboration concept efficiently guides the operator towards task completion. We used this property in the second experiment session to evaluate our system in even more complex scenarios. While in the first session it was sufficient for the astronaut to select the correct commands, the operator was now required to manually parameterize some of the commands to optimize the task outcome. By that, the robot has been successfully commanded to perform complex maintenance and adjustment tasks in the simulated Martian solar farm. In this work, we evaluate the preliminary results of the space-ground experiments and discuss the feedback we received from the astronauts and its impact on future space telerobotics UI design.}
}
  5. Lii, N. Y., Leidner, D., Birkenkampf, P., Pleintinger, B., Bayer, R. and Krueger, T. (2017). Toward Scalable Intuitive Teleoperation of Robots for Space Deployment with the METERON SUPVIS Justin Experiment . Symposium on Advanced Space Technologies in Robotics and Automation. Source
    BibTeX
    @inproceedings{lii2017scalable,
  title = {Toward Scalable Intuitive Teleoperation of Robots for Space Deployment with the METERON SUPVIS Justin Experiment},
  author = {Lii, Neal Y. and Leidner, Daniel and Birkenkampf, Peter and Pleintinger, Benedikt and Bayer, Ralph and Krueger, Thomas},
  booktitle = {Symposium on Advanced Space Technologies in Robotics and Automation},
  year = {2017},
  url = {https://elib.dlr.de/113125/}
}
  6. Artigas, J., Riecke, C., Weber, B., Stelzer, M., Balachandran, R., Schaetzle, S., Bayer, R., Steinmetz, B.-M., Vogel, J., Brunner, B., Albu-Schäffer, A., Guk, M., Zaborovskyi, V., Kondratiev, A., Muliukha, V., Silinenko, A. and Shmakov, O. (2016). Force-Feedback Teleoperation of On-Ground Robots from the International Space Station in the Frame of the KONTUR-2 Experiment . International Symposium on Artificial Intelligence, Robotics and Automation in Space. Source
    BibTeX
    @inproceedings{artigas2016force,
  title = {Force-Feedback Teleoperation of On-Ground Robots from the International Space Station in the Frame of the KONTUR-2 Experiment},
  author = {Artigas, Jordi and Riecke, Cornelia and Weber, Bernhard and Stelzer, Martin and Balachandran, Ribin and Schaetzle, Simon and Bayer, Ralph and Steinmetz, Bernhard-Michael and Vogel, Jörg and Brunner, Bernhard and Albu-Schäffer, Alin and Guk, M. and Zaborovskyi, V. and Kondratiev, A. and Muliukha, V. and Silinenko, A. and Shmakov, O.},
  booktitle = {International Symposium on Artificial Intelligence, Robotics and Automation in Space},
  pages = {1166-1173},
  year = {2016},
  doi = {10.1109/icra.2016.7487246},
  abstract = {This paper presents a new robot controller for space telerobotics missions specially designed to meet the requirements of KONTUR-2, a German & Russian telerobotics mission that addressed scientific and technological questions for future planetary explorations. In KONTUR-2, Earth and ISS have been used as a test-bed to evaluate and demonstrate a new technology for real-time telemanipulation from space. During the August 2015' experiments campaign, a cosmonaut teleoperated a robot manipulator located in Germany, using a force-feedback joystick from the Russian segment of the International Space Station (ISS). The focus of the paper is on the design and performance of the bilateral controller between ISS joystick and Earth robot. The controller is based on a 4-Channels architecture in which stability is guaranteed through passivity and the Time Delay Power Network (TDPN) concept. We show how the proposed approach successfully fulfills mission requirements, specially those related to system operation through space links and internet channels, involving time delays and data losses of different nature.}
}
  7. Lii, N. Y., Leidner, D., Schiele, A., Birkenkampf, P., Pleintinger, B. and Bayer, R. (2015). Simulating an Extraterrestrial Environment for Robotic Space Exploration: The METERON SUPVIS-Justin Telerobotic Experiment and the SOLEX Proving Ground . Symposium on Advanced Space Technologies in Robotics and Automation. Source
    BibTeX
    @inproceedings{lii2015simulating,
  title = {Simulating an Extraterrestrial Environment for Robotic Space Exploration: The METERON SUPVIS-Justin Telerobotic Experiment and the SOLEX Proving Ground},
  author = {Lii, Neal Y. and Leidner, Daniel and Schiele, André and Birkenkampf, Peter and Pleintinger, Benedikt and Bayer, Ralph},
  booktitle = {Symposium on Advanced Space Technologies in Robotics and Automation},
  year = {2015},
  url = {https://elib.dlr.de/98352/}
}
  8. Stelzer, M., Birkenkampf, P., Brunner, B., Steinmetz, B.-M., Vogel, J. and Kühne, S. (2017). Software Architecture and Design of the Kontur-2 Mission . IEEE Aerospace Conference. Source
    BibTeX
    @inproceedings{stelzer2017software,
  title = {Software Architecture and Design of the Kontur-2 Mission},
  author = {Stelzer, Martin and Birkenkampf, Peter and Brunner, Bernhard and Steinmetz, Bernhard-Michael and Vogel, Jörg and Kühne, Stefan},
  booktitle = {IEEE Aerospace Conference},
  pages = {1-17},
  year = {2017},
  doi = {10.1109/aero.2017.7943816},
  abstract = {This paper describes the software architecture and design of the space segment, communication and ground segment software of the Kontur-2 project, which aimed to study the feasibility of planetary exploration through telepresence. The main research objectives in Kontur-2 were the development and in-flight verification of a space qualified two degree of freedom (DoF) force-feedback joystick (RJo) inside the Zvezda Service Module of the International Space Station (ISS), the implementation of telepresence technologies, and the study of human performance when controlling a force feedback joystick in microgravity. The project was conducted from 2012 to 2015 by a consortium consisting of the German Aerospace Center (DLR), the Russian Federal Space Agency (ROSCOSMOS), The Russian State Scientific Center for Robotics and Technical Cybernetics (RTC), S. P. Korolev Rocket and Space Corporation Energia (RSC “Energia”) and the Yuri A. Gagarin State Scientific Research-and-Testing Cosmonaut Training Center (GCTC). DLR conducted two sets of experiments in which a cosmonaut on board the ISS used RJo to perform different tasks with robots located on-ground. The first was conducted with a 2-DoF robot equipped with a camera system, a task board and torque sensors that allowed the cosmonaut to perceive reactive forces caused by contacts with the environment. For the second set of experiments a humanoid robot was utilized to perform a tele-handshake, as well as a cooperative task between the cosmonaut on ISS and colleagues at RTC in St. Petersburg. To realize these experiments, the consortium developed onboard and on-ground software which are described in this paper. The space segment software consists of the control software for RJo and user interfaces on a laptop to guide the cosmonaut efficiently through the experiments. A state machine was designed for these user interfaces to capture state changes during the experiment execution. This way only relevant contextual information is provided to the cosmonaut. On RJo, a component framework has been deployed combining a data-centric architecture with a CCSDS Space Packet interface. Additionally, the communication software has been designed to support a direct multi-channel connection between ground control and ISS using the S-band radio equipment of the consortium. During contact to ISS, the ground operators used the ground segment software at DLR for experiment support, supervision, maintenance and data logging. The visual feedback from the camera system required by the cosmonaut to perform the experiments was provided by a low-latency video stream through a communication channel with very restricted bandwidth. 23 experiment sessions were carried out in 2015 utilizing the Kontur-2 software, which helped to validate telepresence technologies and study human factors for space applications.}
}
  9. Bayer, R. (2015). The Thermal Design of the KONTUR-2 Force Feedback Joystick. elib.dlr.de/112503
    BibTeX
    @misc{bayer2015thermal,
  title = {The Thermal Design of the KONTUR-2 Force Feedback Joystick},
  author = {Bayer, Ralph},
  howpublished = {Presentation at the European Space Mechanisms and Tribology Symposium},
  year = {2015},
  url = {https://elib.dlr.de/112503/}
}
  10. Carey, W., Krueger, T., Wormnes, K., Grenouilleau, J., Ferreira, E., Nergaard, K., van der Hulst, F., den Exter, E. and Gerdes, L. (2022). METERON Analog-1: A Touch Remote . International Astronautical Congress, IAC-22-A3.2B.4. Source
    BibTeX
    @inproceedings{carey2022meteron,
  title = {METERON Analog-1: A Touch Remote},
  author = {Carey, William and Krueger, Thomas and Wormnes, Kjetil and Grenouilleau, Jessica and Ferreira, Edmundo and Nergaard, Kim and van der Hulst, Frank and den Exter, Emiel and Gerdes, Levin},
  booktitle = {International Astronautical Congress},
  number = {IAC-22-A3.2B.4},
  year = {2022},
  url = {https://elib.dlr.de/190115/}
}
  11. Lii, N. Y., Leidner, D., Schiele, A., Birkenkampf, P., Pleintinger, B. and Bayer, R. (2015). Command Robots from Orbit with Supervised Autonomy . SAE Technical Paper Series. Source
    BibTeX
    @inproceedings{lii2015command,
  title = {Command Robots from Orbit with Supervised Autonomy},
  author = {Lii, Neal Y. and Leidner, Daniel and Schiele, André and Birkenkampf, Peter and Pleintinger, Benedikt and Bayer, Ralph},
  booktitle = {SAE Technical Paper Series},
  pages = {53-54},
  publisher = {ACM},
  year = {2015},
  doi = {10.1145/2701973.2702022},
  abstract = {The on-going work at German Aerospace Center (DLR) and European Space Agency (ESA) on the Meteron Supvis-Justin space telerobotic experiment utilizing supervised autonomy is presented. The Supvis-Justin experiment will employ a tablet UI for an astronaut on the International Space Station (ISS) to communicate task level commands to a service robot. The goal is to explore the viability of supervised autonomy for space telerobotics. For its validation, survey, navigation, inspection, and maintenance tasks will be commanded to DLR's service robot, Rollin' Justin, to be performed in a simulated extraterrestrial environment constructed at DLR. The experiment is currently slated for late 2015-2016.}
}
  12. Ferrell, W. R. (1964). Remote Manipulation with Transmission Delay . IEEE Transactions on Human Factors in Electronics. Source
    BibTeX
    @article{ferrell1964remote,
  title = {Remote Manipulation with Transmission Delay},
  author = {Ferrell, William R.},
  journal = {IEEE Transactions on Human Factors in Electronics},
  volume = {HFE-6},
  pages = {24-32},
  publisher = {Institute of Electrical and Electronics Engineers (IEEE)},
  school = {Massachusetts Institute of Technology, Department of Mechanical Engineering},
  type = {Ph.D. thesis},
  year = {1964},
  doi = {10.1109/thfe.1965.6591253},
  abstract = {The effect on the performance of both simple and complex tasks of inserting transmission delay between the master and slave elements of a remote manipulator was studied. Most operators spontaneously adopted an effective “move-and-wait” strategy to cope with delay. A modification of a method proposed earlier enabled task-completion time to be accurately predicted from data taken when there was no delay. With the move-and-wait strategy, completion time depends on the length of sequences of open-loop movements. The number of such moves for positioning in one dimension as a function of task difficulty was found to agree with that predicted by a simple statistical model.}
}
  13. Landzettel, K., Preusche, C., Albu-Schaffer, A., Reintsema, D., Rebele, B. and Hirzinger, G. (2006). Robotic On-Orbit Servicing - DLR's Experience and Perspective . IEEE/RSJ International Conference on Intelligent Robots and Systems. Source
    BibTeX
    @inproceedings{landzettel2006robotic,
  title = {Robotic On-Orbit Servicing - DLR's Experience and Perspective},
  author = {Landzettel, Klaus and Preusche, C. and Albu-Schaffer, Alin and Reintsema, D. and Rebele, Bernhard and Hirzinger, G.},
  booktitle = {IEEE/RSJ International Conference on Intelligent Robots and Systems},
  pages = {4587-4594},
  publisher = {IEEE},
  year = {2006},
  doi = {10.1109/iros.2006.282164}
}
  14. Sarkarati, M., Spada, M., Merri, M., Raymaekers, J. and Nergaard, K. (2014). METERON CCSDS MO Compliant Telerobotic Services Talk DTN . SpaceOps Conference. Source
    BibTeX
    @inproceedings{sarkarati2014meteron,
  title = {METERON CCSDS MO Compliant Telerobotic Services Talk DTN},
  author = {Sarkarati, Mehran and Spada, Mariella and Merri, Mario and Raymaekers, Jens and Nergaard, Kim},
  booktitle = {SpaceOps Conference},
  volume = {0},
  year = {2014},
  doi = {10.2514/6.2014-1871},
  abstract = {Robotic systems are very diverse in their capabilities and characteristics. The software used for their monitoring and control is often custom developed and proprietary. When participating in a space exploration mission, the robotic system constitutes one important element of the overall mission infrastructure. Its operations become accordingly part of the overall mission operations and must be closely coordinated with other elements, such as ground communication networks, ground stations, relay satellites, orbiting spacecraft and in future scenarios manned orbiting stations around the target body. The monitoring and control (M&C) and planning systems of such complex space missions, expose often very different characteristics than the dedicated software systems used during the development, testing and validation of involved robotic elements on the ground. Collaborative mission operation scenarios, in which robotic agents of different space agencies must operate in close coordination, add more complexions to the design of M&C of such missions. The objective of the Multipurpose End-To-End Robotics Operations Network, in short METERON, project is to prepare for future robotic exploration missions by evaluating and demonstrating relevant concepts and technologies in the areas of Communications, Operations and Robotics. The task of integrating robotic elements, participating in METERON experiments, in the overall METERON operations and communications infrastructure can be optimised considerably, if the common robotic capabilities, relevant to mission operations, can be provided through standardised telerobotic service interfaces. The objective of the CCSDS Telerobotic working group, which has started its work in 2013, is to specify such standardised services, which shall abstract from the proprietary interfaces of each robotic system and from the adopted communication protocol. By the same token an initial set of harmonised Robotic Services have been specified for the METERON Operational Environment (MOE), which shall be used for the first time in the upcoming METERON OPSCOM-2 experiment in 2014. The design of the METERON Robotic Services has been based on the service oriented design principles of the CCSDS Mission Operations (MO) framework. METERON Robotic Services support multiple communication protocols including the DTN Bundle Protocol, since evaluation and demonstration of communication aspects related to Delay/Disruption Tolerant Network (DTN) are among main objectives of some of METERON experiments. METERON Robotic Services will be in the context of the OPSCOM-2 experiment the first in-orbit demonstration of the corresponding CCSDS MO monitoring and control services and can serve as proof of concept for the envisaged CCSDS Telerobotic services.}
}
  15. Stoica, A. and Keymeulen, D. (2006). Humanoids in support of lunar and planetary surface operations . IEEE Aerospace Conference. Source
    BibTeX
    @inproceedings{stoica2006humanoids,
  title = {Humanoids in support of lunar and planetary surface operations},
  author = {Stoica, Adrian and Keymeulen, Didier},
  booktitle = {IEEE Aerospace Conference},
  pages = {1-7},
  publisher = {IEEE},
  year = {2006},
  doi = {10.1109/aero.2006.1655721},
  abstract = {This paper presents a vision of humanoid robots as human's key partners in future space exploration, in particular for construction, maintenance/repair and operation of lunar/planetary habitats, bases and settlements. It integrates this vision with the recent plans for human and robotic exploration, aligning a set of milestones for operational capability of humanoids with the schedule for the next decades and development spirals in the Project Constellation. These milestones relate to a set of incremental challenges, for the solving of which new humanoid technologies are needed. A system of systems integrative approach that would lead to readiness of cooperating humanoid crews is sketched. Robot fostering, training/education techniques, and improved cognitive/sensory/motor development techniques are considered essential elements for achieving intelligent humanoids. A pilot project using small-scale Fujitsu HOAP-2 humanoid is outlined}
}
  16. Backes, P. (1993). Supervised Autonomy for Space Telerobotics . JPL Open Repository. Source
    BibTeX
    @inproceedings{backes1993supervised,
  title = {Supervised Autonomy for Space Telerobotics},
  author = {Backes, P.},
  publisher = {JPL Open Repository},
  year = {1993},
  url = {https://dataverse.jpl.nasa.gov/dataset.xhtml?persistentId=hdl:2014/36631}
}
  17. Hebert, P., Ma, J., Borders, J., Aydemir, A., Bajracharya, M., Hudson, N., Shankar, K., Karumanchi, S., Douillard, B. and Burdick, J. (2015). Supervised Remote Robot with Guided Autonomy and Teleoperation (SURROGATE): a framework for whole-body manipulation . IEEE International Conference on Robotics and Automation. Source
    BibTeX
    @inproceedings{hebert2015supervised,
  title = {Supervised Remote Robot with Guided Autonomy and Teleoperation (SURROGATE): a framework for whole-body manipulation},
  author = {Hebert, Paul and Ma, Jeremy and Borders, James and Aydemir, Alper and Bajracharya, Max and Hudson, Nicolas and Shankar, Krishna and Karumanchi, Sisir and Douillard, Bertrand and Burdick, Joel},
  booktitle = {IEEE International Conference on Robotics and Automation},
  pages = {5509-5516},
  publisher = {IEEE},
  year = {2015},
  doi = {10.1109/icra.2015.7139969},
  abstract = {The use of the cognitive capabilties of humans to help guide the autonomy of robotics platforms in what is typically called “supervised-autonomy” is becoming more commonplace in robotics research. The work discussed in this paper presents an approach to a human-in-the-loop mode of robot operation that integrates high level human cognition and commanding with the intelligence and processing power of autonomous systems. Our framework for a “Supervised Remote Robot with Guided Autonomy and Teleoperation” (SURROGATE) is demonstrated on a robotic platform consisting of a pan-tilt perception head, two 7-DOF arms connected by a single 7-DOF torso, mounted on a tracked-wheel base. We present an architecture that allows high-level supervisory commands and intents to be specified by a user that are then interpreted by the robotic system to perform whole body manipulation tasks autonomously. We use a concept of “behaviors” to chain together sequences of “actions” for the robot to perform which is then executed real time.}
}
  18. Ambrose, R. O. and Diftler, M. A. (2001). Robonaut: A Robotic Astronaut Assistant . International Symposium on Artificial Intelligence, Robotics and Automation in Space, 20100033240. Source
    BibTeX
    @inproceedings{ambrose2001robonaut,
  title = {Robonaut: A Robotic Astronaut Assistant},
  author = {Ambrose, Robert O. and Diftler, Myron A.},
  booktitle = {International Symposium on Artificial Intelligence, Robotics and Automation in Space},
  number = {20100033240},
  institution = {NASA},
  address = {Montreal},
  year = {2001},
  url = {https://ntrs.nasa.gov/citations/20100033240},
  abstract = {NASA's latest anthropomorphic robot, Robonaut, has reached a milestone in its capability. This highly dexterous robot, designed to assist astronauts in space, is now performing complex tasks at the Johnson Space Center that could previously only be carried out by humans. With 43 degrees of freedom, Robonaut is the first humanoid built for space and incorporates technology advances in dexterous hands, modular manipulators, lightweight materials, and telepresence control systems. Robonaut is human size, has a three degree of freedom (DOF) articulated waist, and two, seven DOF arms, giving it an impressive work space for interacting with its environment. Its two, five fingered hands allow manipulation of a wide range of tools. A pan/tilt head with multiple stereo camera systems provides data for both teleoperators and computer vision systems.}
}
  19. Keymeulen, D., Myers, J., Newton, J., Csaszar, A., Gan, Q., Hidalgo, T., Moore, J., Sandoval, S., Xu, J., Schon, A., Assad, C. and Stoica, A. (2006). Humanoids for lunar and planetary surface operations. doi.org/10.1109/icsmc.2005.1571549
    BibTeX
    @misc{keymeulen2006humanoids,
  title = {Humanoids for lunar and planetary surface operations},
  author = {Keymeulen, Didier and Myers, John and Newton, Jason and Csaszar, Ambrus and Gan, Quan and Hidalgo, Tim and Moore, Jeff and Sandoval, Steven and Xu, Jiajing and Schon, Aaron and Assad, Chris and Stoica, Adrian},
  howpublished = {Presentation at the IEEE International Conference on Systems, Man and Cybernetics},
  volume = {3},
  pages = {2649-2654},
  year = {2006},
  doi = {10.1109/icsmc.2005.1571549}
}
  20. Stoica, A., Keymeulen, D., Csaszar, A., Gan, Q., Hidalgo, T., Moore, J., Newton, J., Sandoval, S. and Xu, J. (2005). Humanoids for lunar and planetary surface operations . IEEE-RAS International Conference on Humanoid Robots. Source
    BibTeX
    @inproceedings{stoica2005humanoids,
  title = {Humanoids for lunar and planetary surface operations},
  author = {Stoica, Adrian and Keymeulen, Didier and Csaszar, Ambrus and Gan, Quan and Hidalgo, Timothy and Moore, Jeff and Newton, Jason and Sandoval, Steven and Xu, Jiajing},
  booktitle = {IEEE-RAS International Conference on Humanoid Robots},
  pages = {345},
  publisher = {IEEE},
  year = {2005},
  doi = {10.1109/ichr.2005.1573591},
  abstract = {This paper presents a vision of humanoid robots as human's key partners in future space exploration, in particular for construction, maintenance/repair and operation of lunar/planetary habitats, bases and settlements. It integrates this vision with the recent plans for human and robotic exploration, aligning a set of milestones for operational capability of humanoids with the schedule and phases of human space flight system development program for the next decades. These milestones relate to a set of incremental challenges, for the solving of which new humanoid technologies are needed. A system of systems integrative approach that would lead to readiness of cooperating humanoid crews is sketched. Robot fostering, training/education techniques, and improved cognitive/sensory/motor development techniques are considered essential elements for achieving intelligent humanoids. A pilot project in this direction is outlined.}
}
  21. Pedersen, L., Kortenkamp, D., Wettergreen, D. and Nourbakhsh, I. (2003). A Survey of Space Robotics . International Symposium on Artificial Intelligence, Robotics and Automation in Space. Source
    BibTeX
    @inproceedings{pedersen2003survey,
  title = {A Survey of Space Robotics},
  author = {Pedersen, L and Kortenkamp, D and Wettergreen, D and Nourbakhsh, I},
  booktitle = {International Symposium on Artificial Intelligence, Robotics and Automation in Space},
  year = {2003},
  url = {https://ntrs.nasa.gov/citations/20030054507},
  abstract = {In this paper we summarize a survey conducted by NASA to determine the state-of-the-art in space robotics and to predict future robotic capabilities under either nominal and intensive development effort. The space robotics assessment study examined both in-space operations including assembly, inspection, and maintenance and planetary surface operations like mobility and exploration. Applications of robotic autonomy and human-robot cooperation were considered. The study group devised a decomposition of robotic capabilities and then suggested metrics to specify the technical challenges associated with each. The conclusion of this paper identifies possible areas in which investment in space robotics could lead to significant advances of important technologies.}
}
  22. Yoshida, K., Wilcox, B., Hirzinger, G. and Lampariello, R. (2016). Space Robotics . Springer Handbook of Robotics. Source
    BibTeX
    @incollection{yoshida2016space,
  title = {Space Robotics},
  author = {Yoshida, Kazuya and Wilcox, Brian and Hirzinger, Gerd and Lampariello, Roberto},
  booktitle = {Springer Handbook of Robotics},
  series = {Springer Handbooks},
  pages = {1423-1462},
  publisher = {Springer International Publishing},
  year = {2016},
  doi = {10.1007/978-3-319-32552-1_55}
}
  23. Diftler, M. A., Greene, B. D., Joyce, C., De La Pena, N., Noblitt, A. and Ambrose, R. (2011). Robonaut 2 - Initial Activities On-Board the ISS . IEEE Aerospace Conference, 20110023640. Source
    BibTeX
    @inproceedings{diftler2011robonaut,
  title = {Robonaut 2 - Initial Activities On-Board the ISS},
  author = {Diftler, Myron A. and Greene, B. D. and Joyce, Charles and De La Pena, Noe and Noblitt, Alan and Ambrose, Robert},
  booktitle = {IEEE Aerospace Conference},
  number = {20110023640},
  institution = {NASA},
  address = {Big Sky, Montana},
  year = {2011},
  url = {https://ntrs.nasa.gov/citations/20110023640},
  abstract = {Robonaut 2, or R2, arrived on the International Space Station in February 2011 and is currently undergoing testing in preparation for it to become, initially, an Intra-Vehicular Activity (IVA) tool and then evolve into a system that can perform Extra-Vehicular Activities (EVA). After the completion of a series of system level checks to ensure that the robot traveled well on-board the Space Shuttle Atlantis, ground control personnel will remotely control the robot to perform free space tasks that will help characterize the differences between earth and zero-g control. For approximately one year, the fixed base R2 will perform a variety of experiments using a reconfigurable task board that was launched with the robot. While working side-by-side with human astronauts, Robonaut 2 will actuate switches, use standard tools, and manipulate Space Station interfaces, soft goods and cables. The results of these experiments will demonstrate the wide range of tasks a dexterous humanoid can perform in space and they will help refine the methodologies used to control dexterous robots both in space and here on earth. After the trial period that will evaluate R2 while on a fixed stanchion in the US Laboratory module, NASA plans to launch climbing legs that when attached to the current on-orbit R2 upper body will give the robot the ability to traverse through the Space Station and start assisting crew with general IVA maintenance activities. Multiple control modes will be evaluated in this extra-ordinary ISS test environment to prepare the robot for use during EVAs. Ground Controllers will remotely supervise the robot as it executes semi-autonomous scripts for climbing through the Space Station and interacting with IVA interfaces. IVA crew will locally supervise the robot using the same scripts and also teleoperate the robot to simulate scenarios with the robot working alone or as an assistant during space walks.}
}
  24. Imaida, T., Yokokohji, Y., Doi, T., Oda, M. and Yoshikawa, T. (2003). Ground-Space Teleoperation of a Robot Arm Mounted on Engineering Test Satellite No. VII by Direct Bilateral Coupling under a Long Time Delay Condition . Journal of the Robotics Society of Japan, 3. Source
    BibTeX
    @article{imaida2003groundb,
  title = {Ground-Space Teleoperation of a Robot Arm Mounted on Engineering Test Satellite No. VII by Direct Bilateral Coupling under a Long Time Delay Condition},
  author = {Imaida, Takashi and Yokokohji, Yasuyoshi and Doi, Toshitsugu and Oda, Mitsushige and Yoshikawa, Tsuneo},
  journal = {Journal of the Robotics Society of Japan},
  volume = {21},
  number = {3},
  pages = {309--320},
  year = {2003},
  doi = {10.7210/jrsj.21.309},
  abstract = {A bilateral teleoperation experiment with Engineering Test Satellite No.VII (ETS-VII) was conducted on November 22, 1999. Round-trip time for communication between the NASDA ground station and ETS-VII was approximately six to seven seconds. We constructed a bilateral teleoperator that is stable even under such a long time delay. Several experiments, such as slope tracing task and peg-in-hole task, were carried out. Task performance was compared between bilateral mode and unilateral mode with force telemetry data visually displayed on a screen. All tasks were possible by bilateral control without any visual information. Experimental results showed that kinesthetic force feedback to the operator is helpful even under such long time delay and improves the performance of the task.}
}
  25. Oda, M. (1999). Mission of ETS-VII . Journal of the Robotics Society of Japan, 8. Source
    BibTeX
    @article{oda1999space,
  title = {Mission of ETS-VII},
  author = {Oda, Mitsushige},
  journal = {Journal of the Robotics Society of Japan},
  volume = {17},
  number = {8},
  pages = {1055--1061},
  year = {1999},
  doi = {10.7210/jrsj.17.1055}
}
  26. Landzettel, K., Brunner, B., Schreiber, G., Steinmetz, B.-M. and Hirzinger, G. (1999). The NASDA-DLR Joint Robotics Experiment GETEX . Journal of the Robotics Society of Japan, 8. Source
    BibTeX
    @article{landzettel1999nasdab,
  title = {The NASDA-DLR Joint Robotics Experiment GETEX},
  author = {Landzettel, Klaus and Brunner, Bernhard and Schreiber, Gerhard and Steinmetz, Bernhard-Michael and Hirzinger, Gerd},
  journal = {Journal of the Robotics Society of Japan},
  booktitle = {International Symposium on Artificial Intelligence, Robotics and Automation in Space},
  volume = {17},
  number = {8},
  pages = {1086--1095},
  year = {1999},
  doi = {10.7210/jrsj.17.1086},
  abstract = {This paper describes the GETEX (German ETS-VII EXperiments) joint robotics experiments, which has been conducted in April '99 at the first free-floating space robot on NASDA's ETS-VII satellite [1] . All the experiments have been performed and controlled from DLR's ground control system for space robotics applications which was connected to NASDA's ground station in Tsukuba. The telerobotic system combines sensor-based task-level teleprogramrning (as the basis for autonomy) with the features of teleoperation and shared autonomy. The hierarchical system structure is shown as well as the flexibility in programming and controlling each kind of space robotics application. This approach has led to a modular task-directed programming scheme, called Modular A & R Controller (MARCO), which provides a very flexible architecture to adapt the application-specific requirements to a given controlling scheme. A peg-in-hole experiment, using VR methods and the “vision & force” control scheme, by closing sensor control loops directly on-board (force) and via the ground control system (vision) is explained. During GETEX we conducted experiments with relevance to the behavior of ETS-VII in free motion mode in order to verify the existing dynamic models.}
}
  27. Tomlinson, Z., Gallagher, W., Cassidy, J., Roberts, B., Facciol, K. and Easley, J. (2021). Lessons for Future In-Space Telerobotic Servicing from Robotic Refueling Mission . IEEE Aerospace Conference, 20210022255. Source
    BibTeX
    @inproceedings{tomlinson2021lessons,
  title = {Lessons for Future In-Space Telerobotic Servicing from Robotic Refueling Mission},
  author = {Tomlinson, Zakiya and Gallagher, William and Cassidy, Justin and Roberts, Brian and Facciol, Kristen and Easley, Joseph},
  booktitle = {IEEE Aerospace Conference},
  number = {20210022255},
  pages = {1-17},
  institution = {NASA},
  year = {2021},
  doi = {10.1109/aero53065.2022.9843499},
  abstract = {The Robotic Refueling Mission (RRM) was a multi-phased technology development effort by the National Aeronautics & Space Administration (NASA) and the Canadian Space Agency (CSA). The program leveraged the existing robotic systems and expertise of the International Space Station (ISS) program and the tool design and satellite servicing expertise of NASA's Exploration & In-space Services (NExIS) Projects Division at Goddard Space Flight Center (GSFC) to evaluate new hardware and techniques for on-orbit telerobotic servicing. Between 2011 and 2022, two external ISS payloads housed over a dozen robotic tools and adapters designed to service a variety of existing and novel satellite interfaces. Robot operators at NASA's Johnson Space Center (JSC) and CSA used the Special Purpose Dexterous Manipulator (SPDM) to retrieve and operate these tools for tasks such as cutting wires or multi-layer insulation blanketing, removing valve caps, mating electrical connectors, transferring fluids, and performing visual inspections inside a vehicle. Each phase of RRM involved years of preparation. Tool and interface designs were prototyped and evaluated using both NASA and Canadian ground robotic systems. Procedures were developed by GSFC engineers and vetted in partnership with JSC and CSA robot operators. GSFC engineers were trained to provide real-time support during on-orbit operations. These preparatory efforts and the successful on-orbit evaluations yielded an array of lessons for future in-space telerobotic missions. Designing robotic tools for the space environment requires special consideration of materials, indicators, and differences between ground and flight use cases and environments. When there is a limited window for on-orbit operations, devoting time and high-fidelity hardware to ground testing can be critical. Needs during potential troubleshooting are more essential to camera view quality, frame rate, and position requirements than nominal operations. Detailed hardware manuals, nominal and contingency procedures, along with clearly defined operations team roles and protocols are vital for efficiency. RRM also demonstrated how the ISS can be utilized to increase the technology readiness levels required for future missions and led to additional technology partnerships between NExIS and the ISS program. The lessons from RRM are currently being applied to designs, operations concepts, and ground test methodology for missions such as On-orbit Servicing, Assembly, and Manufacturing 1 (OSAM-1) and Mars Sample Return.}
}
  28. Lippay, A. L. (1977). Multi-Axis Hand Controller for the Shuttle Remote Manipulator System . The Annual Conference on Manual Control, 19790009332. Source
    BibTeX
    @inproceedings{lippay1977multi,
  title = {Multi-Axis Hand Controller for the Shuttle Remote Manipulator System},
  author = {Lippay, Andrew L.},
  booktitle = {The Annual Conference on Manual Control},
  number = {19790009332},
  organization = {CAE Electronics, Montreal},
  address = {Cambridge, Massachusetts},
  year = {1977},
  url = {https://ntrs.nasa.gov/citations/19790009332},
  abstract = {The Shuttle Remote Manipulator System, has a articulated arm of 50 ft. length with six motor-driven joints. The basic purpose is to establish physical contact with various space hardware items and maneuver these to the desired position and attitude with respect to the Orbiter, nulling out relative velocities and stabilizing the free-body system by managing residual energies. The normal operating mode is resolved-motion end-point rate control by man-in-loop command. The translational freedoms are defined so that the End Effector (EEFTR) of the arm will move in planes parallel to the principal translational planes of the Orbiter, at a rate commanded by the displacement of the Translation Hand Controller in the corresponding freedom and direction. The rotational freedoms are rate-controlled by the Rotation Hand Controller about pivot axes parallel to Orbiter roll, pitch and yaw, originating at the EEFTR reference point.}
}
  29. Bensch, O., Bensch, L., Nilsson, T., Saling, F., Sadri, W. M., Hartmann, C., Hecking, T. and Kutz, J. N. (2024). Towards a Reliable Offline Personal AI Assistant for Long Duration Spaceflight . International Astronautical Congress, IAC-24-B3.7.12. Source
    BibTeX
    @inproceedings{bensch2024reliable,
  title = {Towards a Reliable Offline Personal AI Assistant for Long Duration Spaceflight},
  author = {Bensch, Oliver and Bensch, Leonie and Nilsson, Tommy and Saling, Florian and Sadri, Wafa M. and Hartmann, Carsten and Hecking, Tobias and Kutz, J. Nathan},
  booktitle = {International Astronautical Congress},
  number = {IAC-24-B3.7.12},
  pages = {336-346},
  year = {2024},
  doi = {10.52202/078364-0039}
}
  30. Ellery, A. (2019). Tutorial Review on Space Manipulators for Space Debris Mitigation . Robotics, 2. Source
    BibTeX
    @article{ellery2019tutorial,
  title = {Tutorial Review on Space Manipulators for Space Debris Mitigation},
  author = {Ellery, Alex},
  journal = {Robotics},
  volume = {8},
  number = {2},
  pages = {34},
  year = {2019},
  doi = {10.3390/robotics8020034},
  abstract = {Space-based manipulators have traditionally been tasked with robotic on-orbit servicing or assembly functions, but active debris removal has become a more urgent application. We present a much-needed tutorial review of many of the robotics aspects of active debris removal informed by activities in on-orbit servicing. We begin with a cursory review of on-orbit servicing manipulators followed by a short review on the space debris problem. Following brief consideration of the time delay problems in teleoperation, the meat of the paper explores the field of space robotics regarding the kinematics, dynamics and control of manipulators mounted onto spacecraft. The core of the issue concerns the spacecraft mounting which reacts in response to the motion of the manipulator. We favour the implementation of spacecraft attitude stabilisation to ease some of the computational issues that will become critical as increasing level of autonomy are implemented. We review issues concerned with physical manipulation and the problem of multiple arm operations. We conclude that space robotics is well-developed and sufficiently mature to tackling tasks such as active debris removal.}
}
  31. Arney, D., Mulvaney, J., Williams, C., Sutherland, R. and Stockdale, C. (2022). In-Space Servicing, Assembly, and Manufacturing (ISAM) State of Play, 2022 Edition . NASA, 20220010995. Source
    BibTeX
    @techreport{nasa2022space,
  title = {In-Space Servicing, Assembly, and Manufacturing (ISAM) State of Play, 2022 Edition},
  author = {Arney, Dale and Mulvaney, John and Williams, Christina and Sutherland, Richard and Stockdale, Christopher},
  number = {20220010995},
  institution = {NASA},
  year = {2022},
  url = {https://ntrs.nasa.gov/citations/20220010995},
  abstract = {The future of spaceflight will yield increasingly more ambitious missions to support civil, national security, and commercial space sectors. Achieving these ambitious missions is not feasible using the traditional paradigm of launching an integrated, fully functioning system on a single launch vehicle. For example, science and human exploration missions will desire payloads that are larger than any foreseeable launch vehicle fairing, national security missions will require persistent assets that are mobile and resilient, and commercial space missions will require cost-effective ways to update to the latest technology on orbit.

In-space Servicing, Assembly, and Manufacturing (ISAM) can vastly expand the performance, availability, and lifetime of space systems compared to the traditional paradigm of launching an asset with no intent to ever touch it again. ISAM capabilities foster an ecosystem that changes the space operations paradigm, creating the foundation for sustainable exploration and serving as a multiplier for other capabilities like space logistics, power generation, and reusability.

Previous achievements in ISAM have enabled ambitious human and robotic space missions. NASA’s International Space Station (ISS) operations and maintenance, servicing missions of the Hubble Space Telescope (HST), and Northrop Grumman’s Mission Extension Vehicle (MEV) demonstrate the dramatic operational missions that can be achieved using ISAM capabilities. Many current and upcoming flight demonstrations are advancing areas that will enable the next generation of civil, national security, and commercial space missions.

This document describes the current state of ISAM missions, capabilities, and developments. Compiling and organizing the available ISAM capabilities will help mission designers incorporate ISAM technologies into their concepts, create the starting point for technology development plans and roadmaps, and provide technologists a survey of the field they are developing. This document divides the ISAM capabilities into 11 capability areas that describe the functions or activities that would be performed in space using ISAM.

This version of the ISAM State of Play is part of a continuing journey to encourage the use of ISAM capabilities in space. Compiling and organizing the current state of ISAM provides a simple resource for those working in the ISAM ecosystem to ensure that the advancements being made build upon the investments of the past. The state of play is ever changing as new capabilities are developed, and this document will be periodically updated to ensure that it is relevant to those who need it in the future.}
}
  32. Bualat, M. G., Smith, T., Fong, T. W., Smith, E. E. and Wheeler, D. W. (2018). Astrobee: A New Tool for ISS Operations . SpaceOps Conference. Source
    BibTeX
    @inproceedings{bualat2018astrobee,
  title = {Astrobee: A New Tool for ISS Operations},
  author = {Bualat, Maria G. and Smith, Trey and Fong, Terrence W. and Smith, Ernest E. and Wheeler, D. W.},
  booktitle = {SpaceOps Conference},
  address = {Marseille, France},
  year = {2018},
  doi = {10.2514/6.2018-2517},
  abstract = {The Astrobee}
}
  33. (2026). DLR: Rollin' Justin. dlr.de/en/rm/research/robotic-systems/humanoids/rollin-justin
    BibTeX
    @misc{dlrrollin,
  title = {DLR: Rollin' Justin},
  organization = {dlr.de},
  year = {2026},
  url = {https://www.dlr.de/en/rm/research/robotic-systems/humanoids/rollin-justin}
}