Stereo Vision
Stereo vision is the range sensor for every Mars rover flown. There is no lidar and no radar on any of them: the terrain model that hazard detection, path selection and arm placement all consume is a point cloud produced by correlating two camera images [1]. The subsystem has to turn two images into metric range quickly enough on a radiation tolerant processor that driving is not stalled, and the environment works against it: Martian soil can be nearly textureless, the sun angle can erase shading, and a short baseline fixed on a mast or a deck sets a geometric ceiling on range accuracy [1][16]. Its cost on flight processors set the drive rate of every mission before Mars 2020 [4].
Design space and why flown systems converged
Section titled “Design space and why flown systems converged”Dense two-frame stereo methods differ in matching cost, aggregation, optimization and refinement [13]. The binocular principle itself, that depth can be recovered from disparity with no monocular pattern cue, was shown with random-dot stereograms in 1960 [9]. The planetary line starts at JPL in the 1970s: the Robotics Research Vehicle system calibrated a pinhole camera model by least squares, searched along the projected epipolar segment between a near and far bound with normalized cross-correlation, and triangulated by minimizing distance to the two rays; it was a laboratory system at about 2 m range [8]. The statistical treatment followed in 1992, with a maximum-likelihood framework for area-correlation disparity and a near-real-time embedded implementation on the Robby testbed vehicle [10].
Flown systems converged on one family: small-window SAD correlation on image pyramids, because each pyramid level cuts compute by a factor of eight and the processors were 12 to 20 MHz parts [12]. The 2002 software documentation of that chain gives rectification with CAHVORE camera models, a difference of Gaussians prefilter, a 7x7 correlator over a disparity range derived from a depth interval, and peak, left-right and blob filters; a navigation cycle on a 12 MHz R3000 took 42 s, of which correlation was 3.3 s [12]. Semi-Global Matching, which aggregates mutual-information costs along multiple 1D paths, was benchmarked at 1.0 to 1.3 s on a 2.8 GHz Xeon for one indoor test pair and flown on no planetary mission [17]. Its runtime and the survey comparisons are on indoor or airborne scenes, not rover imagery [13][17].
The algorithm
Section titled “The algorithm”The flight pipeline is an area-based correlator using the sum of absolute differences over a rectangular window, typically 7x7, searched along the epipolar line [3][5]. The steps are rectification using a geometric lens calibration, bandpass filtering, the SAD correlation over the disparity search range, consistency checks, subpixel disparity by quadratic interpolation of the SAD scores, and projection to XYZ range images [1][3]. A left-right uniqueness check and a threshold on the disparity gradient remove matches that are not supported by both views [5].
The variant used from 2011 onward adds SAD5: the score at each disparity is the SAD1 score of the center pixel plus the lowest two SAD1 scores of the four corner pixels at that disparity level, which improves range at object edges and range discontinuities [5].
Image sizes and what sets them
Section titled “Image sizes and what sets them”Full frame readout from the MER cameras is about 5 s per 1024x1024 12 bit image, so images for navigation are binned vertically in the CCD and read out at 256x1024, then averaged down to 256x256, and stereo matching runs at 256x256 [1]. The output is a 256x256 disparity image in which each pixel either encodes the 3D location of the terrain seen at that pixel in the rectified left image, or is unknown [2].
| Vehicle and camera | Field of view | Typical 3D points per pair |
|---|---|---|
| Spirit, front and rear hazard cameras | 125 degrees | about 15,000 |
| Opportunity, navigation cameras | 45 degrees | about 48,000 |
Source: [2]. Instrument placement uses a different configuration: one or two front hazard camera pairs at 1024x1024, downsampled to 512x512 and cropped to 420x412 for a coarse view, followed by a full resolution 241x241 subframe of the patch of terrain the arm will touch.
Camera geometry decides what the stereo can see. The wide field of view hazard cameras were sized to see more than the full width of the rover a short distance ahead, which is what obstacle avoidance and turn-in-place safety checks need, but their wide field of view and narrow baseline limit useful lookahead to 3 to 4 m. The navigation cameras see further with a narrower field of view and wider baseline, but their field of view is only wide enough to verify one candidate path several meters ahead [1]. The panoramic cameras are not used for autonomous navigation [1].
Range accuracy against distance
Section titled “Range accuracy against distance”Range error is set by the accuracy of the subpixel disparity estimate, which is why the correlator’s subpixel stage rather than its integer search decides how good the range is [3]. Evaluated for the flight camera geometries at 0.25 pixel correlation accuracy [7]:
| MER and MSL navigation cameras | MER panoramic cameras | MSL panoramic cameras | |
|---|---|---|---|
| Baseline | 0.20 m | 0.30 m | 0.20 m |
| Resolution | 1024x1024 | 1024x1024 | 1200x1200 |
| Field of view | 45 x 45 degrees | 16 x 16 degrees | 6 x 6 to 50 x 50 degrees |
| Range error at 50 m | 2.5 m | 0.60 m | 0.20 m at maximum zoom |
Source: [7], Table 1. Range error grows steeply with standoff, which is why the hazard camera lookahead limit is a geometry limit rather than a software one, and why traversability assessment weights nearby map cells more heavily than distant ones [1].
Accuracy: what limits disparity
Section titled “Accuracy: what limits disparity”Measured against synthetic pairs, image noise is nearly irrelevant to correlation stereo. Three structural effects dominate: foreshortening of non-fronto-parallel terrain (about 0.09 pixel standard deviation), camera misalignment after vibration (0.05 to 0.10 pixel), and the quadratic subpixel fit, which biases disparities toward integers by 0.04 to 0.05 pixel; combined they reach about 0.3 pixel, reducible to 0.10 to 0.15 pixel by pre-warping, pre-shifting and a quartic fit [11]. Pixel locking appears as artificial ripples in flat ground; an affine window adaptation cut RMS subpixel error by 78 percent on a ceiling and 86 percent on a floor in synthetic room imagery, with no runtime reported [24]. Validation of the MER correlator against surveyed brick targets covered 1 to 14 m on laboratory and yard scenes, not Mars imagery [14], and the ground-side MER correlator was spot checked on high-contrast survey targets only [15]. The ground pipeline adds nine rejection filters and image pyramids on a workstation, not on the flight processor [15].
Long-range geometry is unforgiving. For a 50 cm baseline, 1.04 mrad pixels and 1.53 m sensor height, one-sigma range error at 100 m is about 3.27 m while height error is about 5 cm, so height rather than range is the usable far cue on a vehicle of that class [19]. Range error scales as range squared over baseline; a 2 m baseline with 2592 pixels over a 12 degree field gave 14 cm range error at 85 m, and a helicopter test with a 1.7 m baseline held better than 1 percent depth error to 467 m but 4.85 percent at 516 m [30]. The helicopter test was post-flight on a laptop with ad hoc extrinsic calibration [30]. Prefilter choice matters at the margin: the bilateral background-subtraction prefilter raised correct disparity from 68.5 to 72.4 percent on one benchmark pair [3], and a night-driving study found that only cooled infrared focal planes held disparity noise at 0.06 to 0.12 pixel against a 0.3 pixel requirement [25]. That study concerns a terrestrial military vehicle and has no lunar counterpart [25].
Families beyond the flown correlator
Section titled “Families beyond the flown correlator”Obstacle detectors on stereo range. Seven independent detectors (positive, negative, trunk, slope, range density, water, overhang) found 20 of 21 surveyed obstacles with no false detections on a 170 m course at 1 m/s, run off line on a workstation [18]. A decade of JPL terrain mapping merged single-frame elevation, roughness and cost into a 100 m by 100 m map at 40 cm cells, and used four resolution swaths when full resolution was too costly [19]. Both are terrestrial programs with no planetary hardware [18][19].
Stereo-based motion estimation. Visual odometry on the rovers tracks features between Navcam stereo pairs. It converged on 97 percent of Spirit steps and 95 percent of Opportunity steps and cost two to three minutes per step on the RAD6000 [20]. The theory behind it shows that stereo ego-motion error grows as distance to the 3/2 power without an orientation sensor [21]. MSL-VO restructured the tracker around the pyramid; its speedups were measured on a non-flight computer [32]. On Curiosity, driving with processing in parallel raised the test drive rate from 51.5 to 77.5 m/hr on a Mars Yard testbed, with no flight results reported in that paper [31].
FPGA stereo. The correlator moved off the CPU on Perseverance [4]. Earlier JPL FPGA work integrated stereo and visual odometry on the Athena prototype rover; the MSL-equivalent timings in that paper were scaled from RAD6000 figures, not measured on a RAD750 [22]. A Virtex-5 plus Tilera manycore compute element built for lidar hazard detection on the Morpheus lander reported only gimbal pointing, and its dynamic pointing missed the error budget [23].
Learned stereo. Analysis of 16,768 Perseverance Navcam in-drive pairs [26] found that flight block matching leaves holes at solar phase angles under 20 degrees and on low-texture terrain. Zero-shot RAFT-Stereo and IGEV++ matched flight accuracy against a pseudo-truth network while raising coverage by 11.7 to 29.8 percent depending on condition [26]. A wavelet-enhanced network reported Abs Rel 0.089 against 0.103 for CRE-Stereo on simulated Zhurong-like imagery [27]. A fine-tuned MASt3R on 50,000 synthetic lunar south pole pairs raised slope correlation from 0.21 to 0.80, on synthetic test data only [29]. An SGM pipeline with superpixel refinement produced traversability maps from Devon Island images without ground truth [28]. None of these has run on a flight or space-qualified processor [26][27][28].
Recent flight and ground results
Section titled “Recent flight and ground results”Perseverance AutoNav. Perseverance moved stereo and visual odometry to a second RAD750 Vision Compute Element with a Xilinx V5QV FPGA, and widened the navigation cameras to 96x73 degrees with the neutral density filters removed, so exposures take 5 ms rather than 250 ms [34]. AutoNav evaluated 15,691 m of the first 17,682 m of driving, 88.7 percent through sol 753, against 6.1 percent for Curiosity [34]. The ENav account through sol 1312 (28 October 2024) reports 32.1 km driven at an average of 92 m/h, with the FPGA stereo feeding ACE collision checking [33]. The CPU usage figures there come from one 34 m drive on sol 753 [33].
Stereo mosaics for absolute localization. In flight on sol 859, Perseverance matched post-drive stereo orthomosaics against a HiRISE map using a census transform on the Ingenuity base station processor. The match took 32 s on the co-processor, 99 percent of benchmark cases fell within 0.93 m, and moving the images over the 10 KB/s serial link took 30 minutes [35]. The accuracy is scored against truth derived from the same class of orbital map [35].
Ground reconstruction products. Zhurong site digital elevation models were built at 0.02 m spacing out to 15 m, where total stereo measurement error is 0.108 m. Localization error against a HiRISE-visible track fell from 3.11 percent of traverse for dead reckoning to 1.75 percent for Affine-SIFT visual localization and 0.50 percent with orbital matching at four waypoints, the last largely manual [36]. The Mars 2020 Landform product aligns Hazcam, Navcam and Mastcam-Z stereo across rover positions and fuses it with orbital data into a 1 km mesh, with no independent accuracy assessment [42]. The VECTOR tool inspects bundle adjustment reprojection errors in Ingenuity Flight 26 stereo reconstruction, with no before and after error figures [41].
Mapping and small-rover stereo. The converging covariance map stores five covariance terms per cell. For the CADRE rovers, a 5 cm baseline pair 17.5 cm above ground, it detects obstacles over 48 cm at 5 m and runs at 80 ms per update on a Snapdragon 820; CADRE had not flown when it was written [37]. The IDEFIX navigation cameras for the MMX Phobos rover have a 6 cm baseline and 122 degree field, and are characterized on the ground only [38]. The Rashid-1 CASPEX cameras were calibrated and their stereo reconstruction exercised in a sand box; the rover was lost in the 25 April 2023 landing failure, so no lunar image exists [39].
Learned depth and datasets for the lunar poles. A RAFT-Stereo plus Gaussian splatting map reconstructed a 120 m synthetic south pole traverse with 2.8 cm mean height error, on synthetic data and a desktop GPU [40]. SPICE-HL3 supplies 1.3 million images in four low-angle lighting conditions with motion capture truth for testing stereo-inertial and visual odometry methods, over gravel coarser than lunar regolith [43].
Cost on flight hardware
Section titled “Cost on flight hardware”| Platform | Configuration | Time per stereo pair |
|---|---|---|
| MER flight processor, 20 MHz RAD6000 | 256x256 SAD | 24 to 30 s |
| Xeon 5160 at 3 GHz | rectification, bilateral subtraction filter, disparity | 7024 ms total, of which 74 ms disparity |
| Core 2 Quad at 2.4 GHz | same pipeline | 8827 ms total, of which 87 ms disparity |
| Virtex-4 LX160 at 66 MHz | rectification, bilateral filter, SAD1 at 1024x768 | 15 Hz fed directly from Camera Link, 12 Hz when raw imagery is loaded over the PCI bus |
| Perseverance Vision Compute Element, Virtex5QV | stereo correlation and visual odometry | 22 million disparities per second |
Sources: [1] for MER, [5] for the FPGA and desktop comparison, [4] for Perseverance. The FPGA figure in [5] is for a 1024x768 SAD1 system consuming 134 block RAMs and 49,924 slices on a Virtex-4 LX160 class part; a 512x384 SAD5 system takes 162 block RAMs and 38,581 slices. On the general purpose processors in that comparison, the bilateral subtraction filter rather than the correlation dominates, at 6947 ms of the 7024 ms total.
The per-step cost across missions tracks the pixel count rather than the clock alone [4]:
| Sojourner | Spirit and Opportunity | Curiosity | Perseverance | |
|---|---|---|---|---|
| Stereo pixels processed per step | 20 | 10,000 to 50,000 | 40,000 to 200,000 | 240,000 to 1,200,000 |
| Image pairs per step | 1 | 1 to 2 | 4 | 1 |
| Autonomous navigation pause per step | not reported | about 120 s | about 120 s | typically 0 if not steering |
Source: [4], Table 1. Perseverance processes about six times as many pixels per step as Curiosity in less time because the correlation runs in the Virtex-5 FPGA on the Vision Compute Element rather than on the RAD750.
The imaging software that feeds the correlator is itself a scheduling problem. MER Imaging Services holds requests in priority queues, two per priority level, and hands each image to a separate post-processing task as soon as the pixels are read so the camera control task can start the next exposure. Image buffers are a limited shared pool, and ground imaging commands queue when none is free [6].
Failure modes
Section titled “Failure modes”Texture starvation. Opportunity could not get acceptable range data from its hazard cameras at Meridiani: the fine texture of the rock-free soil gave inadequate texture for correlation at hazard camera resolution and baseline. The workaround was to use navigation camera stereo instead, and because Meridiani is largely obstacle free, it was sufficient to check the traversability of the nominal path forward and stop the vehicle if a hazard was detected [1]. This is the same terrain property that starves visual odometry, and it is the dominant flight failure mode for both.
Range discontinuities. Correlation windows that straddle a depth edge produce a blended disparity. SAD5 exists specifically to improve range at object edges and range discontinuities, and the disparity gradient threshold exists to discard the results that remain wrong [5].
Calibration. Range depends on the camera model. The MER images are labeled with geometric camera models shortly after acquisition, mapping pixel space to the body-fixed frame shared by all subsystems, so a calibration change propagates into every consumer of the range data [6].
What is not established
Section titled “What is not established”No work in the held set measures flight stereo range accuracy against independent truth on Mars: the camera paper’s range errors are propagated from geometry under an assumed 0.25 pixel correlation accuracy [16], and the Perseverance deep stereo study scores every method against a network’s output [26]. The cost of any learned or global method on a RAD750 or Virtex-5 class part is not reported in the cited papers [26][27]. Visual odometry figures for Mars are convergence rates, since no truth existed there [20]. Neither the MER imaging software paper nor the FPGA studies measured end-to-end drive-rate effects on Mars from the stereo stage alone [6][22].
References
- Maimone, M., Johnson, A., Cheng, Y., Willson, R. and Matthies, L. (2004). Autonomous Navigation Results from the Mars Exploration Rover (MER) Mission
. International Symposium on Experimental Robotics. Source
BibTeX
@inproceedings{maimone2004autonomous, title = {Autonomous Navigation Results from the Mars Exploration Rover (MER) Mission}, author = {Maimone, Mark and Johnson, Andrew and Cheng, Yang and Willson, Reg and Matthies, Larry}, booktitle = {International Symposium on Experimental Robotics}, address = {Singapore}, year = {2004}, url = {https://dataverse.jpl.nasa.gov/dataset.xhtml?persistentId=hdl:2014/41077} } - Maimone, M. W., Leger, P. C. and Biesiadecki, J. J. (2007). Overview of the Mars Exploration Rovers' Autonomous Mobility and Vision Capabilities
. IEEE International Conference on Robotics and Automation, Space Robotics Workshop. Source
BibTeX
@inproceedings{maimone2007overview, title = {Overview of the Mars Exploration Rovers' Autonomous Mobility and Vision Capabilities}, author = {Maimone, Mark W. and Leger, P. Chris and Biesiadecki, Jeffrey J.}, booktitle = {IEEE International Conference on Robotics and Automation, Space Robotics Workshop}, address = {Rome, Italy}, year = {2007}, url = {https://www-robotics.jpl.nasa.gov/media/documents/mer_autonomy_icra_2007.pdf} } - Ansar, A., Castano, A. and Matthies, L. (2004). Enhanced Real-time Stereo Using Bilateral Filtering
. Proceedings, International Symposium on 3D Data Processing, Visualization and Transmission. Source
BibTeX
@inproceedings{ansar2004enhanced, title = {Enhanced Real-time Stereo Using Bilateral Filtering}, author = {Ansar, Adnan and Castano, Andres and Matthies, Larry}, booktitle = {Proceedings, International Symposium on 3D Data Processing, Visualization and Transmission}, pages = {455--462}, year = {2004}, doi = {10.1109/tdpvt.2004.1335273}, abstract = {In recent years, there have been significant strides in increasing quality of range from stereo using global techniques such as energy minimization. These methods cannot yet achieve real-time performance. However, the need to improve range quality for real-time applications persists. All real-time stereo implementations rely on a simple correlation step which employs some local similarity metric between the left and right image. Typically, the correlation takes place on an image pair modified in some way to compensate for photometric variations between the left and right cameras. Improvements and modifications to such algorithms tend to fall into one of two broad categories: those which address the correlation step itself (e.g., shiftable windows, adaptive windows) and those which address the preprocessing of input imagery (e.g. band-pass filtering, Rank, Census). Our efforts lie in the latter area. We present in this paper a modification of the standard band-pass filtering technique used by many SSD- and SAD-based correlation algorithms. By using the bilateral filter of Tomasi and Manduchi [(1998)], we minimize blurring at the filtering stage. We show that in conjunction with SAD correlation, our new method improves stereo quality at range discontinuities while maintaining real-time performance.} } - Rankin, A., Del Sesto, T., Hwang, P., Justice, H., Maimone, M., Verma, V. and Graser, E. (2023). Perseverance Rapid Traverse Campaign
. IEEE Aerospace Conference. Source
BibTeX
@inproceedings{rankin2023perseverance, title = {Perseverance Rapid Traverse Campaign}, author = {Rankin, Arturo and Del Sesto, Tyler and Hwang, Pauline and Justice, Heather and Maimone, Mark and Verma, Vandi and Graser, Evan}, booktitle = {IEEE Aerospace Conference}, pages = {1-16}, address = {Big Sky, Montana}, year = {2023}, doi = {10.1109/aero55745.2023.10115835}, abstract = {Over the first 13 months of the Mars 2020 mission, the Perseverance rover traversed nearly 5 km along the Jezero Crater floor. Near the end of that period, the Science team was anxious to relocate to the ancient Delta region near the crater rim, over 5 km away. A Rapid Traverse Campaign was planned that would prioritize use of Perseverance's autonomous navigation software to drive at an unprecedented high pace and minimize science activities. The Rapid Traverse Campaign started in March 2022 and lasted 31 Martian days. During the campaign, Perseverance drove over 5 km in 24 drives, during which its autonomy software planned 94.8% of its overall driving, enabling it to set several new planetary rover driving records. Perseverance exceeded the longest daily drive distance record achieved by a previous planetary rover (219 meters) 11 times and set new records for the longest multi-sol drive distance in a single plan (528.7 meters) and the longest continuation drive (699.9 meters) by operating without human drive path input during 3 sols of driving. This paper details the planning and execution of the Rapid Traverse Campaign.} } - Villalpando, C. Y., Morfopoulos, A., Matthies, L. and Goldberg, S. (2011). FPGA Implementation of Stereo Disparity with High Throughput for Mobility Applications
. IEEE Aerospace Conference. Source
BibTeX
@inproceedings{villalpando2011fpga, title = {FPGA Implementation of Stereo Disparity with High Throughput for Mobility Applications}, author = {Villalpando, Carlos Y. and Morfopoulos, Arin and Matthies, Larry and Goldberg, Steven}, booktitle = {IEEE Aerospace Conference}, pages = {1--11}, address = {Big Sky, Montana}, year = {2011}, doi = {10.1109/aero.2011.5747269}, abstract = {High speed stereo vision can allow unmanned robotic systems to navigate safely in unstructured terrain, but the computational cost can exceed the capacity of typical embedded CPUs. In this paper, we describe an end-to-end stereo computation co-processing system optimized for fast throughput that has been implemented on a single Virtex 4 LX160 FPGA. This system is capable of operating on images from a 1024 × 768 3CCD (true RGB) camera pair at 15 Hz. Data enters the FPGA directly from the cameras via Camera Link and is rectified, pre-filtered and converted into a disparity image all within the FPGA, incurring no CPU load. Once complete, a rectified image and the final disparity image are read out over the PCI bus, for a bandwidth cost of 68 MB/sec. Within the FPGA there are 4 distinct algorithms: Camera Link capture, Bilinear rectification, Bilateral subtraction pre-filtering and the Sum of Absolute Difference (SAD) disparity. Each module will be described in brief along with the data flow and control logic for the system. The system has been successfully fielded upon the Carnegie Mellon University's National Robotics Engineering Center (NREC) Crusher system during extensive field trials in 2007 and 2008 and is being implemented for other surface mobility systems at JPL.} } - Litwin, T. E. and Maki, J. N. (2005). Imaging Services Flight Software on the Mars Exploration Rovers
. IEEE International Conference on Systems, Man and Cybernetics. Source
BibTeX
@inproceedings{litwin2005imaging, title = {Imaging Services Flight Software on the Mars Exploration Rovers}, author = {Litwin, Todd E. and Maki, Justin N.}, booktitle = {IEEE International Conference on Systems, Man and Cybernetics}, volume = {1}, pages = {895--900}, address = {Waikoloa, Hawaii}, year = {2005}, doi = {10.1109/icsmc.2005.1571260}, abstract = {The imaging services module of the Mars Exploration Rovers' on-board flight software is responsible for providing image data to the rest of the system. It acquires images from a suite of cameras, performs on-board image processing, labels the results with metadata, and delivers the final products to a diverse set of consumers, both on board and on the ground. The demands for flexibility and speed led to a design involving multiple tasks and a large set of parameters controlling the acquisition of the images, the on-board processing, and the method of product delivery.} } - Helmick, D. M., Angelova, A., Livianu, M. and Matthies, L. H. (2007). Terrain Adaptive Navigation for Mars Rovers
. IEEE Aerospace Conference. Source
BibTeX
@inproceedings{helmick2007terrain, title = {Terrain Adaptive Navigation for Mars Rovers}, author = {Helmick, Daniel M. and Angelova, Anelia and Livianu, Matthew and Matthies, Larry H.}, booktitle = {IEEE Aerospace Conference}, pages = {1-11}, address = {Big Sky, Montana}, year = {2007}, doi = {10.1109/aero.2007.352684}, abstract = {A navigation system for Mars rovers in very rough terrain has been designed, implemented, and tested on a research rover in Mars analog terrain. This navigation system consists of several technologies that are integrated to increase the capabilities compared to current rover navigation algorithms. These technologies include: goodness maps and terrain triage, terrain classification, remote slip prediction, path planning, high-fidelity traversability analysis (HFTA), and slip-compensated path following. The focus of this paper is not on the component technologies, but rather on the integration of these components. Results from the onboard integration of several of the key technologies described here are shown. Additionally, the results from independent demonstrations of several of these technologies are shown. Future work will include the demonstration of the entire integrated system.} } - Yakimovsky, Y. and Cunningham, R. (1978). A system for extracting three-dimensional measurements from a stereo pair of TV cameras
. Journal of Geophysical Research: Planets, 2. Source
BibTeX
@article{yakimovsky1978system, title = {A system for extracting three-dimensional measurements from a stereo pair of TV cameras}, author = {Yakimovsky, Y. and Cunningham, R.}, journal = {Journal of Geophysical Research: Planets}, volume = {7}, number = {2}, pages = {195-210}, publisher = {Elsevier BV}, year = {1978}, doi = {10.1016/0146-664x(78)90112-0} } - Julesz, B. (1960). Binocular Depth Perception of Computer-Generated Patterns
. The Bell System Technical Journal. Source
BibTeX
@article{julesz1960binocular, title = {Binocular Depth Perception of Computer-Generated Patterns}, author = {Julesz, Bela}, journal = {The Bell System Technical Journal}, volume = {39}, pages = {1125-1162}, year = {1960}, doi = {10.1002/j.1538-7305.1960.tb03954.x}, abstract = {The perception of depth involves monocular and binocular depth cues. The latter seem simpler and more suitable for investigation. Particularly important is the problem of finding binocular parallax, which involves matching patterns of the left and right visual fields. Stereo pictures of familiar objects or line drawings preclude the separation of interacting cues, and thus this pattern-matching process is difficult to investigate. More insight into the process can be gained by using unfamiliar picture material devoid of all cues except binocular parallax. To this end, artificial stereo picture pairs were generated on a digital computer. When viewed monocularly, they appear completely random, but if viewed binocularly, certain correlated point domains are seen in depth. By introducing distortions in this material and testing for perception of depth, it is possible to show that pattern-matching of corresponding points of the left and right visual fields can be achieved by first combining the two fields and then searching for patterns in the fused field. By this technique, some interesting properties of this fused binocular field are revealed, and a simple analog model is derived. The interaction between the monocular and binocular fields is also describea. A number of stereo images that demonstrate these and other findings are presented.} } - Matthies, L. H. (1992). Stereo vision for planetary rovers: Stochastic modeling to near real-time implementation
. International Journal of Computer Vision, 1. Source
BibTeX
@article{matthies1992stereo, title = {Stereo vision for planetary rovers: Stochastic modeling to near real-time implementation}, author = {Matthies, Larry H.}, journal = {International Journal of Computer Vision}, volume = {8}, number = {1}, pages = {71-91}, year = {1992}, doi = {10.1007/bf00126401}, abstract = {JPL has achieved the first autonomous cross-country robotic traverses to use stereo vision, with all computing onboard the vehicle. This paper describes the stereo vision system, including the underlying statistical model and the details of the implementation. It is argued that the overall approach provides a unifying paradigm for practical domain-independent stereo ranging.} } - Xiong, Y. and Matthies, L. (1997). Error Analysis of a Real-Time Stereo System
. IEEE Computer Society Conference on Computer Vision and Pattern Recognition. Source
BibTeX
@inproceedings{xiong1997error, title = {Error Analysis of a Real-Time Stereo System}, author = {Xiong, Yalin and Matthies, Larry}, booktitle = {IEEE Computer Society Conference on Computer Vision and Pattern Recognition}, pages = {1087-1093}, publisher = {IEEE Comput. Soc}, year = {1997}, doi = {10.1109/cvpr.1997.609465} } - Goldberg, S. B., Maimone, M. W. and Matthies, L. (2002). Stereo Vision and Rover Navigation Software for Planetary Exploration
. IEEE Aerospace Conference. Source
BibTeX
@inproceedings{goldberg2002stereo, title = {Stereo Vision and Rover Navigation Software for Planetary Exploration}, author = {Goldberg, Steven B. and Maimone, Mark W. and Matthies, Larry}, booktitle = {IEEE Aerospace Conference}, volume = {5}, pages = {2025--2036}, address = {Big Sky, Montana}, year = {2002}, doi = {10.1109/aero.2002.1035370}, abstract = {NASA's Mars Exploration Rover (MER) missions will land twin rovers on the surface of Mars in 2004. These rovers will have the ability to navigate safely through unknown and potentially hazardous terrain, using autonomous passive stereo vision to detect potential terrain hazards before driving into them. Unfortunately, the computational power of currently available radiation hardened processors limits the amount of distance (and therefore science) that can be safely achieved by any rover in a given time frame. We present overviews of our current rover vision and navigation systems, to provide context for the types of computation that are required to navigate safely. We also present baseline timing results that represent a lower bound in achievable performance (useful for systems engineering studies of future missions), and describe ways to improve that performance using commercial grade (as opposed to radiation hardened) processors. In particular, we document speedups to our stereo vision system that were achieved using the vectorized operations provided by Pentium MMX technology. Timing data were derived from implementations on several platforms: a prototype Mars rover with flight-like electronics (the Athena Software Development Model (SDM) rover), a RAD6000 computing platform (as will be used in the 2003 MER missions), and research platforms with commercial Pentium III and Sparc processors. Finally, we summarize the radiation effects analysis that suggests that commercial grade processors are likely to be adequate for Mars surface missions, and discuss the level of speedup that may accrue from using these instead of radiation hardened parts.} } - Scharstein, D. and Szeliski, R. (2002). A Taxonomy and Evaluation of Dense Two-Frame Stereo Correspondence Algorithms
. International Journal of Computer Vision. Source
BibTeX
@article{scharstein2002taxonomy, title = {A Taxonomy and Evaluation of Dense Two-Frame Stereo Correspondence Algorithms}, author = {Scharstein, Daniel and Szeliski, Richard}, journal = {International Journal of Computer Vision}, volume = {47}, pages = {7-42}, year = {2002}, doi = {10.1023/a:1014573219977} } - Ansar, A. I., Steele, R. D. and Steinke, R. C. (2005). Performance analysis and validation of a stereo vision system
. IEEE International Conference on Systems, Man and Cybernetics. Source
BibTeX
@inproceedings{ansar2005performance, title = {Performance analysis and validation of a stereo vision system}, author = {Ansar, Adnan I. and Steele, Robert D. and Steinke, Robert C.}, booktitle = {IEEE International Conference on Systems, Man and Cybernetics}, volume = {2}, pages = {1409-1416}, year = {2005}, doi = {10.1109/icsmc.2005.1571344}, abstract = {This paper presents an in-depth performance analysis and validation of a correlation based stereo vision system being used as part of the ongoing 2003 Mars Exploration Rover flight mission. Our analysis includes the effects of correlation window size, pyramidal image down-sampling, vertical misalignment, focus, maximum disparity, stereo baseline, and range ripples. A key element of validation is to determine the stereo localization error both analytically and experimentally. We study both down-range and cross-range error and verify that while camera calibration inaccuracy contributes to both, stereo correlation error affects only the former. Error contributions of subpixel interpolation, vertical misalignment, and foreshortening on stereo correlation are examined carefully. A novel method using bricks with reflective metrology targets and a mast-mounted stereo camera system enabled experimental measurements of the stereo disparity error. The standard deviation of the down-range disparity error was measured at sigma=0.32 pixel for high-resolution 1024times768 camera images. The result is critical in evaluating accurate rover navigation and instrument placement within given error budgets} } - Deen, R. G. and Lorre, J. (2005). Seeing in three dimensions: correlation and triangulation of Mars Exploration Rover imagery
. IEEE International Conference on Systems, Man and Cybernetics. Source
BibTeX
@inproceedings{deen2005seeing, title = {Seeing in three dimensions: correlation and triangulation of Mars Exploration Rover imagery}, author = {Deen, Robert G. and Lorre, Jean}, booktitle = {IEEE International Conference on Systems, Man and Cybernetics}, volume = {1}, pages = {911-916}, year = {2005}, doi = {10.1109/icsmc.2005.1571262}, abstract = {The Mars Exploration Rovers (MER) uses a man-in-the-loop system for control in most cases. While capable of some autonomous driving, all arm operations and most drives are planned on the ground. Planning these operations requires a precise knowledge of the terrain surrounding of the rover: where are the rocks, the sand, and the hazards. This terrain is derived from images taken by stereoscopic cameras. This paper describes in detail the middle parts of the ground-based terrain derivation process: correlation, which finds matching points in the stereo pair, and triangulation, which converts those points to XYZ coordinates. The algorithms and free parameters are described, followed by a discussion of the results obtained, the problems encountered, and possible avenues for future development.} } - Maki, J. N., Bell III, J. F., Herkenhoff, K. E., Squyres, S. W., Kiely, A., Klimesh, M., Schwochert, M., Litwin, T., Willson, R., Johnson, A., Maimone, M., Baumgartner, E., Collins, A., Wadsworth, M., Elliot, S. T., Dingizian, A., Brown, D., Hagerott, E. C., Scherr, L., Deen, R., Alexander, D. and Lorre, J. (2003). Mars Exploration Rover Engineering Cameras
. Journal of Geophysical Research: Planets, E12. Source
BibTeX
@article{maki2003mars, title = {Mars Exploration Rover Engineering Cameras}, author = {Maki, Justin N. and Bell III, J. F. and Herkenhoff, K. E. and Squyres, Steven W. and Kiely, A. and Klimesh, M. and Schwochert, Mark and Litwin, T. and Willson, R. and Johnson, A. and Maimone, M. and Baumgartner, E. and Collins, A. and Wadsworth, M. and Elliot, S. T. and Dingizian, A. and Brown, David and Hagerott, E. C. and Scherr, L. and Deen, R. and Alexander, Doug and Lorre, J.}, journal = {Journal of Geophysical Research: Planets}, volume = {108}, number = {E12}, pages = {8071}, year = {2003}, doi = {10.1029/2003je002077}, abstract = {NASA's Mars Exploration Rover (MER) Mission will place a total of 20 cameras (10 per rover) onto the surface of Mars in early 2004. Fourteen of the 20 cameras are designated as engineering cameras and will support the operation of the vehicles on the Martian surface. Images returned from the engineering cameras will also be of significant importance to the scientific community for investigative studies of rock and soil morphology. The Navigation cameras (Navcams, two per rover) are a mast‐mounted stereo pair each with a 45° square field of view (FOV) and an angular resolution of 0.82 milliradians per pixel (mrad/pixel). The Hazard Avoidance cameras (Hazcams, four per rover) are a body‐mounted, front‐ and rear‐facing set of stereo pairs, each with a 124° square FOV and an angular resolution of 2.1 mrad/pixel. The Descent camera (one per rover), mounted to the lander, has a 45° square FOV and will return images with spatial resolutions of ∼4 m/pixel. All of the engineering cameras utilize broadband visible filters and 1024 × 1024 pixel detectors.} } - Hirschmuller, H. (2005). Accurate and Efficient Stereo Processing by Semi-Global Matching and Mutual Information
. IEEE Computer Society Conference on Computer Vision and Pattern Recognition. Source
BibTeX
@inproceedings{hirschmuller2005accurate, title = {Accurate and Efficient Stereo Processing by Semi-Global Matching and Mutual Information}, author = {Hirschmuller, Heiko}, booktitle = {IEEE Computer Society Conference on Computer Vision and Pattern Recognition}, volume = {2}, pages = {807-814}, publisher = {IEEE}, year = {2005}, doi = {10.1109/cvpr.2005.56}, abstract = {This paper considers the objectives of accurate stereo matching, especially at object boundaries, robustness against recording or illumination changes and efficiency of the calculation. These objectives lead to the proposed semi-global matching method that performs pixelwise matching based on mutual information and the approximation of a global smoothness constraint. Occlusions are detected and disparities determined with sub-pixel accuracy. Additionally, an extension for multi-baseline stereo images is presented. There are two novel contributions. Firstly, a hierarchical calculation of mutual information based matching is shown, which is almost as fast as intensity based matching. Secondly, an approximation of a global cost calculation is proposed that can be performed in a time that is linear to the number of pixels and disparities. The implementation requires just 1 second on typical images.} } - Rankin, A., Huertas, A. and Matthies, L. (2005). Evaluation of stereo vision obstacle detection algorithms for off-road autonomous navigation
. AUVSIs Unmanned Systems North America. Source
BibTeX
@inproceedings{rankin2005evaluation, title = {Evaluation of stereo vision obstacle detection algorithms for off-road autonomous navigation}, author = {Rankin, Arturo and Huertas, Andres and Matthies, Larry}, booktitle = {AUVSIs Unmanned Systems North America}, publisher = {JPL Open Repository}, year = {2005}, url = {https://dataverse.jpl.nasa.gov/dataset.xhtml?persistentId=hdl:2014/38362} } - Rankin, A. L., Huertas, A. and Matthies, L. H. (2009). Stereo vision based terrain mapping for off-road autonomous navigation
. Unmanned Systems Technology XI. Source
BibTeX
@inproceedings{rankin2009stereo, title = {Stereo vision based terrain mapping for off-road autonomous navigation}, author = {Rankin, Arturo L. and Huertas, Andres and Matthies, Larry H.}, booktitle = {Unmanned Systems Technology XI}, series = {SPIE Proceedings}, volume = {7332}, pages = {733210}, publisher = {SPIE}, year = {2009}, doi = {10.1117/12.819099}, abstract = {Successful off-road autonomous navigation by an unmanned ground vehicle (UGV) requires reliable perception and representation of natural terrain. While perception algorithms are used to detect driving hazards, terrain mapping algorithms are used to represent the detected hazards in a world model a UGV can use to plan safe paths. There are two primary ways to detect driving hazards with perception sensors mounted to a UGV: binary obstacle detection and traversability cost analysis. Binary obstacle detectors label terrain as either traversable or non-traversable, whereas, traversability cost analysis assigns a cost to driving over a discrete patch of terrain. In uncluttered environments where the non-obstacle terrain is equally traversable, binary obstacle detection is sufficient. However, in cluttered environments, some form of traversability cost analysis is necessary. The Jet Propulsion Laboratory (JPL) has explored both approaches using stereo vision systems. A set of binary detectors has been implemented that detect positive obstacles, negative obstacles, tree trunks, tree lines, excessive slope, low overhangs, and water bodies. A compact terrain map is built from each frame of stereo images. The mapping algorithm labels cells that contain obstacles as nogo regions, and encodes terrain elevation, terrain classification, terrain roughness, traversability cost, and a confidence value. The single frame maps are merged into a world map where temporal filtering is applied. In previous papers, we have described our perception algorithms that perform binary obstacle detection. In this paper, we summarize the terrain mapping capabilities that JPL has implemented during several UGV programs over the last decade and discuss some challenges to building terrain maps with stereo range data.} } - Cheng, Y. and Maimone, M. (2005). Visual odometry on the Mars Exploration Rovers
. IEEE International Conference on Systems, Man and Cybernetics. Source
BibTeX
@inproceedings{cheng2005visual, title = {Visual odometry on the Mars Exploration Rovers}, author = {Cheng, Yang and Maimone, Mark}, booktitle = {IEEE International Conference on Systems, Man and Cybernetics}, volume = {1}, pages = {903-910}, publisher = {IEEE}, year = {2005}, doi = {10.1109/icsmc.2005.1571261}, abstract = {NASA's Mars Exploration Rovers (MER) was designed to traverse in Viking Lander-I style terrains: mostly flat, with many small non-obstacle rocks and occasional obstacles. During actual operations in such terrains, onboard position estimates derived solely from the onboard inertial measurement unit and wheel encoder-based odometry achieved well within the design goal of at most 10% error. However, MER vehicles were also driven along slippery slopes tilted as high as 31 degrees. In such conditions an additional capability was employed to maintain a sufficiently accurate onboard position estimate: visual odometry. The MER visual odometry system comprises onboard software for comparing stereo pairs taken by the pointable mast-mounted 45 degree FOV navigation cameras (NAV-CAMs). The system computes an update to the 6-DOF rover pose (x, y, z, roll, pitch, yaw) by tracking the motion of autonomously-selected "interesting" terrain features between two pairs of stereo images, in both 2D pixel and 3D world coordinates. A maximum likelihood estimator is applied to the computed 3D offsets to produce a final, corrected estimate of vehicle motion between the two pairs. In this paper we describe the visual odometry algorithm used on the Mars Exploration Rovers, and summarize its results from the first year of operations on Mars.} } - Olson, C., Matthies, L., Schoppers, M. and Maimone, M. (2000). Robust Stereo Ego-Motion for Long Distance Navigation
. IEEE Conference on Computer Vision and Pattern Recognition. Source
BibTeX
@inproceedings{olson2000robust, title = {Robust Stereo Ego-Motion for Long Distance Navigation}, author = {Olson, C. and Matthies, L. and Schoppers, M. and Maimone, M.}, booktitle = {IEEE Conference on Computer Vision and Pattern Recognition}, volume = {2}, pages = {453-458}, publisher = {IEEE Comput. Soc}, year = {2000}, doi = {10.1109/cvpr.2000.854879}, abstract = {Several methods for computing observer motion from monocular and stereo image sequences have been proposed. However, accurate positioning over long distances requires a higher level of robustness than previously achieved. This paper describes several mechanisms for improving robustness in the context of a maximum-likelihood stereo ego-motion method. We demonstrate that even a robust system will accumulate super-linear error in the distance traveled due to increasing orientation errors. However, when an absolute orientation sensor is incorporated, the growth is reduced to linear in the distance traveled, grows much more slowly in practice. Our experiments, including a trial with 210 stereo pairs, indicate that these techniques can achieve errors below 1% of the distance traveled. This method has been implemented to run on-board a prototype Mars rover.} } - Howard, T. M., Morfopoulos, A., Morrison, J., Kuwata, Y., Villalpando, C., Matthies, L. and McHenry, M. (2012). Enabling continuous planetary rover navigation through FPGA stereo and visual odometry
. AIAA SPACE and Astronautics Forum. Source
BibTeX
@inproceedings{howard2012enabling, title = {Enabling continuous planetary rover navigation through FPGA stereo and visual odometry}, author = {Howard, Thomas M. and Morfopoulos, A. and Morrison, Jack and Kuwata, Yoshiaki and Villalpando, C. and Matthies, L. and McHenry, M.}, booktitle = {AIAA SPACE and Astronautics Forum}, pages = {1-9}, publisher = {IEEE}, year = {2012}, doi = {10.1109/aero.2012.6187041}, abstract = {Safe navigation under resource constraints is a key concern for autonomous planetary rovers operating on extraterrestrial bodies. Computational power in such applications is typically constrained by the radiation hardness and energy consumption requirements. For example, even though the microprocessors used for the Mars Science Laboratory (MSL) mission rover are an order of magnitude more powerful than those used for the rovers on the Mars Exploration Rovers (MER) mission, the computational power is still significantly less than that of contemporary desktop microprocessors. It is therefore important to move safely and efficiently through the environment while consuming a minimum amount of computational resources, energy and time. Perception, pose estimation, and motion planning are generally three of the most computationally expensive processes in modern autonomy navigation architectures. An example of this is on the MER where each rover must stop, acquire and process imagery to evaluate its surroundings, estimate the relative change in pose, and generate the next mobility system maneuver [1]. This paper describes improvements in the energy efficiency and speed of planetary rover autonomous traverse accomplished by converting processes typically performed by the CPU onto a Field Programmable Gate Arrays (FPGA) coprocessor. Perception algorithms in general are well suited to FPGA implementations because much of processing is naturally parallelizable. In this paper we present novel implementations of stereo and visual odometry algorithms on a FPGA. The FPGA stereo implementation is an extension of [2] that uses "random in linear out" rectification and a higher-performance interface between the rectification, filter, and disparity stages of the stereo pipeline. The improved visual odometry component utilizes a FPGA implementation of a Harris feature detector and sum of absolute differences (SAD) operator. The FPGA implementation of the stereo and visual odometry functionality have demonstrated a performance improvement of approximately three orders of magnitude compared to the MER-class avionics. These more efficient perception and pose estimation modules have been merged with motion planning techniques that allow for continuous steering and driving to navigate cluttered obstacle fields without stopping to perceive. The resulting faster visual odometry rates also allow for wheel slip to be detected earlier and more reliably. Predictions of resulting improvements in planetary rover energy efficiency and average traverse speeds are reported. In addition, field results are presented that compare the performance of autonomous navigation on the Athena planetary rover prototype using continuous steering or driving and continuous steering and driving with GESTALT traversability analysis using the FPGA perception and pose estimation improvements.} } - Villalpando, C. Y., Werner, R. A., Carson III, J. M., Khanoyan, G., Stern, R. A. and Trawny, N. (2013). A hybrid FPGA/Tilera compute element for autonomous hazard detection and navigation
. IEEE Aerospace Conference. Source
BibTeX
@inproceedings{villalpando2013hybrid, title = {A hybrid FPGA/Tilera compute element for autonomous hazard detection and navigation}, author = {Villalpando, Carlos Y. and Werner, Robert A. and Carson III, John M. and Khanoyan, Garen and Stern, Ryan A. and Trawny, Nikolas}, booktitle = {IEEE Aerospace Conference}, pages = {1--9}, publisher = {IEEE}, year = {2013}, doi = {10.1109/aero.2013.6496977}, abstract = {To increase safety for future missions landing on other planetary or lunar bodies, the Autonomous Landing and Hazard Avoidance Technology (ALHAT) program is developing an integrated sensor for autonomous surface analysis and hazard determination. The ALHAT Hazard Detection System (HDS) consists of a Flash LIDAR for measuring the topography of the landing site, a gimbal to scan across the terrain, and an Inertial Measurement Unit (IMU), along with terrain analysis algorithms to identify the landing site and the local hazards. An FPGA and Manycore processor system was developed to interface all the devices in the HDS, to provide high-resolution timing to accurately measure system state, and to run the surface analysis algorithms quickly and efficiently. In this paper, we will describe how we integrated COTS components such as an FPGA evaluation board, a TILExpress64, and multi-threaded/multi-core aware software to build the HDS Compute Element (HDSCE). The ALHAT program is also working with the NASA Morpheus Project and has integrated the HDS as a sensor on the Morpheus Lander. This paper will also describe how the HDS is integrated with the Morpheus lander and the results of the initial test flights with the HDS installed. We will also describe future improvements to the HDSCE.} } - Stein, A. N., Huertas, A. and Matthies, L. H. (2006). Attenuating stereo pixel-locking via affine window adaptation
. IEEE International Conference on Robotics and Automation. Source
BibTeX
@inproceedings{stein2006attenuating, title = {Attenuating stereo pixel-locking via affine window adaptation}, author = {Stein, Andrew N. and Huertas, Andres and Matthies, Larry H.}, booktitle = {IEEE International Conference on Robotics and Automation}, pages = {914-921}, publisher = {IEEE}, year = {2006}, doi = {10.1109/robot.2006.1641826}, abstract = {For real-time stereo vision systems, the standard method for estimating sub-pixel stereo disparity given an initial integer disparity map involves fitting parabolas to a matching cost function aggregated over rectangular windows. This results in a phenomenon known as pixel-locking, which produces artificially-peaked histograms of sub-pixel disparity. These peaks correspond to the introduction of erroneous ripples or waves in the 3D reconstruction of truly flat surfaces. Since stereo vision is a common input modality for autonomous vehicles, these inaccuracies can pose a problem for safe, reliable navigation. This paper proposes a new method for sub-pixel stereo disparity estimation, based on ideas from Lucas-Kanade tracking and optical flow, which substantially reduces the pixel-locking effect. In addition, it has the ability to correct much larger initial disparity errors than previous approaches and is more general as it applies not only to the ground plane. We demonstrate the method on synthetic imagery as well as real stereo data from an autonomous outdoor vehicle} } - Owens, K. and Matthies, L. (1999). Passive Night Vision Sensor Comparison for Unmanned Ground Vehicle Stereo Vision Navigation
. IEEE Workshop on Computer Vision Beyond the Visible Spectrum: Methods and Applications. Source
BibTeX
@inproceedings{owens1999passive, title = {Passive Night Vision Sensor Comparison for Unmanned Ground Vehicle Stereo Vision Navigation}, author = {Owens, K. and Matthies, L.}, booktitle = {IEEE Workshop on Computer Vision Beyond the Visible Spectrum: Methods and Applications}, pages = {59-68}, publisher = {IEEE Comput. Soc}, year = {1999}, doi = {10.1109/cvbvs.1999.781095}, abstract = {One goal of the "Demo III" unmanned ground vehicle program is to enable autonomous nighttime navigation at speeds of up to 10 m.p.h. To perform obstacle detection at night with stereo vision will require night vision cameras that produce adequate image quality for the driving speeds, vehicle dynamics, obstacle sizes, and scene conditions that will be encountered. This paper analyzes the suitability of four classes of night vision cameras (3-5 /spl mu/m cooled FLIR, 8-12 /spl mu/m cooled FLIR, 8-12 /spl mu/m uncooled FLIR, and image intensifiers) for night stereo vision, using criteria based on stereo matching quality, image signal to noise ratio, motion blur and synchronization capability. We find that only cooled FLIRs will enable stereo vision performance that meets the goals of the Demo III program for nighttime autonomous mobility.} } - Liu, Y., Rothrock, B., Swan, R. M., Ono, H., Scherer, S. and Ono, M. (2026). Interplanetary Generalization and Adaptation of Deep Stereo on Mars
. IEEE Aerospace Conference. Source
BibTeX
@inproceedings{liu2026interplanetary, title = {Interplanetary Generalization and Adaptation of Deep Stereo on Mars}, author = {Liu, Yifei and Rothrock, Brandon and Swan, R. Michael and Ono, Hosei and Scherer, Sebastian and Ono, Masahiro}, booktitle = {IEEE Aerospace Conference}, publisher = {JPL Open Repository}, year = {2026}, doi = {10.48577/jpl.6btrp7}, abstract = {Stereo vision is critical for the autonomous operation ofthe Curiosity and Perseverance rovers on Mars. These rovers employa classical stereo algorithm based on block-matching to estimatedepth from a pair of navigation cameras, which is used to build a3D map of the terrain. The choice of this algorithm was motivatedprimarily by the computational constraints of radiation-hardenedflight hardware and heritage from previous Mars rover missions.However, recent data from Perseverance have revealed cases wherestereo perception was significantly degraded, leading to multipleinstances where autonomous navigation was no longer possible.This work analyzes the Flight Stereo performance of Perseveranceand identifies several contributing factors that lead to degradedperformance, primarily featureless terrain and imaging at low solarphase angle. Using a dataset curated from the navigation camerasover the entire Mars 2020 mission to date, we provide an empiricalanalysis of stereo performance under these conditions and evaluateadvanced learning-based methods that mitigate these failures.Representative deep stereo approaches, including cost-volume anditerative optimization methods, transformer-based architectures, andfoundation models, show strong zero-shot performance on real Marsimagery. We further present a domain adaptation strategy thatleverages foundation models as pseudo ground truth, showing itsclose alignment with Flight Stereo while providing dense disparityestimates. Finally, we analyze computational trade-offs to assessthe feasibility of deploying these algorithms on emerging spacequalifiedhardware. We hope these results will inform designconsiderations for stereo vision systems in future rover missions andcontribute toward more robust vision-based autonomous navigation.} } - Li, J., Wang, J., Li, M., Liu, Y., Wang, Y. and Xu, H. (2025). M3Depth: Wavelet-Enhanced Depth Estimation on Mars via Mutual Boosting of Dual-Modal Data. arxiv.org/abs/2505.14159v2
BibTeX
@misc{li2025m3depth, title = {M3Depth: Wavelet-Enhanced Depth Estimation on Mars via Mutual Boosting of Dual-Modal Data}, author = {Li, Junjie and Wang, Jiawei and Li, Miyu and Liu, Yu and Wang, Yumei and Xu, Haitao}, year = {2025}, url = {https://arxiv.org/abs/2505.14159v2} } - Lu, Y.-S., Arana-Catania, M., Upadhyay, S. and Felicetti, L. (2025). Stereovision Image Processing for Planetary Navigation Maps with Semi-Global Matching and Superpixel Segmentation. arxiv.org/abs/2509.05645
BibTeX
@misc{lu2025stereovision, title = {Stereovision Image Processing for Planetary Navigation Maps with Semi-Global Matching and Superpixel Segmentation}, author = {Lu, Yan-Shan and Arana-Catania, Miguel and Upadhyay, Saurabh and Felicetti, Leonard}, year = {2025}, url = {https://arxiv.org/abs/2509.05645} } - Grethen, C., Gasparini, S., Morin, G., Lebreton, J., Marti, L. and Sánchez-Gestido, M. (2025). Adapting Stereo Vision From Objects To 3D Lunar Surface Reconstruction with the StereoLunar Dataset
. arXiv. Source
BibTeX
@article{grethen2025adapting, title = {Adapting Stereo Vision From Objects To 3D Lunar Surface Reconstruction with the StereoLunar Dataset}, author = {Grethen, Clementine and Gasparini, Simone and Morin, Géraldine and Lebreton, J. and Marti, Lucas and Sánchez-Gestido, Manuel}, journal = {arXiv}, year = {2025}, doi = {10.48550/arxiv.2510.18172}, abstract = {Accurate 3D reconstruction of lunar surfaces is essential for space exploration. However, existing stereo vision reconstruction methods struggle in this context due to the Moon's lack of texture, difficult lighting variations, and atypical orbital trajectories. State-of-the-art deep learning models, trained on human-scale datasets, have rarely been tested on planetary imagery and cannot be transferred directly to lunar conditions. To address this issue, we introduce LunarStereo, the first open dataset of photorealistic stereo image pairs of the Moon, simulated using ray tracing based on high-resolution topography and reflectance models. It covers diverse altitudes, lighting conditions, and viewing angles around the lunar South Pole, offering physically grounded supervision for 3D reconstruction tasks. Based on this dataset, we adapt the MASt3R model to the lunar domain through fine-tuning on LunarStereo. We validate our approach through extensive qualitative and quantitative experiments on both synthetic and real lunar data, evaluating 3D surface reconstruction and relative pose estimation. Extensive experiments on synthetic and real lunar data validate the approach, demonstrating significant improvements over zero-shot baselines and paving the way for robust cross-scale generalization in extraterrestrial environments.} } - Sternberg, D. C., Setterfield, T. P., Bailey, E. S., Ansar, A. I. and Johnson, A. E. (2019). Demonstration of stereo vision for deorbit descent and landing
. Annual AAS Guidance and Control Conference. Source
BibTeX
@inproceedings{sternberg2019demonstration, title = {Demonstration of stereo vision for deorbit descent and landing}, author = {Sternberg, David C. and Setterfield, Timothy P. and Bailey, Erik S. and Ansar, Adnan I. and Johnson, Andrew E.}, booktitle = {Annual AAS Guidance and Control Conference}, publisher = {JPL Open Repository}, address = {Breckinridge, Colorado}, year = {2019}, url = {https://dataverse.jpl.nasa.gov/dataset.xhtml?persistentId=hdl:2014/50664} } - Rankin, A., Holloway, A., Sabel, A., Patel, N. and Maimone, M. W. (2022). Visual Odometry Thinking While Driving for the Curiosity Mars Rover's Three-Year Test Campaign: Impact of Evolving Constraints on Verification and Validation
. IEEE Aerospace Conference, 20230005759. Source
BibTeX
@inproceedings{rankin2022visual, title = {Visual Odometry Thinking While Driving for the Curiosity Mars Rover's Three-Year Test Campaign: Impact of Evolving Constraints on Verification and Validation}, author = {Rankin, Arturo and Holloway, Alexandra and Sabel, Anna and Patel, Nikunj and Maimone, Mark W.}, booktitle = {IEEE Aerospace Conference}, number = {20230005759}, pages = {1-10}, institution = {NASA}, year = {2022}, doi = {10.1109/aero53065.2022.9843487}, abstract = {Over the first 9 years of the Mars Science Laboratory (MSL) Curiosity rover's surface mission, more than 87% of its driving was performed using Visual Odometry (VO). The benefits of using VO during driving are that it minimizes rover position uncertainty and can be used to monitor wheel slip, halting a drive if excessive wheel slip is occurring. The VO implementation onboard Curiosity acquires and processes VO images in between drive steps while the rover is stationary. A VO Thinking While Driving (VTWD) flight software capability has been developed to enable the processing of VO images during rover driving, increasing the distance Curiosity can drive using VO during a given time period up to as much as 1.75x total distance. Verification and Validation (V&V) of this capability has been challenging due to impacts from the COVID-19 pandemic and unavailability of the JPL Mars Yard outdoor test site. The VTWD V&Vtest procedures were modified to use a small indoor space with Mars-like terrain. This paper describes the 3 year V&V effort under challenging conditions to approve the VTWD capability for use on the Curiosity rover.} } - Johnson, A. E., Goldberg, S. B., Cheng, Y. and Matthies, L. H. (2008). Robust and Efficient Stereo Feature Tracking for Visual Odometry
. IEEE International Conference on Robotics and Automation. Source
BibTeX
@inproceedings{johnson2008robust, title = {Robust and Efficient Stereo Feature Tracking for Visual Odometry}, author = {Johnson, Andrew E. and Goldberg, Steven B. and Cheng, Yang and Matthies, Larry H.}, booktitle = {IEEE International Conference on Robotics and Automation}, pages = {39-46}, publisher = {IEEE}, year = {2008}, doi = {10.1109/robot.2008.4543184}, abstract = {Visual odometry can augment or replace wheel odometry when navigating in high slip terrain which is quite important for autonomous navigation on Mars. We present a computationally efficient and robust visual odometry algorithm developed for the Mars Science Laboratory mission. This algorithm is a significant improvement over the algorithm developed for the Mars Exploration Rover Mission because it is at least four time more computationally efficient and it tracks significantly more features. The core of the algorithm is an integrated motion estimation and stereo feature tracking loop that allows for feature recovery while guiding feature correlation search to minimize computation. Results on thousands of terrestrial and Martian stereo pairs show that the algorithm can operate with no initial motion estimate while still obtaining subpixel attitude estimation performance.} } - Toupet, O., Ono, M., del Sesto, T., Maimone, M. and McHenry, M. (2026). Enhanced Autonomous Navigation on the Perseverance Mars Rover
. IEEE Transactions on Field Robotics. Source
BibTeX
@article{toupet2026enhanced, title = {Enhanced Autonomous Navigation on the Perseverance Mars Rover}, author = {Toupet, Olivier and Ono, Masahiro and del Sesto, Tyler and Maimone, Mark and McHenry, Michael}, journal = {IEEE Transactions on Field Robotics}, volume = {3}, pages = {83--124}, year = {2026}, doi = {10.1109/tfr.2025.3636366}, abstract = {This paper presentsEnhanced Autonomous Navigation, orENav, the autonomous driving algorithm of NASA’s Mars roverPerseverance. A unique challenge for the autonomous driving ofPerseveranceis to meet strict safety and performance requirements in a highly uncertain environment with only a single-core CPU with extremely limited computing resources. ENav overcame this challenge with a novel two-stage path selection approach that balances the path optimality and computational efficiency, combined with a unique collision checking algorithm that conservatively approximates computationally expensive kinematic settling (Otsu et al., 2020). In addition, ENav provides robustness against slip by expanding the bounding boxes for wheels used by the collision check. These new features, together with FPGA-accelerated vision processing, enabledPerseveranceto autonomously drive on substantially more rock-dense terrains and increased the average daily driving distance by an order of magnitude compared to its predecessors, theCuriosity, Spirit, andOpportunityrovers. Perseverance has set several new records for autonomous driving on Mars, breaking those previously held by the Opportunity rover. As of the 1312th Martian day since landing, or 28 October 2024 on the Earth calendar, ∼90% of the 32.1 km of driving has used ENav to evaluate the terrain. This paper provides detailed documentation of the ENav algorithm, as well as its implementation, testing, deployment, and driving results on Mars.} } - Verma, V., Maimone, M., Gaines, D. M., Francis, R., Estlin, T., Kuhn, S. R., Rabideau, G. R., Chien, S. A., McHenry, M., Graser, E., Rankin, A. and Thiel, E. (2023). Autonomous Robotics is Driving Perseverance Rover's Progress on Mars
. The International Journal of Robotics Research. Source
BibTeX
@article{verma2023autonomous, title = {Autonomous Robotics is Driving Perseverance Rover's Progress on Mars}, author = {Verma, V. and Maimone, M. and Gaines, Daniel M. and Francis, Raymond and Estlin, T. and Kuhn, Stephen R and Rabideau, G. R. and Chien, S. A. and McHenry, M. and Graser, Evan and Rankin, A. and Thiel, Ellen}, journal = {The International Journal of Robotics Research}, publisher = {JPL Open Repository}, year = {2023}, doi = {10.1126/scirobotics.adi3099} } - Verma, V., Nash, J., Saldyt, L., Dwight, Q., Wang, H., Myint, S., Biesiadecki, J., Maimone, M., Tumbar, A., Ansar, A., Kubiak, G. and Hogg, R. (2024). Enabling Long and Precise Drives for the Perseverance Mars Rover via Onboard Global Localization
. IEEE Aerospace Conference. Source
BibTeX
@inproceedings{verma2024enabling, title = {Enabling Long and Precise Drives for the Perseverance Mars Rover via Onboard Global Localization}, author = {Verma, Vandi and Nash, Jeremy and Saldyt, Lucas and Dwight, Quintin and Wang, Haoda and Myint, Steven and Biesiadecki, Jeffrey and Maimone, Mark and Tumbar, Andrei and Ansar, Adnan and Kubiak, Gerik and Hogg, Robert}, booktitle = {IEEE Aerospace Conference}, address = {Big Sky, Montana}, year = {2024}, url = {https://www-robotics.jpl.nasa.gov/media/documents/2024_Global_Localization_IEEE_Aero.pdf} } - Wan, W., Wang, J., Liu, Z., Di, K., Peng, M., Wang, Y., Liu, B., Hu, G., Han, S., Li, X., Yu, T., Li, L. and Liu, C. (2025). Visual Localization and Topographic Mapping for Zhurong Rover in Tianwen-1 Mars Mission
. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing. Source
BibTeX
@article{wan2025visual, title = {Visual Localization and Topographic Mapping for Zhurong Rover in Tianwen-1 Mars Mission}, author = {Wan, Wenhui and Wang, Jia and Liu, Zhaoqin and Di, Kaichang and Peng, Man and Wang, Yexin and Liu, Bin and Hu, Guolin and Han, Shaojin and Li, Xin and Yu, Tianyi and Li, Lichun and Liu, Chuankai}, journal = {IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing}, volume = {18}, pages = {6393--6408}, year = {2025}, doi = {10.1109/jstars.2025.3540377}, abstract = {This article presents the methods, results, and applications of visual localization and topographic mapping for the Zhurong rover in China's Tianwen-1 Mars mission. New techniques, namely vignette correction and online camera mast calibration, were developed for the images captured by the rover's navigation and terrain cameras (NaTeCam) to enhance the mapping qualities. During surface operations, photogrammetric techniques were routinely applied at the traveling waypoints. The NaTeCam images were obtained to generate 3-D topographic mapping products at the centimeter level to support obstacle analysis and rover path planning. Cross-site visual localization at adjacent waypoints was conducted to reduce the accumulated position errors in the onboard dead-reckoning result. Visual localization refinement was also employed by matching the digital orthophoto map (DOM) from the rover images and the orbital basemap at waypoints where distinct terrain features were available to further eliminate the localization errors. Compared with the reference rover traverse extracted from a high-resolution imaging science experimental image in which the actual rover track is visible, the accuracies of the rover traverses generated from cross-site visual localization and DOM matching refinement are estimated to be 1.75% and 0.50% of the traverse length. The actual total traverse length is 2009 m, which is 88 m longer than 1921 m from the odometer reading, revealing a forward wheel slipping caused by the downslope trend of the southward traverse. The visual localization and topographic mapping results of the Zhurong rover have greatly supported the surface exploration planning, object approaching, and scientific investigations in the mission.} } - Werner, L., Proença, P., Nüchter, A. and Brockers, R. (2023). Covariance Based Terrain Mapping for Autonomous Mobile Robots
. IROS IEEE/RSJ International Conference on Intelligent Robots. Source
BibTeX
@inproceedings{werner2023covariance, title = {Covariance Based Terrain Mapping for Autonomous Mobile Robots}, author = {Werner, Lennart and Proença, Pedro and Nüchter, Andreas and Brockers, Roland}, booktitle = {IROS IEEE/RSJ International Conference on Intelligent Robots}, publisher = {JPL Open Repository}, year = {2023}, doi = {10.48577/jpl.jgveh8}, abstract = {Autonomous mobile robots operating in unknown terrain have to guide their drive decisions through perception. Local mapping and traversability analysis are essential for safe rover operation and low level locomotion. This paper deals with the challenge of building a local, robot-centric map from onboard range sensing. We introduce a new highly-efficient converging covariance based map update for estimating a digital elevation map. The map is capable of representing sub-cell size obstacles and can be queried for height, the presence of obstacles, and slope in constant time. Extensive experiments in simulation and real world test campaigns are used to evaluate the performance of our approach. The presented mapping technique outperforms state of the art grid-mapping algorithms for computationally constraint systems in run time, memory usage, precision and resolvable obstacle size. Further application scenarios are discussed. A variant of this map approach is projected to be used on NASA’s 2024 CADRE mission to the Moon.} } - Vernazza, P., Jorda, L., Tardivel, S., Baroukh, J., Groussin, O., Pimorin, C., Poulain, A., Chabaud, P.-Y., Robert, E., Bertrand, J., Lalucaa, V., Remetean, E., Théret, N., Virmontois, C., Murdoch, N., Le Mouélic, S., Beck, P., Rüsch, O., Flahaut, J., Charnoz, S., Lasue, J., Barucci, M.-A., Doressoundiram, A., Michel, P. and Ulamec, S. (2025). Surface science on Phobos with the navigation cameras of the MMX IDEFIX rover
. Progress in Earth and Planetary Science, 46. Source
BibTeX
@article{vernazza2025surface, title = {Surface science on Phobos with the navigation cameras of the MMX IDEFIX rover}, author = {Vernazza, Pierre and Jorda, Laurent and Tardivel, Simon and Baroukh, Julien and Groussin, Olivier and Pimorin, Celia and Poulain, Axel and Chabaud, Pierre-Yves and Robert, Emilie and Bertrand, Jean and Lalucaa, Valerian and Remetean, Emile and Théret, Nicolas and Virmontois, Cedric and Murdoch, Naomi and Le Mouélic, Stéphane and Beck, Pierre and Rüsch, Ottaviano and Flahaut, Jessica and Charnoz, Sebastien and Lasue, Jeremie and Barucci, Maria-Antonietta and Doressoundiram, Alain and Michel, Patrick and Ulamec, Stephan}, journal = {Progress in Earth and Planetary Science}, volume = {12}, number = {46}, year = {2025}, doi = {10.1186/s40645-025-00708-4}, abstract = {Abstract A CNES/DLR rover called IDEFIX will be deployed on the surface of Phobos in late 2028 or early 2029 as part of JAXA’s Martian Moons eXploration (MMX) mission. The goal of the rover is to travel across the surface of Phobos for at least 100 days, with autonomous guidance provided by the navigation cameras (a stereo pair). By taking stereoscopic images of the area around IDEFIX up to the horizon (resolution of a few millimeters at 1 m), the navigation cameras aim to provide answers to a number of scientific questions, such as the origin of the color dichotomy, the nature of space weathering processes, and the extent of dust transport and exogenous contamination on Phobos. Here we present the IDEFIX navigation cameras, including their ground calibrations, and the science questions they are intended to address. We also present the architecture of the data processing pipeline being developed at the Laboratoire d’Astrophysique de Marseille to address the science objectives of the navigation cameras.} } - Théret, N., Cucchetti, E., Robert, E., Virmontois, C., Millancourt, C., Douaglin, Q., Amsili, A. and Belloir, J.-M. (2024). Enhanced Image Processing for the CASPEX Cameras Onboard the Rashid-1 Rover
. Space Science Reviews. Source
BibTeX
@article{theret2024enhanced, title = {Enhanced Image Processing for the {CASPEX} Cameras Onboard the {Rashid-1} Rover}, author = {Théret, Nicolas and Cucchetti, Edoardo and Robert, Emilie and Virmontois, Cédric and Millancourt, Charles and Douaglin, Quentin and Amsili, Alice and Belloir, Jean-Marc}, journal = {Space Science Reviews}, volume = {220}, pages = {70}, year = {2024}, doi = {10.1007/s11214-024-01091-0} } - Vila, G. C., Dai, A. and Gao, G. (2026). Semantic Segmentation and Depth Estimation for Real-Time Lunar Surface Mapping Using 3D Gaussian Splatting
. IEEE Aerospace Conference. Source
BibTeX
@inproceedings{vila2026semantic, title = {Semantic Segmentation and Depth Estimation for Real-Time Lunar Surface Mapping Using 3D Gaussian Splatting}, author = {Vila, Guillem Casadesus and Dai, Adam and Gao, Grace}, booktitle = {IEEE Aerospace Conference}, pages = {1-14}, year = {2026}, doi = {10.1109/aero66936.2026.11519804}, abstract = {Navigation and mapping on the lunar surface require robust perception under challenging conditions, including poorly textured environments, high-contrast lighting, and limited computational resources. This paper presents a real-time mapping framework that integrates dense perception models with a 3D Gaussian Splatting (3DGS) representation. We first benchmark several models on synthetic datasets generated with the LuPNT simulator, selecting a stereo dense depth estimation model based on Gated Recurrent Units for its balance of speed and accuracy in depth estimation, and a convolutional neural network for its superior performance in detecting semantic segments. Using ground truth poses to decouple the local scene understanding from the global state estimation, our pipeline reconstructs a 120 -meter traverse with a geometric height accuracy of approximately 3 cm, outperforming a traditional point cloud baseline without LiDAR. The resulting 3DGS map enables novel view synthesis and serves as a foundation for a full SLAM system, where its capacity for joint map and pose optimization would offer significant advantages. Our results demonstrate that combining semantic segmentation and dense depth estimation with learned map representations is an effective approach for creating detailed, large-scale maps to support future lunar surface missions.} } - Fygenson, R., Jawad, K., Li, I., Ayoub, F., Deen, R. G., Davidoff, S., Moritz, D. and Hess-Flores, M. (2024). Opening the Black Box of 3D Reconstruction Error Analysis with VECTOR
. IEEE Visualization and Visual Analytics. doi.org/10.1109/vis55277.2024.00065
BibTeX
@misc{fygenson2024opening, title = {Opening the Black Box of 3D Reconstruction Error Analysis with VECTOR}, author = {Fygenson, Racquel and Jawad, Kazi and Li, Isabel and Ayoub, Francois and Deen, Robert G. and Davidoff, Scott and Moritz, Dominik and Hess-Flores, Mauricio}, booktitle = {IEEE Visualization and Visual Analytics}, pages = {286-290}, year = {2024}, doi = {10.1109/vis55277.2024.00065}, abstract = {Reconstruction of 3D scenes from 2D images is a technical challenge that impacts domains from Earth and planetary sciences and space exploration to augmented and virtual reality. Typically, reconstruction algorithms first identify common features across images and then minimize reconstruction errors after estimating the shape of the terrain. This bundle adjustment (BA) step optimizes around a single, simplifying scalar value that obfuscates many possible causes of reconstruction errors (e.g., initial estimate of the position and orientation of the camera, lighting conditions, ease of feature detection in the terrain). Reconstruction errors can lead to inaccurate scientific inferences or endanger a spacecraft exploring a remote environment. To address this challenge, we present VECTOR, a visual analysis tool that improves error inspection for stereo reconstruction BA. VECTOR provides analysts with previously unavailable visibility into feature locations, camera pose, and computed 3D points. VECTOR was developed in partnership with the Perseverance Mars Rover and Ingenuity Mars Helicopter terrain reconstruction team at the NASA Jet Propulsion Laboratory. We report on how this tool was used to debug and improve terrain reconstruction for the Mars 2020 mission.} } - Vona, M. (2025). The Landform Contextual Mesh: Automatically Fusing Surface and Orbital Terrain for Mars 2020. arxiv.org/abs/2509.18330v1
BibTeX
@misc{vona2025landform, title = {The Landform Contextual Mesh: Automatically Fusing Surface and Orbital Terrain for Mars 2020}, author = {Vona, Marsette}, year = {2025}, url = {https://arxiv.org/abs/2509.18330v1} } - Rodríguez-Martínez, D., van der Meer, D., Song, J., Bera, A., Pérez-del-Pulgar, C. J. and Olivares-Mendez, M. A. (2026). SPICE-HL3: Single-Photon, Inertial, and Stereo Camera Dataset for Exploration of High-Latitude Lunar Landscapes
. Scientific Data. Source
BibTeX
@article{rodriguezmartinez2026spice, title = {{SPICE-HL3}: Single-Photon, Inertial, and Stereo Camera Dataset for Exploration of High-Latitude Lunar Landscapes}, author = {Rodríguez-Martínez, David and van der Meer, Dave and Song, Junlin and Bera, Abhishek and Pérez-del-Pulgar, Carlos J. and Olivares-Mendez, Miguel Angel}, journal = {Scientific Data}, volume = {13}, pages = {374}, year = {2026}, doi = {10.1038/s41597-026-06668-8}, abstract = {Abstract Exploring high-latitude lunar regions presents a challenging visual environment for robots. The low sunlight elevation angle and minimal light scattering result in a visual field dominated by a strong contrast featuring long, dynamic shadows. Reproducing these conditions on Earth requires sophisticated simulators and specialized facilities. We introduce a unique dataset recorded at the LunaLab from the SnT - University of Luxembourg, an indoor test facility designed to replicate the optical characteristics of multiple lunar latitudes. Our dataset includes images, inertial measurements, and wheel odometry data from robots navigating different trajectories under multiple illumination scenarios, simulating high-latitude lunar conditions from dawn to nighttime with and without the aid of headlights, resulting in 88 distinct sequences containing a total of 1.3 M images. Data was captured using a stereo RGB-inertial sensor, a monocular monochrome camera, and, for the first time, a novel single-photon avalanche diode (SPAD) camera. We recorded both static and dynamic image sequences, with robots navigating at slow (5 cm/s) and fast (50 cm/s) speeds. All data is calibrated, synchronized, and timestamped, providing a valuable resource for validating perception tasks from vision-based autonomous navigation to scientific imaging for future lunar missions targeting high-latitude regions or those intended for robots operating across perceptually degraded environments.} }