Depth2Pose: A Pose-Based Benchmark for Monocular Depth Estimation without Ground-Truth Depth

Viktor Kocur · Sithu Aung · Gabrielle Flood · Yaqing Ding · Lukáš Bujňák · Torsten Sattler · Zuzana Kukelova

About

This repository contains the full evalautation code the depth2pose framework for estimting monocular depth estimators. Our paper was accepted in the E&D track of NeurIPS 2026.

TLDR: Using standard metrics to evaluate MDEs requires known ground truth depth, which may be hard to acquire. In the Depth2Pose framework we evaluate the depth via two-view relative pose accuracy using the RePoseD framework. This approach only requires a collection of images from which a COLMAP reconstruction can be obtained. By design, depth2pose focuses its evaluation on the quality of geometrically relevant parts of the images and thus captures the usefulness of the estimated depth for downstream geometric tasks.

Dataset download

You can download the full D2P dataset used in the paper from hugging face

Installation

See INSTALL.md.

Evaluation

Running Full Evaluation

To run the full evaluation for all MDEs we provide code to create array jobs for a SLURM cluster. If a cluster is not available the scripts should default to local execution.

Modify the file datasets/d2p_benchmark.json and datasets/standard_benchmark.json with paths to the datasets.

Then you can run the following:

# extract pairs
python datasets/colmap.py --config_path datasets/d2p_benchmark.json
python datasets/colmap.py --config_path datasets/standard_benchmark.json

# match pairs (use matchers/splg.py to obtain SP+LG matches in the same way)
python matchers/loma_matching.py --config_path datasets/d2p_benchmark.json
python matchers/loma_matching.py --config_path datasets/standard_benchmark.json

# infer depth
python spawn_infer_depth.py  --config_path datasets/d2p_benchmark.json
python spawn_infer_depth.py  --config_path datasets/standard_benchmark.json

# evaluate depth for the standard benchmark
python eval_depth.py  --config_path datasets/d2p_benchmark.json

# eval pose estimation accuracy
python spawn_eval_pose.py  --config_path datasets/d2p_benchmark.json --matches loma_2048_noresize
python spawn_eval_pose.py  --config_path datasets/standard_benchmark.json --matches loma_2048_noresize

# eval methods that directly regress depth and relative pose
python spawn_eval_regressed_pose.py --config_path datasets/d2p_benchmark.json
python spawn_eval_regressed_pose.py --config_path datasets/standard_benchmark.json

# finally extract results
python utils/extract_results.py  --config_path datasets/d2p_benchmark.json --matches loma_2048_noresize
python utils/extract_results.py  --config_path datasets/standard_benchmark.json --matches loma_2048_noresize

Except for colmap.py you will need to modify the args to select the correct partition and set time limits.

Adding a new MDE

To add a new MDE you need to add the required code (into depth_estimators folder) implementing the base class in depth_estimators/base.py. Then you also have to modify ``depth_estimators/infer_depth.py` to include the MDE along with its variants similar to the other methods.

To run the mde you need the perform all the steps. But when running:

python spawn_infer_depth.py  --config_path datasets/d2p_benchmark.json
python spawn_infer_depth.py  --config_path datasets/standard_benchmark.json

You add the following arg based on the name of the depth in ALL_MDEs_DICT:

--model_name your_model_name

This will run only the new model. You may run pose estimation with depth provided for only one model and then export the results with the same command as for the full evaluation.