Bad results on ETH3D

[mattiadurso]

Hi guys,
I am trying to compare GLOMAP performances with GTSfM. I can duplicate the results of GLOMAP (and COLMAP) runned on ETH3D undistorted scenes, while for GTSfM I am getting very bad results for 22 out of 25 scenes. For both I use SIFT as feature extractor. I am running the command in scripts/eth3d_benchmark.sh that is

python gtsfm/runner/run_scene_optimizer_colmaploader.py 
                    --mvs_off 
                    --config unified 
                    --correspondence_generator_config_name sift 
                    --share_intrinsics 
                    --images_dir {scene_path}/images 
                    --colmap_files_dirpath {scene_path}/dslr_calibration_undistorted 
                    --num_workers 1 
                    --num_matched 5 
                    --max_frame_lookahead 10 
                    --worker_memory_limit 32GB 
                    --output_root {scene_path}/gtsfm 
                    --max_resolution 760 

where {scene_path} is, for example, .../Datasets/ETH3D_Stereo/DSLR_undistorted_gtsfm/door.

According to the code and the argument --colmap_files_dirpath I am passing the GT intrinsics to the model, but when I check the camera.txt file of ba_output folder I get a different camera model and other parameters (eventhough they are basically zero). Consider the scene "door", the following is the GT camera.txt file

0 PINHOLE 6212 4140 3415.67 3414.36 3119.82 2054.28

while in GTSfM I get

0 RADIAL 1140 760 626.8293303284069 572.5361880231809 377.1142028985508 4.782895950634459e-09 -9.97228226196883e-09

As far as I get, the GT params are [fx, fy, cx,cy], while for GTSfM output I get (scaled) [fx, cx, cy, k1, k1]. Moreover, the camera model changes from pinhole to radial.

Considering I can reproduce the GLOMAP's paper results, I assume my evaluation code is correct. I have the following questions:

• Am I missing something?
• Is that the proper way to pass intrisics to the pipeline?

Thanks in advance!

[travisdriver]

Did you try with SuperPoint/SuperGlue? That should give you much better results on ETH3D.