Reproducibility of GMA on Sintel and KITTI test

[zwei-lin]

Thanks for the great code !
I try to reproduce the Sintel and KITTI test results reported in the paper. However, I got 1.58 on Sintel clean, 2.64 on Sintel final for GMA(our), and 5.14 on KITTI for GMA(p only). The results seem worse than those reported in the paper (1.39 on Sintel clean, 2.47 on Sintel final for GMA(our), and 4.93 on KITTI for GMA(p only)).
Is it because you find the best iteration checkpoint on the validation set, while I use the last iteration checkpoint? If so, may I know the validation set you choose?

[zyfan1]

Hi,
Have you solved this problem yet? I have also had cases where the reproduction did not work out, but the paper mentions that it was done with 12 iterations

[zwei-lin]

Not yet. I emailed the authors and they said that they are not sure about this issue. They did not run multiple experiments.

[az-ja]

Hi,
I could reproduce GMA on Sintel after I trained a few times (4-5 times). In most of my trials, the clean results were about 1.55.
Once I just repeated the experiment and I was close to GMA finally. I did not try on KITTI though.