Poor performance of the other question.

[ZebinX]

Hi, @gordonhu608

Thanks for great work.

I tried inference_chat.py to ask some other questions, like the comparison of the locations, but G2VLM seems could not answer properly. I'm not sure if I use model correctly, could you help me figure it? My case:

Fig:

Question:
which one is closer to me? The blue chair or the white shoes
Answer:
The observer is 1.4 meters away from the center of shoes (red point)

It seems can only answer the question about the depth.

Thank you!

[winnk123]

I met the same problem with you. I guess that the model seems to be overfitting in these datasets.

The question I ask is " Where is the far doorway relative to the long kitchen island from the camera viewpoint? Describe its left-right and front-behind relationship?"

The answer is "The center of kitchen island (red point) lies about 4.0 meters away from you". However there is no red point in this image. And the answer is wrong.