Question about inference_chat.py

[stjohn2007]

Hi, thank you for releasing this great work!

I tried running inference_chat.py, but I encountered the following error:

G2VLM/modeling/g2vlm/g2vlm.py", line 25, in <module>
from pi3.models.layers.transformer_head import Pi3TransformerDecoder, Pi3LinearPts3d, Pi3ContextTransformerDecoder
ModuleNotFoundError: No module named 'pi3'

I tried copying the pi3 directory from the Pi3 repository (https://github.com/yyfz/Pi3/tree/main), but it seems that Pi3TransformerDecoder, Pi3LinearPts3d, and Pi3ContextTransformerDecoder do not exist in the original pi3 directory, so the same error still occurs.

If there are additional implementations or files required to run inference_chat.py, could you please provide them?

Thank you in advance!

[yukki555]

Try setting up in terminal: export PYTHONPATH=/your/path/to/G2VLM/modeling:$PYTHONPATH
This works when importing pi3 and you can always consider this solution first when encountering similar issues in any project.

[stjohn2007]

@Jinhsi555
Thanks! The previous issue is resolved and the script now runs without errors.
However, the output from G2VLM-2B-MoT still seems incorrect.

Below is the full output from inference_chat.py:

> python inference_chat.py --model-path models/G2VLM-2B-MoT
Warning, cannot find cuda-compiled version of RoPE2D, using a slow pytorch version instead
Qwen2 has config.num_hidden_layers 28
<All keys matched successfully>
[test] total_params: 4.538215076B

If the table (red point) is positioned at 2.6 meters, estimate the depth of the clothes (blue point).  Calculate or judge based on the 3D center points of these objects. The unit is meter. Submit your response as one numeric value only.
Please answer the question using a single word or phrase.
output torch.Size([1, 3, 392, 518])
answer:  !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

As shown, the model does not generate a valid answer and only returns multiple ! characters.

I noticed that in the current implementation templated_question may be replaced by an empty string (args.question), so I commented out the corresponding lines. However, the output remains unchanged.

# if args.question is not None: 
#     templated_question = args.question

Is there anything I may be missing to get a valid result, or is this expected behavior for now?

My environment:

• python: 3.10
• cuda: 12.1

Package                   Version
------------------------- ------------
accelerate                1.12.0
addict                    2.4.0
aiofiles                  24.1.0
annotated-doc             0.0.4
annotated-types           0.7.0
anyio                     4.12.0
asttokens                 3.0.1
attrs                     25.4.0
bitsandbytes              0.48.2
blinker                   1.9.0
brotli                    1.2.0
certifi                   2025.11.12
charset-normalizer        3.4.4
click                     8.3.1
comm                      0.2.3
configargparse            1.7.1
contourpy                 1.3.2
cycler                    0.12.1
dash                      3.3.0
decorator                 5.2.1
decord                    0.6.0
easydict                  1.13
einops                    0.8.1
exceptiongroup            1.3.1
executing                 2.2.1
fastapi                   0.124.0
fastjsonschema            2.21.2
ffmpy                     1.0.0
filelock                  3.19.1
flash-attn                2.7.4.post1
flask                     3.1.2
fonttools                 4.61.0
fsspec                    2025.9.0
gitdb                     4.0.12
gitpython                 3.1.45
gradio                    5.38.2
gradio-client             1.11.0
groovy                    0.1.2
h11                       0.16.0
httpcore                  1.0.9
httpx                     0.28.1
huggingface-hub           0.29.1
idna                      3.11
importlib-metadata        8.7.0
ipython                   8.37.0
ipywidgets                8.1.8
itsdangerous              2.2.0
jedi                      0.19.2
jinja2                    3.1.6
joblib                    1.5.2
jsonschema                4.25.1
jsonschema-specifications 2025.9.1
jupyter-core              5.9.1
jupyterlab-widgets        3.0.16
kiwisolver                1.4.9
markdown-it-py            4.0.0
markupsafe                2.1.5
matplotlib                3.7.0
matplotlib-inline         0.2.1
mdurl                     0.1.2
mpmath                    1.3.0
narwhals                  2.13.0
nbformat                  5.10.4
nest-asyncio              1.6.0
networkx                  3.3
ninja                     1.13.0
numpy                     1.24.4
nvidia-cublas-cu12        12.1.3.1
nvidia-cuda-cupti-cu12    12.1.105
nvidia-cuda-nvrtc-cu12    12.1.105
nvidia-cuda-runtime-cu12  12.1.105
nvidia-cudnn-cu12         9.1.0.70
nvidia-cufft-cu12         11.0.2.54
nvidia-curand-cu12        10.3.2.106
nvidia-cusolver-cu12      11.4.5.107
nvidia-cusparse-cu12      12.1.0.106
nvidia-nccl-cu12          2.21.5
nvidia-nvjitlink-cu12     12.9.86
nvidia-nvtx-cu12          12.1.105
open3d                    0.19.0
opencv-python             4.7.0.72
orjson                    3.11.5
packaging                 25.0
pandas                    2.3.3
parso                     0.8.5
pexpect                   4.9.0
pillow                    11.3.0
platformdirs              4.5.1
plotly                    6.5.0
prompt-toolkit            3.0.52
protobuf                  6.33.2
psutil                    7.1.3
ptyprocess                0.7.0
pure-eval                 0.2.3
pyarrow                   11.0.0
pydantic                  2.11.10
pydantic-core             2.33.2
pydub                     0.25.1
pygments                  2.19.2
pyparsing                 3.2.5
pyquaternion              0.9.9
python-dateutil           2.9.0.post0
python-multipart          0.0.20
pytz                      2025.2
pyyaml                    6.0.2
referencing               0.37.0
regex                     2025.11.3
requests                  2.32.3
retrying                  1.4.2
rich                      14.2.0
rpds-py                   0.30.0
ruff                      0.14.8
safehttpx                 0.1.7
safetensors               0.4.5
scikit-learn              1.7.2
scipy                     1.10.1
semantic-version          2.10.0
sentencepiece             0.1.99
sentry-sdk                2.47.0
setuptools                80.9.0
shellingham               1.5.4
six                       1.17.0
smmap                     5.0.2
stack-data                0.6.3
starlette                 0.50.0
sympy                     1.13.1
threadpoolctl             3.6.0
tokenizers                0.21.4
tomlkit                   0.13.3
torch                     2.5.1+cu121
torchaudio                2.5.1+cu121
torchvision               0.20.1+cu121
tqdm                      4.67.1
traitlets                 5.14.3
transformers              4.49.0
triton                    3.1.0
typer                     0.20.0
typing-extensions         4.15.0
typing-inspection         0.4.2
tzdata                    2025.2
urllib3                   2.6.1
uvicorn                   0.38.0
wandb                     0.23.1
wcwidth                   0.2.14
websockets                15.0.1
werkzeug                  3.1.4
wheel                     0.45.1
widgetsnbextension        4.0.15
zipp                      3.23.0

Thank you in advance!

[ZebinX]

@Jinhsi555
Hi,

Thanks for your excellent work. However, I have encountered the same problem as "the output from G2VLM-2B-MoT still seems incorrect".

My configs are as follows:

• machine: H20
• python: 3.9.1
• torch: 2.8.0
• CUDA: 12.3