Out of memory when using 8 * A800-80G for video inference

[caichaoxiang]

Hi,

I followed your instructions and used 8 * A800-80G for video evaluation, but an OOM error was reported during the evaluation phase. Is there something wrong with the settings? My script instructions are as follows:

/Myenv/bin/torchrun --nproc-per-node=8 /Myproj/LongVU/eval/eval_videomme.py --data_path /Mydata/VideoLLaMA2/datasets/eval/videomme --version llama3 --model_path /Myproj/checkpoints/LongVU_Llama3_2_3B

[andyhandsom6]

I also encounter OOM while fine-tuning the model using H800-80Gs. My strategy is to set max_num_frames a little bit lower (400->350, for example).
However, I'd like to listen to the authors' comments towards this.

[tcm03]

I also encounter OOM while fine-tuning the model using H800-80Gs. My strategy is to set max_num_frames a little bit lower (400->350, for example). However, I'd like to listen to the authors' comments towards this.

I used H100 with 96GB but still encountered OOM issues with videos around 55 mins.
When you modified max_num_frames lower, did you encounter any reduction in performance?

[andyhandsom6]

I used H100 with 96GB but still encountered OOM issues with videos around 55 mins. When you modified max_num_frames lower, did you encounter any reduction in performance?

Sorry, nothing to comment since I can't run the model at orinigal max_num_frames. It encounters OOM all the time.