Host memory leak from visualisation

[DavidFletcherShef]

I have found a host memory leak from the visualiser but cannot work out how to fix it.

I noticed this after leaving on the screen a similation which had finished but the visualiser was still open. Returning to the computer after several hours I found all memory and most of the swap space had been consumed. I've tested this on two different computers with different GPUs, both running Slackware Linux 15, Nvidia driver 550.142, Cuda V12.1.66.

The issue can be repeated using anything that opens the visualiser. It is related to display of text (e.g. time step of similation), maybe something about FreeType. To test this the circles tutorial will generate the issue. After completion while it is sitting there doing nothing with the visualiser still open a log of memory use every 5 seconds shows a continuous increase in resident size (RSS), the non-swapped physical memory used.

  PID   RSS    VSZ
13817 323468 9656716
13817 324260 9656716
13817 325052 9656716
13817 325844 9656716
13817 330468 9660812
13817 331260 9660812
13817 332052 9660812
13817 332580 9660812
13817 333372 9660812
13817 334164 9660812
13817 334956 9660812
13817 335484 9660812
13817 336276 9660812
13817 337068 9660812
13817 337860 9660812
13817 338388 9660812
13817 339180 9660812
13817 339972 9660812
13817 340764 9660812
13817 341292 9660812
13817 342084 9660812
13817 342876 9660812
13817 343668 9660812
13817 344196 9660812
13817 344988 9660812
13817 345780 9660812
13817 346572 9660812
13817 347100 9660812
13817 347892 9660812
13817 348684 9660812
13817 349476 9660812
13817 350004 9660812
13817 350796 9660812
13817 351588 9660812
13817 352380 9660812
13817 352908 9660812
13817 353700 9660812
13817 354492 9660812
13817 355284 9660812
13817 355812 9660812

Running valgrind shows many issues, but the ones that appear important are:

==14100== 3,851,085 bytes in 3,473 blocks are definitely lost in loss record 1,960 of 1,963
==14100==    at 0x48445FF: calloc (vg_replace_malloc.c:1117)
==14100==    by 0x627AF8: flamegpu::visualiser::Text::recomputeTex() (in /home/dif/Documents/2025/FLAME_Circles/build/bin/Release/template)
==14100==    by 0x6280FA: flamegpu::visualiser::Text::setString(char const*, ...) (in /home/dif/Documents/2025/FLAME_Circles/build/bin/Release/template)
==14100==    by 0x5FA893: flamegpu::visualiser::Visualiser::run() (in /home/dif/Documents/2025/FLAME_Circles/build/bin/Release/template)
==14100==    by 0xA633883: ??? (in /usr/lib64/libstdc++.so.6.0.29)
==14100==    by 0xA297E44: start_thread (in /lib64/libpthread-2.33.so)
==14100==    by 0xA9DA4AE: clone (in /lib64/libc-2.33.so)
==14100== 
==14100== 4,521,174 bytes in 3,475 blocks are definitely lost in loss record 1,961 of 1,963
==14100==    at 0x48445FF: calloc (vg_replace_malloc.c:1117)
==14100==    by 0x627AF8: flamegpu::visualiser::Text::recomputeTex() (in /home/dif/Documents/2025/FLAME_Circles/build/bin/Release/template)
==14100==    by 0x6280FA: flamegpu::visualiser::Text::setString(char const*, ...) (in /home/dif/Documents/2025/FLAME_Circles/build/bin/Release/template)
==14100==    by 0x5FA86D: flamegpu::visualiser::Visualiser::run() (in /home/dif/Documents/2025/FLAME_Circles/build/bin/Release/template)
==14100==    by 0xA633883: ??? (in /usr/lib64/libstdc++.so.6.0.29)
==14100==    by 0xA297E44: start_thread (in /lib64/libpthread-2.33.so)
==14100==    by 0xA9DA4AE: clone (in /lib64/libc-2.33.so)

Possibly also related is an issue about free and delete for setString which calls recomputeTex():

==14100== 3480 errors in context 8 of 223:
==14100== Mismatched free() / delete / delete []
==14100==    at 0x48429AB: operator delete(void*, unsigned long) (vg_replace_malloc.c:814)
==14100==    by 0x628079: flamegpu::visualiser::Text::setString(char const*, ...) (in /home/dif/Documents/2025/FLAME_Circles/build/bin/Release/template)
==14100==    by 0x5FA893: flamegpu::visualiser::Visualiser::run() (in /home/dif/Documents/2025/FLAME_Circles/build/bin/Release/template)
==14100==    by 0xA633883: ??? (in /usr/lib64/libstdc++.so.6.0.29)
==14100==    by 0xA297E44: start_thread (in /lib64/libpthread-2.33.so)
==14100==    by 0xA9DA4AE: clone (in /lib64/libc-2.33.so)

Looking in build/_deps/flamegpu_visualiser-src/src/flamegpu/visualiser/ui/Text.cpp the relevant function is recomputeTex(). If I replace the first line with return; so that the function does nothing then the memory consumption issue goes away (although of course no text is displayed any longer).

However, I can't see what's wrong with it. malloc is there, but memory is freed before the function returns. FT_Done_Glyph is used to clean up glyphs[i].image. Possibly FT_BitmapGlyph bit which is created from glyphs[i].image also needs to be cleaned up? I can't get to the bottom of it.

The full valgrind log is attached if that's useful to understand it better. Many thanks for any help with this.

valgrind-out.txt

[Robadob]

Thanks, I'll take a look when I get a chance. It wouldn't surprise me if there's an accidental memory leak in the code which renders the FPS to the bottom-right corner of the screen.

[ptheywood]

Running valgrind shows many issues, but the ones that appear important are:

Valgrind with CUDA returns a lot of false-positives / bits of CUDA state which CUDA does not clean up after itself, which makes it a bit of a chore to filter out what is actually useful (e.g. all the still reachable blocks in libcuda.so).
Due to CUDA and c++ destructor limitations we cannot always safely call cuda methods either, so there will be some host leakage we cannot fix (but this should not be in the render loop).

I not sure that we've ever explicitly checked the visualiser for leaks.

---

I've reproduced the leak locally (using the boids example with .join removed), and have potentially found 2 of the leaks (or at least a leak and some UB in Text).

It might be a little while before I've gone through the full valgrind logs to see if there are any more fixes we need to make.

With the 2 minor fixes so far, definately lost reduced from 381,666 to 30,672 bytes (although there may be variance in this between runs depending on how many frames were rendered etc):

valgrind --leak-check=full --show-leak-kinds=all --track-origins=yes --verbose --log-file=valgrind-out.txt ./bin/RelWithDebInfo/boids_spatial3D -s 500 -r 12 

==78075== LEAK SUMMARY:
==78075==    definitely lost: 381,666 bytes in 376 blocks
==78075==    indirectly lost: 1,282,618 bytes in 35,639 blocks
==78075==      possibly lost: 46,152,048 bytes in 169 blocks
==78075==    still reachable: 70,379,521 bytes in 19,418 blocks
==78075==         suppressed: 56 bytes in 1 blocks

==99165== LEAK SUMMARY:
==99165==    definitely lost: 30,672 bytes in 98 blocks
==99165==    indirectly lost: 1,169,757 bytes in 33,213 blocks
==99165==      possibly lost: 46,264,909 bytes in 2,595 blocks
==99165==    still reachable: 70,379,489 bytes in 19,418 blocks
==99165==         suppressed: 56 bytes in 1 blocks

[DavidFletcherShef]

Yes, I agree, there's lots of false positives. Thanks for working on this, it's good to know you were able to replicate it.