RAM consumption

[FredPraca]

Since the version 7, it seems that RAM consumption grew.
When uploading three photos of about 8Mb, php-fpm RAM rises to about 250Mb and more.
On my 2Gb RAM VM, uploading few photos consumes a lot of RAM and php-fpm gets killed.

Were there changes in the way photo handling that would explain this ?

[ildyria]

Hi @FredPraca
What setup are you using? docker compose or directly zip or clone and composer?

The zip/clone+composer should not have impact.
The main difference between version 6 and version 7 is on the new docker images with frankenphp (those the old php-fpm setup is still available with -legacy tag.

We also switch to jobs instead of direct processing. This allows you to have queue workers in the background dealing with the processing of images. It might be possible that the image is kept in memory at different stages and not cleared :| Though 250 Mb is a lot. :|

[FredPraca]

Hi @ildyria ,
I use the standard clone+composer install so the change with Docker image shouldn't have any impact.

But the switch to jobs seems to be the ideal culprit :)
I'll try to investigate to find when it appears.
As a first clue, on the webpage, the state is still "Uploading" when the php-fpm processes grow in memory.

[ildyria]

As a first clue, on the webpage, the state is still "Uploading" when the php-fpm processes grow in memory.

Yes, in the php-fpm mode (legacy/non-worker flow), the last chunk also has the processing step.

[FredPraca]

Just to be sure, you mean that running through php-fpm is considered legacy ?

[ildyria]

You can still run it via nginx + php-fpm/apache + mod php. This is also what is under the docker image with tag edge-legacy or v7.x.x-legacy, and how I am developing it locally.

However we now proposes a docker image based on frankenphp with strong speed promises:

"According to our benchmarks, 3.5x faster than FPM on API Platform apps."

Instead of loading the full framework, env etc at every request, this is persisted across runs and only the request object is hydrated. This present a risk of memory leaks, which is why every x requests the runner is restarted, however the speed increase is noticeable.

We combine the frankenphp image with workers (see docker compose https://github.com/LycheeOrg/Lychee/blob/master/docker-compose.yaml#L447 ) to offload the processing of images in async using the jobs above mentioned.

So I would not say it is 100% legacy, but we strongly suggest using docker as it also reduces the possibilities of bug and improve reproducibility in setups. (Also I just spent half an hour debugging the install of a non-dev friend via screenshots because he did not want to use docker and was facing all kind of permissions issues 😅 ).