Can't upload large files (>= 4.5Mb)

[johan-jnn]

Hi everyone !
Just encountered an issue while using pagecms for my portfolio:

Context

I'm was trying to create a music-only media using the cms.
No issue until I tryed to upload my first music (4.5Mb) :

Note that for a music that is 2.5Mb large there is no problem.

My .pages.yml

Request information

// Payload :
{
  "type": "media",
  "name": "background-musics",
  "content": "<base64 encoded mp3>"
}

Can help, maybe

After some investigation, I deduced that you're using Vercel as the hosting service and apparently they are limiting the upload request size to 4.5Mb.
It's apparently possible to bypass this limitation as described by Vercel it-self:
https://vercel.com/kb/guide/how-to-bypass-vercel-body-size-limit-serverless-functions#upload-directly-to-the-source

[TextAware]

Just commenting to say this has been causing me quite the headache. Thanks for filing the issue & hoping it can be resolved.

[carlosjdelgado]

I have read Vercel's article and I want to make a reflection.

The problem with uploading directly to GitHub from the browser is that it would temporarily leave the access token exposed. This can be solved by later calling the GitHub API to invalidate it, but an attacker who manages to gain access as a contributor to the CMS can manipulate the calls to keep the uninvalidated token and have full write access to the repository, which would be a big problem.

Taking this into account, I can't think of a solution that could help resolve this issue. Using a Vercel Blob would be possible, but that adds infrastructure and I'm not sure if it's feasible.

[johan-jnn]

@carlosjdelgado But could the solution be to wait for the blob to upload in the vercel's server and then upload to GitHub (inside the "onUploadComplete" method on vercel's example) ?

[carlosjdelgado]

free blob storage in vercel has a data transfer limit of 10GB/month, I don't know if it will be enough for the public instance, I suspect not because for a file of, for example, 10MB you have to use 20MB of transfer (10MB write, 10MB delete)

@hunvreus could you tell us if it's feasible to use blob storage as an intermediate support to bypass the Vercel limit?

[carlosjdelgado]

I've had an idea and have already submitted the pull request directly. One way to bypass the restriction is to upload the file in 4MB chunks and store them in the database. Once uploaded, the file is assembled in memory and then uploaded to GitHub. There are more technical considerations and comments that define this idea in the pull request description (#408 ).