Skip to content

feat: add a stand-alone upscale endpoint to the server - #2026

Open
nbeerbower wants to merge 3 commits into
leejet:masterfrom
Schneewolf-Labs:upscale-endpoint
Open

nbeerbower wants to merge 3 commits into
leejet:masterfrom
Schneewolf-Labs:upscale-endpoint

Conversation

@nbeerbower

Copy link
Copy Markdown

Summary

Adds POST /sdcpp/v1/upscale, so an ESRGAN model can be run on its own over
HTTP without a generation wrapped around it.

The capability is already in the library — sd-cli -M upscale (#865) runs an
upscaler with no diffusion model, no text encoder and no sampling — but the
only way to reach it from the server was hires inside an image generation.
That has two problems:

  • It pays for a generation nobody asked for, and a plain upscale cannot be
    expressed at all, because hires.denoising_strength must be greater than
    zero.
  • It is not always survivable. A hires request against Qwen-Image-2.1 aborts
    the server in GGML_ASSERT(a->ne[2] == b->ne[2]) at ggml.c:4557.

The new route is synchronous, like the sdapi ones — the work is a couple of
seconds with no model loading behind it, so it does not need the async job
machinery. It takes the generation context mutex, so an upscale and a
generation never use the device at the same time, and it resolves the model by
the same name capabilities already reports under upscalers.

POST /sdcpp/v1/upscale
{"image": "<base64>", "upscaler": "RealESRGAN_x4plus_anime_6B",
 "repeats": 1, "tile_size": 128, "output_format": "png"}

Two small additions to capabilities so a client can offer this only where it
will work:

  • upscale — a server started without --hires-upscalers-dir has the route
    but nothing to run.
  • model on each upscalers entry — that list also carries the built-in
    scaling filters (None, Lanczos, the latent modes), which are choices for
    the second stage of a generation rather than models that can be run alone.
    Without this a client cannot tell the two apart, and asking to upscale with
    "None" is a confusing failure.

Both are documented in examples/server/api.md alongside the endpoint.

Related Issue / Discussion

Exposes the stand-alone upscale mode added in #865 over the server API.

Additional Information

Verified against a CUDA build on an RTX A6000, server loaded with
Qwen-Image-2.1 and --hires-upscalers-dir pointing at
RealESRGAN_x4plus_anime_6B.pth:

$ curl -sS -H 'Content-Type: application/json' --data-binary @req.json \
    http://127.0.0.1:1234/sdcpp/v1/upscale
real    0m2.207s
{"upscaler":"RealESRGAN_x4plus_anime_6B","scale":4,"repeats":1,
 "width":2048,"height":2048,"output_format":"png","images":[...]}

512x512 in, 2048x2048 out, 2.2 s including the model load. Matches what
sd-cli -M upscale produces for the same input.

Refusals checked: empty body, missing image, unreadable image, and a name
that is not a model-backed upscaler each return 400 with a message rather than
failing later.

capabilities on that server reports "upscale": true and marks
RealESRGAN_x4plus_anime_6B with "model": true while the built-in filters
carry "model": false.

Checklist

The library has had standalone upscaling all along -- `sd-cli -M upscale`
runs an ESRGAN model with no diffusion model, no text encoder and no
sampling, 512x512 to 2048x2048 in about two seconds -- but over HTTP it
could only be reached as the hires stage of an image generation. That means
paying for a generation nobody wanted, and it is not always survivable: a
hires request against Qwen-Image-2.1 dies in
GGML_ASSERT(a->ne[2] == b->ne[2]) at ggml.c:4557 and takes the server with
it. A plain upscale cannot be expressed through hires in any case, since
hires denoising strength must be greater than zero.

The route is synchronous, like the sdapi ones, because two seconds of work
does not need the async job machinery. It holds the generation context mutex
so an upscale and a generation never use the device at the same time,
resolves the model by the same name `capabilities` reports under
`upscalers`, and answers in the shape the other image endpoints use.

  POST /sdcpp/v1/upscale
  {"image": "<base64>", "upscaler": "RealESRGAN_x4plus_anime_6B",
   "repeats": 1, "tile_size": 128, "output_format": "png"}

Two additions to `capabilities` so a client can offer this only where it
works: an `upscale` flag, since a server started without
--hires-upscalers-dir has the route but nothing to run; and a `model` flag
on each `upscalers` entry, because that list also carries the built-in
scaling filters (None, Lanczos, the latent modes), which are choices for the
second stage of a generation rather than models that can be run on their
own. Both are documented in examples/server/api.md alongside the endpoint.
@leejet

leejet commented Sep 23, 2026

Copy link
Copy Markdown
Owner

I can't push to this branch due to the organization's permissions. Could you apply the attached patch to your PR branch?

It contains the review fixes:

  • Reject cumulative upscale dimensions above 8192 pixels per axis before upscaling, with overflow-safe checks.
  • Detect RGB ESRGAN models from metadata, exclude latent upscalers from the standalone endpoint, and report compatible models in capabilities.
  • Validate output formats and parameters before loading the upscaler, returning 400 for invalid requests.

The patch is based on 79f69234 and includes two commits: the fixes and formatting. Apply it with:

git am upscale-endpoint-fixes.patch

upscale-endpoint-fixes.patch

@nbeerbower

Copy link
Copy Markdown
Author

Applied with git am, both commits are on the branch. Thanks for the fixes — the RGB ESRGAN detection in particular is a better answer than the model flag I had, since that only separated models from the built-in filters and not compatible models from incompatible ones.

Verified on the same setup as before (CUDA, RTX A6000, Qwen-Image-2.1 with --hires-upscalers-dir):

  • 512x512 to 2048x2048 with RealESRGAN_x4plus_anime_6B still takes ~2.2 s and returns the same result.
  • capabilities reports "upscale": true, marks that model "image_upscale": true, and carries max_upscale_width/max_upscale_height of 8192.
  • "upscaler": "Latent" is now refused: {"error":"no compatible image upscaler called Latent"}.
  • "repeats": 4 on a 512px image is refused before anything loads: {"error":"upscaled dimensions must not exceed 8192 x 8192"}.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants