feat: add a stand-alone upscale endpoint to the server - #2026
Open
nbeerbower wants to merge 3 commits into
Open
nbeerbower wants to merge 3 commits into
nbeerbower wants to merge 3 commits into
Conversation
The library has had standalone upscaling all along -- `sd-cli -M upscale`
runs an ESRGAN model with no diffusion model, no text encoder and no
sampling, 512x512 to 2048x2048 in about two seconds -- but over HTTP it
could only be reached as the hires stage of an image generation. That means
paying for a generation nobody wanted, and it is not always survivable: a
hires request against Qwen-Image-2.1 dies in
GGML_ASSERT(a->ne[2] == b->ne[2]) at ggml.c:4557 and takes the server with
it. A plain upscale cannot be expressed through hires in any case, since
hires denoising strength must be greater than zero.
The route is synchronous, like the sdapi ones, because two seconds of work
does not need the async job machinery. It holds the generation context mutex
so an upscale and a generation never use the device at the same time,
resolves the model by the same name `capabilities` reports under
`upscalers`, and answers in the shape the other image endpoints use.
POST /sdcpp/v1/upscale
{"image": "<base64>", "upscaler": "RealESRGAN_x4plus_anime_6B",
"repeats": 1, "tile_size": 128, "output_format": "png"}
Two additions to `capabilities` so a client can offer this only where it
works: an `upscale` flag, since a server started without
--hires-upscalers-dir has the route but nothing to run; and a `model` flag
on each `upscalers` entry, because that list also carries the built-in
scaling filters (None, Lanczos, the latent modes), which are choices for the
second stage of a generation rather than models that can be run on their
own. Both are documented in examples/server/api.md alongside the endpoint.
Owner
|
I can't push to this branch due to the organization's permissions. Could you apply the attached patch to your PR branch? It contains the review fixes:
The patch is based on git am upscale-endpoint-fixes.patch |
Author
|
Applied with Verified on the same setup as before (CUDA, RTX A6000, Qwen-Image-2.1 with
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds
POST /sdcpp/v1/upscale, so an ESRGAN model can be run on its own overHTTP without a generation wrapped around it.
The capability is already in the library —
sd-cli -M upscale(#865) runs anupscaler with no diffusion model, no text encoder and no sampling — but the
only way to reach it from the server was
hiresinside an image generation.That has two problems:
expressed at all, because
hires.denoising_strengthmust be greater thanzero.
hiresrequest against Qwen-Image-2.1 abortsthe server in
GGML_ASSERT(a->ne[2] == b->ne[2])atggml.c:4557.The new route is synchronous, like the
sdapiones — the work is a couple ofseconds with no model loading behind it, so it does not need the async job
machinery. It takes the generation context mutex, so an upscale and a
generation never use the device at the same time, and it resolves the model by
the same name
capabilitiesalready reports underupscalers.Two small additions to
capabilitiesso a client can offer this only where itwill work:
upscale— a server started without--hires-upscalers-dirhas the routebut nothing to run.
modelon eachupscalersentry — that list also carries the built-inscaling filters (
None,Lanczos, the latent modes), which are choices forthe second stage of a generation rather than models that can be run alone.
Without this a client cannot tell the two apart, and asking to upscale with
"None" is a confusing failure.
Both are documented in
examples/server/api.mdalongside the endpoint.Related Issue / Discussion
Exposes the stand-alone upscale mode added in #865 over the server API.
Additional Information
Verified against a CUDA build on an RTX A6000, server loaded with
Qwen-Image-2.1 and
--hires-upscalers-dirpointing atRealESRGAN_x4plus_anime_6B.pth:512x512 in, 2048x2048 out, 2.2 s including the model load. Matches what
sd-cli -M upscaleproduces for the same input.Refusals checked: empty body, missing
image, unreadableimage, and a namethat is not a model-backed upscaler each return 400 with a message rather than
failing later.
capabilitieson that server reports"upscale": trueand marksRealESRGAN_x4plus_anime_6Bwith"model": truewhile the built-in filterscarry
"model": false.Checklist