diff --git a/ARCHITECTURE.md b/ARCHITECTURE.md index 4bc337f..c717771 100644 --- a/ARCHITECTURE.md +++ b/ARCHITECTURE.md @@ -322,8 +322,8 @@ The packed planes are copied to a mappable buffer and returned as a `Frame`. Whe | payload (`array`) | received blocks, padded with four zero words; on Apple silicon the CPU writes them in place, with the block offsets (D53) | grows on demand, 64 KiB at least | | block offsets (`array`) | word offset of every block, `u32::MAX` if missing | 49 KiB | | pyramid (`array`) | the four bands of every level of every component, back to back | 63.6 MiB | -| packed planes (`array`) | 8-bit planes, four pixels per word, rows padded to words | 11.9 MiB | -| readback (`MAP_READ`) | copy of the packed planes for the CPU, created by the first decode that reads them back | 11.9 MiB | +| packed planes (`array`) | 8-bit planes, four pixels per word, rows padded to 256 bytes, planes at multiples of 512 bytes | 12.1 MiB | +| readback (`MAP_READ`) | copy of the packed planes for the CPU, created by the first decode that reads them back | 12.1 MiB | | RGBA words (`array`) | one word per pixel, created on the first RGB decode | 31.6 MiB | | RGB readback (`MAP_READ`) | copy of the RGBA words, likewise | 31.6 MiB | | last coarsest level (`array`) | the coarsest level's bands of every component as the last decode left them, copied before a decode that conceals; created on the first (D66) | 383 KiB | @@ -420,7 +420,7 @@ The wall-clock time exceeds the sum of the stages, because submitting work and n ### Portability -The shaders, the decoder's and the encoder's, use only what WebGPU guarantees: no subgroup operations, no 8-bit or 16-bit storage, no `f16`, no immediates (push constants), no storage textures. One shader is the exception, on a path of its own: on Metal, where the adapter offers it, `idwt_planes.wgsl` stores the planes into `r8unorm` storage textures (D52), and the portable copies stay the fallback; likewise the frames written where the shaders read them on Apple silicon need `MAPPABLE_PRIMARY_BUFFERS` (D53). Neither is asked for under strict WebGPU. All buffers are plain storage or uniform buffers and every workgroup stays within 256 invocations and 16 KiB. The encoder reads pictures in textures as sampled `r8unorm` and `rg8unorm` textures with `textureGather` and a non-filtering sampler, and pictures in RGB as sampled `rgba8unorm` and `bgra8unorm` textures with `textureLoad` (D65), which WebGPU has without any feature. The encoder's rate control adds with 32-bit integer atomics, which WGSL has without any feature. Two tests hold this to account: +The shaders, the decoder's and the encoder's, use only what WebGPU guarantees: no subgroup operations, no 8-bit or 16-bit storage, no `f16`, no immediates (push constants), no storage textures. One shader is the exception, on a path of its own: on Metal, where the adapter offers it, `idwt_planes.wgsl` stores the planes into `r8unorm` storage textures (D52), and the portable copies stay the fallback; likewise the frames written where the shaders read them on Apple silicon need `MAPPABLE_PRIMARY_BUFFERS` (D53). Neither is asked for under strict WebGPU. All buffers are plain storage or uniform buffers and every workgroup stays within 256 invocations and 16 KiB. The encoder reads pictures in textures as sampled `r8unorm` and `rg8unorm` textures with `textureGather` and a non-filtering sampler, and pictures in RGB as sampled `rgba8unorm` and `bgra8unorm` textures with `textureLoad` (D65), which WebGPU has without any feature. The encoder's rate control adds with 32-bit integer atomics, which WGSL has without any feature. Two tests hold this to account, and a unit test checks one rule on the real backend: - `tests/shaders.rs` validates the WGSL of every shader with naga using an empty capability set, without a GPU, and checks the entry points, the workgroup sizes and memory. - `tests/gpu.rs` and `tests/gpu_encoder.rs` create a device with `InstanceFlags::STRICT_WEBGPU_COMPLIANCE` and the default limits and decode a vector on it, and encode pictures on it. @@ -489,7 +489,7 @@ Nothing in a browser may block, so the player uses only the parts of the decoder - `GpuContext::from_adapter` creates the device on an adapter that can draw on the canvas. `GpuContext::new`, which blocks, does not exist in the browser build. - `Decoder::record_rgb_to_texture` records the decode and the colour conversion into the player's command encoder and puts the picture into the canvas's texture: by a copy where the canvas allows it and is a multiple of 64 pixels wide, and otherwise by a draw (`draw.wgsl`: one triangle over the canvas, which has the size of the picture, so a fragment reads one word; decisions D40 and D42). One submission per frame, and the picture never leaves the GPU. - Error scopes cannot be waited for, so in the browser build `with_error_scope` just runs its work. The player catches what the browser reports instead (`on_uncaptured_error`) and shows the first error on the page with the next frame. Pictures wider or taller than the browser can draw (`max_texture_dimension_2d`, 8192 by default) are refused when the file is loaded. -- The methods that read a frame back or time it wait for the GPU, and are not for the browser. The decoder makes its read-back buffers only when a decode first reads back, so the player has none: 43.5 MiB fewer at 4K. +- The methods that read a frame back or time it wait for the GPU, and are not for the browser. The decoder makes its read-back buffers only when a decode first reads back, so the player has none: 43.8 MiB fewer at 4K. - In a browser, dropping a wgpu buffer only lets go of the JavaScript object, and the GPU memory waits for the garbage collector. `Decoder` therefore destroys its large buffers when it is dropped, and the player drops the old stream before it loads the next file: about 110 MiB for a 4K stream. - The player keeps one copy of the file in the module's memory and walks its frames in place. When it loads a file it reads the frames from the first up to one that is cut off, or shorter than the start of frame header every frame begins with; it plays those, and says why it does not play the rest. It loads files of up to 1 GiB and pictures of up to 8192x4320 pixels' worth (D39). - A lost device ends the stream, and the next file or sample gets a new player with a new device. A panic in the module leaves its objects in use, so the page then asks to be reloaded and disables its controls. @@ -649,7 +649,7 @@ Short-form ADRs, newest last. Each entry is dated. ### 2026-10-23 — D5: Toolchain and lints -**Decision.** Rust edition 2024, resolver 3, `rust-version` 1.85 (to be revisited when wgpu is added). Workspace lints: `unsafe_code = "forbid"` (the FFI crate will define its own lint table when it needs `unsafe`) and `missing_docs = "warn"`. CI turns warnings into errors through `clippy -D warnings`. Crates are `publish = false` because releases are GitHub Releases. +**Decision.** Rust edition 2024, resolver 3, `rust-version` 1.85 (to be revisited when wgpu is added). Workspace lints: `unsafe_code = "forbid"` (the FFI crate will define its own lint table when it needs `unsafe`) and `missing_docs = "warn"`. CI turns warnings into errors through `clippy -D warnings`. Crates are `publish = false` because releases are GitHub Releases. (Superseded in part by D74: `emberwave-core` and `emberwave-cli` are on crates.io.) ### 2026-10-23 — D6: Workflow conventions @@ -1884,3 +1884,19 @@ The last frame's detail, always, gains everything on a still picture and loses u **Consequence.** No client-visible change: no function, name or behaviour changes. A change that adds a C function without updating the header fails the header test, one that exports another name fails this one. +### 2026-11-18 — D74: emberwave-core and emberwave-cli on crates.io + +**Context.** The crates were `publish = false` (D5) and the releases GitHub Releases alone. Rust users could use the library and the command line tool only through a git dependency or a release archive. The maintainer asked what crates.io was on 2026-11-04, was recommended the two crates, and approved publishing them that night ("네, 올려"). The first upload was refused until the maintainer's crates.io e-mail address was verified; it went through on 2026-11-05. + +**Decision.** `emberwave-core` and `emberwave-cli` 0.18.1 are on crates.io (https://crates.io/crates/emberwave-core, https://crates.io/crates/emberwave-cli). `emberwave-ffi` and `emberwave-web` stay unpublished: C and Swift users take the GitHub release and the XCFramework, and the browser takes the web build. The workspace keeps `publish = false`, and the two crates set their own metadata: keywords, categories, `readme = "../../README.md"` and a copy of `LICENSE` in each crate directory, since a package holds nothing outside its directory. A release publishes the two after its tag's release is out, core first (`CONTRIBUTING.md`). + +**Why.** `cargo install emberwave-cli` and `cargo add emberwave-core` are the way Rust users look for software; the crates were ready and the cost is one more step of a release. + +**How it is done.** +- The upload uses a token that the maintainer made on crates.io with the scopes `publish-new` and `publish-update` and a 30-day expiry (to about 2026-12-04), stored by their own `cargo login`. It is never read, printed or passed on. +- The approval covered the first publish. A later version needs the maintainer's approval again, or Trusted Publishing: in each crate's settings on crates.io, GitHub with owner `with-developer`, repository `emberwave` and workflow `release.yml`, after which the release workflow could publish without a long-lived token. + +**Not done.** `THIRD_PARTY_NOTICES.md` is not in the packages: `cargo package --list` shows `LICENSE` and no notices file, where the release archives carry both (D51). Whether a crate has to carry PyroWave's notice is the maintainer's to judge, not settled here. Trusted Publishing is not set up. + +**Consequence.** A published version cannot be taken back, only yanked, so a release that publishes is a step to confirm. The workspace version, `EMBERWAVE_VERSION` and the crates' dependency on `emberwave-core` move together (D69). + diff --git a/BENCHMARKS.md b/BENCHMARKS.md index 3681ba8..f1139e1 100644 --- a/BENCHMARKS.md +++ b/BENCHMARKS.md @@ -1,6 +1,6 @@ # Benchmarks -How long emberwave takes to decode a frame, and to encode one. The tables under [Results](#results) are written by `scripts/bench.sh` from its measurements; everything else on this page is written by hand. +How long emberwave takes to decode a frame, and to encode one. The tables under [Results](#results) are written by `scripts/bench.sh` from its measurements, and the table under [Estimates for other devices](#estimates-for-other-devices) by `scripts/perf_model.py`; everything else on this page is written by hand. ## Reproducing @@ -23,7 +23,7 @@ scripts/bench_lavapipe.sh # optional: software Vulkan in a Linux conta - **Upstream** is the commit `testdata/README.md` pins; the tables were made at 79a3232, which has neither of upstream's fixes to its encoder of 2026-10-26; those left its decoder's output and its encoder's time as they were (decision D56). It is timed by `refgen bench` (`scripts/refgen/refgen.mm`) around upstream's C API, in the same way: give the decoder the frame's packet, encode the decode into a command buffer, commit it and wait. Its output, three 8-bit planes, stays in GPU memory, so it compares with emberwave's `--output yuv`. - **Encoding from textures** starts from an R8 texture per plane, made from each picture before the measurement (`bench-encode --gpu --textures`, `Encoder::encode_textures`), and ends with the frame's bytes in CPU memory like the encoding from CPU memory below. - **Encoding from BGRA** starts from the same pictures converted to BGRA, as most captures hold them, in CPU memory (`bench-encode --gpu --rgb`, `Encoder::encode_rgb`) or in a BGRA texture each, made before the measurement (`--rgb --textures`, `TexturePicture::Rgb`), and ends like the others. Its table comes with the next full run: the results below are older. -- **Encoding** goes from the picture's 8-bit planes in CPU memory to the frame's bytes in CPU memory: `Encoder::encode` for emberwave, `emberwave-cli bench-encode`; `pyrowave_encoder_encode_cpu`, counting the packets, which waits for the GPU, and `pyrowave_encoder_packetize` for upstream, `refgen bench-encode`. Both upload the planes and read the frame back within the time. The pictures are 20 frames of the same scene, made by `scripts/gen_bench_data.sh --sources`, at the budgets of the decoder's streams of the same size; both tools read them into memory first and take them in turn, so that encodes follow each other as decodes do. Upstream's encoder is timed at its default precision, an FP16 wavelet pyramid, and at FP32, which is what emberwave's uses. It hides its command buffer, so only its total is known. The CPU encoder, at a quarter of a second for a 4K frame, is timed over 60 frames instead of 600. +- **Encoding** goes from the picture's 8-bit planes in CPU memory to the frame's bytes in CPU memory: `Encoder::encode` for emberwave, `emberwave-cli bench-encode`; `pyrowave_encoder_encode_cpu`, counting the packets, which waits for the GPU, and `pyrowave_encoder_packetize` for upstream, `refgen bench-encode`. Both upload the planes and read the frame back within the time. The pictures are 20 frames of the same scene, made by `scripts/gen_bench_data.sh --sources`, at the budgets of the decoder's streams of the same size; both tools read them into memory first and take them in turn, so that encodes follow each other as decodes do. Upstream's encoder is timed at its default precision, an FP16 wavelet pyramid, and at FP32, which is what emberwave's uses. It hides its command buffer, so only its total is known. The CPU encoder, at a fifth of a second for a 4K frame, is timed over 60 frames instead of 600. ## Results @@ -245,7 +245,7 @@ Two more runs with `EMBERWAVE_BENCH_SPIN=1`, at 60 frames a second, the CPU awak - at FP32: 1.47 to 1.48 ms against 0.38 to 0.39 ms at 1080p, and 2.31 to 2.63 ms against 0.82 ms at 4K. That is 2.8 to 3.9 times as long, and its GPU time grows 2.1 to 3.2 times. The slowdown belongs to wavelet decoding on this GPU at this cadence, not to emberwave. On the same streams at that cadence, emberwave's decodes into textures still took 1.3 to 1.5 times as long as upstream's at FP32, more than frame after frame (issue #86). Decision D48 in [`ARCHITECTURE.md`](ARCHITECTURE.md) splits that time up: about 1 ms of it is Metal's own at that cadence, which any decoder pays. -- **What a client sees.** A client draws every display refresh, and its network threads wake between frames, so it lies between these cadences. On this Mac, emberwave's decoding is quicker than HEVC's only when the GPU is kept busy. [In a display loop](#in-a-display-loop-on-the-mac) times that on this Mac. +- **What a client sees.** A client draws every display refresh, and its network threads wake between frames, so it lies between these cadences. In the runs of release 0.11.0 above, emberwave's decoding was quicker than HEVC's only when the GPU was kept busy. [In a display loop](#in-a-display-loop-on-the-mac) times that on this Mac with release 0.18.1: into the planes emberwave is quicker at 1440p and 4K there, and about as quick at 1080p. - **HEVC's decoding grows with the bytes.** At emberwave's bytes per frame it took 2.4 to 4.7 times as long as at Moonlight's rates, 12.3 to 12.7 ms at 4K. Raising HEVC's bit rate costs the client time. - **After a loss**, an HEVC client needs a keyframe: 600 to 730 kB at 4K, which took 5.6 to 7.8 ms to decode. Every PyroWave frame is one. - **BGRA** is VideoToolbox's colour conversion, which Moonlight does not use: it draws NV12. Into RGB, emberwave took 27 to 49 % less than HEVC into BGRA at 60 frames a second. @@ -284,7 +284,7 @@ The two runs' means differed by 4 % or less, except in five rows. HEVC's encodin - Between frames, emberwave's decodes slowed down 1.6 to 4.8 times, and HEVC's by 5.3 % at most. - Into the planes, emberwave then took 2.7 times as long as HEVC into NV12 at 1080p: 5.88 to 5.96 against 2.19 to 2.20 ms. At 1440p it took 2.3 to 2.4 times as long, and at 4K 1.5 times: 9.37 to 9.39 against 6.27 to 6.43 ms. - On the Mac the two took about as long at that cadence. -- **What the cadence adds hardly grows with the size.** At 60 frames a second, a decode into the planes took 4.6 to 4.7 ms longer than frame after frame at 1080p, 5.4 to 5.9 ms at 1440p and 4.9 ms at 4K. On the M3 Max that was about 1.7 ms at 1080p and 2.8 to 3.0 ms at 4K. Issue #86 follows where it goes. +- **What the cadence adds hardly grows with the size.** At 60 frames a second, a decode into the planes took 4.6 to 4.7 ms longer than frame after frame at 1080p, 5.4 to 5.9 ms at 1440p and 4.9 ms at 4K. On the M3 Max that was about 1.7 ms at 1080p and 2.8 to 3.0 ms at 4K. Issue #86, closed since, followed where it went (decision D48). - **HEVC's decoding grows with the bytes.** At emberwave's bytes per frame it took 2.0 to 2.3 times as long as at Moonlight's rates. At 4K that is 12.9 ms, longer than emberwave's decode even at 60 frames a second. - **Keyframes** took no longer than the other frames: 6.2 ms at 4K for 632 to 636 kB. On the Mac they took about twice as long. - **BGRA**, VideoToolbox's colour conversion, which Moonlight does not use, makes HEVC's decoding take 1.9 to 2.1 times as long on the phone. At 60 frames a second, emberwave into RGB took 1.7 to 1.8 times as long as HEVC into BGRA at 1080p, 1.4 times at 1440p, and 21 to 23 % less at 4K. @@ -363,7 +363,7 @@ At 60 frames a second the runs differ more than the two ways do. A scratch progr One round at 1080p, frame after frame, was disturbed for both, 0.54 and 0.58 ms, and is left out of the ranges. - **Frame after frame** the stores take 8 to 12 % less at 4K, 0.08 to 0.12 ms, about all that leaving out the copies promised ([#86](https://github.com/with-developer/emberwave/issues/86)), and about 0.02 ms, 5 %, less at 1080p. -- **At 60 frames a second** the difference is within the noise of these runs: at 4K the median round gained 0.16 ms, about 5 %, while single rounds went from 0.29 ms worse to 0.40 ms better, and at 1080p nothing is measurable. Leaving out the copies had suggested about 0.3 ms at 4K. The gap to upstream at that pace (decision D48) remains for the next items of #86. +- **At 60 frames a second** the difference is within the noise of these runs: at 4K the median round gained 0.16 ms, about 5 %, while single rounds went from 0.29 ms worse to 0.40 ms better, and at 1080p nothing is measurable. Leaving out the copies had suggested about 0.3 ms at 4K. The gap to upstream at that pace (decision D48) remained for the next items of #86, which is closed; what is left of it is #150. - **The copies cost the same as before.** Timed alternately with the program built from `main`, the branch's copies took the same time frame after frame, 0.96 to 0.97 ms at 4K against 0.96 to 0.98 ms, so splitting the inverse transform into two shaders cost nothing. ## Frames written where the shaders read them @@ -498,7 +498,7 @@ Each row is a Chrome stream and, beside it, emberwave at the same bytes a frame. - **At the same bytes the codecs with predicted frames lead in PSNR.** On the scene by 15.4 to 17.7 dB of luma, on the noisy copy by 3.1 to 4.6 dB, at every size and rate that ran, with emberwave given more bytes than the predicted frames cost. On the scene the codecs sit at 44.4 to 46.1 dB, at the ceiling that the scene's grain sets, from 8 kB a frame on (3.5 kB after the key frame); emberwave needs 400 kB a 1080p frame to reach 46.8 dB. The noisy copy leaves the codecs only the noise to code, and they lead by about 4 dB. The earlier comparison on a single key frame of a game, [above](#picture-quality), had HEVC 1.9 to 3.5 dB ahead of emberwave on luma with 2 to 10 % fewer bytes, where nothing could be predicted: the lead here is the prediction's. - **That is what emberwave trades for.** Every frame of it stands alone, so a lost frame costs nothing after it and a decode needs no earlier frame, and the GPU encoder takes 0.8 to 1.9 ms a frame on this Mac ([encoding above](#results)) where Chrome's encoders took 6 to 9 ms at 1080p and 21 to 24 ms at 4K with the hardware ones, 1.9 to 17 ms with its software AV1 and H.264 encoders in real-time mode, and 114 to 610 ms with its software HEVC encoder. The table does not say what the codecs' lead is worth to a viewer: PSNR counts every pixel alike, the pictures are synthetic, and the encoders were in real-time mode. - **Decoding in Chrome.** At Moonlight's rates a frame took 0.9 to 5.3 ms to come out of `VideoDecoder` one at a time on the scene, and 1.2 to 4.7 ms on the noisy copy, with all frames at once 0.4 to 1.9 and 0.4 to 3.6 ms; at emberwave's bytes, 1.0 to 2.7 ms on the scene and 2.2 to 12.6 ms on the noisy copy. emberwave's own decode in earlier Chrome versions on this Mac, through WebGPU, took 0.70 to 1.00 ms at 1080p and 1.69 to 1.88 ms at 4K into an RGB texture, frame after frame ([In the browser](#in-the-browser-phones-included)), and 4.50 to 5.67 ms at 4K at a stream's pace. The two do different work: `VideoDecoder`'s hardware decoders run beside the GPU and hand over video frames that are not yet drawn, and the page measures the time to the frame, not to the screen. -- **For D67's chroma weights these numbers say little.** At the same bytes emberwave's chroma PSNR sits further above its luma PSNR than the codecs' in most rows: Cb in 12 of the 15 rows of the scene and all 15 of the noisy copy, Cr in all 30. On the scene the gap of Cb over luma is 4.2 to 12.6 dB for emberwave and 2.9 to 9.5 for the codecs, on the noisy copy 4.9 to 9.1 against 1.3 to 5.6; HEVC at 1080p is the exception on the scene (6.5 and 6.6 dB against 7.3 and 8.3). That is the direction in which D67's tilt would move bits at low budgets, from chroma to luma. But PSNR counts every band alike, the scene's grain is in luma only, which widens every codec's gap, and the pictures are synthetic, with the chroma of bars and a gradient: this is not evidence that the tilt is worth it. Game footage and a perceptual metric are what D67 asks for; the weights stay upstream's. +- **For D67's chroma weights these numbers say little.** At the same bytes emberwave's chroma PSNR sits further above its luma PSNR than the codecs' in most rows: Cb in 12 of the 15 rows of the scene and all 15 of the noisy copy, Cr in all 30. On the scene the gap of Cb over luma is 4.2 to 12.6 dB for emberwave and 2.9 to 9.5 for the codecs, on the noisy copy 4.9 to 9.1 against 1.3 to 5.6; HEVC at 1080p is the exception on the scene (6.5 and 6.6 dB against 7.3 and 8.3), and at 4K and 576 Mbit/s the two are level (9.5 dB each). That is the direction in which D67's tilt would move bits at low budgets, from chroma to luma. But PSNR counts every band alike, the scene's grain is in luma only, which widens every codec's gap, and the pictures are synthetic, with the chroma of bars and a gradient: this is not evidence that the tilt is worth it. Game footage and a perceptual metric are what D67 asks for; the weights stay upstream's. - **Not measured.** NVENC's own streams, which differ from VideoToolbox's and libaom's; moving game footage; the iPhone; that AV1 decoded in hardware (the page asks for it, and Chrome does not say); the rows listed below. Rows that did not run: diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index d151092..eb3b701 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -85,4 +85,4 @@ After a deliberate change to what the encoder writes, `scripts/gen_encoder_vecto ## Licence -emberwave is MIT licensed, and contributions are accepted under the same licence. [`LICENSE`](LICENSE) is the MIT licence alone; PyroWave's licence, and what is in the repository but not covered by the licence, are in [`THIRD_PARTY_NOTICES.md`](THIRD_PARTY_NOTICES.md), which ships wherever `LICENSE` does (decision D51). +emberwave is MIT licensed, and contributions are accepted under the same licence. [`LICENSE`](LICENSE) is the MIT licence alone; PyroWave's licence, and what is in the repository but not covered by the licence, are in [`THIRD_PARTY_NOTICES.md`](THIRD_PARTY_NOTICES.md), which ships in the release archives (decision D51); the crates on crates.io carry `LICENSE` alone. diff --git a/Cargo.toml b/Cargo.toml index afb24a0..ecdb42b 100644 --- a/Cargo.toml +++ b/Cargo.toml @@ -13,7 +13,9 @@ edition = "2024" license = "MIT" repository = "https://github.com/with-developer/emberwave" rust-version = "1.87" -# Releases are GitHub Releases only; keep cargo from publishing by accident. +# Releases are GitHub Releases; emberwave-core and emberwave-cli also go to +# crates.io (decision D74) and do not inherit this; it keeps cargo from +# publishing the other crates by accident. publish = false [workspace.dependencies] diff --git a/README.ko.md b/README.ko.md index b942028..be9004c 100644 --- a/README.ko.md +++ b/README.ko.md @@ -132,7 +132,7 @@ Apple M3 Max(Metal)에서 2026-10-26에 측정한 프레임당 디코딩 시간 iPhone 17의 Safari에서는 [벤치마크 페이지](https://with-developer.github.io/emberwave/bench.html)가 1080p 프레임을 2.4 ms, 4K 프레임을 8.1 ms에 GPU 메모리의 RGB로 디코딩합니다([`BENCHMARKS.md`](BENCHMARKS.md#in-the-browser-phones-included)). XCFramework를 쓰는 네이티브 [벤치마크 앱](crates/emberwave-ffi/examples/ios)은 같은 프레임을 Metal 텍스처 한 장에는 1.6 ms와 6.2~6.3 ms, Y·Cb·Cr 텍스처에는 1.3 ms와 4.8 ms에 디코딩합니다([`BENCHMARKS.md`](BENCHMARKS.md#on-the-iphone-natively)). -같은 앱으로 Moonlight가 쓰는 하드웨어 HEVC 디코더도 Moonlight의 기본 비트레이트에서 측정했습니다. 프레임을 연달아 디코딩하면 emberwave의 텍스처 디코딩이 M3 Max에서 2.2~3.9배(4K 기준 1.0 ms 대 2.6~3.4 ms), iPhone 17에서 1.4~1.7배(4.5 ms 대 HEVC 6.2~6.3 ms) 빠릅니다(위 측정의 하루 뒤에 잰 값). 하지만 프레임이 초당 60개씩 들어오면, GPU는 프레임 사이에 쉬면서 느려지는 반면 미디어 엔진은 그렇지 않습니다. 이때 Mac에서는 둘이 비슷하고, iPhone에서는 HEVC가 1.5~2.7배 빠릅니다([`BENCHMARKS.md`](BENCHMARKS.md#against-hevc-on-the-same-device)). +같은 앱으로 Moonlight가 쓰는 하드웨어 HEVC 디코더도 Moonlight의 기본 비트레이트에서 측정했습니다. 프레임을 연달아 디코딩하면 emberwave의 텍스처 디코딩이 M3 Max에서 2.2~3.9배(4K 기준 1.0 ms 대 2.6~3.4 ms), iPhone 17에서 1.4~1.7배(4.5 ms 대 HEVC 6.2~6.3 ms) 빠릅니다(위 측정의 하루 뒤에 잰 값). 하지만 프레임이 초당 60개씩 들어오면, GPU는 프레임 사이에 쉬면서 느려지는 반면 미디어 엔진은 그렇지 않습니다. 이 측정에서 Mac의 두 디코더는 비슷했지만, 0.18.1 라이브러리를 화면 출력 루프와 함께 재면 1440p와 4K에서는 emberwave의 플레인 디코딩이 HEVC보다 빠릅니다([`BENCHMARKS.md`](BENCHMARKS.md#in-a-display-loop-on-the-mac)). iPhone에서는 HEVC가 1.5~2.7배 빠릅니다([`BENCHMARKS.md`](BENCHMARKS.md#against-hevc-on-the-same-device)). ## 목표 @@ -155,7 +155,7 @@ iPhone 17의 Safari에서는 [벤치마크 페이지](https://with-developer.git 1.0.0의 날짜는 정하지 않았습니다. 1.0.0은 C·Rust API를 호환되게 유지하겠다는 약속이고, 아래 네 가지가 갖춰지면 태그를 달며, 그 판단은 관리자가 합니다. -- [ ] **안정된 C·Rust API.** 정리된 것: C API가 프로그램을 깨지 않고 늘어나는 규칙(결정 D69), Rust API의 모양(D70), 공유 라이브러리가 헤더의 함수만 내보내는 것(macOS에서 Objective-C 이름 4개만 예외, D73). 남은 것: 라이브러리 이름에 아직 버전이 없고, Rust API의 공개 항목 26개가 wgpu 타입을 받거나 돌려주므로 wgpu의 새 주 버전이 나오면 깨지며, 클라이언트 자신의 Metal·Vulkan·Direct3D 객체를 받는 호출은 실제로 쓰이기 전까지 잠정입니다. 앞의 두 가지는 [#165](https://github.com/with-developer/emberwave/issues/165)에 질문으로 올려 두었습니다. +- [ ] **안정된 C·Rust API.** 정리된 것: C API가 프로그램을 깨지 않고 늘어나는 규칙(결정 D69), Rust API의 모양(D70), 공유 라이브러리가 헤더의 함수만 내보내는 것(macOS에서 Objective-C 이름 4개만 예외, D73). 남은 것: 라이브러리 이름에 아직 버전이 없고, Rust API의 공개 항목 약 서른 개가 wgpu 타입을 받거나 돌려주므로(결정 D70) wgpu의 새 주 버전이 나오면 깨지며, 클라이언트 자신의 Metal·Vulkan·Direct3D 객체를 받는 호출은 실제로 쓰이기 전까지 잠정입니다. 앞의 두 가지는 [#165](https://github.com/with-developer/emberwave/issues/165)에 질문으로 올려 두었습니다. - [ ] **실제로 쓰이는 클라이언트 하나.** 별도 저장소의 브라우저 클라이언트 `emberwave-stream`이 Mac의 화면을 같은 Mac의 탭으로 스트리밍하는 것까지 됩니다. 로컬 네트워크를 건너는 것은 아직 보이지 못했습니다. 두 번째 기기가 필요하거나, 루프백 에뮬레이터로 대신해도 된다는 판단이 필요합니다. - [ ] **스트림 속도에서 업스트림과 같은 수준의 디코딩.** 4K는 도달했고, 1080p는 0.15~0.3 ms 느립니다([#150](https://github.com/with-developer/emberwave/issues/150)). - [ ] **패키지.** crates.io에는 `emberwave-core`와 `emberwave-cli` 0.18.1이 올라가 있습니다. 아직 없는 것은 브라우저 플레이어용 패키지와, Swift용으로 저장소 루트에 두는 `Package.swift`입니다(결정 D64: 릴리스 워크플로를 바꿔야 합니다). diff --git a/README.md b/README.md index dabdb51..9c604f6 100644 --- a/README.md +++ b/README.md @@ -132,7 +132,7 @@ Encoding a 4K frame within 1.2 MB takes 1.90 ms on the GPU, from the planes in C In Safari on an iPhone 17 the [benchmark page](https://with-developer.github.io/emberwave/bench.html) decodes a 1080p frame in 2.4 ms and a 4K frame in 8.1 ms, to RGB in GPU memory ([`BENCHMARKS.md`](BENCHMARKS.md#in-the-browser-phones-included)). Natively, through the XCFramework, the [benchmark app](crates/emberwave-ffi/examples/ios) decodes them into a Metal texture in 1.6 and 6.2 to 6.3 ms, and into Y, Cb and Cr textures in 1.3 and 4.8 ms ([`BENCHMARKS.md`](BENCHMARKS.md#on-the-iphone-natively)). -The same app times the hardware HEVC decoder that Moonlight uses, at Moonlight's default bit rates. Frame after frame, emberwave decodes into textures 2.2 to 3.9 times as quickly on the M3 Max, 1.0 against 2.6 to 3.4 ms at 4K, and 1.4 to 1.7 times as quickly on the iPhone 17, 4.5 against HEVC's 6.2 to 6.3 ms, in runs a day after those above. When frames come 60 a second, the GPU slows down while it idles between them, and the media engine does not. The two then take about as long on the Mac, and on the phone HEVC is 1.5 to 2.7 times as quick ([`BENCHMARKS.md`](BENCHMARKS.md#against-hevc-on-the-same-device)). +The same app times the hardware HEVC decoder that Moonlight uses, at Moonlight's default bit rates. Frame after frame, emberwave decodes into textures 2.2 to 3.9 times as quickly on the M3 Max, 1.0 against 2.6 to 3.4 ms at 4K, and 1.4 to 1.7 times as quickly on the iPhone 17, 4.5 against HEVC's 6.2 to 6.3 ms, in runs a day after those above. When frames come 60 a second, the GPU slows down while it idles between them, and the media engine does not. In those runs the two then took about as long on the Mac; the 0.18.1 library, timed beside a display loop, decodes into the planes quicker than HEVC at 1440p and 4K ([`BENCHMARKS.md`](BENCHMARKS.md#in-a-display-loop-on-the-mac)). On the phone HEVC is 1.5 to 2.7 times as quick ([`BENCHMARKS.md`](BENCHMARKS.md#against-hevc-on-the-same-device)). ## Goals @@ -155,7 +155,7 @@ Next, each in a release when it is ready: 1.0.0 has no date. It is the promise that the C and Rust APIs stay compatible, and it will be tagged when the four things below are true; the maintainer makes that call. -- [ ] **Stable C and Rust APIs.** Settled: the rules by which the C API grows without breaking programs (decision D69), the shape of the Rust API (D70), and the shared library exporting the header's functions and nothing else, apart from four Objective-C names on macOS (D73). Open: the library's name carries no version yet, 26 public items of the Rust API take or return wgpu's types, which a new major version of wgpu would break, and the calls on a client's own Metal, Vulkan and Direct3D objects are provisional until a client has used them. The first two are put as a question in [#165](https://github.com/with-developer/emberwave/issues/165). +- [ ] **Stable C and Rust APIs.** Settled: the rules by which the C API grows without breaking programs (decision D69), the shape of the Rust API (D70), and the shared library exporting the header's functions and nothing else, apart from four Objective-C names on macOS (D73). Open: the library's name carries no version yet, about thirty public items of the Rust API take or return wgpu's types (decision D70), which a new major version of wgpu would break, and the calls on a client's own Metal, Vulkan and Direct3D objects are provisional until a client has used them. The first two are put as a question in [#165](https://github.com/with-developer/emberwave/issues/165). - [ ] **One client in real use.** `emberwave-stream`, the browser client in a repository of its own, streams a Mac's screen to a tab on the same Mac. Across a local network it is not shown yet: that needs a second device, or the loopback emulator accepted in its place. - [ ] **Decoding at a stream's pace level with upstream's.** 4K is there; 1080p is 0.15 to 0.3 ms behind ([#150](https://github.com/with-developer/emberwave/issues/150)). - [ ] **Packages.** On crates.io: `emberwave-core` and `emberwave-cli`, 0.18.1. Not yet: a package for the browser player, and a `Package.swift` at the root of the repository for Swift (decision D64: the release workflow would have to change). diff --git a/crates/emberwave-ffi/README.md b/crates/emberwave-ffi/README.md index 2934373..71ace0f 100644 --- a/crates/emberwave-ffi/README.md +++ b/crates/emberwave-ffi/README.md @@ -105,7 +105,7 @@ swift run --package-path crates/emberwave-ffi/examples/swift \ emberwave-decode testdata/noise_192x136_420.pwv1 out.y4m ``` -`scripts/check_xcframework.sh` builds the C examples and the Swift example against every library of the XCFramework. It also builds the Swift example from `Emberwave.xcframework.zip` as a binary target, as an app's package takes a release's archive. CI runs them on a Mac and in an iOS Simulator, and links them for devices. They have not run on an iPhone or an iPad yet, where an app has to be signed by its developer. The Swift example is for macOS 12, the oldest that the Swift Build of Xcode 27 takes; the library runs on macOS 11. +`scripts/check_xcframework.sh` builds the C examples and the Swift example against every library of the XCFramework. It also builds the Swift example from `Emberwave.xcframework.zip` as a binary target, as an app's package takes a release's archive. CI runs them on a Mac and in an iOS Simulator, and links them for devices. The examples have not run on an iPhone or an iPad, where an app has to be signed by its developer; the benchmark app has (decision D35). The Swift example is for macOS 12, the oldest that the Swift Build of Xcode 27 takes; the library runs on macOS 11. ## Use diff --git a/crates/emberwave-ffi/examples/ios/README.md b/crates/emberwave-ffi/examples/ios/README.md index 859cad0..3781d9c 100644 --- a/crates/emberwave-ffi/examples/ios/README.md +++ b/crates/emberwave-ffi/examples/ios/README.md @@ -67,7 +67,7 @@ Unlock the phone and leave it on the home screen first. With `-run`, the app sta ## On a Mac -`scripts/bench_app_on_mac.sh` makes the same measurements on a Mac, by the app's own code: it builds `EmberwaveBenchMac`, a command-line front-end, against the XCFramework's library for the Mac, and prints the app's report. The progress goes to the standard error. It needs the XCFramework built by `scripts/build_xcframework.sh`, which has the library for the Mac. With `-display` it decodes beside a display loop instead (issue #81, decision D72): a borderless window covers a screen for about three minutes, drawing the last picture at 60 and 120 Hz, and the report adds the time to the screen; `-short` makes it brief. +`scripts/bench_app_on_mac.sh` makes the same measurements on a Mac, by the app's own code: it builds `EmberwaveBenchMac`, a command-line front-end, against the XCFramework's library for the Mac, and prints the app's report. The progress goes to the standard error. It needs the XCFramework built by `scripts/build_xcframework.sh`, which has the library for the Mac. With `-display` it decodes beside a display loop instead (issue #81, decision D72): a borderless window covers a screen for about three and a half minutes, drawing the last picture at 60 and 120 Hz, and the report adds the time to the screen; `-short` makes it brief. ## In CI diff --git a/scripts/bench.sh b/scripts/bench.sh index 676bd18..683f4a0 100755 --- a/scripts/bench.sh +++ b/scripts/bench.sh @@ -7,7 +7,7 @@ # --frames N frames measured per number (default 600: ten passes over # each 60-frame stream, so that the 99th percentile is not # simply the slowest frame); the CPU encoder, which takes a -# quarter of a second for a 4K frame, gets a tenth of it +# fifth of a second for a 4K frame, gets a tenth of it # --no-upstream do not time upstream's Metal decoder and encoder for # comparison # @@ -82,7 +82,7 @@ for stream in $STREAMS; do fi done -# The CPU encoder takes a quarter of a second for a 4K frame: a tenth of the +# The CPU encoder takes a fifth of a second for a 4K frame: a tenth of the # frames, one at least. CPU_FRAMES=$(( FRAMES >= 10 ? FRAMES / 10 : 1 )) for entry in $ENCODE; do