Skip to content

vp9: validate kf_frame_max_boost_first_factor.den in validate_config - #10

Open
kalt2212 wants to merge 4 commits into
webmproject:mainfrom
kalt2212:patch/vp9-validate-config-boost-den
Open

kalt2212 wants to merge 4 commits into
webmproject:mainfrom
kalt2212:patch/vp9-validate-config-boost-den

Conversation

@kalt2212

Copy link
Copy Markdown

Summary

This ports upstream commit deaac25 ("vp8: Validate ss_number_layers and
factor den in validate_config", 2026-09-28) to the VP9 encoder, which was not
given the same treatment.

  1. VP9 checks .den in [1,1000] for 14 rational config factors but not
    kf_frame_max_boost_first_factor.den. The denominator is used as a divisor:
    cpi->twopass.kf_frame_max_boost_first = (double)num / (double)den.
    With den = 0 (settable via the public vpx_codec_enc_init API):
    num/0.0 -> +inf (clamped to 4.0, benign), but 0.0/0.0 -> NaN, which
    survives both clamp comparisons and poisons two-pass rate-control state.
  2. The rc_twopass_stats_in validation block (for VPX_RC_LAST_PASS) is moved
    after all scalar range checks, mirroring the VP8 reorder.

A sweep of every other *_factor.den divisor in the VP9 encoder confirmed
this is the only missing one.

Impact

Invalid encoder configs are now rejected at init instead of silently accepted.

Testing

  • New unittest EncodeAPI.ValidateConfigVp9 (mirrors EncodeAPI.ValidateConfigVp8):
    fails before the fix (vpx_codec_enc_init with den = 0 returned
    VPX_CODEC_OK), passes after (VPX_CODEC_INVALID_PARAM).
  • Full EncodeAPI.* suite: 66/66 pass.

Patch notes

vp9/vp9_cx_iface.c (+35/-33, mostly the block move), test/encode_api_test.cc (+20).

humifix and others added 4 commits September 23, 2026 13:25
Use the destination stride when accessing the destination buffer in
copy_mem64x64 and rd_pick_intra4x4block. Add a copy_mem64x64
regression test that uses a 16x16 checkerboard and frame buffers with
different strides.

Change-Id: I3bf8b6232185a76236d879b1c6726192a0faccac
Add range checks in vp8 validate_config() for ss_number_layers
(1..VPX_SS_MAX_LAYERS) and kf_frame_max_boost_first_factor.den,
and perform all scalar config range checks before dereferencing
rc_twopass_stats_in.buf when g_pass == VPX_RC_LAST_PASS.

Add unittest.

Bug: 564326761
Change-Id: I57edc6e3f8b51caf666bc40ea20352c2b2b01e64
vpx_codec_decode() can return VPX_CODEC_OK for a corrupted VP8 frame,
e.g. when data is missing from the last token partition. Document that
callers must query VP8D_GET_FRAME_CORRUPTED after every decode.

Bug: chromium:561396583
Change-Id: I07f1e495e81a1ff51bf2502b7a64b8c1744bb764
Port the VP8 fix from deaac25 (Bug 564326761) to the VP9 encoder.
validate_config() checked .den in [1,1000] for 14 rational config
factors but not kf_frame_max_boost_first_factor.den, which is used
as a divisor when setting up two-pass rate control; den=0 yields
inf/NaN in kf_frame_max_boost_first. Also perform all scalar config
range checks before dereferencing rc_twopass_stats_in.buf for
VPX_RC_LAST_PASS, matching the VP8 ordering.

Add EncodeAPI.ValidateConfigVp9 unittest.
@google-cla

google-cla Bot commented Sep 30, 2026

Copy link
Copy Markdown

Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

View this failed invocation of the CLA check for more information.

For the most up to date status, view the checks section at the bottom of the pull request.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants