Skip to content

Release 5.16.2: BIMI x/y attribute fix; narrower dnssec exceptions#254

Merged
seanthegeek merged 5 commits into
mainfrom
honest-test-coverage
May 21, 2026
Merged

Release 5.16.2: BIMI x/y attribute fix; narrower dnssec exceptions#254
seanthegeek merged 5 commits into
mainfrom
honest-test-coverage

Conversation

@seanthegeek

Copy link
Copy Markdown
Contributor

Summary

A self-audit against the principle that tests should assert on observable behavior — not just exercise lines — turned up a handful of tests written purely to chase coverage. Several of them sat on top of unfixed bugs and even labeled themselves as such. This PR removes those tests, fixes the underlying bug they were hiding, and narrows a broad exception handler that one of them depended on.

Bug fix

  • BIMI: forbidden SVG x/y attributes are now actually rejected. get_svg_metadata was reading svg["x"] / svg["y"], but xmltodict prefixes attributes with @, so those keys only match child elements — which real SVGs don't have. Net effect: the existing check_svg_requirements rejection of x/y attributes on the root <svg> element has been a no-op. Switched to @x / @y and fixed a typo where the y-value was being written to metadata["x"].

Test cleanup

  • Removed TestSvgMetadataExtraAttributes (tests/test_bimi.py). Built malformed SVGs with child <x> / <y> / <description> / <overflow> elements to hit branches that are unreachable from real SVG input. One test even openly stated: "The branch wrote to metadata['x'] (not metadata['y']) per the source bug — the goal here is line coverage, not correctness."
  • Replaced with one honest end-to-end test that constructs a real SVG with x="0" y="0" attributes, asserts they're captured, and asserts check_svg_requirements flags them.
  • Removed testKnownGoodMocked (tests/test_init.py). Patched all nine check_* helpers to return canned valid dicts, then asserted the canned values came back. The network counterpart testKnownGood is the real test.
  • Removed testValidationExceptionContinues (tests/test_dnssec.py). Existed only to exercise the broad except Exception clause we're narrowing here.

Convention compliance

  • Narrowed three except Exception clauses in checkdmarc/dnssec.py to except (dns.exception.DNSException, OSError, EOFError) per AGENTS.md's "Don't catch Exception broadly" rule.

Coverage impact

Total coverage 96% (small honest drop). The lost percentage points represent branches that were either unreachable, buggy, or exercised tautologically — nothing of real value.

Test plan

  • ruff check --show-fixes — clean
  • python -m pytest tests/ — 433 passed, 10 skipped (network tests on CI)
  • New testRootXYAttributesCaptured proves the BIMI fix works end-to-end
  • CI passes on this branch

🤖 Generated with Claude Code

Drop tests that asserted on contrived inputs solely to keep coverage
high, and fix the underlying bugs they were papering over.

- checkdmarc/bimi.py: SVG x/y forbidden-attribute capture was reading
  the wrong xmltodict keys ("x"/"y" matches child elements, not
  attributes), so the check_svg_requirements rejection never fired
  on real SVGs. Switch to "@x"/"@y" and fix a typo that wrote the
  y value into metadata["x"].
- checkdmarc/dnssec.py: Narrow three broad "except Exception" clauses
  to (dns.exception.DNSException, OSError, EOFError) per AGENTS.md.
- tests/test_bimi.py: Delete TestSvgMetadataExtraAttributes; it built
  malformed SVGs (with child <x>/<y>/<description>/<overflow> elements)
  to exercise unreachable branches and even asserted on a known source
  bug. Replace with one honest end-to-end test that proves the fixed
  forbidden-attribute path works.
- tests/test_init.py: Delete testKnownGoodMocked — it patched all nine
  check_* helpers to return canned valid dicts then asserted the canned
  values came back. The network counterpart testKnownGood is the real
  test.
- tests/test_dnssec.py: Delete testValidationExceptionContinues — it
  existed only to exercise the broad except we've now narrowed away.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@codecov

codecov Bot commented May 20, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 96.34%. Comparing base (0fb62ac) to head (a8a8748).
✅ All tests successful. No failed tests found.

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #254      +/-   ##
==========================================
- Coverage   96.39%   96.34%   -0.05%     
==========================================
  Files          24       24              
  Lines        5715     5695      -20     
==========================================
- Hits         5509     5487      -22     
- Misses        206      208       +2     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

seanthegeek and others added 2 commits May 20, 2026 19:50
Trailing blank line at end of file. The previous commit ran
ruff check but missed ruff format.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@seanthegeek seanthegeek changed the title Remove coverage-chasing tests; fix BIMI x/y attribute check Release 5.16.2: BIMI x/y attribute fix; narrower dnssec exceptions May 20, 2026
seanthegeek and others added 2 commits May 20, 2026 19:59
The narrowed exception handler in test_dnssec catches the actual
exception types dnspython raises. Exercise it with a real
dns.dnssec.ValidationFailure (the exception thrown on a genuine bad
signature) and assert on the function's contract: invalid DNSSEC
reports as not validated, rather than propagating.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
dns.dnssec re-imports it from dns.exception but doesn't re-export it,
so pyright flags the dns.dnssec.ValidationFailure reference as a
reportPrivateImportUsage error. Use the canonical path.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@seanthegeek seanthegeek merged commit e41b5bc into main May 21, 2026
3 checks passed
@seanthegeek seanthegeek deleted the honest-test-coverage branch May 21, 2026 00:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant