Skip to content

Fix error on emphasis around whitespace character reference (#492)#583

Open
apoorvdarshan wants to merge 1 commit into
hukkin:masterfrom
apoorvdarshan:fix/issue-492-emphasis-whitespace-charref
Open

Fix error on emphasis around whitespace character reference (#492)#583
apoorvdarshan wants to merge 1 commit into
hukkin:masterfrom
apoorvdarshan:fix/issue-492-emphasis-whitespace-charref

Conversation

@apoorvdarshan

Copy link
Copy Markdown

Fixes #492

Root cause

For emphasis whose content is a whitespace character reference, e.g.

* *

markdown-it decodes the reference to literal whitespace in the text token's
content (" "), keeping the original source only in the token's markup
field. The em/strong renderers emitted that literal whitespace directly
between the emphasis markers, producing * *. On re-parse, a
Unicode-whitespace flanking character can no longer open or close emphasis
(CommonMark emphasis rules), so the formatted Markdown renders to different
HTML than the input and the AST safety check aborts with an error.

Reproduction

On the pristine master clone (mdformat 1.0.0):

$ printf -- '* *' | mdformat -
Error: Could not format "-".

Formatted Markdown renders to different HTML than input Markdown. This is a bug
in mdformat or one of its installed plugins. ...

Internally (validation disabled) mdformat produced \* *, whereas:

  • input *&#32;* renders to <p><em> </em></p>
  • output \* * / * * renders to <p>* *</p> (no emphasis) — hence the mismatch.

With the fix:

$ printf -- '*&#32;*' | mdformat -
*&#32;*
$ printf -- '*&nbsp;*' | mdformat -
*&#160;*

Both round-trip to the same HTML as the input.

Fix

In em() and strong() (src/mdformat/renderer/_context.py), re-decimalify
leading/trailing Unicode whitespace of the rendered inner text using the
existing decimalify_leading / decimalify_trailing helpers. This mirrors
exactly what paragraph() already does for whitespace at paragraph
boundaries, so boundary whitespace is kept in character-reference form and the
emphasis markers stay functional. The change is localized to the two emphasis
renderers; non-whitespace content is untouched.

Tests

Added two round-trip cases to tests/data/default_style.md (driven by
tests/test_style.py, which runs the real CLI including the AST check):

  • *&#32;**&#32;*
  • **&nbsp;****&#160;**

Confirmed these fail on unmodified code (the CLI exits non-zero via the AST
check) and pass with the fix. The full test suite passes locally
(4286 passed, 1 skipped), including all CommonMark spec fixtures, and
black / isort / flake8 report no issues on the changed file.

Disclosure: prepared with AI assistance; reviewed and verified locally.

Emphasis whose content is a whitespace character reference such as
`*&hukkin#32;*` triggered the AST safety check: markdown-it decodes the
reference to literal whitespace in the text token, and mdformat emitted
that literal whitespace between the emphasis markers. On re-parse, a
Unicode-whitespace flanking character can no longer open/close emphasis,
so the formatted output rendered to different HTML than the input.

Re-decimalify leading/trailing whitespace of em/strong content, mirroring
the existing handling in `paragraph()`, so boundary whitespace stays in
character reference form and the emphasis markers remain functional.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Error on emphasized HTML space entity

1 participant