Fix patchedast region for implicitly-concatenated f-strings (fixes #879) - #880
Open
stevemadere wants to merge 1 commit into
Open
stevemadere wants to merge 1 commit into
stevemadere wants to merge 1 commit into
Conversation
`_JoinedStr` builds its children as [start_quote, *FormattedValues,
end_quote] and lets `_handle` consume each by forward search. The
end-quote child therefore matches the first closing quote after the last
interpolation -- which, in an implicit concatenation, closes only the
part that held that interpolation, not the whole concatenation.
The node's region was then understated and, worse, the scanner was left
parked inside source text that had not been recognised as a string
literal. A `#` in a trailing part -- `f"{a} one " f"python-rope#52 two"` -- then
reads to `_good_token` as the start of a real comment, so the following
`)` is skipped as commented-out and `consume` raises
MismatchedTokenError (issue python-rope#879).
The truncation itself predates the `#`: a hash-free concatenation such
as `f"{a} one " f"52 two"` also produced a short region, it just failed
silently because nothing downstream needed to match a token past the
truncation point.
`consume_string` already measures the full concatenation before
`_JoinedStr` rewinds the scanner, so close the gap from its end offset
rather than re-scanning. Plain (non-f) strings were never affected
because `_Str` consumes via that same pattern.
Fixes python-rope#879
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Fixes an undocumented string parsing crash-inducing bug in _PatchingASTWalker._JoinedStr()
_JoinedStrbuilds its children as [start_quote, *FormattedValues, end_quote] and lets_handleconsume each by forward search. The end-quote child therefore matches the first closing quote after the last interpolation -- which, in an implicit concatenation, closes only the part that held that interpolation, not the whole concatenation.The node's region was then understated and, worse, the scanner was left parked inside source text that had not been recognised as a string literal. A
#in a trailing part --f"{a} one " f"#52 two"-- then reads to_good_tokenas the start of a real comment, so the following)is skipped as commented-out andconsumeraises MismatchedTokenError (issue #879).The truncation itself predates the
#: a hash-free concatenation such asf"{a} one " f"52 two"also produced a short region, it just failed silently because nothing downstream needed to match a token past the truncation point.consume_stringalready measures the full concatenation before_JoinedStrrewinds the scanner, so close the gap from its end offset rather than re-scanning. Plain (non-f) strings were never affected because_Strconsumes via that same pattern.Fixes #879
Checklist (delete if not relevant):