Skip to content

Fix non-converging indentation of adjacent string constants - #411

Merged
darold merged 2 commits into
darold:masterfrom
edjubert:fix/adjacent-constants-idempotency
Sep 19, 2026
Merged

darold merged 2 commits into
darold:masterfrom
edjubert:fix/adjacent-constants-idempotency

Conversation

@edjubert

Copy link
Copy Markdown
Contributor

What

Fix: adjacent string constants separated by a newline (implicit concatenation) gained one space of indentation on every run, so the output never converged.

Fixes #410.

Before

Formatting the same input three times in a row:

SELECT
    concat('a', x, ' AU '
 'b') AS lib
FROM
    t;
SELECT
    concat('a', x, ' AU '
  'b') AS lib
FROM
    t;
SELECT
    concat('a', x, ' AU '
   'b') AS lib
FROM
    t;

After

SELECT
    concat('a', x, ' AU '
        'b') AS lib
FROM
    t;

Stable from the first run, and the continuation line now follows the current indentation instead of staying at column 1.

Why

tokenize_sql keeps consecutive constants in a single token and preserves the whitespace between them verbatim (the rule commented preserved multiple constants with newline). _add_token then prefixed $sp to every constant of that token without normalizing the whitespace already present. $sp is a single space when the token is not at the start of a line, so each run appended one more space to what the previous run had produced.

The fix rebuilds the separators rather than prefixing them: a single space when the constants are on the same line, the current indentation after a newline. That makes the result independent of the input spacing, which is what idempotency requires.

Tests

  • t/test-files/ex85.sql + t/test-files/expected/ex85.sql cover:
    • two constants on consecutive lines in a select list
    • two constants as the last argument of a function call (the nesting case)
    • three constants chained over three lines
    • two constants in a WHERE comparison
  • Full suite: prove -l t/ passes (93 tests)
  • Output is idempotent, verified over three further runs on the expected file

edjubert and others added 2 commits September 12, 2026 04:32
Adjacent string constants separated by a newline are tokenized as a
single token whose inner whitespace is kept verbatim, then _add_token
prefixes $sp to every constant it holds. Since $sp is a single space
when the token is not at the start of a line, the whitespace already
produced by a previous run was never normalized and one space was
appended on every run, so the output never converged.

Rebuild the separators instead: a single space when the constants are
on the same line, the current indentation after a newline.

Before, formatting three times in a row:
    SELECT concat('a' , x , ' AU '
     'b') AS lib
    SELECT concat('a' , x , ' AU '
      'b') AS lib
    SELECT concat('a' , x , ' AU '
       'b') AS lib

After:
    SELECT
        concat('a', x, ' AU '
            'b') AS lib
@darold
darold merged commit 8a8a5b5 into darold:master Sep 19, 2026
0 of 4 checks passed
@edjubert
edjubert deleted the fix/adjacent-constants-idempotency branch September 19, 2026 18:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Idempotency issue with adjacent string constants

2 participants