Skip to content

feat(vscode): derive the Vue TextMate grammars with Monogram#6085

Draft
johnsoncodehk wants to merge 14 commits into
masterfrom
monogram
Draft

feat(vscode): derive the Vue TextMate grammars with Monogram#6085
johnsoncodehk wants to merge 14 commits into
masterfrom
monogram

Conversation

@johnsoncodehk

@johnsoncodehk johnsoncodehk commented Jun 2, 2026

Copy link
Copy Markdown
Member

Draft / proposal — for evaluation. Replaces the hand-written Vue TextMate grammars and the VS Code language configuration with output generated by Monogram, pinned here as a github dependency.

Architecture

  • Monogram is a pinned github devDependency ("monogram": "github:johnsoncodehk/monogram#<commit>" in extensions/vscode/package.json). Its engine (gen-tm, gen-vscode-config) and the reused HTML base (html.ts) come from the dependency; the Vue grammar definition lives in this repo as vue.monogram.ts.
  • scripts/generate-grammar.ts produces, from that one source, the three TextMate grammars (vue.tmLanguage.json + the vue.directives / vue.interpolations injections) and the language configuration (languages/vue-language-configuration.json). It needs only Monogram's own source — no other deps.
  • npm run gen:grammar regenerates + formats; a grammar.spec.ts guard fails if the committed output drifts from vue.monogram.ts + the pinned commit. (Generation runs via node --import scripts/strip-node-modules-types.mjs, a small load hook, because Node won't type-strip .ts under node_modules; the tests run under vitest, which handles them directly.)

Language configuration is now generated

Monogram gained markup language-config support: for a grammar with markup, generateLanguageConfig derives the tag-aware onEnterRules / indentationRules, region folding from the comment delimiters, comment + embedded-language brackets, and the word pattern — all from the markup data (void / raw-text tag sets, comment delimiters, embed scopes), gated so the token-stream grammars (TS / JS / YAML) are unaffected and the agnostic gate still passes. The result is at parity with the hand-written vue-language-configuration.json.

Test coverage

Monogram's Vue correctness gates move here, run against the generated grammar (extensions/vscode/tests/vue-highlight.spec.ts, on the same vscode-tmlanguage-snapshot engine + the real VS Code embedded grammars): directive / {{ }} injection, embed boundaries (#1666 </script>, #5012 as-cast, #3999 multi-line start tag), interpolation expression scoping, <style lang="X"> dialect embedding, and the reported-issue regression corpus below. npm run test:grammar runs these gates alongside the snapshot, so bumping the pinned monogram commit and breaking tokenization fails the sync workflow.

Reported-issue coverage

The regression corpus verifies the generated grammar against issues filed on the hand-written Vue grammar — it improves on 5 and matches on 18.

issue generated hand-written
#6007/#2096/#520as type assertion in directive value ·
#2060-inline — const a = 1;</script> (content on the close line) embeds + clean close ·
#2060-inline-adjacent — an unterminated union before a same-line </script>, then a second <script setup> block ·
#5660as const cast in a v-for value ·
#4716/#5571as cast followed by another attribute ·
… and 18 more both the generated and hand-written grammar handle (✓ / ✓)
issue generated hand-written
#3400instanceof in {{ }}
#5370typeof x !== in v-if
#5118?. / ?? in {{ }}
#1675 — arrow => in {{ }}
#6039/#4741< operator in {{ }} (not a tag!)
#5722 — negated ternary + quotes in {{ }}
#5538/#2060 — trailing export type before </script>
#3999 — a force-wrapped multi-line <script lang="ts"> start tag keeps the body as the ts family
#4769 — tag name starting with template
#5701{{ inside a <script> string
#6070 — capitalized component then a <style> block
#4291<script lang="tsx"> body embeds the declared source.tsx
#4291-jsx — <script lang="jsx"> body embeds the declared source.js.jsx
generic="T" — generic="T extends U"> type-param list embeds as TS
#4410 — dynamic directive argument :[attr]
#3727.prop modifier shorthand
#2666 — dynamic slot name from a template literal
#2560/#1290type as a v-for loop variable

Keeping in sync with Monogram

.github/workflows/sync-grammar.yml (daily + workflow_dispatch) bumps the pinned monogram commit in package.json to monogram master, runs pnpm install, regenerates, and commits any change — a reviewable, reproducible pin (recorded in pnpm-lock.yaml), replacing the old curl of pre-built JSON.

Behaviour

Regenerated from monogram master: the SFC raw-text close rules are restructured and the block close tags lowercased. The only tokenization change in the snapshot is template-in-template.vue, where a plain </template> close no longer leaks text.pug and now matches every other close tag. The grammar snapshot and the language-config drift guard are the best place to review the concrete impact.

…guage.json

Pure rename + package.json path repoint; the file contents are unchanged from the
hand-written grammars. Split out from the content swap so the rename is tracked
cleanly in history (the next commit replaces the contents with Monogram's output).
Replace the hand-written Vue syntax grammars with ones derived by Monogram from a
single proven Vue/HTML grammar. The main SFC grammar and the two injections
(directives + interpolations) are now generated output.

The injections keep the same scopeNames (vue.directives / vue.interpolations),
selectors, and injectTo set, and remain thin stubs that include
text.html.vue#vue-directives / #vue-interpolations (the rule bodies live in the main
grammar's repository) — the same topology as before.

The <template> body now embeds text.html.derivative (the embedded-HTML-fragment scope
VS Code's HTML extension provides), which is what the interpolation injection targets.

Notable behaviour change: an `as` type assertion in a directive value
(#5012 / #6007 / #2096 / #520) no longer eats the closing quote — the derived grammar
bounds the value with a capture-embed, so `:msg="msg as string"` recovers correctly
after the cast instead of leaking TypeScript across the rest of the tag.

Grammar snapshot regenerated (extensions/vscode/tests/grammar.spec.ts).
…ammar snapshots

Load the real text.html.derivative, source.tsx and source.js.jsx grammars
in the embedded grammar snapshot harness so that the <template> body and
<script lang="tsx|jsx"> paths are actually exercised. Previously only
text.html.basic and plain TS/JS were loaded, so the begin/while SFC block
rules silently fell back to #tag and these paths went untested.

Add fixtures for interpolation inside an element, text-only interpolation,
and JSX/TSX script blocks.
Pin down that static style="..." / style='...' embed source.css and that
bound :style="{ ... }" embeds source.ts, now that the harness loads the real
text.html.derivative + css grammars.
Mirror update-html-data: a scheduled (+ manual) workflow that pulls the
generated vue.tmLanguage.json / vue.directives / vue.interpolations from
johnsoncodehk/monogram, runs the formatter, regenerates the grammar
snapshot, and commits any change.
…Monogram submodule

Vendor johnsoncodehk/monogram as a git submodule and move its vue.ts grammar
definition in as vue.monogram.ts. The Vue TextMate grammars AND the VS Code
language configuration are now generated locally by scripts/generate-grammar.ts
(engine from the submodule; the generation path has no external deps), replacing
the daily curl of pre-built JSON from monogram master.

- `npm run gen:grammar` regenerates and formats the artifacts.
- A grammar.spec.ts guard fails if the committed output drifts from
  vue.monogram.ts + the pinned submodule.
- sync-grammar.yml now bumps the submodule and regenerates.

The language configuration is now derived (Monogram gained markup
language-config support — tag-aware onEnter / indentation, region folding,
brackets), at parity with the hand-written one. The grammar is regenerated from
monogram master: the SFC raw-text close rules are restructured and block close
tags lowercased; the only tokenization change is template-in-template.vue
(a plain </template> close no longer leaks text.pug).
…source

test.yml now checks out the monogram submodule (the grammar tests import it),
and dprint-formats vue.monogram.ts + generate-grammar.ts to the repo style.
Bumps the monogram submodule: generateTmLanguage / generateMarkupInjection now
read grammar.name (its single source of truth) instead of taking it as an
argument. Generated output is unchanged.
monogram is dropping its Vue grammar definition + tests, so its Vue correctness
gates move here, run against the generated grammar:
- directives + {{ }} interpolation injection
- embed boundaries (#1666 </script>, #5012 as-cast, #3999 multi-line start tag)
- interpolation expression scoping (statements suppressed, operators kept, nested re-enters)
- <style lang="X"> dialect embedding at every structural position
- the reported-issue regression corpus (vue-issue-cases)

A tokenize/scopeLookup harness (tests/vueGrammarHarness.ts) runs the generated
grammar + the real VS Code embedded grammars through vscode-tmlanguage-snapshot
(the same engine as grammar.spec.ts). `test:grammar` now also runs these gates,
so a submodule bump that breaks tokenization fails the sync workflow.
The embedded grammars are downloaded (not committed); without awaiting the sync
the harness could read them before grammar.spec.ts downloaded them, racing to
ENOENT in CI. Mirror grammar.spec.ts and await the sync first.
…a submodule

Replaces the extensions/vscode/monogram git submodule with a pinned github
devDependency ("monogram": "github:johnsoncodehk/monogram#<hash>") — simpler to
install (pnpm, no submodules: true in CI, no .gitmodules).

vue.monogram.ts / generate-grammar.ts now import `monogram/…` (the package).
Node refuses to type-strip `.ts` under node_modules, so `gen:grammar` runs via
`node --import scripts/strip-node-modules-types.mjs` (an in-thread load hook that
strips Monogram's `.ts`); vitest handles them already. Generated output is
byte-identical.

sync-grammar.yml now bumps the pinned commit in package.json + pnpm install,
replacing the submodule update.

This comment was marked as low quality.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants