make D1 splitter.ts use sqlite's parser logic by alsuren · Pull Request #15712 · cloudflare/workers-sdk

alsuren · 2026-09-18T16:44:05Z

Fixes #15228 and a bunch of others

There are a bunch of MRs (#15234 #15226 and #15163) that are trying to fix minor bugs in our sqlite parser in splitter.ts.

The fundamental problem with that parser is that it's derived from a generic sql parser that is primarily a PostgreSQL parser rather than a sqlite parser.

I wrote on one of them what the solution is: #15163 (comment)

If we are in the business of writing sql parsers, I think we would be better off faithfully porting the logic from https://github.com/sqlite/sqlite/blob/master/src/complete.c rather than organically trying to recreate it one edge case at a time (maybe translate the state machine to use strings rather than integers for the states?).

That way, each time we get a bug report about our parser, we can point an LLM at the upstream sqlite source code and ask "is this a faithful translation from c to typescript?".

(an LLM based review pointed out that we don't handle EXPLAIN correctly and we don't handle [..] style quotes. I understand that neither of these are very common, and neither are regressions introduced in your MR)

The other advantage would be that workerd already carries a patch to make complete.c expose a sqlite_complete_length() function, which we use to split statements (for use with wrangler d1 execute --remote --file f.sql). If the implementations of these statement splitters are are as similar as possible then we will have fewer discrepancies there too.

The existing code also some funkiness around removing comments, but I know that it is due to this bug: #7739 . We don't need to remove all comments to work around that workerd bug: we can just remove fragments that only contain whitespace and comments.

I asked my agent to include the regression tests from #15234 #15226 and #15163 in our test suite, to make sure their edge-cases are covered.

Tests
- Tests included/updated
- Automated tests not possible - manual testing has been completed as follows:
- Additional testing not necessary because:
Public documentation
- Cloudflare docs PR(s):
- Documentation not necessary because: trivial bugfixes

A picture of a cute animal (not mandatory, but encouraged)

changeset-bot · 2026-09-18T16:44:10Z

🦋 Changeset detected

Latest commit: ffbcdb8

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 3 packages

Name	Type
wrangler	Patch
@cloudflare/vite-plugin	Patch
@cloudflare/vitest-plugin	Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

pkg-pr-new · 2026-09-18T16:47:53Z

@cloudflare/autoconfig

npm i https://pkg.pr.new/@cloudflare/autoconfig@15712

@cloudflare/build-output-utils

npm i https://pkg.pr.new/@cloudflare/build-output-utils@15712

@cloudflare/codemods

npm i https://pkg.pr.new/@cloudflare/codemods@15712

@cloudflare/config

npm i https://pkg.pr.new/@cloudflare/config@15712

@cloudflare/containers-shared

npm i https://pkg.pr.new/@cloudflare/containers-shared@15712

create-cloudflare

npm i https://pkg.pr.new/create-cloudflare@15712

@cloudflare/deploy-helpers

npm i https://pkg.pr.new/@cloudflare/deploy-helpers@15712

@cloudflare/kv-asset-handler

npm i https://pkg.pr.new/@cloudflare/kv-asset-handler@15712

miniflare

npm i https://pkg.pr.new/miniflare@15712

@cloudflare/pages-functions

npm i https://pkg.pr.new/@cloudflare/pages-functions@15712

@cloudflare/pages-shared

npm i https://pkg.pr.new/@cloudflare/pages-shared@15712

@cloudflare/unenv-preset

npm i https://pkg.pr.new/@cloudflare/unenv-preset@15712

@cloudflare/vite-plugin

npm i https://pkg.pr.new/@cloudflare/vite-plugin@15712

@cloudflare/vitest-plugin

npm i https://pkg.pr.new/@cloudflare/vitest-plugin@15712

@cloudflare/workers-auth

npm i https://pkg.pr.new/@cloudflare/workers-auth@15712

@cloudflare/workers-editor-shared

npm i https://pkg.pr.new/@cloudflare/workers-editor-shared@15712

@cloudflare/workers-utils

npm i https://pkg.pr.new/@cloudflare/workers-utils@15712

wrangler

npm i https://pkg.pr.new/wrangler@15712

commit: ffbcdb8

ask-bonk · 2026-09-18T16:54:31Z

P1: Unterminated quoted SQL fragments are silently discarded.

hasUsefulToken remains false when scanning fails at an opening [ or quote. Therefore SELECT 1; [unterminated and SELECT 1; 'unterminated execute only SELECT 1 locally and report success. Preserve those fragments so SQLite reports the invalid statement.

Bracketed identifier:

				return { end: undefined, hasUsefulToken: true };

Quoted value or identifier:

				return { end: undefined, hasUsefulToken: true };

ask-bonk · 2026-09-18T16:54:45Z

I'm Bonk, and I've done a quick review of your PR.

PR #15712 replaces D1 SQL splitting with SQLite's statement-completion state machine.

P1: Unterminated quoted SQL fragments can be silently discarded after a valid statement, allowing partial local execution. Suggested fixes posted: make D1 splitter.ts use sqlite's parser logic #15712 (comment)

github run

workers-devprod · 2026-09-20T16:35:56Z

Codeowners approval required for this PR:

✅ @cloudflare/d1
✅ @cloudflare/wrangler

Show detailed file reviewers

devin-ai-integration

Devin Review found 1 potential issue.

cjol

You’re right, this seems a much better fix than the piecemeal PRs we’ve received. I’ll hold off on the others in favour of this?

workers-devprod

Codeowners reviews satisfied

## pnpm-workspace.yaml (default) ## Dependency Updates | Package | From | To | Type | | --- | --- | --- | --- | | `wrangler` | 4.136.2 | 4.138.0 | minor | ## Release Notes <details> <summary>wrangler (4.136.2 → 4.138.0) — 3 releases</summary> <details> <summary>4.138.0</summary> ### Minor Changes - [#15776](cloudflare/workers-sdk#15776) [`b03f960`](cloudflare/workers-sdk@b03f960) Thanks [@edevil](https://github.com/edevil)! - Add event-code support to temporary Worker deployments Use `wrangler deploy --temporary --event-code <code>` to provision an account for an event. Wrangler requires explicit server acknowledgement before caching the account and keeps the event code out of its cache and telemetry. - [#15817](cloudflare/workers-sdk#15817) [`6e77c53`](cloudflare/workers-sdk@6e77c53) Thanks [@jamesopstad](https://github.com/jamesopstad)! - Allow framework commands to produce Preview Build Output with the experimental config When `cf previews deploy` invokes a framework build command, Preview intent is now preserved. Function-based `cloudflare.config.ts` files receive `isPreview: true`, and generated Build Output is marked as a Preview build. ### Patch Changes - [#15806](cloudflare/workers-sdk#15806) [`8fade73`](cloudflare/workers-sdk@8fade73) Thanks [@NuroDev](https://github.com/NuroDev)! - Standardize Zod validation error output Format validation errors with Zod's built-in `prettifyError()` helper so Miniflare, Wrangler, the Vite plugin, and the Vitest plugin show consistent messages and property paths. - Updated dependencies [[`a71237a`](cloudflare/workers-sdk@a71237a), [`8fade73`](cloudflare/workers-sdk@8fade73)]: - miniflare@5.20260921.1-alpha </details> <details> <summary>4.137.0</summary> ### Minor Changes - [#15778](cloudflare/workers-sdk#15778) [`cd7508c`](cloudflare/workers-sdk@cd7508c) Thanks [@jamesopstad](https://github.com/jamesopstad)! - Generate types during development and supported builds with Vite's `experimental.newConfig` option or Wrangler's `--experimental-new-config` flag (and `--experimental-cf-build-output` for builds) When Wrangler's `--experimental-new-config` flag or Vite's `experimental.newConfig` option is enabled, inferred configuration and runtime declarations are now kept in `.cloudflare/types/index.d.ts`. Vite refreshes them during development and production builds. Wrangler refreshes them during development and when building with both `--experimental-new-config` and `--experimental-cf-build-output`. In the experimental `wrangler.config.ts` format, the `types` option is now top-level because it applies to both commands. ### Patch Changes - [#15765](cloudflare/workers-sdk#15765) [`1bdb96d`](cloudflare/workers-sdk@1bdb96d) Thanks [@th0m](https://github.com/th0m)! - Prepare the required egress sidecar for local Containers without configured images Wrangler dev and Vite dev/preview now pull the required sidecar for Durable Object-managed Containers that select their application image at start time. Previously, these Containers failed to start unless the sidecar image was already cached in Docker. - [#15712](cloudflare/workers-sdk#15712) [`f5605f5`](cloudflare/workers-sdk@f5605f5) Thanks [@alsuren](https://github.com/alsuren)! - Match D1 SQL statement splitting to the local SQLite runtime Wrangler now uses SQLite's statement-completion state machine when splitting D1 SQL files. This keeps trigger, quoted identifier, comment, and keyword handling consistent with local exe …[full notes](https://github.com/cloudflare/workers-sdk/releases/tag/wrangler%404.137.0) </details> …and 1 more release(s) not shown </details> --- *This PR was auto-generated by [catalog-update-action](https://github.com/brandhaug/catalog-update-action).* Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>

fix(wrangler): match SQLite SQL statement splitting

450cdb5

workers-devprod added this to workers-sdk Sep 18, 2026

github-project-automation Bot moved this to Untriaged in workers-sdk Sep 18, 2026

alsuren added 2 commits September 18, 2026 18:09

fix(wrangler): omit comment-only SQL fragments

39d3de2

test(wrangler): cover SQLite splitter regressions

ffbcdb8

alsuren force-pushed the dlaban/15228-make-splitter-use-sqlite-parser-logic branch from fe3fc29 to ffbcdb8 Compare September 18, 2026 17:10

alsuren marked this pull request as ready for review September 20, 2026 16:35

alsuren requested a review from workers-devprod as a code owner September 20, 2026 16:35

workers-devprod requested review from a team and cjol and removed request for a team September 20, 2026 16:35

devin-ai-integration Bot reviewed Sep 20, 2026

View reviewed changes

Comment thread packages/wrangler/src/d1/splitter.ts

cjol approved these changes Sep 21, 2026

View reviewed changes

cjol mentioned this pull request Sep 21, 2026

[wrangler] Recognise compound statement markers without surrounding whitespace #15226

Closed

5 tasks

lambrospetrou approved these changes Sep 23, 2026

View reviewed changes

workers-devprod approved these changes Sep 23, 2026

View reviewed changes

alsuren merged commit f5605f5 into main Sep 23, 2026
76 of 81 checks passed

alsuren deleted the dlaban/15228-make-splitter-use-sqlite-parser-logic branch September 23, 2026 09:21

github-project-automation Bot moved this from Untriaged to Done in workers-sdk Sep 23, 2026

workers-devprod mentioned this pull request Sep 23, 2026

Version Packages #15780

Merged

brandhaug mentioned this pull request Sep 27, 2026

chore(deps): bump wrangler from 4.136.2 to 4.138.0 brandhaug/b2b-saas-starter#511

Merged

Navigation Menu

make D1 splitter.ts use sqlite's parser logic - #15712

make D1 splitter.ts use sqlite's parser logic#15712
alsuren merged 3 commits into
mainfrom
dlaban/15228-make-splitter-use-sqlite-parser-logic

alsuren commented Sep 18, 2026 •

edited

Loading

Uh oh!

changeset-bot Bot commented Sep 18, 2026 •

edited

Loading

Uh oh!

pkg-pr-new Bot commented Sep 18, 2026 •

edited

Loading

Uh oh!

ask-bonk Bot commented Sep 18, 2026

Uh oh!

ask-bonk Bot commented Sep 18, 2026

Uh oh!

workers-devprod commented Sep 20, 2026 •

edited

Loading

Uh oh!

devin-ai-integration Bot left a comment

Uh oh!

Uh oh!

cjol left a comment

Uh oh!

workers-devprod left a comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Conversation

alsuren commented Sep 18, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

changeset-bot Bot commented Sep 18, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🦋 Changeset detected

Uh oh!

pkg-pr-new Bot commented Sep 18, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ask-bonk Bot commented Sep 18, 2026

Uh oh!

ask-bonk Bot commented Sep 18, 2026

Uh oh!

workers-devprod commented Sep 20, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

devin-ai-integration Bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

cjol left a comment

Choose a reason for hiding this comment

Uh oh!

workers-devprod left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

alsuren commented Sep 18, 2026 •

edited

Loading

changeset-bot Bot commented Sep 18, 2026 •

edited

Loading

pkg-pr-new Bot commented Sep 18, 2026 •

edited

Loading

workers-devprod commented Sep 20, 2026 •

edited

Loading