Skip to content

win32: render the IME preedit inline in the terminal grid - #5

Closed
nanasess wants to merge 1 commit into
Codavo:mainfrom
nanasess:win32-ime-inline-preedit
Closed

nanasess wants to merge 1 commit into
Codavo:mainfrom
nanasess:win32-ime-inline-preedit

Conversation

@nanasess

@nanasess nanasess commented Sep 2, 2026

Copy link
Copy Markdown

Problem

The win32 apprt only tells the IME where to put its composition window: WM_IME_STARTCOMPOSITION calls ImmSetCompositionWindow (CFS_POINT) and ImmSetCompositionFontW, then hands the message to DefWindowProcW. Nothing else about composition is handled. That gives three problems for CJK input:

  1. The composition string is drawn by the IME's own window, floating on top of the terminal, instead of inline in the grid where the rest of the line is.
  2. The candidate window is positioned once, when the composition starts, and does not follow the caret as the composition grows or shrinks.
  3. Committed text arrives through DefWindowProcW → WM_IME_CHAR → WM_CHAR → handleTextInput, which puts getModifiers() on the key event. A commit key pressed while Ctrl is held is therefore re-encoded as a fixterms CSI-u sequence. 1e960ee (win32: avoid csi-u for ime text commits) works around that downstream by dropping the modifiers; this change removes the cause.

The core already has the wiring for all of this — Surface.preeditCallback and Surface.keyCallback, both used by the GTK and embedded (macOS) apprts — and the renderer already draws renderer_state.preedit. It was simply never called from src/apprt/win32/: grep -rn 'preeditCallback' src/apprt/win32/ returns nothing.

Change

Handle the IMM32 messages directly and connect them to the core. The core is untouched; the diff is confined to src/apprt/win32/App.zig and src/apprt/win32/Surface.zig (which gains im_composing and ime_char_pending).

Message Handling
WM_IME_STARTCOMPOSITION Don't call DefWindowProcW, so the IME's own composition window never appears. Update composition and candidate window positions.
WM_IME_COMPOSITION GCS_COMPSTR → preeditCallback (renders in the grid); the candidate window is repositioned on every update. GCS_RESULTSTR → keyCallback with .utf8. Returns 0 so no WM_IME_CHAR/WM_CHAR is generated.
WM_IME_ENDCOMPOSITION Clear the preedit.
WM_IME_SETCONTEXT Mask off ISC_SHOWUICOMPOSITIONWINDOW — a second line of defence against the IME drawing the composition itself.
WM_KEYDOWN / WM_KEYUP / WM_SYSKEYUP Not forwarded to the core while composing, matching the GTK apprt's im_composing check.
WM_CHAR / WM_IME_CHAR See below.
WM_KILLFOCUS Cancel the composition (NI_COMPOSITIONSTR / CPS_CANCEL) and reset the preedit and composing state.

Why key releases are suppressed too

With the kitty keyboard protocol's report_events enabled (input/key_encode.zig:113), releases are encoded. Suppressing only presses meant the release of a key that had done nothing but feed the IME still reached the pty on its own. GTK suppresses both.

Why WM_CHAR needs its own guard, and why it can't just be dropped

TranslateMessage synthesizes character messages from WM_KEYDOWN before DispatchMessage runs (App.zig:456), so the WM_KEYDOWN guard cannot stop them — a printable key the IME did not consume still reached the pty. (Control characters were already dropped by handleTextInput's codepoint < 0x20 check, so only printable characters were affected.)

Dropping WM_CHAR unconditionally while composing breaks a different path: an IME that commits through WM_IME_CHAR instead of GCS_RESULTSTR relies on DefWindowProcW turning that into WM_CHAR, and WM_IME_CHAR normally arrives during a composition. So WM_IME_CHAR (0x0286) is handled explicitly: it sets ime_char_pending before calling DefWindowProcW, and the WM_CHAR guard only lets a character through while that flag is set. The flag is cleared by its consumer and on both composition end and cancel — but deliberately not when leaving the WM_IME_CHAR arm, because DefWindowProcW is not specified to always emit the WM_CHAR, and clearing early would drop the commit. A flag left set can at worst let one character through during a composition, which is exactly the behaviour before this change, so it fails safe.

Two deliberate choices

  • Commits do not go through textCallback. That path is completeClipboardPaste, so the text would be wrapped in bracketed paste and applications would see it as a paste rather than typing. Like GTK's imCommit, this puts the UTF-8 on an unmodified KeyEvent with an unidentified key.
  • Surface.imePoint() is not used. It divides by the content scale to return logical coordinates for macOS/GTK, whereas IMM32 wants client physical pixels. The existing cell-geometry calculation is reused instead.

Allocation for the commit string

ImmGetCompositionStringW returns a byte length and the string it writes is not NUL terminated. The preedit read is capped at 256 UTF-16 code units, but GCS_RESULTSTR deliberately is not: the allocation is sized from the length IMM32 reports, with a 1 MiB sanity limit against a misbehaving IME, so long commits from speech or handwriting input are not silently lost. An odd reported length is rounded up so the trailing byte stays inside the buffer, and the returned slice is clamped to the allocation in case the second ImmGetCompositionStringW call reports more than the first.

1e960ee's CSI-u guard is kept, as insurance for IMEs that send WM_IME_CHAR directly.

Testing

Built for Windows from WSL with nix develop -c zig build -Dapp-runtime=win32 -Dtarget=x86_64-windows-gnu on this branch (i.e. on top of 215c9673c).

Behaviour was verified on real hardware (Windows 11, ReleaseFast, WSL2 Gentoo session) from the equivalent branch in my fork. That branch differs from this one only by the WM_MOUSEWHEEL change in #4, which is in a different message handler and does not touch any of this code — git diff between the two is exactly that one hunk.

# Case Result
1 Emacs with nskk, Windows IME off No regression — the path that never enters IMM32 behaves exactly as before
2 MS-IME and CorvusSKK composition Renders inline in the grid; the IME's own composition window no longer appears
3 Candidate window Follows the caret as the composition changes
4 Commit Not wrapped in bracketed paste (checked with cat -v and read -r)
5 Esc cancel, and switching tabs mid-composition No composing state left behind; input still works afterwards

Known limitation, not introduced here: the preedit is drawn on the cursor row, so it is not visible if the cursor is scrolled off screen. That is an existing TODO in the core renderer and is untouched by this change.

Relationship to #4

Independent. This branch is based directly on main (215c9673c) and does not include #4. They touch different arms of surfaceDispatch and can be merged in either order.

AI disclosure

Per AI_POLICY.md: the investigation, the patch and the draft of this description were done with Claude Code assisting. I chose the design (in particular: keyCallback over textCallback, cell geometry over imePoint(), and the ime_char_pending handshake instead of an unconditional WM_CHAR drop), reviewed and edited the result, ran the real-hardware verification above myself, and I understand and stand behind the change.

🤖 Generated with Claude Code

The win32 apprt only told the IME where to place its composition window
(WM_IME_STARTCOMPOSITION -> ImmSetCompositionWindow) and left everything
else to DefWindowProcW. As a result:

  - the composition string was drawn by the IME's own window on top of
    the terminal rather than inline in the grid;
  - the candidate window was positioned once, when the composition
    started, and did not follow the caret;
  - committed text arrived via DefWindowProcW -> WM_IME_CHAR -> WM_CHAR
    -> handleTextInput, which puts getModifiers() on the event, so a
    commit key pressed while Ctrl was held got re-encoded as a fixterms
    CSI-u sequence. 1e960ee ("win32: avoid csi-u for ime text commits")
    works around that downstream.

The core already has the wiring for this -- Surface.preeditCallback and
Surface.keyCallback, both used by the GTK and embedded apprts -- and the
renderer already draws renderer_state.preedit. It was simply never
called from src/apprt/win32 (grep for preeditCallback found nothing).
This handles the IMM32 messages directly and connects them.

Message handling:

  - WM_IME_STARTCOMPOSITION: do not call DefWindowProcW, so the IME's
    own composition window never appears. Update the composition and
    candidate window positions.
  - WM_IME_COMPOSITION: GCS_COMPSTR goes to preeditCallback so it
    renders in the grid, and the candidate window is repositioned on
    every update. GCS_RESULTSTR goes to keyCallback with .utf8. Returns
    0 so no WM_IME_CHAR/WM_CHAR is generated.
  - WM_IME_ENDCOMPOSITION: clear the preedit.
  - WM_IME_SETCONTEXT: mask off ISC_SHOWUICOMPOSITIONWINDOW, as a second
    line of defence against the IME drawing the composition itself.
  - WM_KEYDOWN / WM_KEYUP / WM_SYSKEYUP: not forwarded to the core while
    composing, matching the GTK apprt's im_composing check. Releases
    matter as well as presses: with the kitty keyboard protocol's
    report_events enabled (input/key_encode.zig:113) the release of a
    key that only fed the IME would otherwise reach the pty on its own.
  - WM_CHAR: TranslateMessage synthesizes character messages from
    WM_KEYDOWN before DispatchMessage runs (App.zig:456), so the
    WM_KEYDOWN guard cannot stop them and a printable key the IME did
    not consume still reached the pty. They cannot be dropped
    unconditionally either: an IME that commits through WM_IME_CHAR
    instead of GCS_RESULTSTR depends on DefWindowProcW turning that into
    WM_CHAR. WM_IME_CHAR (0x0286) is therefore handled explicitly, which
    sets ime_char_pending before calling DefWindowProcW, and the WM_CHAR
    guard only lets a character through while that flag is set. The flag
    is cleared by its consumer and on both composition end and cancel,
    but not when leaving the WM_IME_CHAR arm, because DefWindowProcW is
    not specified to always emit the WM_CHAR. A flag left set can at
    worst let a single character through during a composition, which is
    the behaviour before this change.
  - WM_KILLFOCUS: cancel the composition (NI_COMPOSITIONSTR /
    CPS_CANCEL) and reset both the preedit and the composing state, so a
    focus change can never leave input wedged.

Two deliberate choices worth recording:

  - Commits do not go through textCallback. That path is
    completeClipboardPaste, so the text would be wrapped in bracketed
    paste. Like GTK's imCommit, this puts the UTF-8 on an unmodified
    KeyEvent with an unidentified key, so committed text is never
    re-encoded as a control sequence.
  - Surface.imePoint() is not used. It divides by the content scale to
    return logical coordinates for macOS/GTK, whereas IMM32 wants client
    physical pixels, so the existing cell geometry calculation is
    reused.

ImmGetCompositionStringW returns a byte length and the string it writes
is not NUL terminated. The preedit read is capped at 256 UTF-16 code
units, but GCS_RESULTSTR is not: the allocation is sized from the length
IMM32 reports, with a 1 MiB sanity limit against a misbehaving IME, so
long commits from speech or handwriting input are not silently lost. An
odd reported length is rounded up so the trailing byte stays inside the
buffer, and the returned slice is clamped to the allocation in case the
second ImmGetCompositionStringW call reports more than the first.

The CSI-u guard from 1e960ee is kept as insurance for IMEs that send
WM_IME_CHAR directly.

Verified on Windows (ReleaseFast):

  - Emacs with nskk and the Windows IME off behaves exactly as before
    (no regression on the path that does not involve IMM32 at all)
  - MS-IME and CorvusSKK compositions render inline in the grid and the
    IME's own composition window no longer appears
  - the candidate window follows the caret as the composition changes
  - committed text is not wrapped in bracketed paste
  - cancelling with Esc, and switching tabs mid-composition, leave no
    composing state behind

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant