feat(hook): Wayland frontmost-window backends (wlroots + GNOME Shell)

[recchia]

Summary

frontmost_bundle_id() is X11-only today, so on a Wayland session it returns
None for native windows and per-app profiles never fire (XWayland windows
aside). This adds two Wayland backends behind the existing selection, keeping
the X11 path as the universal fallback — no behavior change off Wayland, and
macOS/Windows untouched:

• wlroots — zwlr_foreign_toplevel_management_v1 (sway, Hyprland, river, Wayfire)
• GNOME Shell — a minimal, read-only companion extension that exports the
  focused window's WM_CLASS over D-Bus (Mutter offers no protocol/portal for this)

Implements the Wayland half of #95; complements the X11 backend from #122.

Backend selection

detect_session_kind() (XDG_SESSION_TYPE, falling back to
WAYLAND_DISPLAY/DISPLAY) sets the candidate order; the first that
initializes wins:

• Wayland → wlroots → GNOME extension → X11/XWayland
• X11 / unknown → X11

A candidate returns None when it can't initialize (wlr manager absent on
GNOME, extension not installed, …), so unsupported compositors fall through to
X11 exactly as today. Selection is once-per-process; landing on X11 while on
Wayland logs a hint to install the extension.

Compositor coverage

Compositor	Backend	Identifier
wlroots (sway, Hyprland, river, Wayfire)	wlr foreign-toplevel	xdg app_id (org.mozilla.firefox)
GNOME / Mutter	companion extension (D-Bus)	WM_CLASS (org.gnome.Nautilus, firefox_firefox)
KDE/KWin, others	X11/XWayland fallback	X11 WM_CLASS (XWayland windows only)

Files

• src/linux/wlr_foreign_toplevel.rs — binds the foreign-toplevel manager,
  roundtrips per poll, returns the activated toplevel's app_id.
• src/linux/gnome_shell.rs — blocking zbus proxy onto org.openlogi.Frontmost,
  with a per-call timeout so a stalled Shell can't wedge the poll thread.
• gnome-shell-extension/openlogi-frontmost@openlogi.dev/ — the extension. Reads
  only global.display.focus_window.get_wm_class(); no titles, contents, input,
  or UI. Targets GNOME Shell 45–50.
• src/linux.rs — dispatch refactored to a FrontmostSource trait + ordered
  candidate list. New deps (wayland-client, wayland-protocols-wlr, zbus)
  under the existing cfg(target_os = "linux") target.

Identifier semantics — a design call I'd like your read on

GNOME and X11 both return WM_CLASS, so a profile created on X11 carries over to
GNOME/Wayland unchanged. wlroots returns the native xdg app_id, a different
namespace — and since profile lookup is an exact match, a profile created under
wlroots won't match one created under GNOME/X11. I return each compositor's
native identifier rather than a lossy WM_CLASS guess (stripping reverse-DNS and
re-capitalizing is wrong for many apps); reconciling the namespaces belongs in a
single normalization layer over frontmost_bundle_id(), which I left out to keep
this PR focused. Happy to add that pass, or to standardize on one identifier
across all Linux backends — which do you prefer?

Testing

Validated end-to-end on Ubuntu 26.04, GNOME Shell 50.1, Wayland, rustc 1.96:

• Extension State: ACTIVE; gdbus call … GetFocusedWmClass returns and tracks
  the focused window's WM_CLASS.
• cargo run --example frontmost_app -p openlogi-hook follows focus live across
  native-Wayland apps (Ptyxis → org.gnome.Ptyxis, Nautilus → org.gnome.Nautilus)
  and Firefox (firefox_firefox) — windows the X11 backend reports as None.

Not yet hardware-tested: the wlroots backend. It compiles and follows the
protocol spec, but I don't run a wlroots compositor — a sanity check from a
sway/Hyprland user would be welcome, or I can spin one up before merge.

Install (GNOME)

UUID=openlogi-frontmost@openlogi.dev
DEST="$HOME/.local/share/gnome-shell/extensions/$UUID"
mkdir -p "$DEST"
cp crates/openlogi-hook/gnome-shell-extension/$UUID/{metadata.json,extension.js} "$DEST"/
# Wayland can't hot-reload the shell — log out/in, then:
gnome-extensions enable "$UUID"

Open questions

1. Extension UUID / D-Bus name use openlogi.* as placeholders — what namespace
   do you want? (constants mirrored in gnome_shell.rs.)
2. Extension distribution: bundle-and-document (current), ship to
   extensions.gnome.org, or auto-install from the app?

Checklist

• Linux-only (cfg(target_os = "linux")); macOS/Windows untouched.
• Falls through to X11 when no Wayland backend initializes.
• GNOME backend validated on real hardware (Ubuntu 26.04 / GNOME 50.1 / Wayland).
• wlroots backend validated on a wlroots compositor (help wanted).

[greptile-apps]

Greptile Summary

This PR adds two Wayland frontmost-window backends (wlr_foreign_toplevel for sway/Hyprland/river and a GNOME Shell D-Bus extension for Mutter), refactoring the existing X11 path into a FrontmostSource trait + ordered candidate list so Wayland-native apps can drive per-app profile switching for the first time.

• wlr_foreign_toplevel.rs: event-driven protocol bridged to a synchronous poll via drain_events (25 ms cap) and timed_roundtrip (5 s deadline each) for init; compositor Finished events trigger an automatic reconnect on the next poll.
• gnome_shell.rs: blocking zbus proxy with a 5 s METHOD_TIMEOUT on the connection, and a probe call in connect() so the backend is only selected when the companion extension is actually installed.
• linux.rs: detect_session_kind() + candidate list determines backend order; X11/XWayland is always appended as the universal fallback.

Confidence Score: 5/5

Safe to merge; all changes are Linux-only and the X11 path is untouched, so macOS/Windows and existing X11 users are unaffected.

The new backends are well-isolated behind the FrontmostSource trait, each candidate fails gracefully rather than panicking, timeouts are consistently applied, and the wlr reconnect logic handles the common compositor-reload case. The one noted concern — reconnect holding the session mutex for up to 10 s — is bounded and affects only edge-case concurrent callers, not the nominal single-threaded poll loop.

The reconnect path in wlr_foreign_toplevel.rs (frontmost_bundle_id lines 420–421) is worth a second look before merge; all other files are straightforward.

Important Files Changed

Filename	Overview
crates/openlogi-hook/src/linux/wlr_foreign_toplevel.rs	New wlroots foreign-toplevel backend; well-structured with timed round-trips and reconnect logic. One concern: Session::open() is called while the session mutex is held, potentially blocking concurrent callers for up to 10 s on compositor reload.
crates/openlogi-hook/src/linux/gnome_shell.rs	New GNOME Shell D-Bus backend; correctly applies METHOD_TIMEOUT to the connection so all proxy calls are time-bounded, and uses a probe call in connect() to verify the extension is present before selecting the backend.
crates/openlogi-hook/src/linux.rs	Dispatch layer refactored to FrontmostSource trait + candidate list; session detection and fallback logic are clean. Debug hint for X11 fallback on Wayland omits KDE users.
crates/openlogi-hook/gnome-shell-extension/openlogi-frontmost@openlogi.dev/extension.js	Minimal GNOME Shell extension; correctly exports GetFocusedWmClass over the session bus, guards against no focused window, and cleans up properly in disable().
crates/openlogi-hook/Cargo.toml	Adds wayland-client, wayland-protocols-wlr, and zbus under the linux-only target guard; no cross-platform impact.

Sequence Diagram

sequenceDiagram
    participant P as Process (LazyLock init)
    participant D as detect_frontmost_source()
    participant W as wlr_foreign_toplevel::candidate
    participant G as gnome_shell::candidate
    participant X as x11_candidate

    P->>D: first access to FRONTMOST_SOURCE
    D->>D: detect_session_kind()
    alt Wayland session
        D->>W: candidate()
        W->>W: Session::open() [timed_roundtrip x2, up to 10s]
        alt wlr protocol advertised
            W-->>D: Some(WlrForeignToplevelSource)
            D-->>P: wlr backend selected
        else not advertised
            W-->>D: None
            D->>G: candidate()
            G->>G: connect() + probe call [5s timeout]
            alt extension reachable
                G-->>D: Some(GnomeShellSource)
                D-->>P: gnome-shell backend selected
            else absent
                G-->>D: None
                D->>X: x11_candidate()
                X-->>D: Some/None
                D-->>P: x11 or NullSource
            end
        end
    else X11 / Unknown session
        D->>X: x11_candidate()
        X-->>D: Some/None
        D-->>P: x11 or NullSource
    end

    loop ~1 Hz polling
        P->>+D: frontmost_bundle_id()
        D->>D: FRONTMOST_SOURCE.frontmost_bundle_id()
        note over D: wlr: drain_events (25ms cap), gnome: D-Bus call (5s timeout), x11: X property read
        D-->>-P: "Option<String>"
    end

Loading

_{Reviews (3): Last reviewed commit: "fix(hook): bound wlr frontmost dispatch ..." | Re-trigger Greptile}

[recchia]

Cross-referencing #173/#179: once that packaging lands, open question 2 here (extension distribution) has a natural answer — ship openlogi-frontmost@openlogi.dev/ as an nfpm contents: entry (system-wide path /usr/share/gnome-shell/extensions//) plus an install.sh step. Users would still need gnome-extensions enable + a session restart, but it removes the manual copy. Happy to add that as a follow-up once both PRs are in, whichever merges first.