Hele

Hele — Czech for "look!"

Press a hotkey, drag a box around anything on your screen, ask a question. The answer appears in a bubble floating over the thing you asked about.

Built for two jobs:

• Papers. Select an equation and ask what it means. Answers render Markdown and LaTeX, so you get typeset mathematics instead of backslashes.
• GUI help. Select part of an app and ask "how do I do X here". Hele can draw arrows at the controls it is talking about.

Works with any OpenAI-compatible API (OpenAI, OpenRouter, Ollama, LM Studio, …) or the Claude Code CLI. macOS, Windows and Linux.

---

Use it

1. Press Ctrl+Shift+A — Cmd+Shift+A on macOS.
2. Drag a box.
3. Type a question, or just hit Enter for a plain explanation.

The answer streams in as it is generated. Click the × or press Esc to dismiss it — several bubbles can sit on screen at once, and the app underneath stays fully clickable the whole time.

Esc always works: it backs out of the selector, and it cancels a request that is still running.

Install

Grab a build from Releases, or build it yourself — you need Rust 1.85+ and Node 20+:

npm install
npm run tauri build

macOS also needs Screen Recording permission: System Settings → Privacy & Security → Screen Recording → enable Hele, then restart the app. Hele asks on first launch and tells you if it is missing.

Linux needs the usual Tauri dependencies:

sudo apt install libwebkit2gtk-4.1-dev build-essential curl wget file \
  libxdo-dev libssl-dev libayatana-appindicator3-dev librsvg2-dev \
  libgtk-layer-shell-dev

Set up a model

Hele lives in the menu bar / tray. Open Settings there and pick a backend.

Any OpenAI-compatible API. Set the base URL, model and key — there is no hardcoded provider list:

	Base URL	Key
OpenAI	https://api.openai.com/v1	required
OpenRouter	https://openrouter.ai/api/v1	required
Ollama	http://localhost:11434/v1	leave blank
LM Studio	http://localhost:1234/v1	leave blank

Pick a model that can see images (gpt-4o, llama3.2-vision, …).

Claude Code CLI. Select it and you are done — it reuses whatever login the claude command already has, so there is no key to enter. Slower to produce the first token, since the CLI has to start up and read the images.

Your API key goes in the OS keychain, never into a config file.

Privacy

By default Hele sends two images: your selection at full resolution, and the whole screen with a red box drawn around your selection, so the model can see the surrounding context.

Turn "Send the full screenshot" off and only the pixels inside your selection ever leave the machine.

Screenshots are never written to disk. The one exception is the Claude Code backend, which has to save them for the CLI to read; that folder is deleted as soon as the query ends.

Arrows that point at things

Turn on "Ask the model to point at things" and Hele asks for screen coordinates alongside the answer, then draws arrows to them — useful for "which button do I press".

Models don't always follow the format, so anything unexpected falls back to a normal bubble. It is safe to leave on.

Platform support

	Capture	Hotkey	Overlay
macOS	✅	✅	✅
Windows	✅	✅	✅
Linux / X11	✅	✅	✅
Linux / Hyprland, sway, KDE	✅ portal	✅ portal	✅ layer-shell
Linux / GNOME Wayland	✅	❌ no portal	⚠️ below fullscreen windows

GNOME does not implement the global-shortcuts portal, so the hotkey won't register there. Use the tray menu, or bind hele --capture to a key in GNOME's own keyboard settings.

Transparent windows misbehave on some Linux driver combinations. To check yours in a few seconds:

hele --overlay-test

If you can read the test bubble over your desktop with no grey box around it, you're fine.

Command line

Flag
--capture	Start a selection right away
--overlay-test	Draw a test bubble and arrow on every display
--settings	Open settings

Config lives at ~/.config/hele/config.toml on Linux, ~/Library/Application Support/dev.hele.Hele/ on macOS, and %APPDATA%\hele\Hele\config\ on Windows. Settings shows you the exact path.

Development

npm run tauri dev                                  # run it
npm test                                           # frontend tests
cargo test --manifest-path src-tauri/Cargo.toml    # backend tests

On macOS, run Hele from the .app bundle. The bare binary in target/debug/ starts up but every WebKit process dies immediately, leaving invisible empty windows. tauri dev and tauri build both produce a bundle.

Tests that need real hardware and a real claude binary are skipped by default:

cargo test --manifest-path src-tauri/Cargo.toml --test live -- --ignored

Two things to know before changing anything.

Annotation anchors are stored in physical pixels of the screenshot and converted to CSS pixels only at render time, per monitor. Retina and Windows display scaling both mean physical ≠ logical, and mixing them is how you get a bubble 200px off on the external display. It all lives in coords.rs.

The overlay renders a list of primitives, not "an answer":

type Annotation =
  | { type: "bubble"; anchor: Rect; markdown: string; streaming: boolean }
  | { type: "highlight"; anchor: Rect }
  | { type: "arrow"; from: Point; to: Point; label?: string }

New kinds of annotation are new emitters, not a new renderer.

src-tauri/src/
  coords.rs       physical / logical / per-monitor conversions
  capture.rs      screenshots, cropping, the red marker box, permissions
  windows.rs      overlay, selector and settings windows; gtk-layer-shell
  clickthrough.rs cursor polling and the click-through toggle
  ask.rs          capture → request → stream → annotate
  provider/       the Provider trait and its two backends
  grounded.rs     the streaming JSON-envelope parser
  portal.rs       the XDG GlobalShortcuts portal
src/
  overlay/        the annotation renderer
  selector/       region selector and question box
  shared/         markdown + KaTeX + highlighting, placement, types

Not doing (yet)

No follow-up questions, no OCR, no bundled local models, no auto-updater. The bubble does not follow a window you move.

License

MIT