Skip to content

Configuration reference, server deployment, and twcore upgrade - #183

Merged
fylorn merged 5 commits into
mainfrom
feat/config-manual-upgrade
Sep 24, 2026
Merged

fylorn merged 5 commits into
mainfrom
feat/config-manual-upgrade

Conversation

@fylorn

@fylorn fylorn commented Sep 24, 2026 •

Copy link
Copy Markdown
Contributor

Phase P6 of the remote-core plan: the configuration manual, what it takes to run core on a Linux server, and a way to upgrade it there.

Configuration reference (docs/config.md, docs/config.zh-CN.md)

Every section and field of config.yaml, with defaults, allowed values and examples, plus how a change takes effect: the reload stages, a rejected version keeping the previous one in service, history and rollback, twcore check / config show|set|history|rollback, and the environment variables core reads.

The field tables and built-in rule lists are generated and checked. Everything between <!-- generated: … --> and <!-- /generated --> is rendered from crates/tw-config/tests/manual/schema.rs; cargo test -p tw-config --test manual fails when manual and code disagree. The declaration is verified against the code rather than trusted:

  • field names come from serde (a probe Deserializer records the names a derived Deserialize asks for), so a new field without a row fails with its name;
  • a declared default is written into a minimal section and must parse to the same value as omitting it; a required field must fail without it;
  • enum values (protocol, mode, group type, …) are read from serde;
  • YAML examples in the manuals are parsed as configuration.

Regenerate: UPDATE_CONFIG_DOCS=1 cargo test -p tw-config --test manual. CONTRIBUTING describes it.

Fields documented ahead of the code

Rebased onto #184: listen.control.key is now checked against ControlListen like every other field.

listen.control.remote {enabled, bind, port, allow_from} (P4) is declared Ty::Pending: documented under listen.control, and the test fails as soon as ControlListen reads remote, asking for the pending declaration in crates/tw-config/tests/manual/schema.rs to be replaced with checked!(RemoteType, "{}"); its rows and defaults are then verified. Declared defaults to confirm then: enabled: false, bind: all, allow_from: the private ranges, port generated. The server guide assumes twcore init may or may not write a disabled remote section with a random port (it covers both).

Server deployment

  • docs/server.md (+ zh-CN): install → twcore init → the three config edits → systemd → twcore control-key → add the connection in the desktop app; upgrading; uninstalling.
  • packaging/systemd/twcore.service: user thinkwatch, THINKWATCH_HOME=/var/lib/thinkwatch (StateDirectory, 0700), EnvironmentFile=-/etc/thinkwatch/env for ${VAR}, Restart=on-failure, hardening that leaves only the data directory writable (systemd-analyze security: 1.4 OK), AF_NETLINK kept so an interface name in bind resolves.
  • scripts/install.sh (POSIX sh, shellcheck-clean): arch detection, latest or --version, SHA-256 check, runs the binary once, atomic install to /usr/local/bin, user + data dir + env file + unit, twcore init when there is no config, next steps. Idempotent; never starts or restarts the service. --archive FILE installs from a downloaded tarball.

Release assets

release.yml keeps every existing file and adds, for Linux x86_64 and aarch64:

  • twcore-<target>.tar.gz containing twcore-<target>/{twcore, twcore.service, LICENSE}
  • twcore-<target>.tar.gz.sha256

The workflow also unpacks the tarball the way the installer does. Not triggered here; a workflow_dispatch rehearsal would exercise it without publishing.

twcore upgrade

twcore upgrade [--check] [--restart] [--version X.Y.Z]: GitHub release lookup, version comparison (a pinned version installs even if older, since the server has to match the app), download of the bare twcore-<target> asset, SHA-256 check, a --version run of the new file, atomic rename over the current executable (move-aside on Windows). --restart restarts twcore.service when systemd runs it; otherwise it prints the command. The copy inside the desktop app (.app, AppImage, ThinkWatch Lite install dir) is refused. No new dependencies (reqwest, sha2 already in the tree). Tests cover version comparison, asset selection, checksum verification, the full flow against a local fake GitHub, replacing a running binary, and that release.yml publishes the names upgrade looks for.

Verified

  • cargo fmt, clippy --workspace --all-targets -D warnings, cargo test --workspace.
  • Under systemd on Ubuntu 22.04 aarch64 (Docker): install.sh from a tarball, twcore check as the service user, enable --now, gateway and control socket up under the hardening, live reload of a hand edit, bind: eth0 resolving, reinstall while running, and twcore upgrade --version 0.45.0 with and without --restart against the real GitHub releases (and refusing without root).
  • On macOS: twcore upgrade --check and a real downgrade of a copied binary.

🤖 Generated with Claude Code

fylorn and others added 5 commits September 24, 2026 23:59
A core running on its own (on a server) had no way to move to another
release short of downloading a file and copying it over the binary by
hand. `twcore upgrade` asks GitHub for the latest release, or the one
given with --version, downloads the build for this OS and architecture,
checks its .sha256, runs the new file once to see it reports the version
being installed, and renames it over the current executable. Every step
before the rename leaves the old binary untouched, and the rename is
atomic, so a failed or tampered download never leaves a broken file.

--version installs even an older release: a server has to match the
desktop app's version, which may be behind. --check only reports.
--restart restarts twcore.service when systemd is running it; without
it the command says how, since a restart cuts requests in flight.

The copy inside the desktop app is refused: the app updates it itself,
and the app and core have to come from one commit.

No new dependencies: reqwest and sha2 are already in the tree. Tests run
the whole flow against a local fake GitHub (checksum mismatch, a binary
reporting the wrong version, a missing pinned release), replace a
running binary under itself, and read release.yml to check the asset
names it publishes are the ones upgrade looks for.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…script

Running core on a Linux server and managing it from the desktop app
needs a way to put it there. The release already published bare Linux
binaries (the desktop pipeline takes them); it now also publishes
twcore-<target>.tar.gz for x86_64 and aarch64, holding the binary with
its executable bit, the systemd unit and the licence, each with a
.sha256. The bare binaries and their names are unchanged.

packaging/systemd/twcore.service runs core as a dedicated user with its
data in /var/lib/thinkwatch (StateDirectory, 0700), reads ${VAR} values
from /etc/thinkwatch/env, restarts on failure, and is hardened so it can
write only its data directory (systemd-analyze security: 1.4). It keeps
AF_NETLINK so a `bind` naming an interface still resolves.

scripts/install.sh (POSIX sh) detects the architecture, downloads the
latest or a pinned release, verifies the SHA-256, checks the binary runs
here, installs it atomically into /usr/local/bin, creates the user, the
data directory and the environment file, installs the unit, and runs
`twcore init` when there is no configuration. It is safe to run again:
configuration, environment and data are never touched, and it never
starts or restarts the service itself.

docs/server.md (and zh-CN) walks through install, init, the three config
edits, systemd, `twcore control-key` and connecting the desktop app,
plus upgrading and uninstalling.

The unit and the script were run under systemd on Ubuntu 22.04
(aarch64): install, check, enable --now, live reload, reinstall while
running, and `twcore upgrade --restart`.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…checked

docs/config.md and docs/config.zh-CN.md describe every section and field
of config.yaml, how a change takes effect (the reload stages, the
rejected-config rule, history and rollback, the `twcore config` and
`check` commands) and the environment variables core reads.

The field tables and the built-in rule lists are rendered from
crates/tw-config/tests/manual/schema.rs, and the `manual` test fails when
the manual and the code disagree. The declaration is checked against the
code, not trusted:

- field names come from serde itself, through a probe deserializer that
  records what a derived Deserialize asks for, so a field added to a
  config type without a row in the manual fails with its name;
- a declared default is written into a minimal section and must parse to
  the same thing as leaving the field out, and a required field must
  fail without it;
- enum values are read from serde rather than copied;
- the YAML examples in the manuals are parsed as configuration.

`UPDATE_CONFIG_DOCS=1 cargo test -p tw-config --test manual` rewrites
the generated blocks and leaves the prose alone. CONTRIBUTING explains
the mechanism and lists the release assets.

listen.control.key and listen.control.remote are documented ahead of the
code (they land with the control key and remote access work). They are
declared as pending: rendered in the manual, and the test fails as soon
as the code reads those fields, so the declaration has to be switched to
the real types and their defaults get checked.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A Windows checkout can turn the manuals' line endings into CRLF, and the
generated blocks are rendered with LF.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The control key landed (#184), so `listen.control` is no longer pending:
its table is checked against `ControlListen`, `listen` gains the
`control` row, and the starting configuration shown in the manual has
the key `init` and `serve` now write.

`listen.control.remote` is still pending. A row that points at a pending
section is left out of its parent's field comparison, so `remote` stays
documented under `listen.control`; the moment `ControlListen` reads
`remote`, the pending probe fails and asks for the real type.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@fylorn
fylorn force-pushed the feat/config-manual-upgrade branch from 72e817f to c29aff0 Compare September 24, 2026 16:03
@fylorn
fylorn merged commit b9ff8f6 into main Sep 24, 2026
4 checks passed
@fylorn
fylorn deleted the feat/config-manual-upgrade branch September 24, 2026 17:01
fylorn added a commit that referenced this pull request Sep 25, 2026
…summary

#191 made the publish job write the release once, with GitHub's
generated list of pull requests as its only text and the tag as its
title. A release is now titled "ThinkWatch Core <version>", and its text
has, in order:

- an English summary from release-notes/<version>.md, when that file
  exists;
- a table of the files for each platform;
- the commands that install this version on a Linux server and switch
  an existing installation to it (install.sh --version, twcore upgrade
  --version --restart);
- how to verify a download against its .sha256;
- GitHub's generated list of pull requests.

scripts/release_notes.py builds the text; the publish job fetches the
generated list itself (releases/generate-notes) and hands the finished
text to action-gh-release, which no longer generates anything. The
"only once" rule from #191 stays: a release that already exists keeps
its text. A rehearsal (workflow_dispatch) now writes the text into the
run summary, using the version in Cargo.toml.

scripts/release_notes_test.py checks that the table links exactly the
files in release.yml's FILES list, that the install and upgrade options
exist, and that every file in release-notes/ renders; the Linux CI job
runs it before compiling.

release-notes/0.47.0.md summarizes 0.47.0 from #183-#190; the live
v0.47.0 release page now carries it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
fylorn added a commit that referenced this pull request Sep 25, 2026
…summary

#191 made the publish job write the release once, with GitHub's
generated list of pull requests as its only text and the tag as its
title. A release is now titled "ThinkWatch Core <version>", and its text
has, in order:

- an English summary from release-notes/<version>.md, when that file
  exists;
- a table of the files for each platform;
- the commands that install this version on a Linux server and switch
  an existing installation to it (install.sh --version, twcore upgrade
  --version --restart);
- how to verify a download against its .sha256;
- GitHub's generated list of pull requests.

scripts/release_notes.py builds the text; the publish job fetches the
generated list itself (releases/generate-notes) and hands the finished
text to action-gh-release, which no longer generates anything. The
"only once" rule from #191 stays: a release that already exists keeps
its text. A rehearsal (workflow_dispatch) now writes the text into the
run summary, using the version in Cargo.toml.

scripts/release_notes_test.py checks that the table links exactly the
files in release.yml's FILES list, that the install and upgrade options
exist, and that every file in release-notes/ renders; the Linux CI job
runs it before compiling.

release-notes/0.47.0.md summarizes 0.47.0 from #183-#190; the live
v0.47.0 release page now carries it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
fylorn added a commit that referenced this pull request Sep 25, 2026
…summary (#196)

* ci(release): write the release page from a template, with an English summary

#191 made the publish job write the release once, with GitHub's
generated list of pull requests as its only text and the tag as its
title. A release is now titled "ThinkWatch Core <version>", and its text
has, in order:

- an English summary from release-notes/<version>.md, when that file
  exists;
- a table of the files for each platform;
- the commands that install this version on a Linux server and switch
  an existing installation to it (install.sh --version, twcore upgrade
  --version --restart);
- how to verify a download against its .sha256;
- GitHub's generated list of pull requests.

scripts/release_notes.py builds the text; the publish job fetches the
generated list itself (releases/generate-notes) and hands the finished
text to action-gh-release, which no longer generates anything. The
"only once" rule from #191 stays: a release that already exists keeps
its text. A rehearsal (workflow_dispatch) now writes the text into the
run summary, using the version in Cargo.toml.

scripts/release_notes_test.py checks that the table links exactly the
files in release.yml's FILES list, that the install and upgrade options
exist, and that every file in release-notes/ renders; the Linux CI job
runs it before compiling.

release-notes/0.47.0.md summarizes 0.47.0 from #183-#190; the live
v0.47.0 release page now carries it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* release-notes: summarize 0.48.0

v0.48.0 was published before the template landed, so its page holds
only GitHub's generated list. release-notes/0.48.0.md summarizes it
from #191-#198 so the page can be rewritten from the template:

- upgrade notes: CONTROL_API_VERSION 21, so ThinkWatch Lite 2026.9.16
  (core 0.47.0, protocol 20) does not connect to it and a server used
  with that app stays on 0.47.0; the request store's schema 20, which
  empties the request history on the first start (and again on the way
  back); the /in-flight shape, RequestStarted.session and the removed
  ChatgptUsage fields;
- session and route on the start event, requests a rule decided
  without an upstream, the replayable /in-flight snapshot and the new
  /live fields, and /summary/routes (#197);
- per-group unpriced and no-usage counts, and security log totals
  (#193);
- the signed-in account on a ChatGPT account upstream (#195);
- releases published from one job, the server guide and the crate
  metadata (#191), and the test port fix (#192).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* release-notes(0.48.0): say where the key is kept without naming the keychain

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant