Configuration reference, server deployment, and twcore upgrade - #183
Merged
Merged
Conversation
A core running on its own (on a server) had no way to move to another release short of downloading a file and copying it over the binary by hand. `twcore upgrade` asks GitHub for the latest release, or the one given with --version, downloads the build for this OS and architecture, checks its .sha256, runs the new file once to see it reports the version being installed, and renames it over the current executable. Every step before the rename leaves the old binary untouched, and the rename is atomic, so a failed or tampered download never leaves a broken file. --version installs even an older release: a server has to match the desktop app's version, which may be behind. --check only reports. --restart restarts twcore.service when systemd is running it; without it the command says how, since a restart cuts requests in flight. The copy inside the desktop app is refused: the app updates it itself, and the app and core have to come from one commit. No new dependencies: reqwest and sha2 are already in the tree. Tests run the whole flow against a local fake GitHub (checksum mismatch, a binary reporting the wrong version, a missing pinned release), replace a running binary under itself, and read release.yml to check the asset names it publishes are the ones upgrade looks for. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…script
Running core on a Linux server and managing it from the desktop app
needs a way to put it there. The release already published bare Linux
binaries (the desktop pipeline takes them); it now also publishes
twcore-<target>.tar.gz for x86_64 and aarch64, holding the binary with
its executable bit, the systemd unit and the licence, each with a
.sha256. The bare binaries and their names are unchanged.
packaging/systemd/twcore.service runs core as a dedicated user with its
data in /var/lib/thinkwatch (StateDirectory, 0700), reads ${VAR} values
from /etc/thinkwatch/env, restarts on failure, and is hardened so it can
write only its data directory (systemd-analyze security: 1.4). It keeps
AF_NETLINK so a `bind` naming an interface still resolves.
scripts/install.sh (POSIX sh) detects the architecture, downloads the
latest or a pinned release, verifies the SHA-256, checks the binary runs
here, installs it atomically into /usr/local/bin, creates the user, the
data directory and the environment file, installs the unit, and runs
`twcore init` when there is no configuration. It is safe to run again:
configuration, environment and data are never touched, and it never
starts or restarts the service itself.
docs/server.md (and zh-CN) walks through install, init, the three config
edits, systemd, `twcore control-key` and connecting the desktop app,
plus upgrading and uninstalling.
The unit and the script were run under systemd on Ubuntu 22.04
(aarch64): install, check, enable --now, live reload, reinstall while
running, and `twcore upgrade --restart`.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…checked docs/config.md and docs/config.zh-CN.md describe every section and field of config.yaml, how a change takes effect (the reload stages, the rejected-config rule, history and rollback, the `twcore config` and `check` commands) and the environment variables core reads. The field tables and the built-in rule lists are rendered from crates/tw-config/tests/manual/schema.rs, and the `manual` test fails when the manual and the code disagree. The declaration is checked against the code, not trusted: - field names come from serde itself, through a probe deserializer that records what a derived Deserialize asks for, so a field added to a config type without a row in the manual fails with its name; - a declared default is written into a minimal section and must parse to the same thing as leaving the field out, and a required field must fail without it; - enum values are read from serde rather than copied; - the YAML examples in the manuals are parsed as configuration. `UPDATE_CONFIG_DOCS=1 cargo test -p tw-config --test manual` rewrites the generated blocks and leaves the prose alone. CONTRIBUTING explains the mechanism and lists the release assets. listen.control.key and listen.control.remote are documented ahead of the code (they land with the control key and remote access work). They are declared as pending: rendered in the manual, and the test fails as soon as the code reads those fields, so the declaration has to be switched to the real types and their defaults get checked. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A Windows checkout can turn the manuals' line endings into CRLF, and the generated blocks are rendered with LF. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The control key landed (#184), so `listen.control` is no longer pending: its table is checked against `ControlListen`, `listen` gains the `control` row, and the starting configuration shown in the manual has the key `init` and `serve` now write. `listen.control.remote` is still pending. A row that points at a pending section is left out of its parent's field comparison, so `remote` stays documented under `listen.control`; the moment `ControlListen` reads `remote`, the pending probe fails and asks for the real type. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
fylorn
force-pushed
the
feat/config-manual-upgrade
branch
from
September 24, 2026 16:03
72e817f to
c29aff0
Compare
This was referenced Sep 24, 2026
This was referenced Sep 24, 2026
Merged
fylorn
added a commit
that referenced
this pull request
Sep 25, 2026
…summary #191 made the publish job write the release once, with GitHub's generated list of pull requests as its only text and the tag as its title. A release is now titled "ThinkWatch Core <version>", and its text has, in order: - an English summary from release-notes/<version>.md, when that file exists; - a table of the files for each platform; - the commands that install this version on a Linux server and switch an existing installation to it (install.sh --version, twcore upgrade --version --restart); - how to verify a download against its .sha256; - GitHub's generated list of pull requests. scripts/release_notes.py builds the text; the publish job fetches the generated list itself (releases/generate-notes) and hands the finished text to action-gh-release, which no longer generates anything. The "only once" rule from #191 stays: a release that already exists keeps its text. A rehearsal (workflow_dispatch) now writes the text into the run summary, using the version in Cargo.toml. scripts/release_notes_test.py checks that the table links exactly the files in release.yml's FILES list, that the install and upgrade options exist, and that every file in release-notes/ renders; the Linux CI job runs it before compiling. release-notes/0.47.0.md summarizes 0.47.0 from #183-#190; the live v0.47.0 release page now carries it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
fylorn
added a commit
that referenced
this pull request
Sep 25, 2026
…summary #191 made the publish job write the release once, with GitHub's generated list of pull requests as its only text and the tag as its title. A release is now titled "ThinkWatch Core <version>", and its text has, in order: - an English summary from release-notes/<version>.md, when that file exists; - a table of the files for each platform; - the commands that install this version on a Linux server and switch an existing installation to it (install.sh --version, twcore upgrade --version --restart); - how to verify a download against its .sha256; - GitHub's generated list of pull requests. scripts/release_notes.py builds the text; the publish job fetches the generated list itself (releases/generate-notes) and hands the finished text to action-gh-release, which no longer generates anything. The "only once" rule from #191 stays: a release that already exists keeps its text. A rehearsal (workflow_dispatch) now writes the text into the run summary, using the version in Cargo.toml. scripts/release_notes_test.py checks that the table links exactly the files in release.yml's FILES list, that the install and upgrade options exist, and that every file in release-notes/ renders; the Linux CI job runs it before compiling. release-notes/0.47.0.md summarizes 0.47.0 from #183-#190; the live v0.47.0 release page now carries it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
fylorn
added a commit
that referenced
this pull request
Sep 25, 2026
…summary (#196) * ci(release): write the release page from a template, with an English summary #191 made the publish job write the release once, with GitHub's generated list of pull requests as its only text and the tag as its title. A release is now titled "ThinkWatch Core <version>", and its text has, in order: - an English summary from release-notes/<version>.md, when that file exists; - a table of the files for each platform; - the commands that install this version on a Linux server and switch an existing installation to it (install.sh --version, twcore upgrade --version --restart); - how to verify a download against its .sha256; - GitHub's generated list of pull requests. scripts/release_notes.py builds the text; the publish job fetches the generated list itself (releases/generate-notes) and hands the finished text to action-gh-release, which no longer generates anything. The "only once" rule from #191 stays: a release that already exists keeps its text. A rehearsal (workflow_dispatch) now writes the text into the run summary, using the version in Cargo.toml. scripts/release_notes_test.py checks that the table links exactly the files in release.yml's FILES list, that the install and upgrade options exist, and that every file in release-notes/ renders; the Linux CI job runs it before compiling. release-notes/0.47.0.md summarizes 0.47.0 from #183-#190; the live v0.47.0 release page now carries it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * release-notes: summarize 0.48.0 v0.48.0 was published before the template landed, so its page holds only GitHub's generated list. release-notes/0.48.0.md summarizes it from #191-#198 so the page can be rewritten from the template: - upgrade notes: CONTROL_API_VERSION 21, so ThinkWatch Lite 2026.9.16 (core 0.47.0, protocol 20) does not connect to it and a server used with that app stays on 0.47.0; the request store's schema 20, which empties the request history on the first start (and again on the way back); the /in-flight shape, RequestStarted.session and the removed ChatgptUsage fields; - session and route on the start event, requests a rule decided without an upstream, the replayable /in-flight snapshot and the new /live fields, and /summary/routes (#197); - per-group unpriced and no-usage counts, and security log totals (#193); - the signed-in account on a ChatGPT account upstream (#195); - releases published from one job, the server guide and the crate metadata (#191), and the test port fix (#192). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * release-notes(0.48.0): say where the key is kept without naming the keychain Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Phase P6 of the remote-core plan: the configuration manual, what it takes to run core on a Linux server, and a way to upgrade it there.
Configuration reference (
docs/config.md,docs/config.zh-CN.md)Every section and field of
config.yaml, with defaults, allowed values and examples, plus how a change takes effect: the reload stages, a rejected version keeping the previous one in service, history and rollback,twcore check/config show|set|history|rollback, and the environment variables core reads.The field tables and built-in rule lists are generated and checked. Everything between
<!-- generated: … -->and<!-- /generated -->is rendered fromcrates/tw-config/tests/manual/schema.rs;cargo test -p tw-config --test manualfails when manual and code disagree. The declaration is verified against the code rather than trusted:Deserializerrecords the names a derivedDeserializeasks for), so a new field without a row fails with its name;protocol,mode, grouptype, …) are read from serde;Regenerate:
UPDATE_CONFIG_DOCS=1 cargo test -p tw-config --test manual. CONTRIBUTING describes it.Fields documented ahead of the code
Rebased onto #184:
listen.control.keyis now checked againstControlListenlike every other field.listen.control.remote {enabled, bind, port, allow_from}(P4) is declaredTy::Pending: documented underlisten.control, and the test fails as soon asControlListenreadsremote, asking for the pending declaration incrates/tw-config/tests/manual/schema.rsto be replaced withchecked!(RemoteType, "{}"); its rows and defaults are then verified. Declared defaults to confirm then:enabled: false,bind: all,allow_from:the private ranges,portgenerated. The server guide assumestwcore initmay or may not write a disabledremotesection with a random port (it covers both).Server deployment
docs/server.md(+ zh-CN): install →twcore init→ the three config edits → systemd →twcore control-key→ add the connection in the desktop app; upgrading; uninstalling.packaging/systemd/twcore.service: userthinkwatch,THINKWATCH_HOME=/var/lib/thinkwatch(StateDirectory, 0700),EnvironmentFile=-/etc/thinkwatch/envfor${VAR},Restart=on-failure, hardening that leaves only the data directory writable (systemd-analyze security: 1.4 OK),AF_NETLINKkept so an interface name inbindresolves.scripts/install.sh(POSIX sh, shellcheck-clean): arch detection, latest or--version, SHA-256 check, runs the binary once, atomic install to/usr/local/bin, user + data dir + env file + unit,twcore initwhen there is no config, next steps. Idempotent; never starts or restarts the service.--archive FILEinstalls from a downloaded tarball.Release assets
release.ymlkeeps every existing file and adds, for Linux x86_64 and aarch64:twcore-<target>.tar.gzcontainingtwcore-<target>/{twcore, twcore.service, LICENSE}twcore-<target>.tar.gz.sha256The workflow also unpacks the tarball the way the installer does. Not triggered here; a
workflow_dispatchrehearsal would exercise it without publishing.twcore upgradetwcore upgrade [--check] [--restart] [--version X.Y.Z]: GitHub release lookup, version comparison (a pinned version installs even if older, since the server has to match the app), download of the baretwcore-<target>asset, SHA-256 check, a--versionrun of the new file, atomic rename over the current executable (move-aside on Windows).--restartrestartstwcore.servicewhen systemd runs it; otherwise it prints the command. The copy inside the desktop app (.app, AppImage,ThinkWatch Liteinstall dir) is refused. No new dependencies (reqwest, sha2 already in the tree). Tests cover version comparison, asset selection, checksum verification, the full flow against a local fake GitHub, replacing a running binary, and thatrelease.ymlpublishes the names upgrade looks for.Verified
cargo fmt,clippy --workspace --all-targets -D warnings,cargo test --workspace.install.shfrom a tarball,twcore checkas the service user,enable --now, gateway and control socket up under the hardening, live reload of a hand edit,bind: eth0resolving, reinstall while running, andtwcore upgrade --version 0.45.0with and without--restartagainst the real GitHub releases (and refusing without root).twcore upgrade --checkand a real downgrade of a copied binary.🤖 Generated with Claude Code