Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
23 commits
Select commit Hold shift + click to select a range
616adc8
state: lower a clock floor that a better time source contradicts
claude Nov 17, 2026
319cf6f
Sign the refresh channel's upgrade with the device key
claude Nov 17, 2026
c0ac846
Report a crashed or hung kiosk page as not alive
claude Nov 17, 2026
4dd7e9e
Heal a crashed or hung kiosk page: reload, then restart the kiosk
claude Nov 17, 2026
129e9bf
clock: NTP is the authority; only trusted time raises the floor
claude Nov 17, 2026
f17dc36
Enroll only over https, with a wss socket on the backend's host
claude Nov 17, 2026
8bf1b85
Refuse packages signed in the future of an NTP-synchronised clock
claude Nov 17, 2026
15c3c17
Upload page: say whose clock is wrong when the board has NTP
claude Nov 17, 2026
5836a20
docs: the clock with NTP as the authority, and what the floor is now
claude Nov 17, 2026
55cbc5d
Refuse an insecure backend config at daemon start
claude Nov 17, 2026
ddcf777
Docs: https-only enrollment, insecure_dev, kiosk self-healing
claude Nov 17, 2026
81f82de
Agent self-update: a crash-safe transaction and a real probation
claude Nov 17, 2026
5c44ec3
Updater units: start on an unfinished update, time for the probation
claude Nov 17, 2026
c2cd919
Watchdog pings only while the agent is live
claude Nov 17, 2026
7de3c08
Safe mode: say what it is, and write the counter durably
claude Nov 17, 2026
bf51cbe
Docs: the agent update journal, probation and watchdog as they are
claude Nov 17, 2026
448e849
Merge branch 'fix/clock-keeper' into claude/wizardly-curie-s90b5i
claude Nov 17, 2026
d89ea5d
Merge branch 'fix/selfupdate-safety' into claude/wizardly-curie-s90b5i
claude Nov 17, 2026
c60fd74
Merge branch 'fix/kiosk-liveness-tls' into claude/wizardly-curie-s90b5i
claude Nov 17, 2026
c79ec97
install.sh: the backend URL must be https; point dev installs at --in…
claude Nov 17, 2026
6ae00be
Merge branch 'fix/refresh-stream-auth' into claude/wizardly-curie-s90b5i
claude Nov 17, 2026
badfa69
Drop the signed refresh-channel upgrade
claude Nov 19, 2026
b53f78b
Start the new agent when resuming an interrupted swap without a reboot
claude Nov 19, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 4 additions & 2 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -95,7 +95,9 @@ deploy: build-arm64
ssh -t $(PI) 'sudo bash /tmp/update.sh /tmp/musallahboard-agent-arm64'

# First-time full install on a remote host via the release tarball.
# Usage: make install-remote PI=admin@pi.local TOKEN=xxx BACKEND=http://host:8080
# Usage: make install-remote PI=admin@pi.local TOKEN=xxx BACKEND=https://backend.example.org
# (BACKEND must be https://; for a plain-http dev backend, enroll afterwards on
# the board with `musallahboard-agent enroll ... --insecure-dev`.)
install-remote: package
@if [ -z "$(PI)" ]; then echo "Usage: make install-remote PI=user@host [TOKEN=xxx BACKEND=url]"; exit 1; fi
scp build/musallahboard-agent-$(VERSION)-arm64.tar.gz $(PI):/tmp/
Expand All @@ -104,7 +106,7 @@ install-remote: package
$(if $(TOKEN),--token=$(TOKEN)) $(if $(BACKEND),--backend=$(BACKEND))'

# First-time full install on a remote x86-64 host via the release tarball.
# Usage: make install-remote-x86 HOST=admin@server TOKEN=xxx BACKEND=http://host:8080
# Usage: make install-remote-x86 HOST=admin@server TOKEN=xxx BACKEND=https://backend.example.org
install-remote-x86: package-amd64
@if [ -z "$(HOST)" ]; then echo "Usage: make install-remote-x86 HOST=user@host [TOKEN=xxx BACKEND=url]"; exit 1; fi
scp build/musallahboard-agent-$(VERSION)-amd64.tar.gz $(HOST):/tmp/
Expand Down
10 changes: 9 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -53,8 +53,16 @@ sudo musallahboard-agent enroll --token=<token> --backend=<backend URL>
# Easiest: copy the whole command from the admin portal when you issue the token.
```

The backend URL must be `https://`, and the WebSocket address the backend hands back must be `wss://` on the same host; the agent refuses anything else, because the enrollment reply pins the keys the board trusts content from. (A development backend on plain http needs `--insecure-dev`; see [Development boards](#development-boards-on-plain-http).)

The device will then be enrolled and the board will load automatically once its first content and board app packages have synced. The device is now ready to be used as a kiosk!

### Development boards on plain http

For a backend running on your own machine without TLS, add `--insecure-dev` to the enroll command. It is saved in `/etc/musallahboard/agent.toml` as `insecure_dev = true`, and the agent logs a warning at every start while it is on: anyone on that board's network can impersonate the backend. Never use it on a deployed board.

A board enrolled over http before https was required, without `insecure_dev = true`, keeps showing the content it has but no longer talks to the backend (no sync, no update checks, no remote commands). `musallahboard-agent status`, `systemctl status musallahboard-agent` and the journal say why. Re-enroll it over https with `--force`, or, if it is a development board, add `insecure_dev = true` to its `agent.toml` and `sudo systemctl restart musallahboard-agent`.

## Building

A little unorthodox, but this go project uses a Makefile to build the agent binary. The Makefile has targets for both native and cross-compilation.
Expand All @@ -81,7 +89,7 @@ agent release (or a build of this checkout):

```bash
vm/mbvm create
vm/mbvm enroll --token=<token> --backend=http://localhost:8080
vm/mbvm enroll --token=<token> --backend=http://localhost:8080 --insecure-dev
vm/mbvm reset --agent=0.2.1
```

Expand Down
236 changes: 223 additions & 13 deletions cmd/agent/board.go
Original file line number Diff line number Diff line change
Expand Up @@ -77,19 +77,29 @@ func startBoard(ctx context.Context, logger *slog.Logger, cfg *config.Config, sa
logger.Error("device key unreadable: serving installed content only (no sync, no remote commands)",
"err", keyErr, "keyPath", cfg.KeyPath)
}
// Nothing that talks to the backend is built from an insecure config:
// content sync, its refresh channel and the command WebSocket all take
// their addresses from it (docs/architecture.md, section 11).
transportErr := backendTransport(logger, cfg)
offline := func() string {
if transportErr != nil {
return "not talking to LensBridge: " + transportErr.Error()
}
return "device key unreadable"
}
hour, minute := cfg.UpdateTime()
var syncer *boardsync.Syncer
sched := updates.New(updates.Deps{
Layout: layout, Importer: imp, Hour: hour, Minute: minute, AutoUpdate: cfg.AutoUpdate(), Logger: logger,
Changed: func(info updates.Info) { hub.UpdatesChanged(info) },
Check: func(ctx context.Context) error {
if syncer == nil {
return errors.New("this board cannot reach the release channels (device key unreadable)")
return errors.New("this board cannot reach the release channels (" + offline() + ")")
}
return syncer.CheckChannels(ctx)
},
})
if keyErr == nil {
if keyErr == nil && transportErr == nil {
syncer = boardsync.New(boardsync.Deps{
Cfg: cfg, Key: priv, Layout: layout, Importer: imp, Updates: sched,
AgentVersion: version.Version, Logger: logger,
Expand All @@ -103,7 +113,9 @@ func startBoard(ctx context.Context, logger *slog.Logger, cfg *config.Config, sa
Layout: layout, DeviceID: cfg.DeviceID, AgentVersion: version.Version, Hub: hub,
Sync: func() any {
if syncer == nil {
return map[string]any{"enabled": false, "lastError": "device key unreadable"}
// Enabled as configured, so the board app and `status` show
// the reason as a sync error rather than "sync off".
return map[string]any{"enabled": transportErr != nil && cfg.ContentSync(), "lastError": offline()}
}
return syncer.Status()
},
Expand All @@ -121,6 +133,9 @@ func startBoard(ctx context.Context, logger *slog.Logger, cfg *config.Config, sa
s := ws.State()
return &s
}
if transportErr != nil {
return map[string]any{"connected": false, "lastError": offline()}
}
return nil
},
ServicePort: cfg.ServicePort(),
Expand All @@ -145,10 +160,14 @@ func startBoard(ctx context.Context, logger *slog.Logger, cfg *config.Config, sa
logger.Warn("could not write kiosk url", "err", err)
}
_, _ = daemon.SdNotify(false, daemon.SdNotifyReady)
_, _ = daemon.SdNotify(false, "STATUS=serving the board on "+localserver.ListenAddr)
if transportErr != nil {
_, _ = daemon.SdNotify(false, "STATUS=serving installed content only; refusing the backend: "+transportErr.Error())
} else {
_, _ = daemon.SdNotify(false, "STATUS=serving the board on "+localserver.ListenAddr)
}

logInstalled(logger, layout, cfg)
goRun(wg, func() { watchKiosk(ctx, logger, cdpClient) })
goRun(wg, func() { watchKiosk(ctx, logger, cdpClient, screen.Active) })
goRun(wg, func() {
// First say how an agent update that restarted us went: the rest of
// its batch may be waiting in the inbox, and belongs on the same
Expand All @@ -172,6 +191,30 @@ func startBoard(ctx context.Context, logger *slog.Logger, cfg *config.Config, sa
return nil
}

// backendTransport checks that the config may be used to talk to the
// backend (config.CheckTransport) and says so loudly at every start when it
// may not, or when it may only because of the insecure_dev opt-in. A board
// enrolled over plain http before https was required lands here: it keeps
// showing what it has, and the log, `systemctl status`, `musallahboard-agent
// status` and the board app's sync error all carry the reason and the fix.
func backendTransport(logger *slog.Logger, cfg *config.Config) error {
if err := cfg.CheckTransport(); err != nil {
logger.Error("REFUSING TO TALK TO THE BACKEND: this board's config is not safe to use. "+
"Serving installed content only: no content sync, no release channel checks, no remote commands. "+
"Re-enroll it over https (sudo musallahboard-agent enroll --force --token=... --backend=https://...), "+
"or, for a development board only, add insecure_dev = true to "+defaultConfigPath+" and restart the agent.",
"err", err, "backendUrl", cfg.BackendURL, "websocketUrl", cfg.WebSocketURL)
return err
}
if cfg.InsecureDev() {
logger.Warn("INSECURE DEVELOPMENT BOARD: insecure_dev = true, so this board accepts a plaintext backend. "+
"Anyone on its network can impersonate the backend: send it content, reboot it, read its screen and logs. "+
"Never deploy a board like this.",
"backendUrl", cfg.BackendURL, "websocketUrl", cfg.WebSocketURL)
}
return nil
}

func startUploadServer(ctx context.Context, logger *slog.Logger, cfg *config.Config, layout store.Layout,
imp *importer.Importer, keeper *clock.Keeper, screen *updatescreen.Screen, wg *sync.WaitGroup) {
ln, err := uploadserver.Listen()
Expand Down Expand Up @@ -221,9 +264,28 @@ func startCommandChannel(ctx context.Context, logger *slog.Logger, cfg *config.C
return wsClient
}

// watchKiosk navigates Chromium back to the board if it lands on its own
// error page (the server restarting under it, for instance).
func watchKiosk(ctx context.Context, logger *slog.Logger, c *cdp.Client) {
// Kiosk self-healing (watchKiosk). The page is probed every
// kioskWatchInterval. A crashed renderer is certain after one probe; a page
// that does not answer may be busy (a large poster decoding on a Pi), so it
// gets kioskHungThreshold probes. Then the watcher reloads the page; if the
// page is still dead after that, it restarts the kiosk unit. Restarts back off
// from kioskRestartMinGap to kioskRestartMaxGap, and the backoff only starts
// over after the page has been healthy for kioskRestartResetAfter, so a page
// that never comes back costs at most one reload and one restart an hour.
const (
kioskProbeTimeout = 5 * time.Second
kioskCrashThreshold = 1
kioskHungThreshold = 3
kioskRestartMinGap = 5 * time.Minute
kioskRestartMaxGap = time.Hour
kioskRestartResetAfter = time.Hour
)

// watchKiosk keeps the kiosk on the board: it navigates Chromium back if it
// lands on its own error page (the server restarting under it, for instance)
// and heals a crashed or hung page (kioskHealer).
func watchKiosk(ctx context.Context, logger *slog.Logger, c *cdp.Client, updateActive func() bool) {
h := newKioskHealer(logger, c, updateActive)
t := time.NewTicker(kioskWatchInterval)
defer t.Stop()
for {
Expand All @@ -232,14 +294,162 @@ func watchKiosk(ctx context.Context, logger *slog.Logger, c *cdp.Client) {
return
case <-t.C:
}
cctx, cancel := context.WithTimeout(ctx, 5*time.Second)
target, err := c.FirstPageTarget(cctx)
if err == nil && strings.HasPrefix(target.URL, "chrome-error:") {
logger.Info("kiosk is on an error page; returning it to the board")
_ = c.Call(cctx, target, "Page.navigate", map[string]any{"url": localserver.BaseURL}, nil)
h.tick(ctx)
}
}

// kioskHealer is the watcher's state between ticks. Its browser and systemd
// operations are fields so the escalation can be tested without either.
type kioskHealer struct {
log *slog.Logger
now func() time.Time
updateActive func() bool
target func(context.Context) (cdp.Target, error)
navigate func(context.Context, cdp.Target, string) error
health func(context.Context) error
reload func(context.Context) error
restart func(context.Context) error

fails int // consecutive dead-page probes
reloaded bool // reloaded since the last restart (or since healthy)
restarts int // restarts since the backoff last started over
nextRestart time.Time // no restart before this
healthySince time.Time // zero while the page is dead
waitLogged bool
}

func newKioskHealer(logger *slog.Logger, c *cdp.Client, updateActive func() bool) *kioskHealer {
if updateActive == nil {
updateActive = func() bool { return false }
}
return &kioskHealer{
log: logger,
now: time.Now,
updateActive: updateActive,
target: c.FirstPageTarget,
navigate: func(ctx context.Context, t cdp.Target, url string) error {
return c.Call(ctx, t, "Page.navigate", map[string]any{"url": url}, nil)
},
health: c.PageHealth,
reload: c.PageReload,
restart: func(ctx context.Context) error {
// The same privileged path as the kiosk.restart remote command
// (sudo -n systemctl restart, allowed by the sudoers drop-in).
_, err := (&commands.KioskRestart{}).Execute(ctx, nil, func(string, string, *int) {})
return err
},
}
}

func (h *kioskHealer) tick(ctx context.Context) {
// The update screen is driven over CDP by the importer and may be
// waiting on an agent restart; nothing here may navigate under it.
if h.updateActive() {
h.fails = 0
return
}

cctx, cancel := context.WithTimeout(ctx, kioskProbeTimeout)
target, err := h.target(cctx)
cancel()
if err == nil && strings.HasPrefix(target.URL, "chrome-error:") {
h.log.Info("kiosk is on an error page; returning it to the board")
cctx, cancel := context.WithTimeout(ctx, kioskProbeTimeout)
_ = h.navigate(cctx, target, localserver.BaseURL)
cancel()
return
}
if err != nil {
// No browser, or no page yet: starting, or restarting (perhaps
// because we asked). systemd's Restart=always owns that case.
return
}

cctx, cancel = context.WithTimeout(ctx, kioskProbeTimeout)
err = h.health(cctx)
cancel()
if ctx.Err() != nil {
return
}

threshold := 0
switch {
case errors.Is(err, cdp.ErrPageCrashed):
threshold = kioskCrashThreshold
case errors.Is(err, cdp.ErrPageUnresponsive):
threshold = kioskHungThreshold
case errors.Is(err, cdp.ErrDebuggerUnreachable), errors.Is(err, cdp.ErrNoPageTarget):
// The browser went away between the two calls: not the page's fault,
// and not proof it recovered either.
return
default:
// Answered, or between documents, or the hook threw: the page is
// alive. A navigation in flight is not a failure.
h.healthy()
return
}

h.healthySince = time.Time{}
h.fails++
if h.fails < threshold {
h.log.Warn("kiosk page is not answering", "err", err, "failures", h.fails, "url", target.URL)
return
}
h.heal(ctx, err, target.URL)
}

func (h *kioskHealer) healthy() {
now := h.now()
if h.fails > 0 || h.reloaded {
h.log.Info("kiosk page is answering again")
}
h.fails, h.reloaded, h.waitLogged = 0, false, false
if h.healthySince.IsZero() {
h.healthySince = now
}
if h.restarts > 0 && now.Sub(h.healthySince) >= kioskRestartResetAfter {
h.restarts, h.nextRestart = 0, time.Time{}
}
}

// heal takes the next step for a dead page: a reload, at most once until the
// page answers again or the kiosk restarts; then a restart, at most once per
// backoff gap. A page that recovers each time it is reloaded is reloaded each
// time it dies, which is what it needs; a page that never recovers gets one
// reload and one restart per gap.
func (h *kioskHealer) heal(ctx context.Context, cause error, url string) {
if !h.reloaded {
h.fails, h.reloaded = 0, true
h.log.Warn("kiosk page is dead; reloading it", "err", cause, "url", url)
cctx, cancel := context.WithTimeout(ctx, kioskProbeTimeout)
if err := h.reload(cctx); err != nil {
h.log.Warn("could not reload the kiosk page", "err", err)
}
cancel()
return
}

now := h.now()
if now.Before(h.nextRestart) {
if !h.waitLogged {
h.log.Error("kiosk page is still dead; waiting before restarting the kiosk again so a broken page cannot restart it in a loop",
"err", cause, "url", url, "nextAttempt", h.nextRestart.Format(time.RFC3339))
h.waitLogged = true
}
return
}

h.fails, h.reloaded, h.waitLogged = 0, false, false
gap := min(kioskRestartMinGap<<min(h.restarts, 8), kioskRestartMaxGap)
h.restarts++
h.nextRestart = now.Add(gap)
h.log.Error("kiosk page is still dead after a reload; restarting the kiosk",
"err", cause, "url", url, "restarts", h.restarts, "nextAllowed", h.nextRestart.Format(time.RFC3339))
cctx, cancel := context.WithTimeout(ctx, 30*time.Second)
if err := h.restart(cctx); err != nil {
h.log.Error("could not restart the kiosk", "err", err)
}
cancel()
}

func logInstalled(logger *slog.Logger, l store.Layout, cfg *config.Config) {
Expand Down
Loading
Loading