Skip to content

test: cover device enrollment and provisioning keys end to end - #7353

Merged
otavio merged 11 commits into
masterfrom
test/e2e-device-enrollment
Oct 7, 2026
Merged

otavio merged 11 commits into
masterfrom
test/e2e-device-enrollment

Conversation

@otavio

@otavio otavio commented Oct 6, 2026 •

Copy link
Copy Markdown
Member

Summary

Covers the 35 open items of Domain 8 in shellhub-io/team#243, Device Enrollment / Provisioning Keys:
34 testcontainers items in tests/ and one Playwright item. The PR is tests only and changes no
product code.

tests/
├── environment/                 # WithEnv, Stack.SQL, provisioning-key and device-limit helpers
├── enrollment_test.go           # TestEnrollmentPolicy: one stack, hand-built /api/devices/auth
│   ├── automatic                #   expired, disabled, revoked keys; tags; ephemeral
│   ├── manual                   #   accept charges a use, reject charges none
│   ├── allowlist                #   listed MAC, unlisted MAC, reconcile on re-auth
│   ├── keyless, system keys     #   legacy key attribution, disabled legacy key, system key refused
│   ├── re-registration          #   key's current tags and ephemeral setting
│   ├── limits                   #   exhausted key blocks accept, namespace device limit
│   └── events                   #   recorded fields, decision stamped on accept and reject
├── enrollment_webhook_test.go   # TestEnrollmentWebhook: stub integrator in the server's netns
└── cmd/enrollment-webhook/      # the stub: answers /accept, /reject, /defer, logs each call
ui/apps/console/e2e/
├── provisioning-keys.spec.ts   # reveal a key's plaintext, then enroll a device with it
└── devices.ts                  # device enrollment request, shared with members.spec.ts

The webhook cases cover accept, reject, defer, timeout, the HMAC signature, the payload, the
deferred callback accepting and rejecting, a single-use callback token, the callback TTL,
reconcile on re-auth, the once-a-minute throttle, and the SSRF guard.

Two cases write the database through psql, because no API reaches the state they need. One moves
a key's expiry into the past, since the API sets expiry only in whole days ahead. The other sets
max_devices, which only the cloud sets, although validateDeviceAcceptance enforces any
positive value in every edition. Each commit message carries the reason behind its fixture.

Evidence

  • Before: none of these 35 items had a test that asserted its outcome. To show each new case
    can fail, I broke the server one behavior at a time and ran the suites. Every targeted case
    failed:

    Server change Case that failed
    IsValid ignores disabled, expired, revoked automatic expired, disabled, revoked; legacy disabled
    EnrollWith drops ephemeral; key tags not applied ephemeral, tags, re-registration
    allowlist accepts any MAC unlisted MAC, allowlist reconcile
    reconcile throttle removed throttle
    SSRF guard removed from the webhook client SSRF
    HMAC signed with another secret signature and payload
    webhook reject read as pending reject decision
    key timeout ignored timeout
    callback redeem check and key TTL ignored single-use, TTL
    callback reject applied as accept deferred callback rejects
    decision stamp skipped; event hostname changed events
    reject charges the key; charge is a no-op manual reject, manual accept, usage exhausted
    legacy key not resolved keyless attribution and disabled legacy
    IsSystem checks removed system key refused
    webhook not reconcilable webhook reconcile, throttle
    namespace device limit skipped device limit
    Playwright: enroll with the revealed plaintext plus one character reveal, 400 auth invalid

    After: every case passes on the community stack, through the test compose runner:

    --- PASS: TestEnrollmentPolicy (85.82s)
    --- PASS: TestEnrollmentWebhook (157.82s)
    ok   github.com/shellhub-io/shellhub/tests   1124.633s   (full suite, 0 failures)
    ✓ e2e/provisioning-keys.spec.ts › a key's page reveals the plaintext an agent enrolls with
    

    I ran the Playwright spec on community only, because this machine has no license or Stripe
    keys. The reveal route is community code and CI runs the spec on every edition.

Found along the way, not fixed here: a provisioning key accepts tag names the tag schema rejects,
for example first-enrollment. The server then fails to serve the device it tagged, because its
response no longer matches the OpenAPI schema.

Merge Danger

Door: two-way

Blast Radius: CI

The tests job in qa.yml gains two stacks, about four minutes. The webhook stub image builds
from golang:1.26.8-alpine3.24, the image the agent test build already uses.

@otavio
otavio requested review from a team as code owners October 6, 2026 19:36
@github-actions

github-actions Bot commented Oct 6, 2026 •

Copy link
Copy Markdown

Claude encountered an error after 3m 3s —— View job


I'll analyze this and get back to you.

@github-actions

github-actions Bot commented Oct 6, 2026 •

Copy link
Copy Markdown

Claude encountered an error after 3m 11s —— View job


I'll analyze this and get back to you.

luizhf42
luizhf42 previously approved these changes Oct 6, 2026
@otavio
otavio added this pull request to stack #7361 October 6, 2026 22:16
@otavio
otavio force-pushed the test/e2e-device-enrollment branch from e77597e to 88bb9f4 Compare October 7, 2026 13:31
otavio added 11 commits October 7, 2026 11:47
The enrollment tests need state the product only reaches with time or in the cloud. An expired
provisioning key takes a day, because the API sets an expiry in whole days ahead. A namespace device
limit is set only by the cloud, although validateDeviceAcceptance enforces any positive max_devices
whatever the edition. So the stack runs psql in its postgres container, and the helpers that need
it say which state they stand in for.

SQL pipes the statement through stdin because psql does not substitute :'name' variables in a -c
command. Passing values as variables keeps them out of the statement text.

The provisioning key helpers now list 100 keys per page. A stack shared by many cases holds more
keys than the default page of 10, and the old lookups found a key only while it sorted first.
RequireProvisioningKeyUsesHold polls for five seconds, so a charge that must not happen cannot pass
by arriving after a single read.
The enrollment cases send hand-built requests to the endpoint an agent enrolls through, so each one
controls the identity it presents and reads the outcome back through the API. They share one stack
to keep CI time down. Accepting a device merges it into an accepted device with the same MAC, so
every case gives its devices a MAC and a hostname no other case uses.
An allowlist key decides a device on its own, so it never leaves one pending. The reconcile case
starts from a manual key and turns it into an allowlist, which is how a pending device meets an
allowlist on a later authentication.

The server answers a device's authentication from a cache for 30 seconds, so a re-authentication
reaches the enrollment policy only once that entry expires. awaitStatusOnReauth keeps
re-authenticating until it does.
The store derives the digests of a namespace's legacy and pairing keys from its tenant id, as the
SHA-256 of "system:<tenant>" and "system:pairing:<tenant>". Anyone who knows a tenant id can compute
those plaintexts, so the test presents them, after checking they hash to the listed keys' ids.

The case that disables the legacy key re-enables it in a cleanup as well, because a failure before
the re-enable would leave every later keyless case refused. The cleanup uses context.Background()
since t.Context() is already cancelled when cleanups run.
Removing a device keeps its tags, so the re-registered device holds the tag it had and the one the
key carries now. The test pins both.
Community namespaces have no device limit, so the test sets max_devices to the count of accepted
devices and restores -1 in a cleanup. Lifting the limit lets the next re-authentication accept the
pending device, which shows the limit alone held it back.
The event's fingerprint is checked against a fixed key whose SHA-256 fingerprint came from
ssh-keygen, not from the Go code the server uses to compute it.
The stub runs in the server's network namespace and answers each call with the decision its path
names, after the delay its query asks for. It logs every call it receives as a line of JSON, which
the test reads back. The stack allows the webhook to call 127.0.0.0/8, so the server reaches the
stub on loopback while every other private address stays behind the SSRF guard. The stub binds all
interfaces, so the SSRF case calls it at the server's own network address. The guard is the only
thing that can stop that call.

The timeout case gives the key a one second timeout and the stub a ten second delay, and checks
the enrollment returned within the five second default. The callback window case sleeps three
seconds, past the one second TTL, because the token carries no leeway and nothing observable marks
its expiry. The throttle case re-authenticates every two seconds for up to the 30 second auth
cache plus two reconcile intervals, and checks the integrator was asked again no sooner than a
minute after the previous reconcile.
@otavio
otavio force-pushed the test/e2e-device-enrollment branch from 88bb9f4 to 92c3aed Compare October 7, 2026 14:47
@otavio
otavio merged commit 61c3bce into master Oct 7, 2026
47 of 49 checks passed
@otavio
otavio deleted the test/e2e-device-enrollment branch October 7, 2026 16:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants