Skip to content

test: cover SSH legacy mode end to end - #7366

Merged
otavio merged 5 commits into
test/e2e-ssh-identity-modefrom
test/e2e-ssh-legacy-mode
Oct 7, 2026
Merged

otavio merged 5 commits into
test/e2e-ssh-identity-modefrom
test/e2e-ssh-legacy-mode

Conversation

@otavio

@otavio otavio commented Oct 7, 2026 •

Copy link
Copy Markdown
Member

Summary

Covers Domain 12 (SSH Legacy Mode) of shellhub-io/team#243 with testcontainers tests, plus one Playwright spec for the [both] public key item. It also closes the gaps the 2026-10-02 audit found in the SSH tests that were already ticked.

tests/
├── environment/ssh.go          # SSHHostKey, AwaitDeviceOffline, AgeSessionKeepAlive
├── ssh_helpers_test.go         # runOnDevice, sendSCPTree (scp sink protocol, recursive)
├── ssh_public_key_test.go      # hostname, tag and username rules both ways; the 0.6.0 agent gate
├── ssh_connection_test.go      # ciphers, banners, disconnect cleanup, server close,
│                               #   server restart, keep-alive against the session reaper
├── ssh_test.go                 # builtin, SFTP mkdir/ls, recursive SCP; strengthened cases
└── identity_access_test.go     # handshakeWith: handshake with a whole client config
ui/apps/console/e2e/terminal.spec.ts  # register a key in the console, ssh in with it

Each refusal asserts how the server refused. A filtered key must fail with [none publickey], and the server must log refused the offered public key with the filter or version reason for that SSHID. A wrong password and an unregistered key must fail with unable to authenticate, no longer any error.

Audit gaps closed in the existing tests:

  • SCP upload and download compare content.
  • Host key verification compares against the key mounted into the server, with a mismatch case.
  • Agent reconnects wait for offline before waiting for online, then run a command through the new tunnel.
  • Cipher negotiation reads back the cipher the connection picked, one cipher at a time.
  • "Keepalive prevents timeout" now has a timeout to beat. A server restart orphans one session, the test ages it and a live idle session past SHELLHUB_SESSION_KEEPALIVE_TIMEOUT, and the reaper retires only the orphan.

Not covered:

  • "Session timeout on idle". Neither the gateway nor the agent sets an idle or max timeout on its SSH server, so there is nothing to time out. The old "session timeout behavior" case is renamed to what it shows.
  • "Signal forwarding". This is a product bug, filed as shellhub-io/team#285. The gateway forwards the signal request, but the agent never calls Session.Signals, so the forked gliderlabs session buffers the signal and drops it. A probe trapping INT in an exec never saw the trap fire, with or without a PTY. The old case sent signals and read nothing back, so it is removed.
  • Firewall Rule Evaluation (10 items), "License device limit blocks connection", and the [both] firewall deny item. These are Enterprise. qa.yml runs the community stack, where EvaluateFirewall and EvaluateLicense have no evaluator, so CI would never run such tests.

Observed, not fixed here:

  • Closing a session through POST /api/sessions/:uid/close answers 202 and ends the client's channel, but the gateway keeps the client's connection open and the session active until the client leaves. OpenSSH leaves at once with status 255; a client that keeps its connection holds a dead one. Filed as shellhub-io/team#297.

Evidence

  • Before: 18 of the open items had no test that asserts their outcome. The audit found the SCP, host key, reconnect and auth-failure cases assert less than their names, and unticked signals, ciphers and keep-alive.
    After: every new and changed test passes:

    TestSSH (both transports)            new and strengthened cases pass
    TestSSHPublicKeyRules                6/6
    TestSSHPublicKeyRefusedBelowAgent060 2/2
    TestSSHConnectionBehaviour           12/12
    TestSSHServerRestart                 2/2
    Playwright: Public Keys              1 passed
    

    The full tests/ suite ran 568 subtests and 1 failed: the offline-device banner case waited 30s for the stopped agent's device to go offline. When a tunnel's close never reaches the server, the server only finds it at the next yamux ping, every 35 seconds, so AwaitDeviceOffline now waits 90s. That case then passed 4 times in a row.

  • Probes against a running stack show what the uncovered items do today. The signal trap never fires. Each default cipher negotiates and reads back as itself, and CBC is refused with no common algorithm.

Merge Danger

Door: two-way

Test-only. Nothing under server/, agent/ or pkg/ changes.

Blast Radius: none

@otavio
otavio requested review from a team as code owners October 7, 2026 11:26
@otavio
otavio added this pull request to stack #7361 October 7, 2026 11:26
@otavio otavio changed the title test/e2e ssh legacy mode test: cover SSH legacy mode end to end Oct 7, 2026
@github-actions

github-actions Bot commented Oct 7, 2026 •

Copy link
Copy Markdown

Claude finished @otavio's task in 6m 42s —— View job


Code Review Complete

Reviewed 7 files. 4 inline issues posted. Standards: 0 violations, 4 judgement calls. Spec: 0. Correctness: 0.

Standards

No rule violations. The most useful judgement call: requireKeyRefused rebuilds the client config that handshake (tests/identity_access_test.go:52) already provides (tests/ssh_public_key_test.go:156-160). A one-line suggestion is attached.

Spec

The Spec axis did not run. The PR body and its commits cite no issue (no Fixes: trailer), so there is no spec to review against.

Additional notes

  • The Correctness pass found no defects:
    • sendSCPTree follows the scp -t sink protocol: D/C/E records, one ack per step, content followed by \0.
    • The requested modes and the asserted ones agree under the default umask.
    • Each test mutates only its own compose stack.
    • Every expected error string exists in x/crypto or in server/ssh.
    • Nothing under pkg/ or server/api/services changes, so the cloud repo is unaffected.
  • One robustness point that is not a defect: requireSCPAck reads stdout with no deadline. A remote sink that stalls without closing would hang the test until go test -timeout.
  • Test* doc comments in _test.go files were not flagged. The suite already does this consistently across about 20 files.

To request another review round, comment /review.

Comment thread tests/ssh_public_key_test.go
Comment thread tests/ssh_connection_test.go
Comment thread tests/ssh_connection_test.go
Comment thread tests/ssh_public_key_test.go
@otavio

otavio commented Oct 7, 2026

Copy link
Copy Markdown
Member Author

/review

@github-actions

github-actions Bot commented Oct 7, 2026 •

Copy link
Copy Markdown

Claude encountered an error after 0s —— View job


I'll analyze this and get back to you.

@otavio
otavio force-pushed the test/e2e-ssh-legacy-mode branch from b5db5bf to 2ffed3f Compare October 7, 2026 13:31
otavio added 5 commits October 7, 2026 11:47
…pload

go-scp has no recursive upload, so sendSCPTree speaks the scp sink protocol to `scp -r -t`
on the device. A D record opens a directory, C records carry its files, E closes it, and the
sink acknowledges each with a zero byte. The directories ask for 0750 and 0700, not the 0755
a plain mkdir gives, so reading the modes back shows the sink built them from the records.

The builtin is `cd` read back through `pwd` and `type -t`, which only works when the exec runs
in the device user's shell. SFTP results are read back over a separate exec channel.
The reconnect cases wait for the device to go offline before starting the agent again. Without
that wait, AwaitDeviceOnline can pass on the state from before the stop. Each reconnect then
runs a command through the new tunnel. AwaitDeviceOffline allows 90 seconds. The server marks
a device offline when its last tunnel closes. When that close does not reach the server, the
next yamux ping finds the tunnel dead, and the server pings every 35 seconds.

SCP upload sent 1024 zero bytes and checked nothing. SCP download read into a buffer pre-filled
with zeros. Both now compare content the test wrote, and the upload checks the mode it asked for.

Host key verification compares the key the server presents with the key mounted into the server
container, and a client pinned to another key fails with "ssh: host key mismatch". The case it
replaces only checked that two dials saw the same key.

A wrong password and an unregistered key now have to fail with x/crypto's "unable to authenticate,
attempted methods [...]". Before, a dial that failed for any reason passed, an unreachable
device included. handshakeWith takes a whole client config, so the host key and cipher cases
can pass theirs.

"session timeout behavior" is renamed to what it shows, an exec silent for three seconds
surviving. Neither the gateway nor the agent sets an idle or max timeout on its SSH server,
so there is no timeout to configure.

"connection with signal handling" is removed. It sent SIGINT and SIGTERM and read nothing
back. A probe trapping INT in an exec, with and without a PTY, never saw the trap fire. The
gateway forwards the signal request to the agent, but the agent never calls Session.Signals,
so the forked gliderlabs session buffers the signal and nothing delivers it. Covering signal
forwarding needs that fixed first.
Each filter case uses one key both ways. The key logs in to the device or user its rule names. On
the one the rule does not name, the client fails with "[none publickey]" and the server logs
"refused the offered public key" with ErrEvaluatePublicKey for that SSHID. The hostname filter
is an anchored regex over the device name, so the name goes through regexp.QuoteMeta. A key
can only name tags that exist, so the test creates the tag first. The username case logs in as
daemon, which the offer refuses before the server contacts the device.

The agent reports the version it was built with, through an ldflag. NewAgentContainerAtVersion
passes the version to BuildAgentImage, which sets SHELLHUB_VERSION and tags the image
<run>-<version>, once per version per run. Without its own tag, the old-version build would replace
the run's agent that every other test starts. 0.5.2 sits below the 0.6.0 gate.

The password case checks only the login. The server treats an agent below 0.9.3 as one that
closes its whole channel when an exec ends, and a current agent stamped 0.5.2 does not, so an
exec on it loses its exit status. That comes from stamping the version, not from the gate.
…tart

The test offers each of x/crypto's default ciphers alone and reads back the one the connection
negotiated, and aes128-cbc fails the handshake. This replaces the table case that offered five
ciphers and never checked which one the server picked. Negotiation sits between the client
and the gateway, so it does not run per transport version.

The gateway sends a banner only to explain a refusal, so the banner cases are a malformed SSHID
and an offline device. There is no greeting banner to deliver.

The disconnect case finds `sleep 637` with pgrep '[s]leep 637'. The bracket stops the pattern
from matching the shell running pgrep, whose command line holds it too.

Asked to close a session, the API answers 202 and the client's shell channel ends without an exit
status. The connection stays open, and the session active, until the client disconnects, which an
OpenSSH client does once its last channel closes. TestSessionCloseDelegatesToTheGatewayThatOwnsIt
covers that part.

No idle timeout exists, so the timeout a keep-alive holds off is the reaper's. The job on
sessionReaperCron retires a session whose seen_at falls out of SHELLHUB_SESSION_KEEPALIVE_TIMEOUT,
which the test sets to its two-minute floor. A server restart leaves a session no gateway
keeps alive. Aging that session and an idle one ten minutes, then firing the reaper, retires
the orphan and keeps the idle session, whose agent keep-alives refresh seen_at every second in
this suite. This replaces "keep-alive and heartbeat", which sent keepalive@shellhub.io requests
the server refuses and had no timeout to outlast.
The key is generated in the agent container and used by its OpenSSH client, as the SSH Approval
specs do. Restricting it to root and to the device's name in the form shows the filters reach
the server in a shape that still lets the key in.
@otavio
otavio force-pushed the test/e2e-ssh-legacy-mode branch from 2ffed3f to 5d0a2e6 Compare October 7, 2026 14:47
@otavio
otavio merged commit 1b5f21a into master Oct 7, 2026
36 of 42 checks passed
@otavio
otavio deleted the test/e2e-ssh-legacy-mode branch October 7, 2026 16:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant