Skip to content

test: cover firewall rule evaluation end to end - #7384

Closed
otavio wants to merge 5 commits into
fix/license-accept-fail-openfrom
test/e2e-firewall
Closed

otavio wants to merge 5 commits into
fix/license-accept-fail-openfrom
test/e2e-firewall

Conversation

@otavio

@otavio otavio commented Oct 7, 2026 •

Copy link
Copy Markdown
Member

Summary

Covers the Firewall Rule Evaluation items and the firewall end-to-end item of shellhub-io/team#243 Domain 12 (SSH Legacy Mode). This layer sits on #7379 in the per-domain e2e stack and runs in the validate (tests - enterprise) job #7377 added.

The firewall runs in the banner phase, before authentication. A rule that blocks a connection sends the client the Access Denied banner and leaves this line in the server's log:

Session.Evaluate                          legacy namespace, enterprise or cloud
  checkFirewall
    rules by priority ASC, active only
      first rule whose source_ip, username and hostname|tags all match decides
      none matches → allow
  identity namespace → never reads its rules, access policies decide
 tests/environment/
+  firewall.go        FirewallRule, CreateFirewallRule, DeleteFirewallRule, FirewallRuleCount
   docker_compose.go
+                     EditSSHAccessMode (one stack switches legacy → identity → legacy)
 tests/
+  firewall_test.go   TestEnterpriseFirewall: 14 cases on one stack and one device
   ssh_connection_test.go
+                     requireAccessDenied (banner + server log line), now also used by the license tests
   identity_policy_test.go
+                     createTag, replacing two inline copies
 ui/apps/console/e2e/terminal.spec.ts
+  Firewall Rules › a deny rule created in the console keeps the ssh client out

Each refusal checks for the wrapped error text a firewall rule block your connection as well as the log line, because the server writes blocked by a firewall rule for a deny and for a failed evaluation alike. Each match case has a miss case next to it. The two priority cases create their rules in opposite orders.

Product fix, its own commit: the rule form's priority hint said "Higher values are evaluated first". The store lists rules ORDER BY priority ASC, and the first match decides. A user who followed the hint would put their rules in the reverse order. The hint now says "Lower values are evaluated first", and a RuleModal unit test pins it.

Issue lines covered

Line Item Test
399 Connection allowed when no firewall rules exist TestEnterpriseFirewall/a connection is let through when the namespace has no rules
400 Allow rule matches by source IP, username, hostname TestEnterpriseFirewall/an allow rule matching the source address, the user and the hostname lets the connection through (sits over a deny-all rule, so only a full match lets the client in)
401 Deny rule blocks connection by source IP TestEnterpriseFirewall/a deny rule matching the source address blocks the connection, .../a deny rule for another source address lets the connection through
402 Deny rule blocks connection by username TestEnterpriseFirewall/a deny rule matching the user blocks the connection, .../a deny rule for another user lets the connection through
403 Deny rule blocks by hostname filter TestEnterpriseFirewall/a deny rule matching the hostname blocks the connection, .../a deny rule for another hostname lets the connection through
404 Deny rule blocks by tag filter TestEnterpriseFirewall/a deny rule matching the tag blocks the connection, .../a deny rule for a tag the device does not carry lets the connection through
405 Priority ordering: lower priority wins TestEnterpriseFirewall/an allow rule with a lower priority value wins over a matching deny rule, .../a deny rule with a lower priority value wins over a matching allow rule
406 Inactive rule is skipped TestEnterpriseFirewall/an inactive deny rule is skipped
407 Firewall not evaluated in identity mode TestEnterpriseFirewall/a namespace in the identity mode is not held to its firewall rules
408 License device limit blocks connection already covered by #7377: TestEnterpriseLicensing/device connection/connecting with the fleet over the license limit is refused. The license check runs before the firewall, so the firewall's own copy of the device limit check never decides an enterprise connection
452 End-to-end: create firewall deny rule in UI, then verify SSH blocked terminal.spec.ts › Firewall Rules › a deny rule created in the console keeps the ssh client out (with TestEnterpriseFirewall above for the testcontainers side)

Not covered here:

  • 440, session timeout on idle. Neither the gateway nor the agent sets an idle timeout, so there is nothing to test. The line should be struck.
  • 442, signal forwarding. Still blocked by shellhub-io/team#285: the agent drops forwarded signals.

Evidence

  • Before: no test created a firewall rule. The console told users that higher priority values are evaluated first.
  • After: all run locally, in the test runner and the e2e image:
$ go test -run '^TestEnterprise' -v .        (rebased on #7379)
--- PASS: TestEnterpriseFirewall                 14 cases
--- PASS: TestEnterpriseLicensing
--- PASS: TestEnterpriseWithoutLicenseFile

$ go test -run '^(TestSSHPublicKeyRules|TestAccessPolicyEvaluation)$' .    createTag callers
--- PASS: TestSSHPublicKeyRules
--- PASS: TestAccessPolicyEvaluation

$ ui/scripts/e2e.sh test --edition=enterprise terminal.spec.ts -g "SSH Approval|Public Keys|Firewall Rules"
5 passed

$ vitest run src/pages/firewall-rules        53 passed

Mutation checks, one run each, then reverted. Each one made the cases it targets fail:

deny rules created inactive              → the 4 "deny rule matching …" cases fail
allow rule at priority 300, not 1        → "an allow rule with a lower priority value wins …" fails
inactive rule made active                → "an inactive deny rule is skipped" fails
identity mode switch removed             → "a namespace in the identity mode …" fails
console rule saved as Allow, not Deny    → the Playwright case fails: the client gets a shell
hint text reverted                       → the RuleModal unit test fails

golangci-lint v2.14.0 reports 0 issues with --max-issues-per-linter=0 --max-same-issues=0 on tests, server, gateway, pkg, and agent with and without --build-tags docker. ESLint and tsc --noEmit are clean on the console.

Review

/code-review ran four rounds before this PR was opened:

  1. Spec: clean apart from one minor point. clientAddress depended on an earlier subtest's session, so it now logs in itself. Standards: 3 hard, 8 judgement calls.
    • The EditSSHAccessMode contract was wrong.
    • FirewallRuleCount read only the first page, so it now reads X-Total-Count.
    • Duplicated steps were extracted: requireAccessDenied, createTag, and the native client cleanup in the file-level afterEach.
    • Constants and subtest names were made clearer, and the data clumps became matching/allow/deny.
    • One rejected: the unit test that asserts the hint text. The text is the bug being fixed.
  2. Spec: clean. Standards: 1 hard, 3 judgement calls, all fixed. The EditSSHAccessMode contract was still incomplete (JWT bearer 403, SSHLegacyAllowed, the seeded policy's logins). The "lower priority" names were ambiguous, and two doc comments had gaps.
  3. Standards: 1 minor hard, fixed. The CreateFirewallRule contract listed some reasons for a non-200 response and missed others. Two rejected: DeleteFirewallRule's "fails unless 200" already covers every case, and the shared helpers sit next to their siblings, as tagDevice and dialForBanner already do.
  4. Clean.

Merge Danger

Door: two-way

Tests and one line of console copy. No migration, API or wire change.

Blast Radius: tests

TestEnterpriseFirewall adds about 80 seconds to validate (tests - enterprise). The only change users see is the corrected priority hint in the firewall rule form.

@otavio
otavio added this pull request to stack #7380 October 7, 2026 20:14
@otavio otavio changed the title test/e2e firewall test: cover firewall rule evaluation end to end Oct 7, 2026
@otavio
otavio marked this pull request as ready for review October 7, 2026 20:21
@otavio
otavio requested review from a team as code owners October 7, 2026 20:21
@github-actions

github-actions Bot commented Oct 7, 2026 •

Copy link
Copy Markdown

Claude finished @otavio's task in 7m 8s —— View job


Code Review Complete

Reviewed 10 files. 1 inline issue posted. Standards: 0 violations, 1 judgement call. Spec: did not run. Correctness: 0.

Standards

The FirewallRuleFilter doc comment doesn't say whether a multi-tag filter matches a device carrying any one of the tags or all of them. The server matches on any one (cloud/internal/cloud/services/firewall.go:108-120). See tests/environment/firewall.go:31-32. The other doc comments (EditSSHAccessMode, FirewallRule, CreateFirewallRule, DeleteFirewallRule, FirewallRuleCount) are accurate against the server and cloud routes. The commit messages give the reasons the code leaves out.

Spec

The Spec axis did not run. The PR closes no issue in this repo, and the only reference in its commits is Refs: shellhub-io/team#243, a different repo that this run cannot read.

Additional notes

The correctness pass checked these claims against the source and found them sound:

  • The asserted strings match what the server emits (server/ssh/server/server.go:140, server/ssh/session/errors.go:11, and the access-denied banner).
  • The store orders rules priority ASC and the first active match decides, so the corrected hint matches the server.
  • Patterns are anchored (\A(?:…)\z).
  • The identity-mode case cleans up in LIFO order on contexts that outlive the test.
  • Each openNativeSSH call reads its own output buffer.

To request another review round, comment /review.

Comment thread tests/environment/firewall.go Resolved
otavio added 5 commits October 8, 2026 11:43
Two tests created a tag with the same inline request, and the licensing tests
spelled out the access-denied check (log mark, the banner, the server's log
line) that the firewall tests need too. createTag now sits next to tagDevice,
and requireAccessDenied next to dialForBanner.
The rule form told users that higher priority values are evaluated first. The
server lists a namespace's rules by priority ascending and lets the first match
decide, so a user following the hint would order their rules backwards and a
deny meant to win would lose to the allow it was meant to override.
The firewall rule types live in the private cloud module, which the tests
module cannot import, so the harness carries its own request shape. A rule's
response holds tag objects where the request holds tag names, which is why
CreateFirewallRule reads back only the ID.

EditSSHAccessMode lets one stack serve a case in each access mode: a legacy
namespace can switch to identity and back, where building a second stack
would cost minutes.
Each case asserts the wrapped error text next to the server's log line,
because the server logs "blocked by a firewall rule" both for a rule that
denies and for an evaluation that failed (a bad pattern, a missing license).
Only the error text tells a rule's deny apart.

The source address a rule matches is read from the session the client opens
first, not hardcoded: Docker's port publishing decides what the server sees,
and it differs between rootful and rootless daemons.

Every match case has a miss case beside it, and the two priority cases add
their rules in opposite orders, so neither a matcher that ignores its field nor
evaluation in insertion order passes.

Refs: shellhub-io/team#243
The native client logs in once before the rule exists, so the refusal that
follows can only come from the rule the console created.

Killing the native clients moves to the file's afterEach: a third describe
needed the same cleanup the other two each repeated.

Refs: shellhub-io/team#243
@otavio
otavio force-pushed the test/e2e-firewall branch from c1eab01 to 49d3fa6 Compare October 8, 2026 14:45
@otavio
otavio force-pushed the fix/license-accept-fail-open branch from 7955e5b to fe74a86 Compare October 8, 2026 14:45
@otavio
otavio removed this pull request from stack #7380 October 8, 2026 15:58
@otavio

otavio commented Oct 8, 2026

Copy link
Copy Markdown
Member Author

Consolidated into #7390, which carries this PR's commits so the whole stack reviews and merges together.

@otavio otavio closed this Oct 8, 2026
@otavio
otavio deleted the test/e2e-firewall branch October 8, 2026 16:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant