Smoke test: real-clone + leverage end-to-end against a public campaign (nightly tier)

[obey-agent]

Proposal

Add a smoke-tier integration test that exercises camp clone followed by camp leverage against a real public campaign with real submodule project repos. Runs out of band (just test smoke, nightly CI), not on every PR.

Why this is worth adding

Existing tests/integration/leverage_test.go covers the cobra command surface against synthetic alpha/beta projects built by setupLeverageCampaign. That suite catches flag/output regressions but cannot trigger bugs that only show up against real campaigns:

• Submodule resolution against a real meta-repo with heterogeneous git remotes (SSH and HTTPS).
• git blame and git log output against real commit histories (merge commits, mailmap entries, multi-author dedup).
• scc categorization across realistic language mixes rather than the synthetic single Go file per project.
• Time-based leverage scoring against realistic commit-date distributions rather than a single "initial" commit per project.

These classes of bug are exactly the kind of thing PR review cannot catch and that show up only when a real user runs camp leverage for the first time against their own campaign.

Target campaign

https://github.com/lancekrogers/Obey-Agent-Economy is a real public campaign with proper .campaign/, CLAUDE.md, AGENTS.md, and 11 entries in .gitmodules. ~14.5 MB on disk, mix of Go, TypeScript, Solidity, Python, and Shell across the submodules.

Submodule URL types in the campaign as of writing:

Submodule	URL shape	Cloneable in a fresh container?
projects/agent-coordinator	git@github.com:Blockhead-Consulting/agent-coordinator.git	Yes, with SSH→HTTPS rewrite (see below)
projects/agent-inference	same shape	Yes, with rewrite
projects/agent-defi	same shape	Yes, with rewrite
projects/hiero-plugin	same shape	Yes, with rewrite
projects/dashboard	same shape	Yes, with rewrite
projects/contracts	same shape	Yes, with rewrite
projects/cre-risk-router	git@github.com:lancekrogers/cre-risk-router.git	Yes, with rewrite
lib/forge-std	https://github.com/foundry-rs/forge-std	Yes, no rewrite needed
projects/agent-prediction	local filesystem path	No
projects/obey-platform	local filesystem path	No
projects/agent-bags	local filesystem path	No

The smoke test does NOT recurse-init all submodules. It clones the meta-repo, then inits the explicit list of 8 cloneable submodules. The 3 local-path entries are skipped; the test does not depend on upstream campaign cleanup.

Shape

File layout

• New file: tests/integration/leverage_smoke_test.go

• Build tag header:

  //go:build integration && smoke
  // +build integration,smoke

  Combining the two tags means the file builds only when BOTH integration and smoke are passed. The existing just test integration recipe passes only -tags=integration, so the smoke file is excluded from that run.

Recipe

Add to .justfiles/test.just, alongside the existing integration / integration-verbose / integration-run block:

# Run smoke tests against a real public campaign (slow; not for PR loop)
[no-cd]
smoke:
    go test -v -tags=integration,smoke ./tests/integration/... -timeout 15m

# Run a specific smoke test
[no-cd]
smoke-run TEST:
    go test -v -tags=integration,smoke ./tests/integration/... -run "{{TEST}}" -timeout 15m

The smoke recipe calls go test directly rather than going through internal/buildutil since it is not part of the unit-or-integration dashboard. Adding a smoke action to buildutil is a separate, optional follow-up.

CI cadence

GitHub Actions workflow change (separate from this issue's scope but worth flagging): add a nightly cron workflow that runs just test smoke. PR-triggered workflows continue to run only just test integration. The smoke job should not block any PR.

SSH rewriting in the test container

The container has no SSH keys. Rewrite SSH GitHub URLs to HTTPS at the start of the test:

tc.Shell(t, `git config --global url."https://github.com/".insteadOf "git@github.com:"`)

This is scoped to the container's git config and does not affect the host. Anonymous HTTPS reads are rate-limited (60/hr without auth, 5000/hr with a token); for a once-nightly smoke run the anonymous limit is fine. If rate limits become a problem, plumb GITHUB_TOKEN from the workflow into the container and use https://${GITHUB_TOKEN}@github.com/.

Test body sketch

//go:build integration && smoke
// +build integration,smoke

package integration

import (
    "encoding/json"
    "fmt"
    "strings"
    "testing"

    "github.com/stretchr/testify/assert"
    "github.com/stretchr/testify/require"
)

// Pinned to a known-good revision of lancekrogers/Obey-Agent-Economy.
// Update procedure documented in tests/integration/README.md (TODO):
// when this test starts failing because the upstream campaign moved,
// pick a recent commit SHA from `git log --oneline main` on the upstream,
// verify the test passes against it locally, then update this constant.
const obeyAgentEconomySHA = "<pick a recent main SHA at implementation time>"

// Submodules with public-URL entries that can be cloned anonymously
// after the SSH-to-HTTPS rewrite. The three local-path submodules in the
// upstream .gitmodules are deliberately skipped.
var obeyAgentEconomyPublicSubmodules = []string{
    "projects/agent-coordinator",
    "projects/agent-inference",
    "projects/agent-defi",
    "projects/hiero-plugin",
    "projects/dashboard",
    "projects/contracts",
    "projects/cre-risk-router",
    "lib/forge-std",
}

func TestLeverage_SmokeAgainstObeyAgentEconomy(t *testing.T) {
    if !sccAvailable {
        t.Fatal("scc binary not available in container")
    }

    tc := GetSharedContainer(t)

    // Rewrite SSH GitHub URLs to anonymous HTTPS for the container's git config.
    tc.Shell(t, `git config --global url."https://github.com/".insteadOf "git@github.com:"`)

    root := "/campaigns/obey-agent-economy"

    // Clone the meta-repo only (no recurse), then check out the pinned SHA,
    // then init only the cloneable submodules.
    tc.Shell(t, fmt.Sprintf(`set -e
        git clone https://github.com/lancekrogers/Obey-Agent-Economy %s
        cd %s
        git checkout %s
        git submodule update --init %s
    `, root, root, obeyAgentEconomySHA, strings.Join(obeyAgentEconomyPublicSubmodules, " ")))

    // Run leverage and parse the JSON output.
    out, err := tc.RunCampInDir(root, "leverage", "--json")
    require.NoError(t, err, out)

    jsonStart := strings.Index(out, "{")
    require.GreaterOrEqual(t, jsonStart, 0, out)

    var result struct {
        Campaign map[string]any   `json:"campaign"`
        Projects []map[string]any `json:"projects"`
    }
    require.NoError(t, json.Unmarshal([]byte(out[jsonStart:]), &result))

    // Structural assertions only. No numeric value matches because the
    // upstream campaign evolves and we do not want flaky tests on every commit.
    require.NotNil(t, result.Campaign, "campaign score should be present")
    require.GreaterOrEqual(t, len(result.Projects), 3,
        "expected at least 3 projects from the public submodules")

    for _, p := range result.Projects {
        // Field names should match the actual JSON shape produced by the
        // leverage command; verify against `camp leverage --json` output.
        if code, ok := p["code"].(float64); ok {
            assert.Greater(t, code, float64(0),
                "project %q should have non-zero code lines", p["name"])
        }
    }
}

The exact JSON field names should be verified against the current camp leverage --json output during implementation; the sketch above is illustrative.

Tradeoffs

Concern	Mitigation
Runtime (clone + submodule init + per-project scc + blame)	~1 minute. Smoke tier, not PR tier.
Brittleness from upstream changes	Pin to SHA. Structural assertions only, no numeric matches.
Network dependency on GitHub	Acceptable for a nightly tier; required for any real-clone test.
Anonymous rate limits	60/hr without auth is enough for nightly. Plumb GITHUB_TOKEN if needed.
Maintenance: updating the pinned SHA	One-line change when the smoke test starts failing. Document the procedure.
Three local-path submodules in upstream	Skipped explicitly. Test does not require upstream cleanup.

Out of scope for this issue

• Cleanup of the local-path submodules in lancekrogers/Obey-Agent-Economy. The campaign owner decides whether to publish/remove those at their own pace. The smoke test does not depend on it.
• Adding more smoke tests beyond leverage. If clone + leverage works, the harness exists for other clone-based smoke tests later.
• Making the smoke tier run on every PR. Explicitly nightly or manual only.
• Adding a smoke action to internal/buildutil. The direct go test recipe is sufficient.

Acceptance criteria

• tests/integration/leverage_smoke_test.go exists with //go:build integration && smoke tags.
• .justfiles/test.just has smoke and smoke-run recipes that call go test -tags=integration,smoke ./tests/integration/....
• just test smoke passes locally against the pinned SHA.
• just test integration (without smoke tag) does NOT execute the smoke test, confirming the build-tag gate works.
• CI is configured to run smoke nightly or pre-release (not on every PR). Either a new workflow file or an additional cron schedule on an existing one.
• Docs (tests/integration/README.md if it exists, or inline in the smoke file) note the pinned-SHA update procedure.
• No flaky numeric assertions: assertions are structural (non-zero, contains, length-at-least).

Notes for a picker-upper

• Helper functions: GetSharedContainer(t), tc.Shell(t, ...), tc.RunCamp(...), tc.RunCampInDir(dir, ...) all exist in tests/integration/helpers_container.go and are used by the existing leverage integration tests.
• sccAvailable is a package-level var initialized by NewSharedContainer in main_test.go. Existing leverage tests follow the pattern of if !sccAvailable { t.Fatal(...) }.
• The pinned SHA constant should be at the top of the new file (or in a small smoke_pins.go). When updating, run git ls-remote https://github.com/lancekrogers/Obey-Agent-Economy refs/heads/main to get the current main HEAD and verify the test passes against it locally before bumping.
• The 3 local-path submodules in .gitmodules are the only reason this test cannot use plain --recurse-submodules. If the upstream campaign ever swaps those to public URLs (or removes them), simplify the test to git clone --recurse-submodules and remove the explicit submodule list.