Skip to content

feat: Native AOT tool packages and release pipeline (PR 3 of 3) - #193

Merged
HandyS11 merged 15 commits into
developfrom
feat/aot-packaging
Sep 15, 2026
Merged

HandyS11 merged 15 commits into
developfrom
feat/aot-packaging

Conversation

@HandyS11

@HandyS11 HandyS11 commented Sep 15, 2026 •

Copy link
Copy Markdown
Owner

Summary

Last of three PRs from the Native AOT design (docs/superpowers/specs/2026-09-14-native-aot-design.md). ProjGraph.Cli and ProjGraph.Mcp now ship as RID-specific Native AOT .NET tool packages, with a framework-dependent any fallback. Every native package is smoke-tested on its own platform before a release pushes anything. Installation is unchanged for users (dotnet tool install -g ProjGraph.Cli, dnx ProjGraph.Mcp@x.y.z): the SDK picks the right package.

  • Tool projects.
    • Both set PublishAot=true and ToolPackageRuntimeIdentifiers=win-x64;linux-x64;linux-arm64;linux-musl-x64;linux-musl-arm64;osx-arm64;any.
    • The MCP server is no longer self-contained or single-file.
    • Native debug symbols and tool symbol packages are no longer packed.
  • One smoke path everywhere.
    • .github/scripts/native-tool-smoke.sh builds, runs dotnet pack -r <rid>, installs both tools with dotnet tool install from the local packages, and runs Tests.Smoke.Aot against the JIT build of the same commit.
    • The composite action .github/actions/native-tool-smoke runs it on the runner, or inside mcr.microsoft.com/dotnet/sdk:10.0-alpine for linux-musl-*.
    • The aot-smoke job uses it for linux-x64. The job id is unchanged.
  • pack.yml (new, reusable).
    • pack-native runs one job per RID: windows-latest, ubuntu-latest, ubuntu-24.04-arm, and macos-latest, with the musl legs in Alpine.
    • pack-portable packs the 6 libraries, the 2 pointer packages and the 2 any packages, then smoke-tests the any packages.
    • Every job checks its exact package set with verify-packages.sh.
    • It also runs on PRs that touch packaging, so this PR runs it.
  • publish.yml: prepare → pack → publish.
    • publish checks all 22 packages, then pushes everything except the pointers.
    • It waits until NuGet.org lists every tool sub-package. The IDs are read from the pointers' DotnetToolSettings.xml, and the step fails before any push if a pointer lists a package that wasn't built.
    • Only then does it push the pointers, create the Release, and submit to the MCP Registry.
    • The version comes from the tag via -p:Version; Directory.Build.props is no longer edited.
    • The server.json version is stamped before packing.
    • Permissions are scoped per job, jobs have timeouts, and the tag format is checked on push too.
  • Smoke suite.
    • NativeBuildGuardTests fail if a "native" build is really a JIT apphost or a .dll.
    • MCP server stderr is captured into the test output.
  • CLI. The IL3050 suppression is narrowed to a CreateApp helper.
  • Docs. The CLI and MCP READMEs and the DocFX getting-started page now state the SDK requirement and the native platforms. CLAUDE.md, CONTRIBUTING.md and ARCHITECTURE.md cover the tool packages, every RID sync point, and the new release flow.

The branch also carries the implementation plan (docs/superpowers/plans/2026-09-14-native-aot-pr3-packaging-release.md).

Results (local, linux-x64, SDK 10.0.401)

  • Package sizes: ProjGraph.Cli.linux-x64 ~14.7 MB, ProjGraph.Mcp.linux-x64 ~17.1 MB, ProjGraph.Cli.any ~8.2 MB, pointer packages ~32 KB. Without excluding native symbols, the CLI package would be 32.3 MB.
  • Smoke results, each from a clean clone:
    • The smoke script passed 30/30 on linux-x64.
    • It passed 30/30 for linux-musl-x64 in the Alpine container, using the action's exact docker run command.
    • The pack-portable commands passed 28/28 against the any packages.
  • Installation behaviour:
    • On glibc, dotnet tool install resolves projgraph.cli.linux-x64, and the command is a symlink to the native binary with the exec bit set.
    • On Alpine, it resolves linux-musl-x64.
    • A missing native package fails the install instead of falling back.
  • Existing suites: build has 0 warnings and dotnet format is clean. 1,178 tests pass and Tests.Smoke.Aot reports 30 skipped.
  • actionlint is clean on ci.yml, pack.yml and publish.yml, and shellcheck is clean on all 5 scripts.
  • CI on this PR: all six pack-native legs passed the 30-test smoke suite on the installed packages (win-x64, linux-x64, linux-arm64, linux-musl-x64, linux-musl-arm64, osx-arm64), along with pack-portable (28/28), aot-smoke, and build ×3.

Deviations from the spec

  1. Two extra native RIDs: linux-musl-x64 and linux-musl-arm64. Without them, dotnet tool install and dnx on Alpine resolve the glibc linux-x64 package instead of any, because the RID graph maps musl to glibc. The tool then fails with projgraph: not found; I reproduced this in sdk:10.0-alpine. The release grows from 18 to 22 packages.
  2. pack.yml is a reusable workflow that is also PR-triggered on packaging paths. Otherwise the first Windows, macOS, Arm64 or musl native build would happen on a release tag.
  3. The release smoke installs the tools with dotnet tool install instead of extracting executables from the .nupkg. This tests the pointer's RID resolution and the exec bit. The PR 2 note about running chmod +x on extracted binaries no longer applies. The smoke NuGet config lists the local packages plus NuGet.org, and on Windows the suite runs the .exe files in the tool store (see below).
  4. aot-smoke uses the same pack → install → smoke path instead of dotnet publish.
  5. Native debug symbols aren't packaged (CopyOutputSymbolsToPublishDirectory=false when PublishAot=true). They made the CLI package 32.3 MB instead of 14.7 MB.
  6. IncludeSymbols=false on both tool projects. A pointer package fails to pack a symbols package (NU5017). The six library .snupkg files still ship.
  7. Pointer packages travel as their own pointers artifact, so push order is decided by directory, not by file name.
  8. pack-portable smoke-tests the any packages too, with the native-build guard filtered out.
  9. On Windows, the MCP reference is the JIT apphost ProjGraph.Mcp.exe. The MCP SDK wraps stdio commands in cmd.exe /c, which is fragile with a quoted dotnet.exe host path.
  10. Tests.Integration.Mcp's McpServerProcess now launches the framework-dependent apphost from bin/<cfg>/net10.0/. Without that change, four MCP integration tests fail.
  11. PR 2 review follow-ups are included: the native-build guard, MCP stderr capture, the narrowed IL3050 suppression, split pack steps, and TRX upload on failure.
  12. Plain JIT builds now get AOT feature switches in their runtimeconfig.json, because PublishAot is set in the csproj. This affects the smoke reference and dotnet run. The any packages don't get them and are tested separately.
  13. The MCP server.json version is stamped before packing, not only in publish, via set-server-json-version.sh. Every ProjGraph.Mcp* package embeds .mcp/server.json, so the spec's sed-in-publish would have shipped 0.4.0 inside all 8 MCP packages.
  14. The NuGet.org wait list is read from the pointer packages by pointer-package-ids.sh, and publish fails before any push if a listed package wasn't built. A RID added only to ToolPackageRuntimeIdentifiers therefore stops the release instead of shipping a pointer whose package for that RID doesn't exist.

Found during execution

  • Fallback smoke: unzip -d artifacts/fallback/cli can't create two missing folder levels, which would have failed every release's fallback smoke. The folder is now created first.
  • Package count: the plan's count of the release package folder was wrong (16); the correct number is 20.
  • Stale server.json version in the packaged MCP server.json: deviation 13.
  • RID list in five places with no check against what the pointers list: deviation 14, plus documented sync points in ARCHITECTURE.md and both csproj files.
  • First CI run of this PR (osx-arm64): dotnet tool install always downloads microsoft.netcore.app.host.osx-x64 for RID-specific packages, because the SDK reads tools/any as a pre-net6 framework. The smoke config had no NuGet.org source, so that download failed. NuGet.org is now listed after the local packages; no ProjGraph package with the CI version exists there.
  • First CI run of this PR (win-x64): the SDK writes projgraph.cmd launchers, not .exe files. The Windows smoke run now uses the executables in the tool store, which also makes NativeBuildGuardTests meaningful on Windows.
  • Shell hook: a local shell hook rewrote dotnet commands inside heredocs while I prototyped. Every file was written through the editor, and the tree is checked for dtk/rtk.

Known follow-ups (not blocking)

  • Missing server.json: set-server-json-version.sh exits 0 if server.json is missing. That can't happen in CI today, and a one-line existence check fixes it.
  • Alpine image: pin the Alpine SDK image tag.
  • Stale plan blocks: two of the plan's code and text blocks (Task 3 csproj, Task 6 Step 5 docs) lag the committed files.

Maintainer checklist (after merge; these publish public packages or change settings)

  • NuGet.org API key scope: confirm the NUGET_API_KEY glob covers the 14 new ProjGraph.* package IDs. If it lists explicit IDs, the first native push fails.
  • Make aot-smoke required on develop (still pending from PR 2; the ruleset has no required status checks yet).
  • RC dry run: push a pre-release tag such as v1.2.0-rc.1 on the merge commit. The publish log must show:
    • holds the 20 expected packages and holds the 2 expected packages;
    • Every package the pointers list is present;
    • the package pushes;
    • is listed on NuGet.org for all 14 IDs;
    • then the pointer pushes, the Release, and the MCP Registry.
  • Install checks with the RC, on each native RID:
    • dotnet tool install -g ProjGraph.Cli --version <rc>, then projgraph --help.
    • (printf '%s\n' '{"jsonrpc":"2.0","id":1,"method":"initialize","params":{"protocolVersion":"2025-06-18","capabilities":{},"clientInfo":{"name":"rc-check","version":"1.0"}}}'; sleep 10) | dnx ProjGraph.Mcp@<rc> --yes must print a serverInfo.version starting with <rc>.
    • On Linux and macOS, ls -l ~/.dotnet/tools/projgraph must link into …projgraph.cli.<rid>….
    • For musl: docker run --rm mcr.microsoft.com/dotnet/sdk:10.0-alpine sh -c 'dotnet tool install -g ProjGraph.Cli --version <rc> && ~/.dotnet/tools/projgraph --help'.
  • Fallback: on an unlisted platform (Intel Mac osx-x64 or Windows on Arm win-arm64), the install must resolve projgraph.cli.any and run.
  • MCP Registry: curl -s "https://registry.modelcontextprotocol.io/v0/servers?search=io.github.HandyS11/projgraph" must list the RC version.
  • Transient publish failures: prefer "Re-run failed jobs", which reuses the run's packages. A new dispatch should run from the tag ref.

Exit criteria (spec, PR 3)

  • pack-native is green on all six RIDs and pack-portable is green on this PR, together with build ×3 and aot-smoke.
  • After merge: the RC dry run publishes all 22 packages in the specified order.
  • After merge: dnx and dotnet tool install work on each native RID and on a fallback platform.
  • After merge: the MCP Registry entry resolves.

🤖 Generated with Claude Code

HandyS11 and others added 13 commits September 15, 2026 00:15
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Classic Info-Zip unzip (shipped by ubuntu-latest) can only auto-create
one missing directory level under -d. artifacts/fallback/{cli,mcp}
needs two (fallback and cli/mcp), so both unzip calls in the
"Smoke test the fallback packages" step failed with "checkdir: cannot
create extraction directory", leaving all 28 smoke tests failing with
FileNotFoundException instead of passing. Create artifacts/fallback
first so unzip only has to create the one remaining level.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Add pointer-package-ids.sh, which prints the IDs each pointer package
lists in its DotnetToolSettings.xml. publish now fails before any push
if a pointer lists a package that wasn't built, and it waits on
NuGet.org for exactly those IDs instead of a hand-written list.

Also harden the release workflows from the final review:
- publish: read-only top-level permissions, write permissions only on
  the publish job, job timeouts, tag validation for pushed tags, and
  re-run guidance in the workflow_dispatch comment.
- pack: fallback smoke TRX upload on failure, wider pull request
  paths, and a concurrency group keyed on the release tag.
- set-server-json-version.sh prints its error when no version matches.
- wait-for-nuget.sh caps each request at 30 seconds.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
ARCHITECTURE.md names the four places a RID lives, says publish derives
its wait list from the pointer packages, and updates the CI line and
the release flow (server.json stamps, NuGet.org validation wait). Both
tool project comments point at the other RID lists. CONTRIBUTING.md
warns that a local native pack ships a stale publish folder.

The PR 3 plan's workflow, action, and script blocks now match the
committed files, and it records deviation 14.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Copilot AI lite review requested due to automatic review settings September 15, 2026 00:34

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

pack.yml uses actions/checkout with ref: ${{ inputs.ref }} even on PR-triggered runs where inputs.ref is unset, risking the workflow running against the wrong revision.

Get a fresh assessment by requesting another Copilot review.

Pull request overview

This PR (3/3) completes the Native AOT packaging/release work by turning ProjGraph.Cli and ProjGraph.Mcp into RID-specific Native AOT .NET tool packages (plus a framework-dependent any fallback), adding a single “pack → install → smoke” path across CI and releases, and wiring a safer publish order (push sub-packages first, wait for NuGet indexing, then push pointer packages).

Changes:

  • Add smoke-suite guards for “native really is native” and capture MCP server stderr into failing test output.
  • Switch CLI and MCP tool projects to PublishAot=true + ToolPackageRuntimeIdentifiers=... and update integration tests/docs accordingly.
  • Introduce pack.yml + packaging scripts and restructure publish.yml to build/pack/push in a deterministic, release-safe order.
File summaries
File Description
tests/ProjGraph.Tests.Smoke.Aot/NativeBuildGuardTests.cs New guard tests that fail if “native” points at a JIT apphost or .dll.
tests/ProjGraph.Tests.Smoke.Aot/McpParityTests.cs Writes captured MCP stderr to xUnit output to aid diagnosis on failures.
tests/ProjGraph.Tests.Smoke.Aot/Helpers/McpServerPair.cs Captures native/reference server stderr lines via MCP stdio transport options.
tests/ProjGraph.Tests.Integration.Mcp/Helpers/McpServerProcess.cs Updates integration helper to launch the framework-dependent MCP apphost from bin/<cfg>/<tfm>/.
src/ProjGraph.Mcp/README.md Documents SDK requirement and which platforms get native vs fallback builds.
src/ProjGraph.Mcp/ProjGraph.Mcp.csproj Enables Native AOT tool packaging and disables symbols/publish symbols for pointer/tool packaging correctness/size.
src/ProjGraph.Cli/README.md Documents SDK requirement and native vs fallback platform behavior.
src/ProjGraph.Cli/ProjGraph.Cli.csproj Enables Native AOT tool packaging (RID list, symbols/symbol-copy behavior).
src/ProjGraph.Cli/Program.cs Narrows IL3050 suppression to the CommandApp construction helper.
docs/superpowers/plans/2026-09-14-native-aot-pr3-packaging-release.md Adds the implementation plan detailing the workflow/scripts and release flow.
docfx/guides/getting-started.md Adds Native AOT platform notes and fallback behavior to getting started docs.
CONTRIBUTING.md Updates smoke-test documentation to the package-based smoke script/action path.
CLAUDE.md Updates tool packaging and release documentation and smoke test project description.
ARCHITECTURE.md Documents new pack/publish workflow structure and tool packaging invariants/sync points.
.github/workflows/publish.yml Restructures to prepare → pack → publish, pushes sub-packages first, waits for NuGet indexing, then pushes pointers, then releases/registry publish.
.github/workflows/pack.yml New reusable workflow to build/smoke native RID packages and pack portable/pointer/fallback packages.
.github/workflows/ci.yml Switches aot-smoke to use the composite action (pack → install → smoke).
.github/scripts/wait-for-nuget.sh New script to poll NuGet flat container until expected versions are listed.
.github/scripts/verify-packages.sh New script to assert exact package set presence (guard against missing/extra packages).
.github/scripts/set-server-json-version.sh New script to stamp server.json version before packing/publishing.
.github/scripts/pointer-package-ids.sh New script to derive the NuGet wait/verify list from pointer packages’ DotnetToolSettings.xml.
.github/scripts/native-tool-smoke.sh New script implementing the unified per-RID pack/install/smoke flow.
.github/actions/native-tool-smoke/action.yml New composite action to run the smoke script on runners (and Alpine container for musl).
Review details

Suppressed comments (1)

.github/workflows/pack.yml:101

  • Same issue as above: in pull_request runs inputs.ref is unset, so this checkout may not be pinned to the PR commit. Add a fallback (e.g. github.sha) so pack-portable runs against the correct code when the workflow is PR-triggered.
      - uses: actions/checkout@v7
        with:
          ref: ${{ inputs.ref }}

  • Files reviewed: 22/23 changed files
  • Comments generated: 1
  • Review effort level: Lite

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment on lines +69 to +72
- uses: actions/checkout@v7
with:
ref: ${{ inputs.ref }}

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Made explicit in 71d1496: both checkouts now use ref: ${{ inputs.ref || github.sha }}, with a comment saying which ref each trigger gets.

The previous behaviour was already correct. When ref is empty, actions/checkout uses its default, and for pull_request that's the same merge commit as github.sha. The first run of this PR confirms it: every pack-native leg ran ./.github/actions/native-tool-smoke, which exists only on this branch. The change only puts that intent in the file.

HandyS11 and others added 2 commits September 15, 2026 03:03
dotnet tool install on macOS arm64 downloads microsoft.netcore.app.host.osx-x64
for RID-specific packages (the SDK reads tools/any as a pre-net6 framework), so
the smoke NuGet config now lists NuGet.org after the local packages. On Windows
the tool path holds .cmd launchers, so the smoke suite runs the executables
installed in the tool store.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@HandyS11
HandyS11 merged commit 233d899 into develop Sep 15, 2026
14 checks passed
@HandyS11
HandyS11 deleted the feat/aot-packaging branch September 15, 2026 09:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants