Harden installer: verify via GitHub build attestation instead of curl | sh#201
Conversation
Addresses untitaker#198. Anchor the installer's integrity outside the (mutable) release assets: - Enable `github-attestations` in dist so each release artifact gets a Sigstore-backed SLSA attestation recorded in GitHub's transparency log. - scripts/install.sh now downloads hyperlink-installer.sh to a file and runs `gh attestation verify` on it before executing, instead of piping curl straight into sh. Generated-by: Claude Opus 4.8 (1M context)
Enabling `github-attestations` in dist-workspace.toml requires the generated release workflow to be regenerated (the `plan` job's out-of-date check caught this). Adds the `attestations`/`id-token` write permissions and the `actions/attest-build-provenance@v2` step that dist emits for attested builds. Generated-by: Claude Opus 4.8 (1M context)
|
I think using cargo dist's feature is a good idea, but it's pretty clear to me that the output of this contribution hasn't been reviewed by a human at all. Please fix this PR, and don't reply with more AI-generated comments. If I have to go through more churn reviewing your contributions, I don't think it's worth my time, I have a claude max plan myself. |
First of all let me tell you that the whole comment here was manually written by me. No agents were I finished testing my (previously Draft) PR - and wanted to let you know that it is ready to actual review now - previously it was more of direction, and I hope I was clear in expressing it and showing that this is a quick draft. I will explain in more detail why I could not do it before, but the most important is to guide on how you can verify that I actually run and tested it - as I was initially planning. The change is now quite well tested - you can see artifficial 0.2.2 release https://github.com/potiuk/hyperlink/releases generated automatically with the modified workflows. My agents helped to quickly understand how to do it, what to push how to upgrade version and guided me during tesing. You can see the branch generated here https://github.com/potiuk/hyperlink/commits/attestation-release-test/ - so that you can see how i tested it. Also the output of the workflows that built the release and created attestation for it can be found here https://github.com/potiuk/hyperlink/commits/attestation-release-test/. Unfortunately, the build is red, I did not want to create a new package to test publishing to Those are the things I could test is that attestation works: 
Then I tested that installation in the new script works as well: 
I also tested what happens when 
I am sorry for sed, and vi - I am an old school, bald gay in my 50s and I keep on coming back to my old habits with using vi (not even vim) when i am editing files for quick testing, and In the second case I just modified the --repo (back to your name) and it failed because it could not pull the attestation - i have different version that you have not released yet, so in reality it would be different error, but the script will behave similarly. Of course using TRAP is somewhat old-school as well to delete the temporary file, so you might have other ideas how to do it better. I am happy to listen to any comments and review now - when I undrafted it, there are about a million of ways it could be done. But I am absolutely happy to adjust to your expectations. Just a small suggestion. Just telling "fix it" does not really provide any useful guidance. The best reviews I do (and I do quite a lot of them, even if they are recently mostly agentic assisted) are when I clearly specify what - and most importantly "why" should be fixed. Providing enough context goes a long way, especially if someone wants to help me to solve my security problems. And yeah, I rarely comment when author explicitly marked something as a Draft - maybe it's just me but at least among the open-source mainteiners it's a clear sign "not ready for review". Even the button to "undraft" such PR is called "Make it ready for review". But yeah, others might have different conventions, so maybe it was my mistake to assume that Draft is "not ready". Also BTW (That's I think the forth BTW) - More explanation about the reason why I left the PR in a Draft state (hoping that you will not be tempted to review it, and that you will not be distracted by a fact that I just did it to learn and see how it can be done and it was certainly not fully reviewed - not even mentioning any testing. I knew (my agent told me) that in order to test it I will not only have to generate release.yaml having to install locally dist 0.30.2, but also i will have to create an artifficial release on GH, because there is no other way to test attestation generation and verification easily. So yes - what you saw was - quickly genereated. Including (if you care to look at the history of my pushes) some manual release.yaml override when the initial draft failed - because I simply did not have dist locally installed in the 0.30.2 version - version you have pinned and I was travelling and had very weak internet and I knew it will be a bit of an adventure and some MBs to download. And boy how much of an adventure it was. I am very security conscious, so I did not want to do what happens in the actions of yours - where you just pipe the script to | sh to install it without verifying it's SHA as well (that would be pretty ironic when the whole PR is precisely to fix non-SHA chacked download of this sort, so I left it more as a direction, and finally could use brew, tap-keg based installlation which downloaded about 1GB of dependencies - including old llm version. After using my agent to figure out quickly how to do it - i did it finally where I could download some 2GB - which would have taken me few hours if I tried to do it where I was before: 
So that might finally after all those words written (again - let me remind you - no agents were BTW. While doing testing I also noticed that your workflows generate tons of warnings about deprecated Node20 - which is very, very deprecated (it's already EOL and stoppped receiving security issues), so my suggestion would be for you to upgrade it to Node 24. Luckily my agent helped me very quickly to confirm what I suspected based on my 20+ years of programming experience and almost single-handedly developing one of the most complex CI workflows for Apache Airflow that I've ever laid my hands and eyes on tells. That you need to upgrade your dist to the latest version to get new versions of the actions (which are also EOL and deprecated on their own). As a member of ASF security committee - seeing EOL versions of tools used in CI causes me to immmediately have goose-bumps, so I would strongly, strongly recommmend you to attempt to upgrade it. Actually I would even offer to open PR by my agents to do it and do some testing with it, but I am afraid I will be again scolded and told that I should fix my Draft version of PR because I will have an opportunity to test it properly, you seem to be pretty allergic to the fact that someone openly says that they are using agents to save a lot of time - and rather than spending time on writing manually long messages (Usually my hand-written commit messages were way longer and more detailed that what the agent writes me - I review what the agent writes of course, I do not accept it blindly. If you wish, you can check some of my commits in Airflow from like - 2/3 years ago when there was no agents. Those description by agents are I think an improvement for a number of people - because they - much better - reflect what I wanted to write. As an ADHD person I have a tendency to describe everything, do a lot of digressions when I described my commits, and some people complain that I write too long messages. I found out Agents and AI is a blessing - becuase usually they can concisely describe what I wanted to say - without me struggling what I should, and what I should not write, and how much additional information to give to describe full context. That's my version of ADHD, sorry for that. In the collapsed section, you can find out what my agent has to tell it - I am hiding it so that you do not see it accidentally. Why the warnings with Node 20 and how to fix itHere's the key finding — the actions are entirely dist-controlled, and bumping dist is the clean fix: So the right way to upgrade the actions isn't to hand-edit release.yml (the dist-generate CI check would revert it) — it's a one-line bump of cargo-dist-version to 0.32.0 in dist-workspace.toml, then dist generate. dist 0.32.0 emits all-current, Node-24-compatible actions (its own tested versions), which resolves the deprecation cleanly. Why this over the alternative ([dist.github-action-commits] to pin individual actions on dist 0.30.2): that works but makes you responsible for keeping the pins current and risks mismatching what dist's template expects. Bumping dist lets dist own it. Again - happy to take reviews now - if there is something that I can improve here - feel free. And BTW (5) - I am also happy to use my agents to help you to upgrade to newer, safer dist. Quoting what usually my agents tell me "Just say a word" . |
It's not a review. I pushed back on reviewing an AI-generated change that you yourself didn't review. And I'm asserting you didn't based on your past behavior in this repo. I use AI myself and regularly deal with AI-generated output at work. That doesn't mean i'm willing to spend the same kind of energy, outside of work.
My response isn't a matter of principle. You opened issues without openly stating they were automaticaly created, on the third duplicate it became obvious. There was no such disclaimer, and i feel like my time has been wasted. On the PR itself, I did give directional feedback (yes, that cargo dist feature is what we should use) but I also was very explicit that i'm not going to put more effort into it until a human looked through it. Again, this was me managing my energy based on my past interactions with you. If i didn't explicitly state that, I strongly suspect I would've had a long conversation with your agent until all issues were actually addressed. I think i've been very patient with your conduct in this repo, and it seems you feel entitled to me spending time on this change (that you yourself haven't reviewed) because it's very important to the ASF. I think that attitude is corrosive to the open source community and if you want to continue to engage you need to change it. to the actual PR review:
otherwise lgtm |
|
Thanks — both addressed:
And you're right about the bigger thing. The duplicate issues were a mistake in my automation that I should have caught and disclosed as auto-created — that wasted your time, and I'm sorry. I apologized earlier and I'm happy to apologize again; no entitlement intended, it's your repo and your time. I've already fixed that in the automation I use, so it shouldn't recur. No hard feelings on my side at all — and honestly, I understand the frustration first-hand. On Airflow we've suddenly gone past 550 open PRs, a lot of them AI-generated, and the churn of reviewing changes the author didn't really look at themselves is exhausting. So I get exactly where you're coming from. I'll keep my changes here human-reviewed and disclosed. And to be fully transparent: yes, these words were put together with AI assistance — but I read and reviewed every line before sending (as the footer below says), and they express exactly what I want to say, just shorter, clearer, and without my usual typos. The intent is entirely mine. Drafted-by: Claude Code (Opus 4.8); reviewed by @potiuk before posting |
|
If you are interested - PR apache/infrastructure-actions#973 that will make sure attribution is given on all outgoing messages (BTW. It was also Agent generated as you might guess) . |
|
And once again apologies for the mistake - it's simply, we need to somehow make our life easier and when you single-handedly manage 80 conversations, or 100 PRs to merge from dependabot a week - you have to rely on automation, and the automation might simply have errors - whether it's agentically run or hand-coded, no big difference. And - at least in my - and a number of people in ASF - case, particularly those who co-create https://github.com/apache/airflow-steward - agentic SKILLs that allow us to manage to influx of agentic skills, human is always behind it - taking responsibility for those mistakes. And those will happen with automation. No matter if there is agentic or deterministic automation. |
|
ok this looks good to me. can you document in the README how one would disable the attestation? I think this will be relevant to forgejo users to which the next release will basically be a breaking change (as they probably don't have gh cli or tokens in the runner image) -- but i also just want it there so i can hedge against bugs. I was considering the idea of automatically skipping attestation when the tokens aren't available, but it seems to undermine the entire point of doing this. Also happy to accept a contribution about upgrading node images etc. |
yes i understand this perspective but i generally see a pattern where people try to push these kinds of issues down into individual maintainers of their deptree. But it seems like a game of whack-a-mole to me, and creates (duplicated) work for everybody. I generally think this effort would better be invested into better packaging tooling. Side note, I am generally suspicious of GHA package management. I think the tooling to build secure actions, and to use them securely, is abysmal. You see the cost here: We had to invent our own attestation mechanism built from sticks and stones for something that should have been completely solved by GitHub e2e. Why can I not release an action with bundled release artifacts where attestation just works OOTB? I am starting to believe it's best to abandon third-party actions entirely and use direct invocation of I have also found this issue: axodotdev/cargo-dist#1147 -- if you have a lot of rust-based actions like this (probably not), maybe fixing this first will make the next PR like this one simpler. |
We all are. https://nesbitt.io/2025/12/06/github-actions-package-manager.html - post of my friend from security team of OSSF. It's getting better though - we are about to get access to https://github.com/github-early-access/actions-locked-dependencies - which seems to be somewhat a progres. And, yes attestations are only protecting against some cases - not all of them - but at least we have a trace of where things were build and what was used. So far we are using this agentically coded (and iterated about a 100 times) tool to verify the packages - https://github.com/apache/infrastructure-actions/tree/main/utils/verify_action_build - it attempts to rebuild actions from sources where needed, verify a bunch of edge cases, about a million ways how npm packages can be built (because of course you can have both sources and minified javascript in and the only way to know whether the minified javascript is coming from the sources is to rebuild. Every few days we find another edge case :) Fun. Luckily with agentic SKILLS, it's mostly "Analyze #42" and it explains why reproducibility did not work and produces a new PR adding millionth first variant of building stuff. |
|
Suggestion - about the conditional use. Maybe we can simply base the "no attestation => success" of the action on the presence of I think that would solve your problem of making sure thins will work outside of GH ecosystem without any changes? WDYT @untitaker ? BTW. If you see why writing comments/messages with AI is better for me ... I already updated the message 4 times with small interpunction and typos.... Being fast typer with ADHD, but at the same time being perfectionist, does not help too much. |
|
I think this may be too implicit, I really don't mind the breaking change for Forgejo users. It's important to me that people have a way to opt out of this if the attestation is broken for any number of reasons (maybe some server is down, or attestation is flaky, who knows) |
Base attestation verification on whether the `gh` CLI is present rather than an explicit opt-out. On GitHub-hosted runners `gh` is available and the check runs automatically; on runners without it (e.g. Forgejo or other non-GitHub CI) verification is skipped with a warning so the action works out of the box. Two overrides: - HYPERLINK_SKIP_ATTESTATION=1 skips verification entirely, even when `gh` is present. - HYPERLINK_FORCE_ATTESTATION=1 requires verification and fails when `gh` is missing. Document the behaviour in the README GitHub action section. Generated-by: Claude Opus 4.8 (1M context)
|
I went ahead to show you how it might look like (just pushed) - but i can go back easily (this is also nice thing about agents how cheap such iterations are |
|
I think that's fair, two envvars is fine by me |
|
Yeah. Looked at the docs and It's quite straightforward :D |
|
BTW. It's so much nicer to have even such small design sessions with real people than with agents - but in OSS we were usually quite lonely anyway - but that one reminded me how important it is to create opportunities to do it regularly. Thanks for reminding me that - and sorry again for the initial mishaps :) |
|
no worries, i understand, and what you said about loneliness in oss rings true. i pushed 0.3.0 and will be able to tell in a minute if this actually works e2e (i understand that you tested this already though) |
Yep 🤞 |
|
ok, this is tricky https://github.com/untitaker/hyperlink/actions/runs/28299539927/job/83845520638 i thought GH_TOKEN was provided automatically by actions? can't find a setting to turn that on. |
|
hmm okay, this is now released and functional in 0.3.1. i think the testing had a gap, but it's all good. |
|
Thank you - yes - I think I did not look at that workflow. |
2 participants
What
Addresses #198 — verify the installer against a GitHub build attestation instead of
curl … | sh.github-attestations = trueindist-workspace.toml→ each release artifact gets a Sigstore-backed SLSA attestation recorded in GitHub's transparency log.scripts/install.shdownloadshyperlink-installer.shto a file and runsgh attestation verifyon it before executing.Why
Per the discussion in #198: a checksum shipped beside the script in the same (mutable) release proves nothing — both can be swapped together. A GitHub attestation lives in the transparency log, independent of the release assets, so it can't.
Draft / open questions
gh attestation verifyneedsGH_TOKEN/GITHUB_TOKENin the step env — may need documenting for consumers (or fetching the attestation bundle offline).github-attestationsis enabled.hyperlink-installer.shitself is attested (vs only the archives); may need agithub-attestations-filterstweak.Opened as a draft for discussion — happy to iterate.
Generated-by: Claude Opus 4.8 (1M context)