Repository navigation
Merge master to resolve conflicts with folbricht/desync#323 - #1
Merged
Merged
Conversation
* Always send the configured client certificate to HTTP stores Go's default client certificate selection sends no certificate when the server's CertificateRequest doesn't list the issuing CA. Set GetClientCertificate so the configured certificate is always presented, as curl and OpenSSL-based clients do. Fixes #443 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Check the response body close error in the TLS test golangci-lint's errcheck only exempts deferred Close calls. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Fixes GO-2026-6617, GO-2026-6612, GO-2026-6611, GO-2026-6610 and GO-2026-6603 in its HTTP/2 implementation. x/net v0.60.0 requires Go 1.26, so the minimum Go version goes from 1.25 to 1.26, and the other golang.org/x modules move to their matching releases.
* Plan assembly up front and rearrange data already in the target Replace the sequencer with a plan that decides where every chunk of the target comes from before anything is written, and runs the resulting steps as a graph with explicit dependencies. Every kind of step declares the range of the target it writes and the range it reads, either the content from before assembly or what other steps write, and the order of the steps is derived from that. A seed whose data file is the target itself, found by os.SameFile, is used in place: chunks already in position are skipped, chunks elsewhere in the file are moved to where they belong, and only the rest comes from the store. Moves are ordered so each reads its source before anything overwrites it. Moves in a dependency cycle hold their source in a buffer, filled on demand by the step about to overwrite it, within the memory left under GOMEMLIMIT; a buffer that doesn't fit makes its move take the chunk from the store. Consecutive chunks that shift together become one move, split again only where a cycle runs through it, and such runs are cloned on filesystems with reflinks. Chunks are taken from a source that clones them before one that copies, and from seeds before the target's own data. Seed indexes are validated against their files before assembly instead of reading back what was written; a seed file that changes during assembly is noticed by its size and modification time, and the chunks of the output that don't match are then written from the store again. A target larger than the output is shrunk after assembly, once its data can't be needed anymore. Closes #312 * Fail a cancelled assembly and describe seed backfill in the help Workers that wait for a step exit without error when the context is cancelled, so an assembly stopped by the dispatcher reported success with an incomplete target, which could then be renamed over the output. Return the context's error when steps remain. The extract help still said a seed modified during extraction could corrupt the target. The target is verified and backfilled from the store in that case now. * Name the algorithms behind in-place assembly * Call the memory that breaks move cycles a stash Buffer also named the scratch memory moves copy through. Stash is what Android's block-based OTA updates call it. splitCycles becomes splitRunsInCycles, it splits runs, not cycles. * Document in-place seeds, seed order and seeds changing mid-extract * Copy duplicates of chunks already in place from the target The self-seed only matches positions after the one it fills, which keeps copies within the target from depending on each other in a cycle. A chunk that's already in place before a later duplicate was never used for it, so the duplicate came from the store. Extracting a Debian cloud image over its version from two weeks earlier fetched 3.4k more chunks than master. Nothing writes over chunks in place, so copying from them doesn't depend on other steps and can't form a cycle. The self-seed now also matches them in either direction, limited to the run of chunks in place, and prefers them over an equally long match elsewhere. * Compare null chunks in the target with zeros instead of hashing them Finding the chunks already in place hashed every position of the target, including the zeros that make up most of a typical disk image. Extracting over a target that already holds the output took 2.7 times the CPU of master. Null chunks are compared with zeros now, which brings it back in line, and speeds up verifying the output after a seed changed as well. * Show progress while checking the target, validating seeds and verifying Planning read the whole target and every seed without any output, which on slow storage like eMMC took half a minute for a 5 GB image. Checking the target for chunks already in place, validating the seeds and verifying the output after a seed changed now each show a progress bar, counted in chunks.
#323 moved fileseed.go to assemble-fileseed.go, renamed NewIndexSeed to NewFileSeed, and removed selfseed.go and sequencer.go. The IndexRegenerationSettings changes are carried over to NewFileSeed, which now takes the regeneration settings as a fourth argument. TestExtractWithNonStaticSeeds was dropped in #323 together with SeedSequencer and stays removed. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
folbricht
changed the title
unlightable
added a commit
that referenced
this pull request
Oct 17, 2026
Manually rebased to highlight an actual merge changeset.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Merges current
masterinto this branch to resolve the conflicts with folbricht#323, which reorganized the assembly code.fileseed.gobecameassemble-fileseed.goandNewIndexSeedwas renamed toNewFileSeed. TheIndexRegenerationSettingschanges are carried over:NewFileSeednow takes the regeneration settings as a fourth argument, overrides min/avg/max from the seed index, andRegenerateIndexuses them. Test callers passIndexRegenerationSettings{}.selfseed.goandsequencer.gowere removed in Plan assembly up front and rearrange data already in the target folbricht/desync#323 (the self-seed now lives inassemble-selfseed.goand implementsSeed), so the edits to them are dropped, including theTODO: selfSeed doesn't have a LongestMatchWithnote.TestExtractWithNonStaticSeedswas removed in Plan assembly up front and rearrange data already in the target folbricht/desync#323 together withSeedSequencer. It stays removed; the corrupted-seed regeneration path is still covered by the--regenerate-invalid-seedscases incmd/desync/extract_test.go.assemble_test.gouseschunkers.DefaultChunkSizeMaxin the testify-converted code from master.go build,go vetandgo test ./...pass, and the generateddocs/clipages are unchanged. Merging this into yourmasterupdates folbricht#442.🤖 Generated with Claude Code