feat(services/s3): support sha256 checksum algorithm#7829
Open
wolfv wants to merge 1 commit into
Open
Conversation
Add SHA256 as a selectable checksum algorithm for S3 uploads, alongside the existing CRC32C and MD5 options. When configured, the SHA256 digest of the body is computed and sent via the x-amz-checksum-sha256 header for single uploads, and as the per-part ChecksumSHA256 element when completing multipart uploads. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
dosubot
Bot
added
size:M
wolfv
force-pushed
the
feat/s3-sha256-checksum
branch
from
a50f725 to
a105eaf
Compare
dosubot
Bot
added
size:M
Member
|
Thank you for working on this! My hope is that once we have #6817, we don't need to implement this for every service. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Which issue does this PR close?
Closes #.
Rationale for this change
S3 supports several checksum algorithms for verifying upload integrity (e.g. for buckets with Object Lock). OpenDAL's S3 service already supports
crc32candmd5, but notsha256, which is one of the most commonly used and is accepted by S3 for both single and multipart uploads.What changes are included in this PR?
Sha256variant to the internalChecksumAlgorithmenum, mapping to thex-amz-checksum-sha256header.format_sha256_iterhelper (using thesha2crate) incalculate_checksum."sha256"for thechecksum_algorithmconfig / builder option.ChecksumSHA256element when completing multipart uploads.Are there any user-facing changes?
Yes.
"sha256"is now a valid value for the S3checksum_algorithmoption. Builder and config docs are updated. No breaking changes.AI Usage Statement
This PR was prepared with the assistance of Claude Code (Claude Opus 4.8). All changes were reviewed by the author.