V2 block hashes#8395
Open
arvidn wants to merge 4 commits into
Open
Conversation
Contributor
There was a problem hiding this comment.
Pull request overview
Note
Copilot was unable to run its full agentic suite in this review.
Adds support for caching/consuming precomputed v2 (SHA-256) block hashes via the storage layer so hash jobs can avoid re-reading blocks from disk, and updates tests accordingly.
Changes:
- Introduce per-piece storage of precomputed v2 block hashes in
pread_storageandmmap_storage, consumed byhash/hash2jobs. - Update disk I/O backends (
pread_disk_io,mmap_disk_io) anddisk_cacheto produce/consume these precomputed hashes and drop them on clear-piece. - Adjust tests: move end-to-end v2 block hash validation to
test_disk_ioand remove cache-local v2 assertions/tests.
Reviewed changes
Copilot reviewed 11 out of 11 changed files in this pull request and generated 6 comments.
Show a summary per file
| File | Description |
|---|---|
| test/test_disk_io.cpp | Adds explicit expected v1/v2 hash verification for async hashing. |
| test/test_disk_cache.cpp | Removes v2 hash assertions/tests that depended on cache-owned v2 hashes; adds explanatory comments. |
| test/disk_cache_test_utils.hpp | Updates piece_entry_params construction to pass the new storage field. |
| src/pread_storage.cpp | Implements storage-backed precomputed v2 hash cache (store/take/drop). |
| src/pread_disk_io.cpp | Consumes storage-backed v2 hashes for hash and adds a precomputed fast-path for hash2. |
| src/mmap_storage.cpp | Implements storage-backed precomputed v2 hash cache (store/take/drop) and stores v1/v2 flags. |
| src/mmap_disk_io.cpp | Computes v2 hashes inline during writes, consumes them during hash/hash2, and drops them on clear-piece. |
| src/disk_cache.cpp | Routes v2 block-hash production/consumption through pread_storage instead of cache-local arrays. |
| include/libtorrent/aux_/pread_storage.hpp | Declares precomputed v2 hash APIs and backing container. |
| include/libtorrent/aux_/mmap_storage.hpp | Declares precomputed v2 hash APIs and backing container + v1/v2 flags. |
| include/libtorrent/aux_/disk_cache.hpp | Updates cache entry/params to carry storage weak_ptr; removes cache-local v2 hash snapshot plumbing. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment on lines
+1058
to
+1059
| int const ret = | ||
| j->storage->hash2(m_settings, h, len, a.piece, a.offset, file_mode, j->flags, j->error); |
Comment on lines
+101
to
+109
| std::optional<sha256_hash> pread_storage::take_precomputed_v2_block( | ||
| piece_index_t const piece, int const block) | ||
| { | ||
| std::lock_guard<std::mutex> l(m_precomputed_v2_mutex); | ||
| auto const it = m_precomputed_v2.find(piece); | ||
| if (it == m_precomputed_v2.end()) return std::nullopt; | ||
| auto& entry = it->second; | ||
| if (block >= entry.present.size() || !entry.present.get_bit(block)) return std::nullopt; | ||
| sha256_hash const h = entry.hashes[block]; |
Comment on lines
+142
to
+157
| std::vector<lt::sha256_hash> expected_v2; | ||
| lt::sha1_hash expected_v1; | ||
| if (need_v2) | ||
| { | ||
| expected_v2.resize(blocks_in_piece); | ||
| int const v2_blocks = fs.blocks_in_piece2(p); | ||
| int const v2_size = fs.piece_size2(p); | ||
| for (int b = 0; b < v2_blocks; ++b) | ||
| { | ||
| int const off = b * block_size; | ||
| int const bsize = std::min(block_size, v2_size - off); | ||
| lt::hasher256 hh; | ||
| hh.update({buffer.data() + off, bsize}); | ||
| expected_v2[b] = hh.final(); | ||
| } | ||
| } |
Comment on lines
+371
to
+376
| auto pc = storage->take_precomputed_v2(loc.piece); | ||
| int const to_copy = | ||
| std::min(int(job2.block_hashes.size()), int(pc.hashes.size())); | ||
| for (int idx = 0; idx < to_copy; ++idx) | ||
| if (idx < pc.present.size() && pc.present.get_bit(idx)) | ||
| job2.block_hashes[idx] = pc.hashes[idx]; |
Comment on lines
+1044
to
+1052
| int const blk = a.offset / default_block_size; | ||
| if (auto h = j->storage->take_precomputed_v2_block(a.piece, blk)) | ||
| { | ||
| a.piece_hash2 = *h; | ||
| std::int64_t const read_time = total_microseconds(clock_type::now() - start_time); | ||
| m_stats_counters.inc_stats_counter(counters::disk_hash_time, read_time); | ||
| m_stats_counters.inc_stats_counter(counters::disk_job_time, read_time); | ||
| return {}; | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.