Follow-up: making the custom disk cache actually help (~840 MB/s on 10G/NVMe)

[tjjh89017]

Quick follow-up to #7764, where adding a cache in front of our raw-disk storage made things slower (350-400 MB/s vs 450-600 without, network thoughtput).

(EZIO writes to a raw partition, not a filesystem. Torrent "files" are hex disk offsets, blocks in a piece are physically contiguous. Target is 10G+ LAN + NVMe.)

Why the first cache was slower: it was a shared map + mutex (plus a store_buffer for read-after-write ordering). With aio_threads disk threads all hitting one lock per block, it was a serialization point. The lock ate all of them.

The fix was partitioning instead of locking: Route every block of a piece to one worker thread: hash (storage, piece) (deliberately not the offset) to pick a partition, one thread owns one partition.

• No mutex anywhere: a partition is only touched by its owner, so map/LRU/stats need no locking.
• Read-after-write ordering without store_buffer: same piece always lands on the same thread, so async_read naturally queues behind async_write. Deleted the whole store_buffer.

The reason offset can't go into the hash: async_hash reads a piece's blocks straight from the cache, so scattering them across partitions reintroduces the race you just removed. Keeping a whole piece thread-local is what keeps it lock-free.

Result on 1-on-1 (--cache-size 4096 --aio-threads 16, 60 GiB, NVMe both ends): download rate sustains ~840 MB/s, peaks ~870, 98-100% hit rate. Flush-to-disk is slower (~540 MiB/s) and that's the point, the cache lets the network run ahead of the disk.

Code: https://github.com/tjjh89017/ezio (raw_disk_io.cpp + the lock-free cache).

Thanks @arvidn again always.

[arvidn]

do you shard at piece level, or at some possibly higher granularity if pieces are small?
The main challenge, I would expect, is to still compute piece- block hashes in separate threads. I think it's important to not mix that into the I/O threads. The hash jobs will need to pin the buffers, and for v1 torrent hashes, scan contiguous ranges of pieces regularly.

Doesn't that force a mutex anyway? I suppose each shard could have its own mutex though.

[tjjh89017]

We shard it at the Piece-level. Each piece goes to one and specific worker thread by hash(storage_id, piece).

About piece hashing,
The same thread that owns a partition also runs async_hash for its pieces, and reads the blocks from its own local cache, or read it from the storage implementation. Also, we force it to hash a piece before doing any next operation. The hash job will handle the buffer itself, if the data is not in the cache, we will need to read from the disk again. (Every worker thread will only have one hash job on going, and allocate 16KiB when it's being executed.)

And because of the sharding method, Piece A's write/hash/read will always be on the same thread and ordered (write always first, hash 2nd, read last; this is guaranteed in bittorrent protocol and your implementation, we only read and send the piece after the hash is correct, hashing is always after writing the whole piece)

Each shard/shard cache belongs to specific thread id. (Thread A owns Shard A, Cache A)
In the thread, everything is owned by itself, we don't need mutex here.

store_buffer and buffer_pool are global.
Because reading X is always after writing X in the same thread, we just remove the store_buffer, we will not have any chacne to have to read something not in the disk or cache (write through model)
PS: async_read doesn't read the cache in main thread, but always in worker thread to guarantee the order.

Buffer_pool allocate and free are all in main thread, worker thread only access the buf data.
When we want to release the disk_buffer_holder, we will std::move the buffer_holder back to main thread to ensure only main thread will handle the lifecycle of buffer_pool.
The main thread is the single thread, and the only thread which can call async_* to dispatch the jobs.

According those parts.
Everything in async_* before I post it to my worker thread, is single threaded and still in main thread only. (single thread, lock-free)
Everything in my worker thread is thread local memory region or owned by itself, only specific thread will access/write it. (shard, fully independent, lock-free)
Everything related to buffer_pool and disk_buffer_holder is handled in main thread (Allocateing in async_* before I post to other thread, or std::move the buffer_holder back to main thread to handle freeing.) (single thread, lock-free)

Hope this help you to get the concept of my implementation

[tjjh89017]

By the way, this design has some limitation.
Most of settings are immutable after starting the libtorrent session.
That is really hard to handle the thread count change and the cache size change. (We handle the cache size change, but not aio_threads)

And we reuse the deprecated settings cache_size to determine the cache size in disk_interface instance