Repository navigation
docs: byte counting goes to buffer algos page #447
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: develop
Are you sure you want to change the base?
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -11,7 +11,7 @@ | |
| :page-mode: explanation | ||
| :page-aliases: 5.buffers/5e.algorithms.adoc | ||
|
|
||
| == Measuring | ||
| == Counting Bytes | ||
|
|
||
| cpp:buffer_size[] returns the total number of *bytes* across every buffer in a sequence: | ||
|
|
||
|
|
@@ -20,30 +20,73 @@ cpp:buffer_size[] returns the total number of *bytes* across every buffer in a s | |
| include::example$snippets/5d_buffer_algo.cpp[tag=buffer_size_example,indent=0] | ||
| ---- | ||
|
|
||
| cpp:buffer_length[] returns the *number of buffers* in the sequence: | ||
|
|
||
| cpp:buffer_empty[] reports whether a sequence carries no data, | ||
| either because it holds no buffers or because every buffer has size zero. | ||
| It is a shorthand for `buffer_size(bufseq) == 0`. | ||
|
|
||
| Capy allows passing zero-size buffers to read and write operations. | ||
| This is to cover the situations where the size of the buffer is determined dynamically, | ||
| based on incoming messages. A protocol may require that a sender first communicates the size of a | ||
| message, and then the bytes of the message. | ||
| In such protocols, having nothing to send may be correctly | ||
| represented by sending the number zero, followed by no bytes. | ||
| This would translate into later using a zero-size buffer. | ||
|
|
||
| [CAUTION] | ||
| ==== | ||
| The following two functions have similar names but return different values: | ||
|
|
||
| * cpp:buffer_size[] -- number of bytes across the entire buffer sequence, | ||
| * cpp:buffer_length[] -- number of contiguous buffers in the sequence. | ||
|
|
||
| A three-buffer sequence of 100 bytes each has a cpp:buffer_size[] of 300 and a cpp:buffer_length[] of 3. | ||
| ==== | ||
|
|
||
|
|
||
| == buffer_slice | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. The other headings on this page describe tasks ("Counting Bytes", "Copying", "Partial Transfer Loops"), but these two are bare identifiers (also
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. If this is ok with you, I would address it in a separate PR. A PR that rewrites the entire Buffer algorithms section.
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This is a small change the preserve formatting while the page is in an intermediate state. |
||
|
|
||
| cpp:buffer_slice[] returns a byte sub-range of a buffer sequence, as a value: | ||
|
|
||
| [source,cpp] | ||
| ---- | ||
| include::example$snippets/5d_buffer_algo.cpp[tag=buffer_length_example,indent=0] | ||
| include::example$snippets/5d_buffer_algo.cpp[tag=buffer_slice_include] | ||
|
|
||
| include::example$snippets/5d_buffer_algo.cpp[tag=buffer_slice,indent=0] | ||
| ---- | ||
|
|
||
| [IMPORTANT] | ||
| ==== | ||
| These two names are easy to confuse, so read them as answering different questions: | ||
| cpp:buffer_slice[]`(seq, offset, length)` returns a value that is itself a buffer sequence, so you can pass it to any operation expecting one. Both `offset` and `length` are optional, which makes it a general byte sub-range primitive. Except in the single-buffer case the result borrows `seq`, so the sequence must outlive the slice. | ||
|
|
||
| * cpp:buffer_size[] -- how many bytes of data? This is what I/O operations care about. | ||
| * cpp:buffer_length[] -- how many buffers hold it? This is the element count of the sequence, the analogue of `std::ranges::size`. | ||
| Capy creates and manages buffer handles on your behalf. Those are the sub-range a cpp:buffer_slice[] produces, and the descriptors a type-erased stream passes to the OS. Each is valid only for the window its API documents. | ||
|
|
||
| A three-buffer sequence of 100 bytes each has a cpp:buffer_size[] of 300 and a cpp:buffer_length[] of 3. You want cpp:buffer_size[] far more often. | ||
| ==== | ||
| == consuming_buffers | ||
|
|
||
| cpp:buffer_empty[] reports whether a sequence carries no data, either because it holds no buffers or because every buffer has size zero: | ||
| When transferring data incrementally, cpp:consuming_buffers[] is a cursor that tracks progress: | ||
|
|
||
| [source,cpp] | ||
| ---- | ||
| include::example$snippets/5d_buffer_algo.cpp[tag=buffer_empty_example,indent=0] | ||
| include::example$snippets/5d_buffer_algo.cpp[tag=consuming_buffers_include] | ||
|
|
||
| include::example$snippets/5d_buffer_algo.cpp[tag=read_all,indent=0] | ||
|
mvandeberg marked this conversation as resolved.
|
||
| ---- | ||
|
|
||
| The cursor borrows the underlying sequence and provides: | ||
|
|
||
| * cpp:consuming_buffers::data[data]`()` -- buffer sequence view of the remaining bytes | ||
| * cpp:consuming_buffers::consume[consume]`(n)` -- advance past `n` transferred bytes, in place | ||
|
|
||
| [NOTE] | ||
| ==== | ||
| `read_all` takes its stream by reference and its buffers by value. Both are safe | ||
| here, because the caller owns the stream and awaits the task immediately. | ||
|
|
||
| A coroutine reads its parameters when its body runs, not when the call is | ||
| written. A task that is stored and awaited later outlives its call expression, | ||
| so a reference parameter can dangle by then. That is why the fan-out example on | ||
| xref:4.coroutines/4g.composition.adoc[Concurrent Composition] takes its item by | ||
| value: it collects its tasks first, then awaits them together. | ||
| ==== | ||
|
|
||
| == Copying | ||
|
|
||
| cpp:buffer_copy[] copies data from one buffer sequence to another and returns the number of bytes copied: | ||
|
|
@@ -71,7 +114,7 @@ The algorithm fills target buffers in order, reading from source buffers as need | |
|
|
||
| == Partial Transfer Loops | ||
|
|
||
| Real transfers move some of the bytes, not all of them. Drive the loop with a cpp:consuming_buffers[] cursor, described on xref:5.buffers/5b.composite-buffers.adoc#_consuming_buffers[consuming_buffers]. | ||
| Real transfers move some of the bytes, not all of them. Drive the loop with a cpp:consuming_buffers[] cursor, described in <<_consuming_buffers>>. | ||
|
|
||
| [source,cpp] | ||
| ---- | ||
|
|
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,41 @@ | ||
| // | ||
| // Copyright (c) 2025 Vinnie Falco (vinnie.falco@gmail.com) | ||
| // Copyright (c) 2026 Michael Vandeberg | ||
| // Copyright (c) 2026 Andrzej Krzemieński (akrzemi1@gmail.com) | ||
| // | ||
| // Distributed under the Boost Software License, Version 1.0. (See accompanying | ||
| // file LICENSE_1_0.txt or copy at http://www.boost.org/LICENSE_1_0.txt) | ||
| // | ||
| // Official repository: https://github.com/cppalliance/capy | ||
| // | ||
| = Rationale | ||
| :page-mode: explanation | ||
|
|
||
| == Why `void*` rather than `std::byte`? | ||
|
|
||
| Two concrete forces decide it: | ||
|
|
||
| * *The platform types already use it.* The OS structures Capy maps onto are `void*` or `char*`: `iovec`'s `iov_base`, `WSABUF`'s `buf`. The pointer field never needs reinterpreting. | ||
| * *Callers supply many element types.* Data arrives as `char[]`, `unsigned char[]`, `std::byte[]`, `std::string`, and more. One neutral pointer erases all of them to a single representation. `std::span<std::byte>` would make every caller reinterpret first, and `std::span<void>` is ill-formed. | ||
|
|
||
| This is an argument from layout and from caller convenience, not from semantics. "Raw memory" describes `std::byte` just as well as it describes `void*`. | ||
|
|
||
| == Why concepts rather than one span type? | ||
|
|
||
| `std::span<std::span<std::byte>>` is the reflexive answer for multiple buffers, and it works. The problem is composition. Joining a two-buffer header to a three-buffer body means building a new five-element array, so every composition allocates: | ||
|
|
||
| [source,cpp] | ||
| ---- | ||
| include::example$snippets/5e_rationale.cpp[tag=span_combine,indent=0] | ||
| ---- | ||
|
|
||
| A concept accepts the composite directly, with no allocation and no overload per shape. | ||
|
|
||
| At type-erasure boundaries the tradeoff reverses. Virtual functions need concrete types, so Capy converts to concrete descriptors internally and keeps concepts at the user-facing edge. | ||
|
|
||
| == Why Bidirectional? | ||
|
|
||
| The concepts require bidirectional ranges, not merely forward ranges, for two reasons: | ||
|
|
||
| 1. Some algorithms traverse buffers backwards | ||
| 2. The slice views from cpp:buffer_slice[] and cpp:consuming_buffers::data[]`()` must adjust the first and last buffers' bounds |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This section lost its example. The
buffer_empty_exampleinclude was dropped, but the tagged, tested fragment is still in the snippet file. Could you include it again after this paragraph?There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Would it be ok, if I consequently remove the example from the cpp file.
This example contributes nothing to the user's understanding of the library, and at the same time makes the docs longer.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The
buffer_sizeexample already contains an empty buffer. Could we add a couple of asserts in that example to handle both functions at the same time?I suggest this because
buffer_emptyis the only function without an example on the page. I appreciate streamlining the documentation. I think this satisfies the desire for an example without bloating the page too much.