Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions doc/modules/ROOT/nav.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,7 @@
** xref:5.buffers/5b.composite-buffers.adoc[Composite Buffers]
** xref:5.buffers/5c.system-io-integration.adoc[System I/O Integration]
** xref:5.buffers/5d.buffer-algo.adoc[Buffer Algorithms]
** xref:5.buffers/5e.rationale.adoc[Rationale]
* xref:6.streams/6.intro.adoc[Streams]
** xref:6.streams/6a.concepts.adoc[Stream Concepts]
** xref:6.streams/6b.wrappers.adoc[Type-Erased Wrappers]
Expand Down
7 changes: 4 additions & 3 deletions doc/modules/ROOT/pages/4.coroutines/4g.composition.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -144,9 +144,10 @@ loop that bound it.

In the snippet above nothing dangles: `items` outlives the cpp:when_all[]. The
hazard is what a caller could do -- passing a temporary would leave the stored
task holding a reference to it. Awaiting a task immediately, as `read_all` on
xref:5.buffers/5b.composite-buffers.adoc[Composite Buffers] does, is the case where a
reference parameter is safe.
task holding a reference to it. Awaiting a task immediately, as function `read_all`
from section
xref:5.buffers/5d.buffer-algo.adoc#_consuming_buffers[Buffer Algorithms] does,
is the case where a reference parameter is safe.
====

=== Racing a Deadline
Expand Down
7 changes: 5 additions & 2 deletions doc/modules/ROOT/pages/5.buffers/5.intro.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -29,5 +29,8 @@ designed with the following criteria in mind:
that describe sequences of those handles.
* xref:5.buffers/5c.system-io-integration.adoc[System I/O Integration] --
how a sequence maps onto the structures that native I/O systems expect.
* xref:5.buffers/5d.buffer-algo.adoc[Buffer Algorithms] -- the algorithms
that measure, copy, and consume buffers.
* xref:5.buffers/5d.buffer-algo.adoc[Buffer Algorithms] -- counting bytes,
slicing, copying, and consuming those sequences.
* xref:5.buffers/5e.rationale.adoc[Rationale] -- why the handles use `void*`,
why sequences are concepts, and why those concepts require bidirectional
iteration.
10 changes: 5 additions & 5 deletions doc/modules/ROOT/pages/5.buffers/5a.contiguous-buffers.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -11,11 +11,11 @@
:page-mode: explanation

The most efficient way to provide storage for bytes is a contiguous region: an array.
While it is not always possible to efficiently compose a full message in a single contiguous storage,
While it is not always possible to efficiently compose a full message in a single contiguous region,
it is certainly a good default. Capy offers two types for this purpose:

* cpp:const_buffer[] -- a handle to a contiguous sequence of bytes. I/O operations will never modify these bytes.

Check warning on line 17 in doc/modules/ROOT/pages/5.buffers/5a.contiguous-buffers.adoc

View workflow job for this annotation

GitHub Actions / Antora Docs

[vale_adoc Capy.SimpleTense] Avoid needless future/perfect tense — prefer present simple (style guide C4): 'will '.
* cpp:mutable_buffer[] -- a handle to a contiguous storage for bytes; I/O operations will fill it.
* cpp:mutable_buffer[] -- a handle to a contiguous region for sttoring bytes; I/O operations will fill it.

Check warning on line 18 in doc/modules/ROOT/pages/5.buffers/5a.contiguous-buffers.adoc

View workflow job for this annotation

GitHub Actions / Antora Docs

[vale_adoc Capy.SimpleTense] Avoid needless future/perfect tense — prefer present simple (style guide C4): 'will '.

Both types are present in header `<boost/capy/buffers.hpp>`.

Expand Down Expand Up @@ -58,7 +58,7 @@
----
<1> C array of `char` is fine.
<2> Array of cpp:byte[std::byte] is fine.
<3> Vector of `int` is fine: the integers will be interpreted as bytes during transmission.

Check warning on line 61 in doc/modules/ROOT/pages/5.buffers/5a.contiguous-buffers.adoc

View workflow job for this annotation

GitHub Actions / Antora Docs

[vale_adoc Capy.SimpleTense] Avoid needless future/perfect tense — prefer present simple (style guide C4): 'will '.


== Lifetime Management
Expand All @@ -66,29 +66,29 @@
The contract for read and write operations in Capy is:

* The user manages the lifetime of the byte storage.
* I/O operations receive a handle to the storage and assume that it will be available for the entire duration

Check warning on line 69 in doc/modules/ROOT/pages/5.buffers/5a.contiguous-buffers.adoc

View workflow job for this annotation

GitHub Actions / Antora Docs

[vale_adoc Capy.SimpleTense] Avoid needless future/perfect tense — prefer present simple (style guide C4): 'will '.
of the asynchronous operations.

[source,cpp]
----
include::example$snippets/5a_contiguous_buffers.cpp[tag=good_management,indent=0]
----
<1> OK: The read will be performed at unpredictable time, but because afterwards
<1> OK: The read will be performed at an unpredictable time, but because afterwards

Check warning on line 76 in doc/modules/ROOT/pages/5.buffers/5a.contiguous-buffers.adoc

View workflow job for this annotation

GitHub Actions / Antora Docs

[vale_adoc Capy.SimpleTense] Avoid needless future/perfect tense — prefer present simple (style guide C4): 'will '.
the coroutine needs to be resumed, its frame (containing the array) is guaranteed
to outlive the read operation.

[source,cpp]
----
include::example$snippets/5a_contiguous_buffers.cpp[tag=bad_management,indent=0]
----
<1> Bug: the awaitable rather than `co_await`-ed is returned to be used later.
<1> Bug: the awaitable, rather than being `co_await`ed, is returned to be used later.
At that later time the array will no longer be alive.

Check warning on line 85 in doc/modules/ROOT/pages/5.buffers/5a.contiguous-buffers.adoc

View workflow job for this annotation

GitHub Actions / Antora Docs

[vale_adoc Capy.SimpleTense] Avoid needless future/perfect tense — prefer present simple (style guide C4): 'will '.


== Mutable versus Const Buffers

cpp:mutable_buffer[] represents the storage for bytes, intended to be written. The value
of the bytes is irrelevant, because they will be overwritten. In fact, they may have _indeterminate value_.

Check warning on line 91 in doc/modules/ROOT/pages/5.buffers/5a.contiguous-buffers.adoc

View workflow job for this annotation

GitHub Actions / Antora Docs

[vale_adoc Capy.SimpleTense] Avoid needless future/perfect tense — prefer present simple (style guide C4): 'will '.

cpp:const_buffer[] represents bytes to be read. It is not only storage, it is also values.

Expand All @@ -108,10 +108,10 @@
include::example$snippets/5a_contiguous_buffers.cpp[tag=buffer_conversion,indent=0]
----
<1> The storage starts with indeterminate byte values (intended to be overwritten).
<2> Bug: will read indeterminate byte values, which is undefined behavior.

Check warning on line 111 in doc/modules/ROOT/pages/5.buffers/5a.contiguous-buffers.adoc

View workflow job for this annotation

GitHub Actions / Antora Docs

[vale_adoc Capy.SimpleTense] Avoid needless future/perfect tense — prefer present simple (style guide C4): 'will '.


== Buffer Processing interface
== Buffer Processing Interface

Algorithms operating on buffers use the following interface provided by both
cpp:const_buffer[] and cpp:mutable_buffer[].
Expand Down
98 changes: 14 additions & 84 deletions doc/modules/ROOT/pages/5.buffers/5b.composite-buffers.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -11,9 +11,9 @@
= Composite Buffers
:page-mode: explanation

In real protocols often a message to be transmitted is composed of a couple of chunks prepared separately
In real protocols, a message to be transmitted is often composed of a couple of chunks prepared separately
in separate contiguous regions of storage. For instance, you may separately get message headers, the payload,
and the message checksum. Trying to get them into a single contiguous storage would require the user to do
and the message checksum. Trying to get them into a single contiguous region would require the user to do
unnecessary copying.

In order to address this use case, all read and write operations in Capy allow you to provide ranges
Expand Down Expand Up @@ -61,7 +61,7 @@ Treating a single contiguous buffer as a one-element sequence is intentional, to
Internally, it is treated as a single-element range.

While a structure looks two-dimensional, the bytes across all contiguous buffers
are interpretted and transmitted as a single sequence: first the bytes from the first contiguous buffer,
are interpreted and transmitted as a single sequence: first the bytes from the first contiguous buffer,
then the bytes from the second contiguous buffer, and so on.

The mechanism for handling ranges of buffers assumes that the number of elements will be around four.
Expand All @@ -88,7 +88,7 @@ Concept cpp:ConstBufferSequence[] tests if the type models a const buffer sequen
suitable for write algorithms. It can be one of:

* cpp:const_buffer[],
* cpp:mutable_buffer[] (with caveat that its data has determinate values),
* cpp:mutable_buffer[] (with the caveat that its data has determinate values),
* a bidirectional range of either cpp:const_buffer[] or cpp:mutable_buffer[].

[source,cpp]
Expand All @@ -99,100 +99,30 @@ include::example$snippets/5b_composite_buffers.cpp[tag=test_const_buffer,indent=
These concepts are present in header `<boost/capy/buffers.hpp>`.


=== Iterating Buffer Sequences
=== Navigating Buffer Sequences

Use `begin()` and `end()` from `<boost/capy/buffers.hpp>`:
Header `<boost/capy/buffers.hpp>` also provides a couple of function objects for navigating buffer sequences.

[source,cpp]
----
include::example$snippets/5b_composite_buffers.cpp[tag=iterate,indent=0]
----

These handle both single buffers, returning pointer-to-self, and ranges, returning standard iterators:

[source,cpp]
----
include::example$snippets/5b_composite_buffers.cpp[tag=begin_end_uniform,indent=0]
----

=== buffer_slice


cpp:buffer_slice[] returns a byte sub-range of a buffer sequence, as a value:
In order to iterate over contiguous buffers themselves (as opposed to bytes),
use cpp:begin[capy::begin] and cpp:end[capy::end]:

[source,cpp]
----
include::example$snippets/5b_composite_buffers.cpp[tag=buffer_slice_include]

include::example$snippets/5b_composite_buffers.cpp[tag=buffer_slice,indent=0]
include::example$snippets/5b_composite_buffers.cpp[tag=iterate,indent=0]
----

cpp:buffer_slice[]`(seq, offset, length)` returns a value that is itself a buffer sequence, so you can pass it to any operation expecting one. Both `offset` and `length` are optional, which makes it a general byte sub-range primitive. Except in the single-buffer case the result borrows `seq`, so the sequence must outlive the slice.
These functions treat single contiguous buffers as a one-element range of contiguous buffers:

Capy creates and manages buffer handles on your behalf. Those are the sub-range a cpp:buffer_slice[] produces, and the descriptors a type-erased stream passes to the OS. Each is valid only for the window its API documents.


=== consuming_buffers

When transferring data incrementally, cpp:consuming_buffers[] is a cursor that tracks progress:

[source,cpp]
----
include::example$snippets/5b_composite_buffers.cpp[tag=consuming_buffers_include]

include::example$snippets/5b_composite_buffers.cpp[tag=read_all,indent=0]
include::example$snippets/5b_composite_buffers.cpp[tag=begin_end_uniform,indent=0]
----

The cursor borrows the underlying sequence and provides:

* cpp:consuming_buffers::data[data]`()` -- buffer sequence view of the remaining bytes
* cpp:consuming_buffers::consume[consume]`(n)` -- advance past `n` transferred bytes, in place

[NOTE]
====
`read_all` takes its stream by reference and its buffers by value. Both are safe
here, because the caller owns the stream and awaits the task immediately.

A coroutine reads its parameters when its body runs, not when the call is
written. A task that is stored and awaited later outlives its call expression,
so a reference parameter can dangle by then. That is why the fan-out example on
xref:4.coroutines/4g.composition.adoc[Concurrent Composition] takes its item by
value: it collects its tasks first, then awaits them together.
====

== Rationale


=== Why `void*` rather than `std::byte`?

Two concrete forces decide it:

* *The platform types already use it.* The OS structures Capy maps onto are `void*` or `char*`: `iovec`'s `iov_base`, `WSABUF`'s `buf`. The pointer field never needs reinterpreting.
* *Callers supply many element types.* Data arrives as `char[]`, `unsigned char[]`, `std::byte[]`, `std::string`, and more. One neutral pointer erases all of them to a single representation. `std::span<std::byte>` would make every caller reinterpret first, and `std::span<void>` is ill-formed.

This is an argument from layout and from caller convenience, not from semantics. "Raw memory" describes `std::byte` just as well as it describes `void*`.



=== Why concepts rather than one span type?

`std::span<std::span<std::byte>>` is the reflexive answer for multiple buffers, and it works. The problem is composition. Joining a two-buffer header to a three-buffer body means building a new five-element array, so every composition allocates:
Function cpp:buffer_length[] says how many contiguous buffers are present in the buffer
sequence. It is equivalent to

[source,cpp]
----
include::example$snippets/5b_composite_buffers.cpp[tag=span_combine,indent=0]
include::example$snippets/5b_composite_buffers.cpp[tag=length_equivalent,indent=0]
----

A concept accepts the composite directly, with no allocation and no overload per shape.

At type-erasure boundaries the tradeoff reverses. Virtual functions need concrete types, so Capy converts to concrete descriptors internally and keeps concepts at the user-facing edge.


=== Why Bidirectional?

The concepts require bidirectional ranges, not merely forward ranges, for two reasons:

1. Some algorithms traverse buffers backwards
2. The slice views from cpp:buffer_slice[] and cpp:consuming_buffers::data[]`()` must adjust the first and last buffers' bounds

If your custom sequence offers only forward iteration, wrap it in a type that provides bidirectional access.
69 changes: 56 additions & 13 deletions doc/modules/ROOT/pages/5.buffers/5d.buffer-algo.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@
:page-mode: explanation
:page-aliases: 5.buffers/5e.algorithms.adoc

== Measuring
== Counting Bytes

cpp:buffer_size[] returns the total number of *bytes* across every buffer in a sequence:

Expand All @@ -20,30 +20,73 @@ cpp:buffer_size[] returns the total number of *bytes* across every buffer in a s
include::example$snippets/5d_buffer_algo.cpp[tag=buffer_size_example,indent=0]
----

cpp:buffer_length[] returns the *number of buffers* in the sequence:

cpp:buffer_empty[] reports whether a sequence carries no data,

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This section lost its example. The buffer_empty_example include was dropped, but the tagged, tested fragment is still in the snippet file. Could you include it again after this paragraph?

[source,cpp]
----
include::example$snippets/5d_buffer_algo.cpp[tag=buffer_empty_example,indent=0]
----

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would it be ok, if I consequently remove the example from the cpp file.
This example contributes nothing to the user's understanding of the library, and at the same time makes the docs longer.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The buffer_size example already contains an empty buffer. Could we add a couple of asserts in that example to handle both functions at the same time?

assert(!capy::buffer_empty(combined));
assert(capy::buffer_empty(buf3));

I suggest this because buffer_empty is the only function without an example on the page. I appreciate streamlining the documentation. I think this satisfies the desire for an example without bloating the page too much.

either because it holds no buffers or because every buffer has size zero.
It is a shorthand for `buffer_size(bufseq) == 0`.

Capy allows passing zero-size buffers to read and write operations.
This is to cover the situations where the size of the buffer is determined dynamically,
based on incoming messages. A protocol may require that a sender first communicates the size of a
message, and then the bytes of the message.
In such protocols, having nothing to send may be correctly
represented by sending the number zero, followed by no bytes.
This would translate into later using a zero-size buffer.

[CAUTION]
====
The following two functions have similar names but return different values:

* cpp:buffer_size[] -- number of bytes across the entire buffer sequence,
* cpp:buffer_length[] -- number of contiguous buffers in the sequence.

A three-buffer sequence of 100 bytes each has a cpp:buffer_size[] of 300 and a cpp:buffer_length[] of 3.
====


== buffer_slice

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The other headings on this page describe tasks ("Counting Bytes", "Copying", "Partial Transfer Loops"), but these two are bare identifiers (also consuming_buffers on line 52). How about == Slicing and == Consuming Buffers? The second one keeps the generated ID _consuming_buffers, so <<_consuming_buffers>> on line 108 still resolves.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If this is ok with you, I would address it in a separate PR. A PR that rewrites the entire Buffer algorithms section.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a small change the preserve formatting while the page is in an intermediate state.


cpp:buffer_slice[] returns a byte sub-range of a buffer sequence, as a value:

[source,cpp]
----
include::example$snippets/5d_buffer_algo.cpp[tag=buffer_length_example,indent=0]
include::example$snippets/5d_buffer_algo.cpp[tag=buffer_slice_include]

include::example$snippets/5d_buffer_algo.cpp[tag=buffer_slice,indent=0]
----

[IMPORTANT]
====
These two names are easy to confuse, so read them as answering different questions:
cpp:buffer_slice[]`(seq, offset, length)` returns a value that is itself a buffer sequence, so you can pass it to any operation expecting one. Both `offset` and `length` are optional, which makes it a general byte sub-range primitive. Except in the single-buffer case the result borrows `seq`, so the sequence must outlive the slice.

* cpp:buffer_size[] -- how many bytes of data? This is what I/O operations care about.
* cpp:buffer_length[] -- how many buffers hold it? This is the element count of the sequence, the analogue of `std::ranges::size`.
Capy creates and manages buffer handles on your behalf. Those are the sub-range a cpp:buffer_slice[] produces, and the descriptors a type-erased stream passes to the OS. Each is valid only for the window its API documents.

A three-buffer sequence of 100 bytes each has a cpp:buffer_size[] of 300 and a cpp:buffer_length[] of 3. You want cpp:buffer_size[] far more often.
====
== consuming_buffers

cpp:buffer_empty[] reports whether a sequence carries no data, either because it holds no buffers or because every buffer has size zero:
When transferring data incrementally, cpp:consuming_buffers[] is a cursor that tracks progress:

[source,cpp]
----
include::example$snippets/5d_buffer_algo.cpp[tag=buffer_empty_example,indent=0]
include::example$snippets/5d_buffer_algo.cpp[tag=consuming_buffers_include]

include::example$snippets/5d_buffer_algo.cpp[tag=read_all,indent=0]
Comment thread
mvandeberg marked this conversation as resolved.
----

The cursor borrows the underlying sequence and provides:

* cpp:consuming_buffers::data[data]`()` -- buffer sequence view of the remaining bytes
* cpp:consuming_buffers::consume[consume]`(n)` -- advance past `n` transferred bytes, in place

[NOTE]
====
`read_all` takes its stream by reference and its buffers by value. Both are safe
here, because the caller owns the stream and awaits the task immediately.

A coroutine reads its parameters when its body runs, not when the call is
written. A task that is stored and awaited later outlives its call expression,
so a reference parameter can dangle by then. That is why the fan-out example on
xref:4.coroutines/4g.composition.adoc[Concurrent Composition] takes its item by
value: it collects its tasks first, then awaits them together.
====

== Copying

cpp:buffer_copy[] copies data from one buffer sequence to another and returns the number of bytes copied:
Expand Down Expand Up @@ -71,7 +114,7 @@ The algorithm fills target buffers in order, reading from source buffers as need

== Partial Transfer Loops

Real transfers move some of the bytes, not all of them. Drive the loop with a cpp:consuming_buffers[] cursor, described on xref:5.buffers/5b.composite-buffers.adoc#_consuming_buffers[consuming_buffers].
Real transfers move some of the bytes, not all of them. Drive the loop with a cpp:consuming_buffers[] cursor, described in <<_consuming_buffers>>.

[source,cpp]
----
Expand Down
41 changes: 41 additions & 0 deletions doc/modules/ROOT/pages/5.buffers/5e.rationale.adoc
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
//
// Copyright (c) 2025 Vinnie Falco (vinnie.falco@gmail.com)
// Copyright (c) 2026 Michael Vandeberg
// Copyright (c) 2026 Andrzej Krzemieński (akrzemi1@gmail.com)
//
// Distributed under the Boost Software License, Version 1.0. (See accompanying
// file LICENSE_1_0.txt or copy at http://www.boost.org/LICENSE_1_0.txt)
//
// Official repository: https://github.com/cppalliance/capy
//
= Rationale
:page-mode: explanation

== Why `void*` rather than `std::byte`?

Two concrete forces decide it:

* *The platform types already use it.* The OS structures Capy maps onto are `void*` or `char*`: `iovec`'s `iov_base`, `WSABUF`'s `buf`. The pointer field never needs reinterpreting.
* *Callers supply many element types.* Data arrives as `char[]`, `unsigned char[]`, `std::byte[]`, `std::string`, and more. One neutral pointer erases all of them to a single representation. `std::span<std::byte>` would make every caller reinterpret first, and `std::span<void>` is ill-formed.

This is an argument from layout and from caller convenience, not from semantics. "Raw memory" describes `std::byte` just as well as it describes `void*`.

== Why concepts rather than one span type?

`std::span<std::span<std::byte>>` is the reflexive answer for multiple buffers, and it works. The problem is composition. Joining a two-buffer header to a three-buffer body means building a new five-element array, so every composition allocates:

[source,cpp]
----
include::example$snippets/5e_rationale.cpp[tag=span_combine,indent=0]
----

A concept accepts the composite directly, with no allocation and no overload per shape.

At type-erasure boundaries the tradeoff reverses. Virtual functions need concrete types, so Capy converts to concrete descriptors internally and keeps concepts at the user-facing edge.

== Why Bidirectional?

The concepts require bidirectional ranges, not merely forward ranges, for two reasons:

1. Some algorithms traverse buffers backwards
2. The slice views from cpp:buffer_slice[] and cpp:consuming_buffers::data[]`()` must adjust the first and last buffers' bounds
Loading
Loading