Skip to content

Commit 480cdd2

Browse files
committed
update benchmark results
1 parent c34518a commit 480cdd2

2 files changed

Lines changed: 24 additions & 23 deletions

File tree

bench/README.md

Lines changed: 15 additions & 15 deletions
Original file line numberDiff line numberDiff line change
@@ -25,21 +25,21 @@ ruby bench/report.rb --update-readme
2525

2626
| Message size | inproc | ipc | tcp |
2727
|---|---|---|---|
28-
| 128 B | 1.17M msg/s / 150 MB/s | 400.0k msg/s / 51.2 MB/s | 390.6k msg/s / 50.0 MB/s |
29-
| 512 B | 1.23M msg/s / 629 MB/s | 296.9k msg/s / 152 MB/s | 288.5k msg/s / 148 MB/s |
30-
| 2 KiB | 1.37M msg/s / 2.80 GB/s | 234.3k msg/s / 480 MB/s | 225.6k msg/s / 462 MB/s |
31-
| 8 KiB | 1.38M msg/s / 11.27 GB/s | 105.2k msg/s / 862 MB/s | 106.4k msg/s / 871 MB/s |
32-
| 32 KiB | 1.36M msg/s / 44.52 GB/s | 39.4k msg/s / 1.29 GB/s | 35.9k msg/s / 1.18 GB/s |
28+
| 128 B | 1.64M msg/s / 210 MB/s | 293.5k msg/s / 37.6 MB/s | 307.8k msg/s / 39.4 MB/s |
29+
| 512 B | 1.63M msg/s / 833 MB/s | 253.2k msg/s / 130 MB/s | 231.7k msg/s / 119 MB/s |
30+
| 2 KiB | 2.01M msg/s / 4.12 GB/s | 211.1k msg/s / 432 MB/s | 180.1k msg/s / 369 MB/s |
31+
| 8 KiB | 2.00M msg/s / 16.41 GB/s | 101.0k msg/s / 827 MB/s | 94.5k msg/s / 774 MB/s |
32+
| 32 KiB | 1.99M msg/s / 65.14 GB/s | 34.4k msg/s / 1.13 GB/s | 32.4k msg/s / 1.06 GB/s |
3333

3434
### 3 peers
3535

3636
| Message size | inproc | ipc | tcp |
3737
|---|---|---|---|
38-
| 128 B | 1.39M msg/s / 178 MB/s | 388.7k msg/s / 49.8 MB/s | 382.3k msg/s / 48.9 MB/s |
39-
| 512 B | 1.34M msg/s / 684 MB/s | 283.2k msg/s / 145 MB/s | 277.7k msg/s / 142 MB/s |
40-
| 2 KiB | 1.37M msg/s / 2.80 GB/s | 224.4k msg/s / 460 MB/s | 218.0k msg/s / 446 MB/s |
41-
| 8 KiB | 1.36M msg/s / 11.15 GB/s | 104.2k msg/s / 854 MB/s | 104.1k msg/s / 853 MB/s |
42-
| 32 KiB | 1.36M msg/s / 44.51 GB/s | 36.6k msg/s / 1.20 GB/s | 33.6k msg/s / 1.10 GB/s |
38+
| 128 B | 2.01M msg/s / 257 MB/s | 295.6k msg/s / 37.8 MB/s | 309.8k msg/s / 39.7 MB/s |
39+
| 512 B | 1.98M msg/s / 1.01 GB/s | 221.1k msg/s / 113 MB/s | 227.4k msg/s / 116 MB/s |
40+
| 2 KiB | 2.04M msg/s / 4.19 GB/s | 164.7k msg/s / 337 MB/s | 171.5k msg/s / 351 MB/s |
41+
| 8 KiB | 2.01M msg/s / 16.48 GB/s | 93.7k msg/s / 767 MB/s | 92.8k msg/s / 760 MB/s |
42+
| 32 KiB | 1.98M msg/s / 64.98 GB/s | 32.6k msg/s / 1.07 GB/s | 29.4k msg/s / 964 MB/s |
4343

4444
<!-- END push_pull -->
4545

@@ -58,11 +58,11 @@ Latency is `1 / msgs_s` converted to µs.
5858
<!-- BEGIN req_rep -->
5959
| Message size | inproc | ipc | tcp |
6060
|---|---|---|---|
61-
| 128 B | 8.42 µs | 44.4 µs | 57.7 µs |
62-
| 512 B | 8.43 µs | 49.2 µs | 60.7 µs |
63-
| 2 KiB | 8.06 µs | 51.9 µs | 64.1 µs |
64-
| 8 KiB | 8.14 µs | 57.5 µs | 68.4 µs |
65-
| 32 KiB | 8.04 µs | 71.3 µs | 87.4 µs |
61+
| 128 B | 8.71 µs | 50.6 µs | 64.4 µs |
62+
| 512 B | 8.42 µs | 57.4 µs | 67.6 µs |
63+
| 2 KiB | 8.32 µs | 59.7 µs | 73.4 µs |
64+
| 8 KiB | 8.26 µs | 68.4 µs | 79.8 µs |
65+
| 32 KiB | 8.27 µs | 93.5 µs | 113 µs |
6666

6767
<!-- END req_rep -->
6868

doc/BINDINGS.md

Lines changed: 9 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -24,7 +24,7 @@ between runs.
2424

2525
| Binding | Language / Runtime | How it reaches the wire |
2626
|---|---|---|
27-
| **OMQ** 0.21.0 | MRI Ruby 4.0.2 + YJIT | pure Ruby, Async fibers, no native dep |
27+
| **OMQ** 0.23.1 | MRI Ruby 4.0.2 + YJIT | pure Ruby, Async fibers, no native dep |
2828
| **ffi-rzmq** 2.0.7 | MRI Ruby 4.0.2 + YJIT | FFI → libzmq 4.3.5 |
2929
| **CZTop** 2.0.2 | MRI Ruby 4.0.2 + YJIT | FFI → CZMQ → libzmq 4.3.5 |
3030
| **pyzmq** 26.4.0 | CPython 3.13.5 | Cython → libzmq 4.3.5 |
@@ -35,22 +35,22 @@ between runs.
3535
| Binding | 128 B msg/s | 128 B MB/s | 1024 B msg/s | 1024 B MB/s |
3636
|---|---:|---:|---:|---:|
3737
| JeroMQ (JRuby) | **959,677** | 122.8 | 410,420 | 420.3 |
38-
| OMQ (MRI, pure Ruby) | 350,564 | 44.9 | 217,282 | 222.5 |
38+
| OMQ (MRI, pure Ruby) | 332,030 | 42.5 | 224,148 | 229.5 |
3939
| pyzmq (CPython) | 326,067 | 41.7 | 287,211 | 294.1 |
4040
| CZTop (MRI) | 268,342 | 34.3 | 225,171 | 230.6 |
4141
| ffi-rzmq (MRI) | 39,840 | 5.1 | 32,854 | 33.6 |
4242

4343
### Observations
4444

45-
- **JeroMQ runs away with throughput.** ~2.7× OMQ on small messages, ~1.9× on
45+
- **JeroMQ runs away with throughput.** ~2.9× OMQ on small messages, ~1.8× on
4646
1 KB. The JVM JIT on a tight `send(byte[], 0)` loop against a pure-Java
4747
transport is hard to beat, and JeroMQ is mature (derived from the same
4848
codebase pedigree as libzmq itself).
49-
- **OMQ ties pyzmq on small messages** (326k vs 351k) — a pure-Ruby fiber
49+
- **OMQ edges pyzmq on small messages** (332k vs 326k) — a pure-Ruby fiber
5050
pipeline keeps up with Cython-wrapped libzmq when per-message overhead
5151
dominates. pyzmq pulls ahead at 1 KB because libzmq's C framing is still
5252
faster per byte than Ruby.
53-
- **CZTop is close to OMQ at 1 KB** (230 vs 222 MB/s) — at that size both
53+
- **CZTop matches OMQ at 1 KB** (231 vs 230 MB/s) — at that size both
5454
are bandwidth-bound on loopback, and the FFI-per-call overhead
5555
disappears against the memcpy cost.
5656
- **ffi-rzmq is ~10× slower than everything else.** Not libzmq's fault —
@@ -62,7 +62,7 @@ between runs.
6262

6363
| Binding | 128 B rtt/s | 128 B µs/rtt | 1024 B rtt/s | 1024 B µs/rtt |
6464
|---|---:|---:|---:|---:|
65-
| OMQ (MRI, pure Ruby) | **14,819** | **67.5** | **13,382** | **74.7** |
65+
| OMQ (MRI, pure Ruby) | **14,167** | **70.6** | **12,902** | **77.5** |
6666
| JeroMQ (JRuby) | 14,948 | 66.9 | 13,118 | 76.2 |
6767
| pyzmq (CPython) | 14,232 | 70.3 | 13,925 | 71.8 |
6868
| ffi-rzmq (MRI) | 11,729 | 85.3 | 10,895 | 91.8 |
@@ -82,8 +82,9 @@ between runs.
8282
## Headline takeaways
8383

8484
1. **JeroMQ wins throughput**, by a wide margin on small messages.
85-
2. **OMQ is the fastest Ruby option on both benches** — beats ffi-rzmq and
86-
CZTop on throughput *and* REQ/REP, despite being pure Ruby.
85+
2. **OMQ is the fastest Ruby option overall** — beats ffi-rzmq everywhere,
86+
beats CZTop on small-message throughput and on REQ/REP, ties CZTop at
87+
1 KB throughput. All despite being pure Ruby.
8788
3. **On REQ/REP latency, runtime choice barely matters** — OMQ, JeroMQ and
8889
pyzmq all cluster around 65–75 µs. At that scale the bottleneck is TCP
8990
loopback, not the binding.

0 commit comments

Comments
 (0)