Commit 5628029
perf: Add FSST vs Zstd vs Zstrong string compression benchmark
Summary:
Add a focused benchmark comparing FSST, Zstd, and Zstrong on representative high-cardinality string datasets: URLs, log messages, and UUID-like strings. The benchmark reports compression ratios, compressed sizes, encode throughput, and decode throughput for the same generated inputs.
Current opt benchmark results from `buck2 run 'fbcode//mode/opt' fbcode//dwio/nimble/encodings/benchmarks:fsst_benchmark`:
Compression ratios, higher is better:
`URLs`: FSST 5.17x, Zstd 37.76x, Zstrong 37.46x
`Log Messages`: FSST 4.13x, Zstd 15.41x, Zstrong 15.52x
`UUIDs`: FSST 1.80x, Zstd 2.23x, Zstrong 2.04x
Compressed sizes in bytes:
`URLs`: raw 7,873,015, FSST 1,521,449, Zstd 208,527, Zstrong 210,176
`Log Messages`: raw 10,302,360, FSST 2,492,908, Zstd 668,545, Zstrong 663,962
`UUIDs`: raw 3,993,314, FSST 2,218,804, Zstd 1,787,045, Zstrong 1,953,354
Encode time per iteration:
`URLs`: FSST 15.75ms, Zstd 9.45ms, Zstrong 11.43ms
`Log Messages`: FSST 29.89ms, Zstd 15.43ms, Zstrong 17.94ms
`UUIDs`: FSST 22.63ms, Zstd 33.40ms, Zstrong 49.30ms
Decode time per iteration:
`URLs`: FSST 3.18ms, Zstd 5.39ms, Zstrong 5.96ms
`Log Messages`: FSST 5.54ms, Zstd 6.83ms, Zstrong 7.80ms
`UUIDs`: FSST 12.21ms, Zstd 6.88ms, Zstrong 8.47ms
Takeaway: FSST provides moderate compression and fast decode on repeated URL/log-style strings. Zstd/Zstrong produce much smaller payloads for highly redundant URLs/logs, while UUID-like strings are a less favorable FSST workload.
Reviewed By: duxiao1212
Differential Revision: D1068783731 parent 28849b3 commit 5628029
1 file changed
Lines changed: 606 additions & 0 deletions
0 commit comments