The cuda::experimental::cuco::hyperloglog template exposes a single customization point today: the hash functor. The HyperLogLog algorithm has three points that tend to vary together:
- Hash function (algorithm + seed).
- Bit slicing (which bits become the register index; how rho is
extracted from the remainder).
- Finalizer (HLL++ vs Composite estimator; with different
reduction state requirements).
Different sketch ecosystems pin all three to specific values. Apache Spark, Apache Datasketches, Redis, BigQuery, etc. each define their own combination. A cudax consumer who needs sketches that are binary-compatible with an external library has to fork the impl today, because the hash slot alone doesn't reach into the slicing or the finalizer.
Proposal: replace the bare Hash template parameter with a Policy parameter that bundles all three customization points. Cudax ships one default policy reproducing today's behaviour; downstream consumers supply their own policies for ecosystem-specific conventions without touching cudax kernels.
The change is a rename-in-place at the API boundary (hyperloglog<Key, MR, Scope, Hash> -> hyperloglog<Key, MR, Scope, Policy>); default-instantiated users (hyperloglog<Key>) are unaffected at source and behaviour level.
This is preparation work for enabling GPU-accelerated HLL in Apache Datasketches.
The cuda::experimental::cuco::hyperloglog template exposes a single customization point today: the hash functor. The HyperLogLog algorithm has three points that tend to vary together:
extracted from the remainder).
reduction state requirements).
Different sketch ecosystems pin all three to specific values. Apache Spark, Apache Datasketches, Redis, BigQuery, etc. each define their own combination. A cudax consumer who needs sketches that are binary-compatible with an external library has to fork the impl today, because the hash slot alone doesn't reach into the slicing or the finalizer.
Proposal: replace the bare Hash template parameter with a Policy parameter that bundles all three customization points. Cudax ships one default policy reproducing today's behaviour; downstream consumers supply their own policies for ecosystem-specific conventions without touching cudax kernels.
The change is a rename-in-place at the API boundary (
hyperloglog<Key, MR, Scope, Hash>->hyperloglog<Key, MR, Scope, Policy>); default-instantiated users (hyperloglog<Key>) are unaffected at source and behaviour level.This is preparation work for enabling GPU-accelerated HLL in Apache Datasketches.