Skip to content

Graph analytics catalog: 20 new kernels, and a workspace CI that can pass - #9

Merged
alexy merged 32 commits into
mainfrom
work/analytics-catalog
Sep 20, 2026
Merged

alexy merged 32 commits into
mainfrom
work/analytics-catalog

Conversation

@alexy

@alexy alexy commented Sep 20, 2026 •

Copy link
Copy Markdown
Member

Tier A of docs/goals/graph-analytics-catalog.md: the procedure registry grows from 10 projection kernels to 30, and Nutmeg serves every one of them with no code of its own.

Kernels

Family Procedures
Community louvain, leiden, labelPropagation
Centrality betweenness, closeness, harmonic, eigenvector, katz, hits
Structure kCore, triangleCount, localClusteringCoefficient, bridges, articulationPoints, biconnectedComponents
Similarity nodeSimilarity
Paths and flow spanningTree, maxFlow, minCut
Embeddings fastRP

Each has an independent oracle in its test (brute force, a dense-matrix definition, or exhaustive search) plus hand-computed cases, and cancellation, budget and scratch-release checks. Parallel kernels return bit-identical results and charge identical work at 1, 2, 3 and 8 threads; louvain, leiden and labelPropagation are sequential on purpose, because reproducibility is the contract.

Plumbing

  • run_on_projection and public projection_options, so an embedder runs any kernel by name on its own projection.
  • NodeTable: one result shape for per-node values, for rows a kernel keys itself (node pairs, edges), and for fixed-length Float32 vectors (Arrow FixedSizeList). A table with no rows yields one empty batch carrying its schema.
  • A second positional node argument, target.
  • parallel.rs and meter.rs: fixed blocks combined in order. The other agent's WorkMeter on work/rayon-kernels supersedes Meter; see codex-to-codex.md for the proposed merge order.

CI

The workspace workflow has failed on every run for weeks, for stacked reasons each hidden by the one before:

  1. grust-ladybug linked zstd twice under --all-features on Linux. It is now tested on its own.
  2. The benchmark runner's tests read benchmarks/lsqb/upstream/lsqb, which is gitignored. fetch-upstream.sh now downloads the pinned archive and verifies its size and SHA-256.
  3. One timing test in the runner failed once and passed once on the same code. It is not fixed; its assertion now prints the error it saw.

Superseded pull-request runs now cancel themselves.

Not in this PR

articleRank (an option inside pagerank.rs), and bellmanFord/astar, which need projections that admit negative weights; the design question is recorded in the plan. Nutmeg's run-on-projection branch must merge together with this.

🤖 Generated with Claude Code

alexy and others added 23 commits September 19, 2026 17:25
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
P1 of the analytics catalog plan. Registration and direct execution now read
one catalog of specs; run_on_projection finds a kernel by name and converts
its result to an ArrowResultCursor, and projection_options exposes the
projection reading the registered procedures use. Kernels return their
result rather than a row cursor, so the same value serves both surfaces.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The workspace job has failed on every push: LadybugDB's prebuilt static
library bundles zstd, a whole-workspace --all-features build unifies
features so zstd-sys is linked into the same test binary, and rust-lld
rejects the duplicate ZSTD_* symbols. macOS's linker accepts them, so it
never reproduced locally. Built alone, grust-ladybug does not pull zstd-sys.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…urns

Group 5 of the analytics catalog, first through the whole pipeline: kernel,
Arrow output, registry, Cypher. NodeTable gives per-node results one Arrow
adapter and one row adapter; Meter charges work in blocks. The oracle applies
the definition of a core to 2,482 small graphs, multigraphs and loops included.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…e start

Group 4 of the analytics catalog. The forward algorithm finds each triangle
once on the simple underlying graph; node blocks of equal estimated work run
on the caller's rayon pool into atomic counters, so the counts and the work
charged are the same at any width. parallel.rs is the pattern later kernels
reuse. Nutmeg serves both procedures with no change of its own.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Group 1 of the analytics catalog. One modularity formula serves Newman's
undirected and Leicht-Newman's directed case; moves are applied one at a
time so the result is reproducible at any pool width. The oracle is a
dense-matrix modularity and the brute-force optimum over every partition.
random.rs is the counter-based generator later kernels share.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…ernels

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…nup test saw

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The offline tests load upstream/lsqb, which is gitignored, so they could
never pass on a fresh checkout. They were unreachable while the Ladybug
link failure stopped the job earlier.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@alexy alexy changed the title Analytics catalog: run_on_projection, NodeTable, k-core; fix the failing CI Graph analytics catalog: 20 new kernels, and a workspace CI that can pass Sep 20, 2026
@alexy
alexy merged commit 4163e42 into main Sep 20, 2026
2 checks passed
alexy added a commit to querygraph/nutmeg that referenced this pull request Sep 20, 2026
Lands together with querygraph/grust#9, which adds run_on_projection,
public projection_options and twenty kernels. Nutmeg no longer dispatches
kernels by name itself.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
alexy added a commit that referenced this pull request Sep 20, 2026
Documentation only.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant