Bring the egglog backend to performance parity with egg by yihozhang · Pull Request #1646 · herbie-fp/herbie

yihozhang · 2026-08-24T06:55:27Z

Draft: requires the egglog-experimental build from egraphs-good/egglog-experimental#63 (pinned to egraphs-good/egglog#1005, currently rev 924b5cd0) on PATH. Nightly-infra note: the egglog-herbie Makefile target installs egglog-experimental from upstream main, which lacks :dag and the back-off tags this branch uses — and because Herbie blocks on egglog protocol errors, that binary turns every test into a silent 150 s timeout. Until #63 merges, the target must install from that PR's branch.

What's added

Size-aware scheduling: the rewrite schedule becomes (back-off :node-limit ,node-limit) with an :until guard on (or (bad-merge?) (<= node-limit (get-node-size!))). The scheduler (Give schedulers e-graph access and per-rule match application egraphs-good/egglog#1005) applies each rule's chosen matches before consulting the next rule and checks the live e-node count against the limit, instead of choosing all matches against a stale size; the e-graph is rebuilt once per iteration, as in egg. Node limits count e-nodes (like egg) rather than total table rows. bad-merge-rule now also runs after the rewrite phase, and both the rewrite loop and constant folding stop as soon as bad-merge? fires, so an unsound merge halts the schedule instead of feeding further growth.
Same rule set and scheduling as egg: rules with a bare-variable left-hand side (pow1: a -> (pow a 1)) are expanded into one rule per spec constructor instead of being dropped (egg applies them; skipping them made egglog do less per iteration, run more iterations, and extract ~2× wider root classes at the same node limit). Constant folding saturates with the default scheduler ((saturate (run const-fold :until (bad-merge?)))), like egg's analysis-based folding: it is not rewriting to be rationed, and running it through the back-off instance advanced that scheduler's iteration counter twice per repeat, halving ban lengths relative to egg.
Persistent egglog subprocess: the prelude and all rule declarations are identical for every call in a run, so one subprocess per worker loads them once and each call is isolated with egglog push/pop. Timeout-interrupted calls are detected and the subprocess discarded; dump:egglog still spawns per call so dumps stay self-contained. Bindings are anchored by one indexed herbie-const constructor declared in the prelude instead of one constructor command per binding.
DAG extraction: multi-extract ... :dag let-binds subterms shared across variants instead of expanding each variant to a tree (~12× smaller responses); the response is read directly from the subprocess port and interned with per-shared-subterm memoization.
Calls with no roots skip the subprocess: a program with no real-typed roots (e.g. an FPCore whose output is a vector) produced a series-phase call ending in a zero-expression multi-extract — an egglog parse error that left Herbie blocked on a response until the per-test timeout. run-egglog now returns no variants without consulting the subprocess, matching the egg path. Fixes the Return input vector timeout in bench/arrays.fpcore (150 s → 0.4 s, result identical to egg).
*egglog-variants-limit* is wired into patch.rkt (previously dead code; default keeps the hardcoded 1,000,000 — no behavior change).

Performance (bench/numerics, seed 1, `--threads 1`)

	wall time	rewrite phase	rewrite candidates	mean end error
egglog backend, before this PR	365.7 s	128.3 s	—	4 failing tests
+ size-aware eager scheduling, persistent subprocess, `:dag`	79.9 s	19.3 s	477,707	1.434
+ egg's rule set/scheduling, one rebuild per iteration	74.3 s	19.7 s	357,988	1.444
egg backend (reference)	60.2 s	10.2 s	297,539	1.492

Where the remaining single-threaded 1.24× goes (profile in the branch's egglog-scheduler-design.md): Herbie's reconstruct! scales with candidate count (egglog still returns 1.2× egg's), e-matching over egg-length bans, per-call push/pop cloning of the rule set, and the taylor-lowering path's fixed per-call cost.

Full bench/ evaluation (seed 1, `--threads 8`, 150 s/test, 718 FPCores)

Both backends run sequentially from the same checkout on an idle 128-core machine, 2026-08-31:

	egg	egglog	ratio
wall (8 threads)	17m14s	17m09s	1.00x
peak RSS	19.3 GB	12.1 GB	—
mean end error (bits, 710 paired)	2.337	2.390	—
timeouts (150 s)	2	2	—
crashes	0	0	—

Accuracy is a wash: 51 egglog wins / 61 losses / 598 within 0.1 bits over the 710 tests both scored, with symmetric tails. Summed per-test time is 1.07× egg's — the remaining serial overhead is fully absorbed by parallelism. The failure set is now identical between backends: 5 pre-existing errors and the 2 expected fidget timeouts (prospero, bear), all backend-independent. The regressions in the previous version of this evaluation (1.41× wall, 45 series-phase timeouts, 4 vector-program crashes, multi-GB memory spikes on the proj suite) are gone with the schedule changes and the no-roots fix above.

A replication of the nightly flow (infra/nightly.sh bench reports --threads 2, date seed) completes all 718 tests in 26m14s of benchmarking with the same 7 backend-independent failures.

🤖 Generated with Claude Code

Three changes to the egglog backend, requiring the egglog-experimental build from egraphs-good/egglog-experimental#63: 1. The rewrite schedule uses (back-off :node-limit N :eager-apply 1) with get-node-size! :until guards: the scheduler checks the live e-node count before each rule and stops approving matches at the node limit, instead of choosing all matches and overshooting it by up to 4.7x. Node limits now measure e-nodes, like egg, rather than total table rows. 2. One egglog subprocess per worker instead of one per call: the prelude and all rule declarations are identical across calls within a run, so they are loaded once and each call is isolated with push/pop. The subprocess lives in its own custodian so per-test timeouts cannot reclaim it; an in-flight flag discards it if a call is interrupted mid-protocol. dump:egglog still spawns per call so dumps stay self-contained. 3. Extraction uses multi-extract's :dag output, which let-binds subterms shared across variants instead of expanding every variant to a tree (responses shrink ~12x). The response is read directly from the subprocess port and converted with memoized e1/e2 conversion that interns each shared subterm once, embedding batchrefs as children. Also wires *egglog-variants-limit* into patch.rkt (previously dead; the default keeps the hardcoded 1000000). On bench/numerics (seed 1): report wall time 365.7s -> 84.2s, rewrite phase 128.3s -> 19.7s (egg: 60.2s / 10.4s); final e-graph sizes now match egg's distribution; per-test errors unchanged and one blow-up-induced timeout fixed. Steps 2 and 3 are bit-identical in behavior. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

Resolves conflicts with the batch->block rename (herbie-fp#1641): keeps the persistent-subprocess and :dag extraction paths, renamed to the block API (block-add!, output-block, insert-block/insert-vs, egglog-dag->blockrefs), and keeps *egglog-variants-limit* wired in patch.rkt over the upstream hardcoded literal. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

egglog's num_nodes now excludes relations as well as base-sort functions. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

- Expand rules with a bare-variable left-hand side (pow1: a -> (pow a 1)) into one rule per spec constructor instead of dropping them. egg applies such rules; skipping them made egglog do less per iteration, run more iterations, and extract ~2x wider root classes at the same node limit. - Run constant folding with the default scheduler, not through the back-off instance: it is not rewriting to be rationed, and sharing the instance advanced its iteration counter twice per repeat, halving ban lengths. - Declare one indexed `herbie-const` constructor in the prelude instead of one constructor command per binding (tens of thousands of round trips per run). bench/numerics, seed 1, one thread: 79.9 s -> 74.3 s wall (egg 60.0 s), 477,707 -> 357,988 rewrite candidates (egg 297,539), mean end error 1.444 (egg 1.492). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

The back-off scheduler now always applies each rule's matches before consulting the next rule (the :eager-apply tag was removed upstream), so drop it from the schedule. Stop the rewrite loop as soon as bad-merge? fires: check it in the :until guard alongside the node limit, run bad-merge-rule after the rewrite phase as well as after constant folding, and saturate const-fold with the same guard so an unsound merge cannot keep folding. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

A program with no real-typed roots (e.g. an FPCore whose output is a vector) produced an egglog call ending in (multi-extract N :dag) with zero expressions. egglog rejects that as a parse error, so the reader blocked on a response that never came and the test sat at the per-test timeout. With no roots there is nothing to insert or extract; return no variants without consulting the subprocess, matching the egg backend. Fixes the "Return input vector" timeout in bench/arrays.fpcore (150s -> 0.4s, result identical to the egg backend). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

pavpanchekha

This PR mixes a bunch of different optimizations. Most of them are basically good, I like it, and I'm very excited to get egglog closer to egg performance. I basically see two "big" changes I want:

The process caching idea is way complex and has a lot of sharp edges. Like, what if egglog spins forever. Its owning benchmark times out, next one runs, and what, it just waits in line? I'm not sure, but the lifetime of the egglog process is confusing to me. And I don't like the fact that caching-specific code is spread throughout the codebase. If we're gonna cache the process, I'd rather it be in a single place. Honestly some of this code looks un-reviewed.
The multi-extract code looks basically fine, but it really feels like we should move the dag-resolution code into the egglog-multi-extract method, and then also the fact that that method does its own CLI I/O handling is suspicious. Ideally we'd optimize egglog-send for all cases instead of having a special case use a fast path.

pavpanchekha · 2026-09-19T04:34:17Z

+  ;; declarations are already loaded (see static-egglog-commands), and isolate
+  ;; this call with push/pop. With dump:egglog, spawn a fresh subprocess so
+  ;; every dump file is a complete, replayable session.
+  (define use-persistent? (not (flag-set? 'dump 'egglog)))


And actually rather the dump exist and work the same way whether or not we’re using a persistent session, the reason being, we use these dumps for actually debugging egglog, and we wanted to be faithful to how Herbie actually uses it

In other words, we don’t actually wanna dump the commands Herbie would send for a single call. We want to dump the commands Herbie actually sent, for egglog debugging.

pavpanchekha · 2026-09-21T01:11:59Z

+  (define subproc
+    (cond
+      [use-persistent? (get-persistent-subprocess (static-egglog-commands pform))]
+      [else
+       (define fresh (create-new-egglog-subprocess label))
+       ;; 1. Add the prelude - send directly to egglog.
+       (prelude fresh #:mixed-egraph? #t)
+       fresh]))


Seems better to encapsulate all this in get-persistent-subprocess? All the way from use-persistent? though the branch and creation logic?

pavpanchekha · 2026-09-21T01:12:47Z

+    (when use-persistent?
+      (egglog-send subproc '(push)))


Probably also this? Like, we should make some abstraction over the caching so we don't have to think about it all the time.

pavpanchekha · 2026-09-21T01:21:05Z

+      (unless use-persistent?
+        (apply egglog-send subproc (egglog-step-commands step pform)))


Why not just put these in the prelude?

pavpanchekha · 2026-09-21T01:22:14Z

+      (egglog-multi-extract subproc
+                            `(multi-extract ,extract


Wait what does egglog-multi-extract do that we still need to spell out the multi-extract command here?

pavpanchekha · 2026-09-21T01:27:20Z

+(define (persistent-subprocess-usable? static-commands)
+  (and persistent-subprocess
+       (not persistent-call-in-progress?)
+       (not (port-closed? (egglog-subprocess-input persistent-subprocess)))
+       (eq? (subprocess-status (egglog-subprocess-process persistent-subprocess)) 'running)
+       (equal? persistent-subprocess-key static-commands)))


This all feels quite complex.

pavpanchekha · 2026-09-21T01:29:03Z

+           (define output (expr->e1-pattern (rule-output rule)))
+           (cond
+             [(symbol? input)
+              ;; A bare-variable left-hand side (e.g. pow1: a -> (pow a 1)) matches


Interesting and clever

pavpanchekha · 2026-09-21T01:30:10Z

+                `(rule ((= ,input (,ctor ,@args))) ((union ,input ,output)) :ruleset ,tag))]
+             [else `((rewrite ,(expr->e1-pattern input) ,output :ruleset ,tag))]))))
+
+(define (keyword-like? x)


When exactly does this come up? This seems super hacky.

pavpanchekha · 2026-09-21T01:30:31Z

    ; Add the binding and constructor union to all-bindings for the future rule
    (set! all-bindings (cons curr-var-spec-binding all-bindings))
-    (set! all-bindings (cons `(union (,constructor-name) ,binding-name) all-bindings))
+    (set! all-bindings (cons `(union (herbie-const ,constructor-num) ,binding-name) all-bindings))


pavpanchekha · 2026-09-21T01:33:49Z


+;; Send an extract command and read its single s-expression response directly
+;; from the subprocess port. Responses can be many megabytes, so this avoids
+;; materializing them as intermediate line strings before parsing.


This code also seems really hacky. I get that the intermediate string is probably bad for perf, but it's worth checking what other outputs we ever read from egglog that can't be read. Like, why not extend the benefits of this optimization to everyone?

pavpanchekha · 2026-09-21T01:47:13Z

@yihozhang if you want to address these, great, if you want me to take over the PR that's OK too.

yihozhang and others added 2 commits August 23, 2026 23:54

yihozhang force-pushed the egglog-backend-performance branch from dd5a3a1 to 859c56f Compare August 24, 2026 07:21

yihozhang mentioned this pull request Aug 25, 2026

Size-aware back-off scheduling and DAG extraction output egraphs-good/egglog-experimental#63

Merged

yihozhang and others added 2 commits August 25, 2026 16:55

Clarify what get-node-size! counts in the egglog schedule comment

b6aa24d

egglog's num_nodes now excludes relations as well as base-sort functions. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

This was referenced Aug 28, 2026

Fix scheduler semantics and node counting egraphs-good/egglog#1007

Merged

Fix size-aware scheduling and DAG extraction egraphs-good/egglog-experimental#66

Merged

yihozhang and others added 2 commits August 31, 2026 16:10

yihozhang marked this pull request as ready for review September 1, 2026 00:16

yihozhang changed the title ~~Speed up the egglog backend 4.3x with size-aware scheduling~~ Bring the egglog backend to performance parity with egg Sep 1, 2026

pavpanchekha requested changes Sep 21, 2026

View reviewed changes

Navigation Menu

Bring the egglog backend to performance parity with egg - #1646

Bring the egglog backend to performance parity with egg#1646
yihozhang wants to merge 6 commits into
herbie-fp:mainfrom
yihozhang:egglog-backend-performance

yihozhang commented Aug 24, 2026 •

edited

Loading

Uh oh!

pavpanchekha left a comment

Uh oh!

pavpanchekha Sep 19, 2026

Uh oh!

pavpanchekha Sep 19, 2026

Uh oh!

pavpanchekha Sep 21, 2026

Uh oh!

pavpanchekha Sep 21, 2026

Uh oh!

pavpanchekha Sep 21, 2026

Uh oh!

pavpanchekha Sep 21, 2026

Uh oh!

pavpanchekha Sep 21, 2026

Uh oh!

pavpanchekha Sep 21, 2026

Uh oh!

pavpanchekha Sep 21, 2026

Uh oh!

pavpanchekha Sep 21, 2026

Uh oh!

pavpanchekha Sep 21, 2026

Uh oh!

pavpanchekha commented Sep 21, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

		(unless use-persistent?
		(apply egglog-send subproc (egglog-step-commands step pform)))

Conversation

yihozhang commented Aug 24, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What's added

Performance (bench/numerics, seed 1, --threads 1)

Full bench/ evaluation (seed 1, --threads 8, 150 s/test, 718 FPCores)

Uh oh!

pavpanchekha left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

pavpanchekha commented Sep 21, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

yihozhang commented Aug 24, 2026 •

edited

Loading

Performance (bench/numerics, seed 1, `--threads 1`)

Full bench/ evaluation (seed 1, `--threads 8`, 150 s/test, 718 FPCores)