Skip to content

Cache dependency graph in findBreadcrumbEntries to fix O(n²) build scaling#71

Open
gavinmorrison wants to merge 1 commit into
11ty:mainfrom
gavinmorrison:cache-dependency-graph
Open

Cache dependency graph in findBreadcrumbEntries to fix O(n²) build scaling#71
gavinmorrison wants to merge 1 commit into
11ty:mainfrom
gavinmorrison:cache-dependency-graph

Conversation

@gavinmorrison

Copy link
Copy Markdown

Problem

findBreadcrumbEntries() calls getDependencyGraph(nodes) on every invocation, rebuilding the full dependency graph from collections.all each time. Since this filter runs once per page and the nodes array is the same reference throughout a build, the result is O(n²) scaling — doubling the page count roughly quadruples build time.

Originally reported in #64.

How to reproduce

  1. Create an Eleventy site with @11ty/eleventy-navigation and a base layout that calls collections.all | eleventyNavigationBreadcrumb(...)
  2. Generate 1,000+ pages with eleventyNavigation keys
  3. Run npx eleventy — the benchmark output will show eleventyNavigationBreadcrumb dominating build time

Benchmarks

Tested with @11ty/eleventy@3.1.2 and @11ty/eleventy-navigation@1.0.5:

Pages Before After Speedup
1,000 29.9s 0.95s 31x
2,000 279s (4m 39s) 1.06s 263x

Approach

Cache the dependency graph using a WeakMap with the nodes array as the key. Since collections.all is the same object reference throughout a single Eleventy build, the graph only needs to be built once per build.

Using a WeakMap ensures that between builds (or in --serve mode) when Eleventy creates a new array, the old cache is automatically garbage collected, preventing memory leaks while naturally invalidating the cache.

All 25 existing tests pass.

Fixes #64

findBreadcrumbEntries() rebuilds the full dependency graph from
collections.all on every call. Since this filter runs once per page
and the nodes array is the same reference throughout a build, this
causes O(n²) scaling.

Cache the graph using the nodes reference as a key. The cache
invalidates automatically between builds when collections.all is
recreated.

Benchmarks (minimal Eleventy site, navigation v1.0.5):
- 1,000 pages: 29.9s → 0.64s (47× faster)
- 2,000 pages: 279s → 1.17s (238× faster)

Fixes 11ty#64
@gavinmorrison

Copy link
Copy Markdown
Author

@zachleat are you the maintainer for this project? Is there any chance you would be able to review this PR, please?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

[Request] Add an option to cache DependecyGraph

1 participant