manifest.json uses absolute paths as keys — --update always triggers full rebuild on CI or different clone path

Summary

manifest.json and cache/stat-index.json store absolute file paths as keys. This means --update (incremental re-extraction) always falls back to a full rebuild when the repo is cloned to a different path — including CI environments (GitHub Actions, etc.).

Reproduction

from graphify.detect import detect_incremental
from pathlib import Path

# After a full build at /original/path, copy the repo to /new/path
# then run detect_incremental from /new/path:
result = detect_incremental(Path('.'))
print(result['new_total'])   # reports ALL files as new
print(len(result['deleted_files']))  # reports ALL original files as deleted

Confirmed by copying a 193-file corpus repo from /Users/.../device-managing-portal to /private/tmp/dmp-test:

193 files "deleted" (old absolute paths gone from manifest)
618 files "new" (new absolute paths not in manifest)
Result: full re-extraction triggered instead of incremental

Root cause

manifest.json keys are absolute paths:

{
  "/Users/Varuna_1/git/device-managing-portal/backend/apps/__init__.py": {
    "mtime": 1783744253.177,
    "ast_hash": "d41d8cd98f00b204e9800998ecf8427e",
    "semantic_hash": "d41d8cd98f00b204e9800998ecf8427e"
  }
}

cache/stat-index.json has the same structure — absolute path as key, content hash as value — but the hash is never used as a fallback lookup when the path key doesn't match.

Impact

CI/CD broken: GitHub Actions checks out repos at /home/runner/work/<repo>/ — never matching a developer's local path. Every CI run does a full rebuild.
Multi-machine teams broken: developer A builds the graph, commits graphify-out/, developer B clones and runs --update — full rebuild every time.
graph.json itself is fine: all source_file values are already relative paths. Only the incremental bookkeeping files are affected.

Expected behavior

detect_incremental() should match files by relative path from the repo root (or by content hash as a fallback), not by absolute path. This would make graphify-out/manifest.json portable across machines and clone paths.

Suggested fix

Option A — store relative paths as manifest keys:

# Instead of: str(abs_path)  →  "/Users/.../repo/backend/foo.py"
# Use:         str(abs_path.relative_to(scan_root))  →  "backend/foo.py"

Option B — hash-first lookup: when an absolute path key misses, fall back to matching by ast_hash/semantic_hash on the file content. This is more robust (survives renames too) but requires an inverted index.

Option A is simpler and fixes the CI use case completely.

Environment

graphifyy version: pip show graphifyy → installed 2026-07-17
Python 3.12
macOS 25.5.0

Thanks for the detailed report.

What version does graphify --version report? Could you reproduce with a fresh graphify-out?

This appears to overlap #777 / #1125 and fixed since v0.8.31.

Tried to reproduce this against current v8 (post-0.9.19) to help answer the version question above — results support the overlap-with-#777/#1125 theory:

Fresh manifest, current code: built at path A (save_manifest(..., root=target), the shape every current caller uses — cli extract, watch, and the current skill runbook), keys come out relative (src/main.py). Copied the whole clone (including graphify-out/) to a different path and ran detect_incremental: 0 new, 0 deleted, all unchanged. Portable as designed.

Legacy absolute-keyed manifest (what pre-0.8.31 — or an outdated installed skill runbook that never passed root= — wrote) evaluated at a new clone path: every file reported new, and on a machine where the old absolute paths don't exist they'd all additionally report deleted — which matches the "193 deleted + 618 new" signature exactly.

So the manifest itself looks healthy on current code, and the symptom points to a graphify-out/ whose manifest predates the relativization (upgrading the package doesn't rewrite it until the next successful save — save_manifest does migrate seeded legacy rows to relative keys on its next run with root=).

One part of the report that IS still true on v8: cache/stat-index.json keys are absolute (abs_key = str(resolved) in cache.py), so a clone move silently loses the stat fastpath (full re-hash, though not re-extraction) and dead rows for the old path accumulate. That might be worth splitting into its own focused issue.

fix(watch): write manifest.json to the target's graphify-out (#2316)

fix(watch): write manifest.json to the target's graphify-out (Graphif…

Looks fixed on current v8, flagging with a caveat below.

When a root is passed, manifest.json keys are now rewritten to be relative to that root before the atomic write (graphify/detect.py, save_manifest), with an inline comment explicitly citing "the #777/#1964 portability guarantee". So a manifest built at one absolute path and then read after a repo move/clone-to-a-different-path should resolve by relative key instead of forcing a full rebuild.

Caveat: I verified this at the manifest-write layer (the code path is unambiguous) but didn't complete a full CLI-level before/after repro (build at path A, move to path B, confirm graphify update . reuses the cache instead of rebuilding everything). If anyone can still reproduce a full rebuild after a plain repo move on current v8, please reopen with the exact repro — that would point at a different layer still using absolute keys.

Related but distinct symptom on the same root cause (manifest path-key inconsistency), found 2026-08-09 on graphifyy 0.8.31 — noting it here rather than as a separate issue since the underlying inconsistency (some code paths write absolute keys, others relative, without normalizing) looks like the same class of bug as this one, just observed a different way.

Our case: graphify-out/manifest.json had 641 entries for ~308 real files — 307 in absolute-path form, 334 in relative-path form, many duplicating the same file under both formats with different data (e.g. the relative entry for CLAUDE.md had a stale mtime and an empty semantic_hash; the absolute entry had a current mtime and a real hash). Unlike this issue's symptom (cross-machine/CI clone → always full rebuild, safe-but-wasteful), ours was the opposite failure mode: detect_incremental() sometimes reads the stale duplicate and under-reports changes — it told us "2 files changed" when 146 were actually missing from the semantic cache, i.e. a false negative that could let genuinely-changed content silently go unindexed rather than trigger an unnecessary rebuild.

Workaround used: don't trust detect_incremental() when the manifest might be mixed-format; instead run a full detect() + check_semantic_cache(all_files) (in graphify.cache), which normalizes each path to absolute before lookup and so is immune to the format-mixing — treat that as ground truth for what actually needs (re-)extraction.

Not re-verified against the current release (this was on 0.8.31) — flagging as a data point in case it's useful alongside the CI/clone-path fix proposed here, since a fix that normalizes manifest keys to relative-from-root (Option A in this issue) would very likely also close the gap we hit, if the write path that currently produces absolute keys is the same one responsible for this duplication.

@cabgab-dev

@cabgab-dev thanks, this pointed at a real live gap I'd missed when I checked back in on the original symptom. Opened #3781: both load_manifest and save_manifest's relativize step already collapse an absolute+relative duplicate key down to one entry, but were keeping whichever raw key happened to iterate last in the on-disk JSON rather than whichever was actually current — exactly the "stale duplicate wins, under-reports changes" shape you described. Fixed by breaking the tie on each entry's own seen timestamp instead of iteration order.

@ayushcodes10

Fixed in v0.9.69 (now on PyPI) by #3781 — thanks @ayushcodes10.

Release: https://github.com/Graphify-Labs/graphify/releases/tag/v0.9.69

Navigation Menu

Repository navigation

manifest.json uses absolute paths as keys — --update always triggers full rebuild on CI or different clone path #1964

Summary

Reproduction

Root cause

Impact

Expected behavior

Suggested fix

Environment

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

manifest.json uses absolute paths as keys — --update always triggers full rebuild on CI or different clone path #1964

Description

Summary

Reproduction

Root cause

Impact

Expected behavior

Suggested fix

Environment

Activity

oleksii-tumanov commented on Jul 17, 2026

HerenderKumar commented on Jul 19, 2026

ayushcodes10 commented on Sep 16, 2026

cabgab-dev commented on Sep 23, 2026

ayushcodes10 commented on Sep 23, 2026

safishamsi commented on Sep 26, 2026

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions