Fix discover showing different branch trees across worktrees by earfman · Pull Request #1755 · VirtusLab/git-machete

earfman · 2026-07-21T23:23:44Z

Problem

git machete discover produces a different branch tree depending on which worktree it is run from, even though all worktrees share one .git/machete file by default (machete.worktree.useTopLevelMacheteFile=true). Running discover in one worktree can silently drop branches that are visible from another.

Root cause

discover trims its output to the ~10 most-recently-checked-out branches, ranked via Git.get_latest_checkout_timestamps(). That method parsed git reflog show for HEAD, but HEAD's reflog is per-worktree (<git-dir>/logs/HEAD for the main worktree, <git-dir>/worktrees/<id>/logs/HEAD for each linked one). So the recency ranking only saw the current worktree's checkouts; branches recently used in other worktrees were treated as stale and pruned, differently depending on where the command ran.

Fix

Aggregate every worktree's HEAD reflog (<git-dir>/logs/HEAD plus worktrees/*/logs/HEAD), keeping the most recent checkout timestamp per branch. The ranking is now worktree-independent, so discover produces the same, complete tree regardless of the invoking worktree — keeping the shared-file behaviour (intentional and documented) while making the tree consistent, per the reporter's points 2 and 3.

The reflog files are read directly (a single git reflog show can only report the current worktree's HEAD), decoded as strict UTF-8 with errors="surrogateescape" so a non-ASCII committer name can't raise mid-read.

Testing

New regression test test_discover_is_worktree_independent asserts that discover yields the same managed branches from both the main and a linked worktree (using --checked-out-since for a deterministic trigger); it fails without the fix and passes with it. The existing discover/git/worktree suites pass locally. Added a RELEASE_NOTES.md entry under 3.44.2.

PawelLipski · 2026-07-22T14:47:16Z

Thanks for the contrib! pls fix mypy so that we can also validate test coverage (will be posted automatically by codecov once the tests pass)

And also, why does PR title/description mention status? AFAICS status command itself isn't affected - does it meant the status that's printed by discover itself? still, I'd remove the reference

PawelLipski · 2026-07-22T14:53:22Z

+        if os.path.isdir(worktrees_dir):
+            for worktree_id in sorted(os.listdir(worktrees_dir)):
+                reflog_paths.append(self.get_main_worktree_git_subpath("worktrees", worktree_id, "logs", "HEAD"))
+


Note that depending on machete.worktree.useTopLevelMacheteFile git config key, machete file can be shared b/w worktrees (default), or be stored per-worktree. Still, I think it's reasonable to assume that discovery can happen based on all the reflogs combined, regardless of what worktree it is run from. Worst case, when useTopLevelMacheteFile=false, too many branches will be taken into account in discovery and the user will need to trim down the per-worktree machete file.

Also, when useTopLevelMacheteFile=false, does discover write to the per-worktree machete file (for the worktree where discover is executed)? I'd assume so, but pls verify while you're here

(okay responded below)

In future, if we deem that discovery should happen per-worktree when useTopLevelMacheteFile=false, we can easily conditionally change the logic to the old one. For now, better not introduce too many moving parts, this is complex enough as it is 😅 worktrees (thanks to wide agent adoption...) only opened another cans of worms, thus adding complexity in multiple places

Credit the contributor, drop the issue number, and remove the 'status' reference since only discover is affected.

Annotate the discover_here() helper's return type as Set[str] (was a bare 'set') to satisfy disallow_any_generics under mypy, so the check suite passes.

earfman · 2026-07-22T18:30:06Z

Thanks @PawelLipski, and thanks for the collaborator invite! Pushed two commits that address everything:

mypy — the failure was a bare set return annotation on a helper in the new test; changed it to Set[str], so the suite should go green now (and codecov can post coverage).

Release note — credited myself and dropped the issue number, matching the earlier entries.

status reference — removed from the title and description. You're right that the status command itself isn't affected; it was pointing at the tree discover prints, but agreed it reads as misleading.

Per-worktree file question — confirmed: with useTopLevelMacheteFile=false, discover writes to the per-worktree machete file of the worktree it's run from. The path comes from __get_git_machete_branch_layout_file_path(), which returns the current worktree's git dir in that mode (and the main worktree's in the default shared mode); both the read and the save go through that one resolved _branch_layout_file_path, so this PR doesn't touch it — the change is confined to the reflog aggregation used for recency ranking. Verified end-to-end: discover in the main worktree saved to .git/machete, and in a linked worktree to .git/worktrees/<id>/machete. And exactly as you anticipated, because ranking now combines all worktrees' reflogs, the linked worktree's file did pick up a branch checked out only in the main worktree — the "too many branches, trim the per-worktree file" case you described, not a regression.

The new test uses linked worktrees (git 2.5+). Guard it with the same skipif the other worktree tests use, so it stops failing on the python 3.6 / git 1.8.0 CI lane.

PawelLipski · 2026-07-22T20:49:34Z

+        os.chdir(worktree)
+        from_worktree = discover_here()
+
+        # The two worktrees must agree, and each cross-worktree branch must survive pruning.


Also, sanity check - without the change to git.py, are the two discovered layouts different? otherwise the test would be of limited value

PawelLipski · 2026-07-22T20:51:29Z


 ## New in git-machete 3.44.2

+- fixed: `git machete discover` no longer produces a different branch tree depending on which worktree it is run from; the fresh-branch recency ranking now aggregates HEAD reflogs across all worktrees rather than only the current one (contributed by @earfman)


Actually, pls also include the reporter:

Suggested change

- fixed: `git machete discover` no longer produces a different branch tree depending on which worktree it is run from; the fresh-branch recency ranking now aggregates HEAD reflogs across all worktrees rather than only the current one (contributed by @earfman)

- fixed: `git machete discover` no longer produces a different branch tree depending on which worktree it is run from; the fresh-branch recency ranking now aggregates HEAD reflogs across all worktrees rather than only the current one (reported by @jasonoura, contributed by @earfman)

Probably the first time the given issue has a different reporter & implementer, and neither is me (or someone from my team) 😅 are you guys associated in any way?

PawelLipski · 2026-07-22T20:57:41Z

+
+        # The two worktrees must agree, and each cross-worktree branch must survive pruning.
+        assert from_main == from_worktree, (from_main, from_worktree)
+        assert "feat_wt" in from_main    # checked out only in the linked worktree


Yeah, they'd be different, okay

PawelLipski · 2026-07-22T20:59:05Z

Ouch, there'll be no coverage since it's coming from fork, and secrets are not provided by CI to the builds from forks, lemme think how to resolve it...

PawelLipski · 2026-07-22T21:10:56Z

+
+        # Raw reflog line: "<old-sha> <new-sha> <name> <email> <unix-ts> <tz>\t<subject>".
+        # We read the files directly (rather than N `git reflog show` subprocesses, one per worktree) since
+        # `git reflog show` can only report the current worktree's HEAD; the on-disk format is stable.


Actually yeah, if it comes to format stability - is it declared anywhere in git docs?
I mean, tests pass for a very wide range of versions (1.8 to the recent 2.5x ones), so the format is effectively stable, but curious if git claims it explicitly

(maybe irrelevant, see comment at the end of file)

PawelLipski · 2026-07-22T21:12:53Z

@@ -1460,22 +1460,56 @@ def get_git_timespec_parsed_to_unix_timestamp(self, date: str) -> int:
            raise UnexpectedMacheteException(f"Cannot parse timespec: `{date}`")

    def get_latest_checkout_timestamps(self) -> Dict[str, int]:  # TODO (#110): default dict with 0


Remove this TODO while you're here, it's stale ;p

PawelLipski · 2026-07-22T21:13:36Z

+        # We read the files directly (rather than N `git reflog show` subprocesses, one per worktree) since
+        # `git reflog show` can only report the current worktree's HEAD; the on-disk format is stable.
+        pattern = re.compile("^checkout: moving from (.+) to (.+)$")
+        for reflog_path in reflog_paths:


I'd extract reading of reflogs to a lower-level method which would return the reflog entries as some named tuples, and then get_latest_checkout_timestamps would just use these entries without dealing with file format

Nah, no, see below

PawelLipski · 2026-07-22T21:15:05Z

+                    # (rather than "first entry wins") is order-independent, so it's correct even though the
+                    # reflog files are appended oldest-first - the opposite order to `git reflog show`.
+                    if timestamp > result.get(branch, 0):
+                        result[branch] = timestamp


Also, in fact, would it be very difficult to instead of all this parsing, just run git reflog in each individual directory, and aggregate the results at the end? This is run once per discover (which is a relatively rare operation), and reflog is fast compared to e.g. log (mostly does what you do here anyway)

This way we won't need to take care of parsing raw reflog data, but instead git reflog output which is easier to control

earfman added 3 commits July 21, 2026 16:04

Aggregate all worktree reflogs when ranking branches in discover

9b7d696

Add regression test for worktree-independent discover (VirtusLab#1754)

9d4b8f9

Add release note for VirtusLab#1754

26d8f15

PawelLipski reviewed Jul 22, 2026

View reviewed changes

Comment thread RELEASE_NOTES.md Outdated

PawelLipski added bug Something isn't working discover Relates to how machete infers upstreams branches and discovers the entire tree of branchs deps labels Jul 22, 2026

PawelLipski reviewed Jul 22, 2026

View reviewed changes

earfman added 2 commits July 22, 2026 11:19

Address review: tidy 3.44.2 release note

24a9cd0

Credit the contributor, drop the issue number, and remove the 'status' reference since only discover is affected.

Address review: fix mypy type-arg in discover worktree test

bca1326

Annotate the discover_here() helper's return type as Set[str] (was a bare 'set') to satisfy disallow_any_generics under mypy, so the check suite passes.

earfman changed the title ~~Fix discover/status showing different branch trees across worktrees~~ Fix discover showing different branch trees across worktrees Jul 22, 2026

Skip worktree-independence test on git < 2.5

bfca334

The new test uses linked worktrees (git 2.5+). Guard it with the same skipif the other worktree tests use, so it stops failing on the python 3.6 / git 1.8.0 CI lane.

PawelLipski reviewed Jul 22, 2026

View reviewed changes


		## New in git-machete 3.44.2

		- fixed: `git machete discover` no longer produces a different branch tree depending on which worktree it is run from; the fresh-branch recency ranking now aggregates HEAD reflogs across all worktrees rather than only the current one (contributed by @earfman)

		@@ -1460,22 +1460,56 @@ def get_git_timespec_parsed_to_unix_timestamp(self, date: str) -> int:
		raise UnexpectedMacheteException(f"Cannot parse timespec: `{date}`")

		def get_latest_checkout_timestamps(self) -> Dict[str, int]: # TODO (#110): default dict with 0

Uh oh!

Conversation

earfman commented Jul 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Problem

Root cause

Fix

Testing

Uh oh!

Uh oh!

PawelLipski commented Jul 22, 2026

Uh oh!

PawelLipski Jul 22, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

PawelLipski Jul 22, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

earfman commented Jul 22, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

PawelLipski Jul 22, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

PawelLipski commented Jul 22, 2026

Uh oh!

PawelLipski Jul 22, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

earfman commented Jul 21, 2026 •

edited

Loading

PawelLipski Jul 22, 2026 •

edited

Loading

PawelLipski Jul 22, 2026 •

edited

Loading

PawelLipski Jul 22, 2026 •

edited

Loading

PawelLipski Jul 22, 2026 •

edited

Loading