TraceDiff: Post WF name disambiguation#800
Open
kyle-hoffmeyer wants to merge 5 commits into
Open
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This PR adds name disambiguation to TraceDiff's Wagner-Fischer alignment along with other minor changes.
Problem
Wagner-Fischer aligns child node lists by normalized name. When multiple children share the same name, WF treats them as interchangeable (all have edit cost 0) and picks matches based on positional alignment. This can pair structurally unrelated nodes.
Consider a transformer attention block where both traces have multiple
aten::bmmchildren, but one trace has an extra wrapping layer that shifts positions:Here
bmm_Acomputes Q×K^T (children:aten::transpose,aten::expand) andbmm_Bcomputes Attn×V (children:aten::expand,aten::reshape). They have completely different subtrees but identical names.WF aligns by position and produces:
But if the position shift is different — say trace2 has the
aten::bmmnodes in a different order relative to other ops — WF may instead produce:WF sees both as
aten::bmmwith cost 0 and has no way to distinguish them. The resulting diff falsely reports that bmm_A's children changed (transpose/expand → expand/reshape) when in reality the operations are identical across traces — they were just matched with the wrong counterpart.Solution: Post-WF Disambiguation
A new post-processing pass (
_disambiguate_same_name_candidates) runs after every WF alignment. It identifies ambiguous matches and resolves them using subtree structure.How it works
Step 1 — Index unmatched nodes by name.
Partition WF ops into match/delete/insert. Group deleted indices (trace1 unmatched) and inserted indices (trace2 unmatched) by name.
Step 2 — Detect ambiguity.
For each WF match
(i, j)with name N, check if there are also unmatched nodes named N on either side. If not, the match is unambiguous — skip it.Step 3 — Disambiguate using BFS child-name levels.
When ambiguity exists:
children1[i]when disambiguating trace2 candidates)._bfs_child_name_levelscomputes a fingerprint for each node: a list of sorted tuples, one per BFS depth level, containing the normalized names of all GPU-path children at that depth. Example:_find_bfs_survivorscompares each candidate's fingerprint against the anchor's. A candidate is evicted if any level differs. If exactly one candidate survives and it differs from WF's original choice, a reassignment is recorded.Step 4 — Apply reassignments.
Substitution maps (
swap_i,swap_j) rewrite the ops list: demoted match sides become delete/insert, promoted nodes are consumed from the delete/insert pool, and the new match is emitted in place.Integration
The disambiguation pass is called at the end of
wagner_fischer, so all callers (Phase 2, Phase 3, Phase 4 intraverse_and_merge) get disambiguated results automatically.This new post-WF disambiguation does not change the number of matches/inserts/deletes. It only changes existing matches.
Other Changes
cuda_drivercategory supportcuda_driverevents (e.g.,cuLaunchKernel) are now treated equivalently tocuda_runtimeevents throughout the TraceDiff pipeline:Kernel launch name equivalence
A new mapping (
_KERNEL_LAUNCH_EQUIVALENTS) normalizes vendor-specific kernel launch API names to a common name during comparison:{ "hipModuleLaunchKernel": "__kernel_launch__", "cuLaunchKernel": "__kernel_launch__", }This prevents spurious diffs when comparing HIP and CUDA traces that use different launch APIs for the same operation.
collapse_single_gpu_childstops at kernel-dispatching cpu_opsPreviously,
collapse_single_gpu_childonly stopped at kernel dispatch leaf nodes or nodes with != 1 GPU-path child. Now it also stops atcpu_opnodes that have a kernel dispatch child. This prevents collapsing through the last meaningful cpu_op before a kernel launch, preserving it as a comparison point in the diff.Phase 3 reconciliation preserves Phase 2 matches
When
reconcile_unmatchedexpands wrapper nodes, the resulting children lists are re-indexed. Previously, all children were re-aligned with a fresh WF pass, which could undo correct Phase 2 matches (after the new post-WF disambiguation). Now, Phase 2 match ops are pinned by UID pair, and WF only runs on the unpinned (newly expanded) children.