Problem or use case
Two Python repositories. One imports a function from the other and calls it. Each repository is built on its own, then the two graphs are combined with merge-graphs or global add. The combined graph has no edge between the repositories: the imports_from edge ends on a sourceless placeholder, and the call is not in the graph at all. The same two files scanned as one directory produce both edges.
This is the Python shape of #3152. Java, C++, C# and Swift park a member call whose receiver type lives elsewhere (#3222), and #3384, #3386, #3388, #3390, #3393 extend that per language. Python is missing from the series, and its dependency shape is different: not obj.method() bound through a receiver type, but from module import symbol plus a plain call, which symbol_resolution.py binds by (module_stem, symbol) inside one build and drops otherwise.
Proposed solution
- Python extractor: when
from module import symbol cannot be resolved in the corpus, park it on the importing file node, and park the raw calls that use the imported name (or its alias) on the caller node, names only, in the same spirit as _park_unresolved_member_call.
- Merge pass: for a parked module, find exactly one file node in another repo with that module stem; rewire the
imports_from edge and emit calls edges for the parked names. Ambiguous stems fabricate nothing.
- Add
"python": frozenset({".py"}) to _LANG_SUFFIXES so a Python name never binds to a same-named declaration in another language.
Package-level imports (from pkg.sub import x, import pkg) would follow the same rule with the package directory as the stem; the single-module case above is the minimal one.
Related
#3152, #3222 (mechanism), #3384 #3386 #3388 #3390 #3393 (per-language series, Python missing), #3665 (unresolved call indistinguishable from none), #3414 #3415 #3360 (false cross-repo edges from stub dedupe), #3578 (shared type linking for global add, types only).
Alternatives considered
No response
Area
Extraction or language support
Additional context
lib_repo/config_validator.py
def validator_from_json(data):
return ConfDict(data)
class ConfDict:
def __init__(self, data):
self.data = data
app_repo/app.py
from config_validator import validator_from_json
def load(payload):
return validator_from_json(payload)
$ (cd lib_repo && git init -q && graphify update . --no-cluster) # 4 nodes, 4 edges
$ (cd app_repo && git init -q && graphify update . --no-cluster) # 2 nodes, 2 edges
$ graphify merge-graphs lib_repo/graphify-out/graph.json app_repo/graphify-out/graph.json --out merged.json
merged.json: 7 nodes, 6 edges, 0 edges between the repos.
app_repo::app -imports_from-> app_repo::config_validator # placeholder, no source_file
app_repo::app -contains-> app_repo::app_load # no calls edge at all
Control, both files in one directory:
$ graphify update joint --no-cluster # 6 nodes, 8 edges
app -imports-> config_validator_validator_from_json
app_load -calls-> config_validator_validator_from_json
Expected
After merge-graphs or global add:
app_repo::app -imports_from-> lib_repo::config_validator (the file node whose module stem matches the placeholder, when exactly one repo provides it);
app_repo::app_load -calls-> lib_repo::config_validator_validator_from_json, tagged like _cross_repo_call, under the same single-definition guard the in-repo resolver uses.
Where it stops today
symbol_resolution.py::resolve_python_import_guided_calls resolves from module import symbol by (module_stem, symbol) in the current corpus and drops the call when the module is absent. Nothing is written to graph.json, so no merge-time pass can recover it (same root cause as #3152, and the invisibility #3665 describes).
cross_repo_calls.py reads metadata.unresolved_calls, but only receiver-typed member calls are parked, and _LANG_SUFFIXES lists cpp, csharp, java, swift.
global_graph.py::global_add deduplicates sourceless stubs by label among stubs only; a stub is never compared with a file node of another repo. What that dedupe does produce is the false cross-repo edges of #3414 and #3360: on five internal Python repositories, 831 edges between repos, all through shared stubs like Any, Enum, BaseModel, none between two real files. The one real dependency between those repos (a service importing an in-house library in three files) gave no edge; a joint scan of the same two repos gave 25.
Problem or use case
Two Python repositories. One imports a function from the other and calls it. Each repository is built on its own, then the two graphs are combined with
merge-graphsorglobal add. The combined graph has no edge between the repositories: theimports_fromedge ends on a sourceless placeholder, and the call is not in the graph at all. The same two files scanned as one directory produce both edges.This is the Python shape of #3152. Java, C++, C# and Swift park a member call whose receiver type lives elsewhere (#3222), and #3384, #3386, #3388, #3390, #3393 extend that per language. Python is missing from the series, and its dependency shape is different: not
obj.method()bound through a receiver type, butfrom module import symbolplus a plain call, whichsymbol_resolution.pybinds by(module_stem, symbol)inside one build and drops otherwise.Proposed solution
from module import symbolcannot be resolved in the corpus, park it on the importing file node, and park the raw calls that use the imported name (or its alias) on the caller node, names only, in the same spirit as_park_unresolved_member_call.imports_fromedge and emitcallsedges for the parked names. Ambiguous stems fabricate nothing."python": frozenset({".py"})to_LANG_SUFFIXESso a Python name never binds to a same-named declaration in another language.Package-level imports (
from pkg.sub import x, import pkg) would follow the same rule with the package directory as the stem; the single-module case above is the minimal one.Related
#3152, #3222 (mechanism), #3384 #3386 #3388 #3390 #3393 (per-language series, Python missing), #3665 (unresolved call indistinguishable from none), #3414 #3415 #3360 (false cross-repo edges from stub dedupe), #3578 (shared type linking for
global add, types only).Alternatives considered
No response
Area
Extraction or language support
Additional context
merged.json: 7 nodes, 6 edges, 0 edges between the repos.Control, both files in one directory:
Expected
After merge-graphs or global add:
app_repo::app -imports_from-> lib_repo::config_validator(the file node whose module stem matches the placeholder, when exactly one repo provides it);app_repo::app_load -calls-> lib_repo::config_validator_validator_from_json, tagged like _cross_repo_call, under the same single-definition guard the in-repo resolver uses.
Where it stops today
symbol_resolution.py::resolve_python_import_guided_calls resolves from module import symbol by (module_stem, symbol) in the current corpus and drops the call when the module is absent. Nothing is written to graph.json, so no merge-time pass can recover it (same root cause as #3152, and the invisibility #3665 describes).
cross_repo_calls.py reads metadata.unresolved_calls, but only receiver-typed member calls are parked, and _LANG_SUFFIXES lists cpp, csharp, java, swift.
global_graph.py::global_add deduplicates sourceless stubs by label among stubs only; a stub is never compared with a file node of another repo. What that dedupe does produce is the false cross-repo edges of #3414 and #3360: on five internal Python repositories, 831 edges between repos, all through shared stubs like Any, Enum, BaseModel, none between two real files. The one real dependency between those repos (a service importing an in-house library in three files) gave no edge; a joint scan of the same two repos gave 25.