feat(extract): Add GDScript extractor - #3750
AWCostabile wants to merge 5 commits into
Conversation
Nodes: the script (its class_name, or its autoload name, or the file name), func, signal, const, enum and inner class. Edges: extends -> inherits to the parent script; preload()/load() of a .gd -> imports; signal connect/emit -> uses (plus the connect callback as a reference); calls resolved through the same file, the extends chain, the project's [autoload] table, preload const aliases and the class_name index - all EXTRACTED, because each names its target in source; a call none of those resolves is counted, never guessed, and never handed to the shared name-matching pass. A <page>.md §N.N in a comment becomes a references edge to a section node of that page; a bare §N.N inherits the last page the file named (INFERRED). The project index is built once per project.godot root and cached per process. Registered in the dispatch, the language-family table, the extractor registry and detect's code set; grammar tree-sitter-gdscript 6.x. Version carries the +gms1 local label so the fork build is observable from --version.
A fixture script exercising every declaration shape, extracted inside a tmp_path Godot project (autoloads by res:// and uid://, a preload-only law script, a helper); inheritance and ancestor-call resolution across three scripts; the autoload label; a script outside any project; and the dispatch assertion that no other grammar reaches a .gd file.
There was a problem hiding this comment.
Graphify reviewed this change.
Worth a look — the grounded gate found no coupling regressions or blocking issues, but 3 advisory finding(s) below merit a look before merge.
Graphify review — findings
Adds a GDScript extractor and wires .gd through the pipeline: extract_gdscript in gdscript.py treats one file as one class and emits its func/signal/const/enum/inner-class members, with edges for extends inheritance, preload/load imports, signal connect/emit/emit_signal uses, and call resolution through same-file/ancestor functions plus a per-project [autoload], class_name, and uid:// index (cached per project.godot root). Registers .gd in CODE_EXTENSIONS, the language-family map, and the extractor dispatch/registry so detection and dispatch pick it up; calls resolved by none of the project mechanisms are counted as unresolved_calls rather than fed to the shared name-matcher, and files outside a Godot project still extract with only same-file resolution. Bumps the README grammar count to 38 and adds .gd to the listed extensions.
Worth a look
- Path traversal via unvalidated citation page in doc section resolution —
graphify/extractors/gdscript.py· Escalate · medium- agreed by 2 of 2 members but NOT verified (no proof, no reproducing execution) — consensus is not a verdict; needs human review
- Failed project.godot read leaves _uids uninitialized —
graphify/extractors/gdscript.py:82· Escalate · medium- agreed by 2 of 2 members but NOT verified (no proof, no reproducing execution) — consensus is not a verdict; needs human review
- Bare citations in a comment attach to the last explicit page, not the nearest preceding page —
graphify/extractors/gdscript.py· Escalate · medium- agreed by 2 of 2 members but NOT verified (no proof, no reproducing execution) — consensus is not a verdict; needs human review
Analysis details — impact, health, verification
Impact & health
Graphify review
Impact — 3088 functions depend on the 1025 functions this change touches.
Health — this change adds coupling hotspots:
- new:
extract()— 655 callers, 45 callees - new:
_rebuild_code()— 142 callers, 54 callees - new:
detect()— 112 callers, 15 callees - new:
_extract_generic()— 18 callers, 29 callees - new:
save_manifest()— 40 callers, 11 callees - new:
extract_js()— 87 callers, 4 callees - new:
extract_files_direct()— 17 callers, 20 callees - new:
extract_xaml()— 19 callers, 17 callees - …and 61 more — each is listed as a finding
Verification — 3088 functions in the blast radius were not formally verified this run (proofs are advisory here).
Gate & verification
graphify gate
PASS — objectively clean (no health regressions, tests not run — proofs not run this pass (advisory)). Grounded, not self-assessed.
Advisory (not blocking):
- verification_scope: 2837 function(s) in the blast radius were not formally verified this run
Test selection
Test selection
290 of 290 test file(s) selected (100%) via static blast radius.
Escalated to a full run for safety — the selection is not trustworthy on its own (see below). CI should run the whole suite.
tests/test_affected_cli.py— full-run-safetytests/test_affected_member_seed.py— full-run-safetytests/test_agents_platform.py— full-run-safetytests/test_analyze.py— full-run-safetytests/test_anthropic_custom_endpoint.py— full-run-safetytests/test_antigravity_install.py— full-run-safetytests/test_apm_fallback_version.py— full-run-safetytests/test_architecture_doc.py— full-run-safetytests/test_astro_extraction.py— impact, full-run-safetytests/test_astro_import_ids.py— impact, full-run-safetytests/test_atomic_canvas_export.py— full-run-safetytests/test_atomic_version_stamp.py— full-run-safetytests/test_atomic_writes.py— impact, full-run-safetytests/test_backend_env_isolation.py— full-run-safetytests/test_backend_extras.py— full-run-safetytests/test_benchmark.py— full-run-safetytests/test_benchmark_raw_graph.py— full-run-safetytests/test_build.py— impact, full-run-safetytests/test_build_merge_dedup_scope.py— full-run-safetytests/test_build_merge_hyperedges_and_prune.py— full-run-safetytests/test_build_merge_shrink_guard.py— full-run-safetytests/test_builtin_global_type_refs.py— impact, full-run-safetytests/test_cache.py— full-run-safetytests/test_callflow_html.py— full-run-safetytests/test_cargo_introspect.py— full-run-safetytests/test_carried_hyperedge_remap.py— full-run-safetytests/test_case_sensitive_resolution.py— impact, full-run-safetytests/test_charmap_encoding.py— impact, full-run-safetytests/test_chunking.py— impact, full-run-safetytests/test_cjs_module_extension.py— impact, full-run-safetytests/test_claude_cli_backend.py— impact, full-run-safetytests/test_claude_md.py— full-run-safetytests/test_cli_broken_pipe.py— full-run-safetytests/test_cli_export.py— full-run-safetytests/test_cli_help.py— full-run-safetytests/test_cluster.py— full-run-safetytests/test_codebuddy.py— full-run-safetytests/test_community_hub_labels.py— full-run-safetytests/test_community_labels_skill.py— full-run-safetytests/test_confidence.py— full-run-safetytests/test_corrupt_graph_json.py— full-run-safetytests/test_cpp_nested_and_cli.py— impact, full-run-safetytests/test_cpp_objc_cross_file_calls.py— impact, full-run-safetytests/test_cpp_preprocess.py— full-run-safetytests/test_cross_extension_reexport_self_cycle.py— impact, full-run-safetytests/test_cross_language_call_resolution.py— impact, full-run-safetytests/test_cross_repo_external_call_guards.py— impact, full-run-safetytests/test_cross_repo_member_calls.py— impact, full-run-safetytests/test_cross_repo_shared_types.py— full-run-safetytests/test_csharp_call_site_generic_args.py— impact, full-run-safety- … and 240 more
non-code file(s) changed (
README.md,graphify/extractors/MIGRATION.md,pyproject.toml,tests/fixtures/sample.gd,uv.lock) → running the full suite for safety (a code graph can't see config/fixture/data deps)
changed code file(s) with no mapped test (
README.md,graphify/extractors/MIGRATION.md,graphify/extractors/__init__.py,pyproject.toml) — a coverage gap or a missing link — running the full suite rather than only the selected tests
Selection is safe under the controlled-regression assumption; always-run tests + a periodic full run are the backstops. Advisory — it never changes the check verdict.
· 5 grounded finding(s) anchored inline below; 64 more finding(s) on lines outside this diff (see the check run).
| return out | ||
|
|
||
|
|
||
| def extract_gdscript(path: Path) -> dict: |
There was a problem hiding this comment.
extract_gdscript()
fans out to 12 callees (efferent coupling); 7 callers depend on it (afferent coupling).
Grounded coupling-delta finding (deterministic), not an LLM guess.
| function_bodies: list[tuple[str, Any, Any]] = [] # (nid, body node, def node) | ||
| top_funcs: frozenset[str] = frozenset() | ||
|
|
||
| def declare_member(node, owner_nid: str, owner_stem: str, inner: bool) -> None: |
There was a problem hiding this comment.
declare_member()
fans out to 7 callees (efferent coupling).
Grounded coupling-delta finding (deterministic), not an LLM guess.
| return _script_of(value, path, project) if value.endswith(".gd") else None | ||
| return None | ||
|
|
||
| def emit_extends(node, owner_nid: str) -> None: |
There was a problem hiding this comment.
emit_extends()
fans out to 6 callees (efferent coupling).
Grounded coupling-delta finding (deterministic), not an LLM guess.
| return _read_text(parts[1], source) | ||
| return None | ||
|
|
||
| def handle_attribute(node, caller_nid: str) -> None: |
There was a problem hiding this comment.
handle_attribute()
fans out to 11 callees (efferent coupling).
Grounded coupling-delta finding (deterministic), not an LLM guess.
| return | ||
| unresolved_calls.append({"caller_nid": caller_nid, "callee": f"{'.'.join(names)}.{method}" if names else method, "line": line}) | ||
|
|
||
| def handle_call(node, caller_nid: str) -> None: |
There was a problem hiding this comment.
handle_call()
fans out to 6 callees (efferent coupling).
Grounded coupling-delta finding (deterministic), not an LLM guess.
|
Ahh, I jus saw #1929 after opening, which is pretty thorough 😓... While this PR covers the GDScript half of Godot Support on the current |
Addresses the three advisory findings from the automated review: - the project index initialises its uid table before anything can fail, so a project.godot that cannot be read still leaves an index that answers None for a uid:// instead of raising AttributeError; - a cited documentation page only rewrites the recorded path when it resolves to a real file INSIDE the project root (or its docs/), and a page carrying '..' segments is never looked up at all — the caller already hands a basename, but the resolver now enforces containment itself; - citations are walked in the order they appear in a comment, so a bare section attaches to the page named before it and never to one named later on the same line. One test per finding.
|
Update: I pushed
Added one test per finding, and the full suite runs locally with 5679 passes and the same 23 Windows-only failures as a clean v8 checkout. The five fan-out notes are the usual for a new extractor module (the Zig/Rust ones score about the same), so I've left those as-is. |
There was a problem hiding this comment.
Graphify reviewed this change.
Worth a look — the grounded gate found no coupling regressions or blocking issues, but 2 advisory finding(s) below merit a look before merge.
Graphify review — findings
Adds a GDScript extractor for Godot 4 .gd files, wiring extract_gdscript into the dispatch table, the dotnet-style language-family map, the extractor registry, and CODE_EXTENSIONS. The extractor treats one file as one class and emits Godot-aware edges: extends → inherits to the parent script, preload/load → imports, signal connect/emit → uses (plus references to a callback), and calls resolved through the project's [autoload] table, preload const aliases, and class_name index — with unresolvable bare calls counted as unresolved_calls rather than handed to the name-matching resolver. Builds a per-project.godot index (autoloads, class_name→script, uid://→script, per-script func names) cached per process, and falls back to same-file-only resolution for scripts outside any Godot project.
Worth a look
- _file_index returns stale declarations after a script is edited —
graphify/extractors/gdscript.py:134· Escalate · medium- agreed by 2 of 2 members but NOT verified (no proof, no reproducing execution) — consensus is not a verdict; needs human review
- GDScript res:// references can escape the project root —
graphify/extractors/gdscript.py:97· Escalate · medium- agreed by 2 of 2 members but NOT verified (no proof, no reproducing execution) — consensus is not a verdict; needs human review
Analysis details — impact, health, verification
Impact & health
Graphify review
Impact — 3092 functions depend on the 1029 functions this change touches.
Health — this change adds coupling hotspots:
- new:
extract()— 655 callers, 45 callees - new:
_rebuild_code()— 142 callers, 54 callees - new:
detect()— 112 callers, 15 callees - new:
_extract_generic()— 18 callers, 29 callees - new:
save_manifest()— 40 callers, 11 callees - new:
extract_js()— 87 callers, 4 callees - new:
extract_files_direct()— 17 callers, 20 callees - new:
extract_xaml()— 19 callers, 17 callees - …and 61 more — each is listed as a finding
Verification — 3092 functions in the blast radius were not formally verified this run (proofs are advisory here).
Gate & verification
graphify gate
PASS — objectively clean (no health regressions, tests not run — proofs not run this pass (advisory)). Grounded, not self-assessed.
Advisory (not blocking):
- verification_scope: 2841 function(s) in the blast radius were not formally verified this run
Test selection
Test selection
290 of 290 test file(s) selected (100%) via static blast radius.
Escalated to a full run for safety — the selection is not trustworthy on its own (see below). CI should run the whole suite.
tests/test_affected_cli.py— full-run-safetytests/test_affected_member_seed.py— full-run-safetytests/test_agents_platform.py— full-run-safetytests/test_analyze.py— full-run-safetytests/test_anthropic_custom_endpoint.py— full-run-safetytests/test_antigravity_install.py— full-run-safetytests/test_apm_fallback_version.py— full-run-safetytests/test_architecture_doc.py— full-run-safetytests/test_astro_extraction.py— impact, full-run-safetytests/test_astro_import_ids.py— impact, full-run-safetytests/test_atomic_canvas_export.py— full-run-safetytests/test_atomic_version_stamp.py— full-run-safetytests/test_atomic_writes.py— impact, full-run-safetytests/test_backend_env_isolation.py— full-run-safetytests/test_backend_extras.py— full-run-safetytests/test_benchmark.py— full-run-safetytests/test_benchmark_raw_graph.py— full-run-safetytests/test_build.py— impact, full-run-safetytests/test_build_merge_dedup_scope.py— full-run-safetytests/test_build_merge_hyperedges_and_prune.py— full-run-safetytests/test_build_merge_shrink_guard.py— full-run-safetytests/test_builtin_global_type_refs.py— impact, full-run-safetytests/test_cache.py— full-run-safetytests/test_callflow_html.py— full-run-safetytests/test_cargo_introspect.py— full-run-safetytests/test_carried_hyperedge_remap.py— full-run-safetytests/test_case_sensitive_resolution.py— impact, full-run-safetytests/test_charmap_encoding.py— impact, full-run-safetytests/test_chunking.py— impact, full-run-safetytests/test_cjs_module_extension.py— impact, full-run-safetytests/test_claude_cli_backend.py— impact, full-run-safetytests/test_claude_md.py— full-run-safetytests/test_cli_broken_pipe.py— full-run-safetytests/test_cli_export.py— full-run-safetytests/test_cli_help.py— full-run-safetytests/test_cluster.py— full-run-safetytests/test_codebuddy.py— full-run-safetytests/test_community_hub_labels.py— full-run-safetytests/test_community_labels_skill.py— full-run-safetytests/test_confidence.py— full-run-safetytests/test_corrupt_graph_json.py— full-run-safetytests/test_cpp_nested_and_cli.py— impact, full-run-safetytests/test_cpp_objc_cross_file_calls.py— impact, full-run-safetytests/test_cpp_preprocess.py— full-run-safetytests/test_cross_extension_reexport_self_cycle.py— impact, full-run-safetytests/test_cross_language_call_resolution.py— impact, full-run-safetytests/test_cross_repo_external_call_guards.py— impact, full-run-safetytests/test_cross_repo_member_calls.py— impact, full-run-safetytests/test_cross_repo_shared_types.py— full-run-safetytests/test_csharp_call_site_generic_args.py— impact, full-run-safety- … and 240 more
non-code file(s) changed (
README.md,graphify/extractors/MIGRATION.md,pyproject.toml,tests/fixtures/sample.gd,uv.lock) → running the full suite for safety (a code graph can't see config/fixture/data deps)
changed code file(s) with no mapped test (
README.md,graphify/extractors/MIGRATION.md,graphify/extractors/__init__.py,pyproject.toml) — a coverage gap or a missing link — running the full suite rather than only the selected tests
Selection is safe under the controlled-regression assumption; always-run tests + a periodic full run are the backstops. Advisory — it never changes the check verdict.
· 5 grounded finding(s) anchored inline below; 64 more finding(s) on lines outside this diff (see the check run).
| return out | ||
|
|
||
|
|
||
| def extract_gdscript(path: Path) -> dict: |
There was a problem hiding this comment.
extract_gdscript()
fans out to 12 callees (efferent coupling); 9 callers depend on it (afferent coupling).
Grounded coupling-delta finding (deterministic), not an LLM guess.
| function_bodies: list[tuple[str, Any, Any]] = [] # (nid, body node, def node) | ||
| top_funcs: frozenset[str] = frozenset() | ||
|
|
||
| def declare_member(node, owner_nid: str, owner_stem: str, inner: bool) -> None: |
There was a problem hiding this comment.
declare_member()
fans out to 7 callees (efferent coupling).
Grounded coupling-delta finding (deterministic), not an LLM guess.
| return _script_of(value, path, project) if value.endswith(".gd") else None | ||
| return None | ||
|
|
||
| def emit_extends(node, owner_nid: str) -> None: |
There was a problem hiding this comment.
emit_extends()
fans out to 6 callees (efferent coupling).
Grounded coupling-delta finding (deterministic), not an LLM guess.
| return _read_text(parts[1], source) | ||
| return None | ||
|
|
||
| def handle_attribute(node, caller_nid: str) -> None: |
There was a problem hiding this comment.
handle_attribute()
fans out to 11 callees (efferent coupling).
Grounded coupling-delta finding (deterministic), not an LLM guess.
| return | ||
| unresolved_calls.append({"caller_nid": caller_nid, "callee": f"{'.'.join(names)}.{method}" if names else method, "line": line}) | ||
|
|
||
| def handle_call(node, caller_nid: str) -> None: |
There was a problem hiding this comment.
handle_call()
fans out to 6 callees (efferent coupling).
Grounded coupling-delta finding (deterministic), not an LLM guess.
What does this PR do?
This adds support for GDScript, the scripting language in Godot 4 (hopefully to assist those in indie game development). I use graphify on a single Godot project and the
.gdfiles were being skipped, so most of the codebase was missing from the graph 😢!While GDScript doesn't have import statements, a script can refers to another by:
a) extending it
b) calling
preload("res://path.gd")c) using one of the autoload singletons listed in
project.godotd) using a global
class_nameSo the extractor is built around these four mechanisms.
For each script it creates a node for the file (the label is the
class_nameif there is one, otherwise the autoload name, otherwise the filename), and nodes for the functions, signals, constants, enums and inner classes in it. Methods of an inner class are labelled.name()with amethodlink back to the class, the same way the Zig extractor handles struct methods.What it records between scripts:
extends Xbecomes aninheritsrelation pointing at the parent script. Built-in engine classes such asNodedon't have a script file, so nothing is emitted for them.preload()andload()of a.gdpath are recorded asimportsof that file. This is the important one. Scripts that don't declare aclass_namecan only be reached this way.signal.connect(callback),signal.emit()andemit_signal("name")are recorded asusesfrom the enclosing function to the signal. The callback handed toconnectis also noted as areferencesrelation.extendschain.Autoload.method(),Alias.method()whereAliasis apreloadconstant, andClassName.method()are looked up in the script they name..new()on any of those is recorded asreferencesagainst the script.movement.md §2.14links the enclosing function (or the file) to asectionnode for that page asreferences. The project I work on does this in almost every file, so it was worth the few lines. If a later comment in the same file just says§2.15, it is attached to the last page that file mentioned and markedINFERRED. Everything else above isEXTRACTED.Anything that can't be resolved with those rules (engine methods like
add_child, calls on$Node, calls on local variables) is counted inunresolved_callsand left out. I did try passing them through asraw_callsso the shared cross-file resolver could match them by name, but on real code that mostly connected_readyand_processbetween scripts that have nothing to do with each other. Leaving them out gave a much cleaner graph as far as I see it.The autoload table, the
class_nameindex and the per-file function and signal names are read once perproject.godotand cached for the lifetime of the process. Autoloads written asuid://are resolved through.gd.uidfiles which Godot generates next to each script. If a given script isn't inside a Godot project at all it still gets extracted (just without cross-file resolution).*On ids: functions, signals and classes use
make_id(stem, name)so the existing remap pass handles them. Constants and enums include a kind segment (make_id(stem, "const", name)). I had a fixture withconst Helperandfunc helper()in the same file and they merged into one node, becausemake_idlowercases.Type of change
Files changed
graphify/extractors/gdscript.pyis new.graphify/extractors/__init__.py,graphify/extract.py(re-export,_DISPATCH,_LANG_FAMILY_BY_EXT) andgraphify/detect.py(CODE_EXTENSIONS) register the suffix.CODE_EXTENSIONS, so it didn't need a change.pyproject.tomlanduv.lockaddtree-sitter-gdscript>=6.1,<7. It's a small wheel on PyPI and works with thetree-sitter0.25 already pinned.tree-sitter-language-packwould also do it but that pulls in 160 grammars.[dm], that's fine with me (the tests already skip when the module is absent).tests/test_languages.pyandtests/test_extractors_registry.py, with the fixture attests/fixtures/sample.gd.README.md, and the count was bumped to 38.graphify/extractors/MIGRATION.md.How was this tested?
The 23 failures are already there on
v8at 20a20d3 before my changes, same tests by name. They look Windows-specific to me: the read-only file case intest_atomic_writes, the symlink cases intest_non_regular_files,test_ollama_retry_cap, the install and uninstall scope tests, two intest_hooks, two intest_watch, one terraform path test, but none of them involve.gd.The new tests set up a small Godot project in
tmp_pathwith two autoloads (one declared asuid://), a script that has noclass_nameand can only be reached by preload, and a helper. They check the imports, inheritance and calls across three scripts, the signal wiring, the autoload label, a script outside any project, and that.gdis routed to this extractor only.I also ran it on the project it was written for, which has 174 scripts (174 file nodes, 2842 functions and 281
imports). Grepping the source finds 288 preload/load targets. Six of the seven that don't show up are examples inside doc comments, and the last is a pair of scripts that preload each other, which the undirected graph keeps as a single connection. There are 477 citation links to 152 section nodes, nothing dangling and no absolute paths in any id.graphify pathbetween two scripts that only connect through a third returns the two-hop chain.Graphify-specific checklist
Additional Information
I used
importsfor preloads rather thanimports_from, because a preload brings in the whole script and the relation points at the file node. Ifimports_fromis what you'd expect there, that's a simple enough change. The doc citation nodes havefile_type: "doc"andsource_fileset to the markdown page (resolved underdocs/when that directory exists). I didn't find an existing node type that fit so I usedsection, but I am happy to rename it if you prefer.PS: Cheers for a great project 👍