Repository navigation
Conversation
- Rebuild the original build tree: ZAR resources become text .zrd files in the folders their compiler recorded (84 directories, including subfolders), interp.zbd becomes the gamegen *.gs and support\*.gw scripts with their original times, and sound banks become data\common\sounds WAVs with stored lower-quality variants. - Keep record order, residue bytes and timestamps in .zstudio layouts tied to source hashes. Accept an output only after it packs back byte-identically; other families (GameZ, anim, texture packs, image.zbd) are carried verbatim for now. All 58 files of the 1999 corpus, and the 1998 corpus, reconstruct and pack back exactly. - Define a lossless zReader text syntax (bit-exact floats, Latin-1 strings, dangling keys) and engine-exact gamegen script tokenization, with line-numbered parse errors. - Pack into a separate folder with staged, re-parsed, all-or-nothing writes, or verify without writing; report identical, changed and failed outputs. - Open text .zrd sources in the ZRD editor and save them as text, compile them on archive import, and show scripts as token text. - Add Tools menu commands and the source_reconstruct, source_pack and source_status MCP tools (85 tools), with documentation and tests.
|
@codex review |
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 55a97caf26
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| { | ||
| string target = SourceProject.Resolve(destination, result.Path); | ||
| Directory.CreateDirectory(Path.GetDirectoryName(target)!); | ||
| File.Move(SourceProject.Resolve(staging, result.Path), target, true); |
There was a problem hiding this comment.
When repacking an existing destination, a failure on any later File.Move—for example because a target is locked or the disk fills—or while writing the marker occurs after earlier targets and stale files have already been replaced or deleted. The operation then throws despite leaving a mixed old/new pack and potentially destroying the last known-good output; retain rollback copies or publish the completed staging tree with an atomic swap.
AGENTS.md reference: AGENTS.md:L17-L17
Useful? React with 👍 / 👎.
| Directory.CreateDirectory(Path.GetDirectoryName(target)!); | ||
| File.Move(SourceProject.Resolve(staging, result.Path), target, true); |
There was a problem hiding this comment.
When a previously packed destination contains a junction/reparse point in an output subdirectory, RejectLinks(destination) only inspects the destination and its ancestors, so this directory creation and move follow the nested link and can overwrite a file outside the selected pack folder. Validate every existing target parent immediately before publication without following reparse points, including the stale-output deletion path.
AGENTS.md reference: AGENTS.md:L17-L17
Useful? React with 👍 / 👎.
| doc.SourceSyntax = "gamegen-script"; string text = System.Text.Encoding.Latin1.GetString(bytes); | ||
| var lines = Sources.GameGenScriptText.Tokenize(text); |
There was a problem hiding this comment.
When opening a large .gs/.gw source—up to the global 512 MiB document limit—this decodes the entire byte array to UTF-16, after which Tokenize creates replacement, split, and token strings before enforcing any aggregate bound; the .zrd sibling similarly calls Encoding.Latin1.GetString before its node limit. A comment-heavy or single-row file can therefore exhaust memory during ordinary GUI or MCP opening instead of producing a diagnostic, so impose a byte/character limit before conversion or parse incrementally.
AGENTS.md reference: AGENTS.md:L13-L13
Useful? React with 👍 / 👎.
| var previous = JsonSerializer.Deserialize<PackMarker>(await File.ReadAllBytesAsync(marker, token), SourceProject.Json); | ||
| foreach (var old in previous?.Outputs ?? []) | ||
| if (!results.Any(r => r.Path.Equals(old.Path, StringComparison.OrdinalIgnoreCase))) { string stale = SourceProject.Resolve(destination, old.Path); if (File.Exists(stale)) File.Delete(stale); } |
There was a problem hiding this comment.
When the selected destination was previously packed from a different source project, the mere presence of zstudio-pack.json allows reuse, and the deserialized Origin is never compared with manifest.Origin.Fingerprint. Publication then deletes every old output absent from the current project and overwrites overlaps, so choosing the wrong prior pack folder silently destroys another project's packed files; reject an origin mismatch or require an empty destination.
Useful? React with 👍 / 👎.
| token.ThrowIfCancellationRequested(); var output = manifest.Outputs[i]; progress?.Report(new(i, manifest.Outputs.Count, output.Path)); | ||
| try | ||
| { | ||
| var built = await Task.Run(() => Build(projectRoot, output, token), token); |
There was a problem hiding this comment.
When a shared source changes between output iterations, each Build rereads live disk state independently, so the staged set need not correspond to any project revision. For example, soundsh.zbd can capture an edited WAV, then if that WAV is restored before soundsm.zbd is built, the derived bank accepts the original hash and cached variant; every output reopens and the incoherent set is published. Snapshot source bytes/hashes for the whole run or recheck them all before publication.
AGENTS.md reference: AGENTS.md:L17-L17
Useful? React with 👍 / 👎.
| string prefix = Path.TrimEndingDirectorySeparator(Path.GetFullPath(root)) + Path.DirectorySeparatorChar; | ||
| if (ViewModel.Documents.FirstOrDefault(d => d.IsDirty && Path.GetFullPath(d.Path).StartsWith(prefix, StringComparison.OrdinalIgnoreCase)) is { } dirty) | ||
| throw new StudioCommandException("unsaved_changes", $"Save or discard the edits to {Path.GetFileName(dirty.Path)} before packing."); | ||
| using var cancellation = CancellationTokenSource.CreateLinkedTokenSource(token); operation = cancellation; CancelOperationItem.IsEnabled = true; |
There was a problem hiding this comment.
When the user opens another folder while packing or verification is running, this token is linked only to the command token; OpenRootAsync replaces the workspace without canceling operation. The old project's pack can therefore continue publishing after its owning workspace and dirty-document check are obsolete, and its completion status is posted into the new workspace. Link the operation to the captured workspace lifetime or cancel it during root replacement.
AGENTS.md reference: AGENTS.md:L41-L41
Useful? React with 👍 / 👎.
| Directory.CreateDirectory(projectRoot); | ||
| Context context = new(projectRoot, token); |
There was a problem hiding this comment.
When reconstruction is canceled or an I/O error occurs after this directory is created, files already written by prior iterations are left behind without a completed manifest. A retry with the same destination then fails the nonempty-folder check, forcing the user to identify and delete partial generated data manually; stage reconstruction in a temporary sibling or track and remove the operation's files on failure.
Useful? React with 👍 / 👎.
| var files = Corpus(corpusRoot); | ||
| // The recovered build layout is RECOIL's; MechWarrior 3 data uses other formats and folders. | ||
| if (files.FirstOrDefault(f => FormatRegistry.Probe(f.Path) is { Family: FormatFamily.GameZ, Version: 27 } or { Family: FormatFamily.Animation, Version: 39 }) is { Path: not null } mw3) | ||
| throw new InvalidDataException($"{mw3.Relative} is MechWarrior 3 data; source reconstruction supports RECOIL."); |
There was a problem hiding this comment.
When the selected source is empty or contains only arbitrary files, this code merely finds no MW3 header and proceeds to create a successful project—an empty folder yields zero outputs, while unrelated files become passthrough outputs—and the UI reports that every output packed byte-identically. Require positive RECOIL evidence such as the expected prepared-script/archive structure before creating the destination so a wrong folder is not presented as a reconstructed RECOIL source tree.
AGENTS.md reference: AGENTS.md:L17-L17
Useful? React with 👍 / 👎.
| if (extension.Equals(".zrd", StringComparison.OrdinalIgnoreCase) && Sources.ZrdText.LooksLikeText(prefix)) | ||
| return new(FormatFamily.Zrd, null, Recognition.Supported, SourceZrdDescription); | ||
| if (extension.Equals(".gw", StringComparison.OrdinalIgnoreCase) || extension.Equals(".gs", StringComparison.OrdinalIgnoreCase)) | ||
| return new(FormatFamily.Scripts, null, Recognition.Supported, SourceScriptDescription); |
There was a problem hiding this comment.
When a valid ZAR is named .gs/.gw, or named .zrd with a first payload word outside 1–4, these extension branches return before the definitive archive-footer check below. The same bytes that were previously recognized as an archive are consequently tokenized or parsed as source text and fail or display bogus content; move the structural ZAR probe ahead of filename-based source recognition.
AGENTS.md reference: AGENTS.md:L19-L19
Useful? React with 👍 / 👎.
| project = root, manifest.Game, manifest.Version, origin = manifest.Origin, | ||
| families = manifest.Outputs.GroupBy(o => o.Family).ToDictionary(g => g.Key, g => g.Count()), | ||
| notes = manifest.Notes.Take(32).Select(n => Bounded(n, 512)).ToArray(), noteCount = manifest.Notes.Count, notesTruncated = manifest.Notes.Count > 32, | ||
| outputs = Page(manifest.Outputs, a, o => o.Path, o => new { o.Path, o.Family, o.Bytes, o.Sha256 }).Data |
There was a problem hiding this comment.
When a project has a large but permitted manifest field—for example a multi-megabyte origin.name, family, or output path—this response passes it through verbatim even though only notes and row counts are bounded. A single row can therefore allocate or emit most of the 64 MiB manifest through MCP before any final protocol-size rejection; truncate each returned string with disclosed original lengths, including the corresponding PackResult path fields.
AGENTS.md reference: AGENTS.md:L13-L13
Useful? React with 👍 / 👎.
|
Closing: the source-project design is changing. Projects will no longer be byte-identical, and they will keep no extra metadata. Models become glTF with PNG textures, lower-quality sound and texture variants are regenerated when packing, and anim.zbd is compiled from its .zrd/.zan sources. The work will return in a new PR after local testing. |
1 participant
Problem and resulting behavior
RECOIL's shipped ZBD files are build outputs. The studio built them with a gamegen tool from a source tree (
data\beside a folder of.gs/.gwscripts), repackaging shared textures, models, animations and resources for every mission. This PR is the first part of reconstructing that source tree so zStudio can work on sources like a world editor and pack the game files back.Tools → Reconstruct source project… (MCP
source_reconstruct) rebuilds a project from a shipped data folder:zrdr.zbd,mN\zrdr.zbd) become text.zrdfiles in the directories their compiler recorded. Every member stores the temporary file it was compiled from, for exampleD:\battlesportdev\data\m1\zrdr\envmodels\fueE3B0.TMP, so 84 original directories are recovered, including subfolders (envmodels,vtol,semi,explosns…) that member names do not carry.interp.zbdbecomes the original build scripts,gamegen\*.gsandgamegen\support\*.gw, with their recorded modification times.data\common\sounds\*.wavfrom the high-quality bank; medium/low variants are stored and used while their source is unchanged.anim.zbd, texture packs andimage.zbdare carried verbatim until their sources are reconstructed in follow-up PRs.Every output is verified to pack back byte-identically before it is accepted. All 58 files of the 1999 corpus and the 1998 corpus reconstruct and pack back exactly.
Tools → Pack ZBD files… / Verify source project (MCP
source_pack) builds every game file from the project on disk. Outputs are staged and reopened through the shared readers, and nothing is written if any output fails. Each output is reported as identical, changed (with the edited sources) or failed.source_statussummarizes a project.Formats
f32:XXXXXXXXfor NaN payloads), keeps the int/float distinction the engine enforces, supports Latin-1 escapes and preserves dangling keys and order. Parse errors name the line.CZInterp::TokenizeLine, retail 0x4C13C0), including comma-only empty tokens..zstudiolayouts and apply only while a source's content hash matches.Text
.zrdsources open in the shared ZRD editor and save as text; importing one into an archive compiles it. Scripts open as token text..zstudiometadata stays out of Files/Search.Research basis
A read-only study of the reconstructed engine, both corpora and the retail exe established what survives: compile paths, the complete gamegen scripts in
interp.zbd,anim.zbddependency stamps (448 sources with times) and.flt/.tifnames in GameZ. It also established what is lost: the original text syntaxes, OpenFlight attributes and full-resolution textures. Details are indocs/source-project.mdanddocs/status.md.Next parts
Texture packs and
image.zbdwith.tifsources,anim.zbdwith its.zrd/.zansources, and GameZ worlds with OpenFlight.fltsources and a new GameZ writer.Validation
.zrdediting and saving, unsaved-edit refusal, verify and pack.