From 7a7ec3c9dc2ff0f9cae14e606c123f92505e72a7 Mon Sep 17 00:00:00 2001 From: r Date: Fri, 25 Sep 2026 14:57:51 +0000 Subject: [PATCH] fix(bin/doctor.sh): gate self-test scratch copy now carries the whole model directory MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The gate self-test's scratch copy previously carried only .mpr + mprcontents/. A --target=deploy build (the same target the real gate runs, in place, against the project as it sits on disk) also resolves widget/theme/design-property references out of theme/, resources/, widgets/ and javasource/ sitting beside the .mpr — a thin copy reported hundreds to 1000+ false errors as a "dirty baseline" while the real gate passed with 0 errors on the identical model (two field reports: an Atlas project on Mendix 11.14.0, and a project with a custom theme on Mendix 11.12.4). Now copies the whole directory holding the .mpr (dirname "$MPR" — the model's own root on a single-tree checkout, or app/ on a two-tree one) minus .git/, deployment/, node_modules/ and .mpr-snapshots/, none of which mxbuild reads, and prints the copied size (du -sh) so the cost is visible. The existing real-basename fix (copying every sibling, including the .mpr, as itself) is unchanged. mxbuild --target=check was ruled out: grepped for it across bin/, project-bin/, skills/ and bug-logs/ and found no evidence it exists or is used anywhere in this toolkit — --target=deploy is the only target ever invoked. A sibling .mpr dropped into the project's own directory was also ruled out: MPR v2 stores an internal record of the model's basename in mprcontents/, and a second .mpr there would collide with the real model's own mprcontents/. Verified against two constructed fixtures (single-tree and .mpr-under-app/) with a stub mxbuild that logs the scratch directory's contents on every invocation: both layouts copy in mprcontents/, theme/, resources/, widgets/, javasource/ and the .mpr itself, exclude all four named directories (including when nested one level under app/ on the two-tree layout), report the copied size, reach a "pass" verdict, and leave no scratch directory behind afterward. Not run against a real mxbuild in this environment. Co-Authored-By: Claude Fable 5.1 Claude-Session: https://claude.ai/code/session_01VJgWP5vEoAsNsJYqCDMGNw --- CHANGELOG.md | 1 + bin/doctor.sh | 63 ++++++++++++++++++++++++++++++++++++++++++++------- 2 files changed, 56 insertions(+), 8 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 30ebae9..e1729f8 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -16,6 +16,7 @@ three commits past it), and a bug report can name a release instead of a sha nob Sections dated before 2026-09-19 predate the cycle and stay as they are. ## Unreleased +- fix(bin/doctor.sh): **the gate self-test's scratch copy now carries the whole model directory, not just `.mpr` + `mprcontents/`.** A `--target=deploy` build (the same target the real gate runs, in place, against the project as it sits on disk) resolves widget/theme/design-property references out of `theme/`, `resources/`, `widgets/` and `javasource/` sitting beside the `.mpr`; a thin copy reported hundreds to over a thousand `Could not find widget` / design-property errors as a "dirty baseline" while the real gate passed with 0 errors on the identical model. Now copies the directory holding the `.mpr` (single-tree root or `app/` on a two-tree checkout) minus `.git/`, `deployment/`, `node_modules/` and `.mpr-snapshots/`, and prints the copied size (`du -sh`) so the cost is visible. Keeps the existing real-basename fix. A sibling `.mpr` in the project's own directory was ruled out (collides with the project's own `mprcontents/`); `mxbuild --target=check` was ruled out (no evidence it exists anywhere in this toolkit's usage) — field reports, issue #127 and a macOS custom-theme project - learn(skills/learned-file-upload-widget.md): **a file upload mxcli can author, with proof that it uploads.** The Mendix File Uploader 2.5.0 bound to a `System.FileDocument` specialisation: the MDL shape (entities, grants, create/delete microflows, advanced formats), which upload widgets mxcli cannot author (classic FileManager, PDS uploader), the two traps with workarounds (a simple-mode `allowedfileformat` passes exec and fails `mx check` with CE0463; an uploader DESCRIBE will not re-exec, `exposes 2 datasources`), and the six-step upload instrument. Field run on stock v0.24.0: the section-4 MDL taken verbatim from the skill gave `mx check` 0 errors, 2/2 files stored, 2/2 downloads sha256-equal, `.csv` rejected with 0 rows; both traps reproduce unchanged on v0.24.0 — a Mendix app-rebuild project - new(skills/mendix-best-practices-index.md): **one row per Mendix best-practice area: the Mendix docs page, the bundled `assess-quality` section, the toolkit skill that applies it before the write, and the `mxcli lint` rule that catches it after exec.** An index, not a copy — the practice text stays on the Mendix pages (17 URLs verified HTTP 200 on 2026-09-25) and in the mxcli-bundled skill; Mendix's own Best Practice Recommender rules (MXP001–016) anchor the performance rows, and four rows say out loud that no lint rule exists and the preflight checklist is the only check. Routed `all` agents, stages 3/5/6, group reference — Maurits Visser - learn(skills/microflow-preflight.md, agents/mdl-agent.md): **a microflow now gets a tier before any MDL — Simple, Guided or Split-first — and Split-first means a posted split plan (thin orchestrator + one `SUB_` per responsibility, with signatures) that the user confirms first.** Prompted by a colleague's session refusing a long microflow as "too difficult" while the same task went through on a stronger model: the piece was too big, not the task. mdl-agent gains two rules — never hand back "too difficult", hand back the split plan; escalate one failing `SUB_` by name after one retry, never the whole script. Also records the mxcli team's answer on positioning: a standalone `mxcli layout` command — which on v0.24.0 and upstream main (2026-09-25) arranges domain models only (`--dry-run` on a scratch copy of a PoC model: 10 entity moves, no microflow), so the no-`@position` rule and the if-branch workaround stand until a microflow mode ships; noted in the bug ledger and `learned-microflow-patterns.md` — Maurits Visser diff --git a/bin/doctor.sh b/bin/doctor.sh index 1875419..65935b2 100755 --- a/bin/doctor.sh +++ b/bin/doctor.sh @@ -924,6 +924,36 @@ fi # The project's own .mpr/mprcontents are never touched — this runs entirely inside a scratch # directory, removed on every exit path via a RETURN trap (bash), not just the happy path. # +# THE SCRATCH COPY IS THE WHOLE MODEL DIRECTORY, NOT JUST .mpr + mprcontents/. Two field +# reports (an Atlas project on Mendix 11.14.0 with mxcli v0.23.0, Windows 11; and a project +# with a custom theme on Mendix 11.12.4, macOS) both saw this self-test FAIL with a dirty +# baseline — 1000+ `Could not find widget ...` / design-property errors, every one of them +# from Atlas_Web_Content page templates or the custom theme — while the real gate (exec.sh / +# verify-model.sh, which run mxbuild --target=deploy IN PLACE against the project as it sits +# on disk) passed with 0 errors on the identical model. A --target=deploy build resolves +# widget/theme/design-property references out of theme/, resources/, widgets/ and javasource/ +# sitting BESIDE the .mpr; a copy of only .mpr + mprcontents/ cannot see any of them, so it +# fails on resources that were never actually broken. Options weighed and why they lost: +# A. mxbuild --target=check (skip resource resolution). No evidence this target exists — +# grepped bin/, project-bin/, skills/, bug-logs/: every real gate in this toolkit runs +# --target=deploy, and nothing here has ever invoked --target=check. +# B. A sibling .mpr copied into the project's OWN directory. Unsafe, not chosen: MPR v2 +# stores its content in mprcontents/ beside the .mpr, so a second .mpr dropped next to +# the real one collides with the project's own mprcontents/ — this would risk corrupting +# the very model it is meant to leave untouched. +# (Symlinks into the project tree instead of copying were also considered and rejected: +# Git Bash on Windows silently COPIES through a symlink instead of linking it, unless +# MSYS=winsymlinks is set on that machine — nothing here can assume it is.) +# C. Copy the whole project tree. Chosen, scoped down: copy the directory that holds the +# .mpr (dirname "$MPR" — the project root on a single-tree checkout, app/ on a two-tree one, +# CLAUDE.md "Shipping an instrument" rule 2), minus .git/, deployment/, node_modules/ and +# .mpr-snapshots/ — none of which mxbuild reads, and the last of which can itself hold +# several full mprcontents/ copies. The model's real basename is preserved automatically, +# because every sibling is copied as-is (a renamed .mpr makes mxbuild bail before it writes +# an error file at all — reported 2026-09-22, the basename fix below). This measures nothing +# it cannot afford: the copied size is printed (du -sh) so a user sees the cost before the +# next run, not after. +# # Skipped under --quick (two extra mxbuild runs); force it with `bin/doctor.sh --gate-selftest # [project-dir]`, which also works stood alone without waiting through the rest of doctor. # Bounded by DOCTOR_GATE_TIMEOUT (default 300s) via mxtk_mxbuild_error_count. exec.sh's own @@ -938,6 +968,7 @@ head_ "Gate self-test (can the mxbuild gate actually see an error?)" gate_selftest() { local scratch scratch_mpr t0 t1 elapsed mdl model_dir base_count bad_count rc timeout_s + local copy_excl copy_ok entry entry_name x copy_size timeout_s="${DOCTOR_GATE_TIMEOUT:-300}" t0=$(date +%s) @@ -964,18 +995,34 @@ gate_selftest() { } trap 'rm -rf "$scratch" 2>/dev/null' RETURN - # Keep the model's REAL basename in the scratch dir: mprcontents/ (copied verbatim below) - # carries an internal record of it, and a renamed copy makes mxbuild bail BEFORE it writes - # any error file — which this self-test would then report as "gate cannot read mxbuild's - # error file", a false FAIL on a healthy gate. (Reported 2026-09-22 by Yvann.) + # Copy the WHOLE model directory — .mpr, mprcontents/, theme/, resources/, widgets/, + # javasource/, everything a --target=deploy build reads off disk beside the .mpr — minus + # what it never reads. See the section header above ("THE SCRATCH COPY IS THE WHOLE MODEL + # DIRECTORY") for the two field reports this fixes and the options it was weighed against. + # The model's REAL basename is preserved automatically because every sibling, the .mpr + # included, is copied as itself: mprcontents/ carries an internal record of the .mpr's own + # name, and a renamed copy makes mxbuild bail BEFORE it writes any error file at all — which + # this self-test would then report as "gate cannot read mxbuild's error file", a false FAIL + # on a healthy gate (reported 2026-09-22 by Yvann). + model_dir="$(dirname "$MPR")" + copy_excl=".git deployment node_modules .mpr-snapshots" + copy_ok=1 + for entry in "$model_dir"/* "$model_dir"/.[!.]*; do + [ -e "$entry" ] || continue + entry_name="$(basename "$entry")" + for x in $copy_excl; do + [ "$entry_name" = "$x" ] && continue 2 + done + cp -R "$entry" "$scratch/" 2>/dev/null || copy_ok=0 + done scratch_mpr="$scratch/$(basename "$MPR")" - if ! cp "$MPR" "$scratch_mpr" 2>/dev/null; then - bad "gate self-test: could not copy the model into the scratch dir" + if [ "$copy_ok" -ne 1 ] || [ ! -e "$scratch_mpr" ]; then + bad "gate self-test: could not copy the model directory into the scratch dir" GATE_SELFTEST_LINE="fail (copy failed)" return 0 fi - model_dir="$(dirname "$MPR")" - [ -d "$model_dir/mprcontents" ] && cp -r "$model_dir/mprcontents" "$scratch/mprcontents" 2>/dev/null + copy_size="$(du -sh "$scratch" 2>/dev/null | awk '{print $1}')" + note "copied ${copy_size:-?} of project resources into the scratch dir (excluding .git, deployment/, node_modules/, .mpr-snapshots/)" # (a) Baseline: the gate must resolve SOME integer off this model, clean or not — "?" here # means the gate cannot read mxbuild's own output, which is the original F-042-class defect.