diff --git a/CHANGELOG.md b/CHANGELOG.md index 40eb79c..0922df5 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -83,6 +83,7 @@ Sections dated before 2026-09-19 predate the cycle and stay as they are. - learn(skills/learned-mdl-preflight.md): **STOP row 26: DataGrid 2 column and `sort by` paths over an association.** Write the column path unquoted, and do not sort over an association on ≤ v0.24.0. Both wrong forms pass `check --references` and fail CE1613 at build. — field report from a colleague's naming-conventions project (mxcli v0.23.0, Mendix 11.12.4) - learn(skills/lint-that-actually-runs.md): **three more lint vocabulary facts, verified on v0.24.0, with a before/after rule.** `microflow_type` is `MICROFLOW`/`NANOFLOW`/`RULE`, and `microflows()` yields all three. SQL `PERSISTENT` is Starlark `Persistent`. `activity_count` counts top-level objects (splits, loops, annotations), not actions, and `activities_for()` does not see loop bodies. — field report from a colleague's naming-conventions project (mxcli v0.23.0, Mendix 11.12.4) - learn(skills/learned-microflow-patterns.md): **expression functions take positional arguments only — `toString(from: $X)` is not a syntax error, it's a silent misparse.** MDL's expression grammar has no named-argument form for function calls; `label: value` parses as `label` (a bare unresolved identifier) COLON-divided by `value` — COLON being the OQL division operator. Verified on mxcli v0.24.0 / Mendix 11.12.1, scratch copy of a PoC model: `toString(from: $Temperature)` and `formatDateTime($D, pattern: 'yyyy-MM-dd')` both pass `mxcli check --references` silently (exit 0) because the collapsed argument count still matches the function's arity; `substring(from: $S, index: 0, length: 3)` fails, but with a misleading `E006` ("expects 2 to 3 argument(s), got 1") that never names labelled arguments as the cause. The likely source: labelled-colon syntax is genuine MDL elsewhere (page `Attribute:`/`Action: MICROFLOW Name(Param: val)`, `@anchor(from: bottom, to: top)`) — just never inside an expression — workshop feedback, Sep 2026 +- fix(bin/doctor.sh): **the gate self-test's scratch copy now carries the whole model directory, not just `.mpr` + `mprcontents/`.** A `--target=deploy` build (the same target the real gate runs, in place, against the project as it sits on disk) resolves widget/theme/design-property references out of `theme/`, `resources/`, `widgets/` and `javasource/` sitting beside the `.mpr`; a thin copy reported hundreds to over a thousand `Could not find widget` / design-property errors as a "dirty baseline" while the real gate passed with 0 errors on the identical model. Now copies the directory holding the `.mpr` (single-tree root or `app/` on a two-tree checkout) minus `.git/`, `deployment/`, `node_modules/` and `.mpr-snapshots/`, and prints the copied size (`du -sh`) so the cost is visible. Keeps the existing real-basename fix. A sibling `.mpr` in the project's own directory was ruled out (collides with the project's own `mprcontents/`); `mxbuild --target=check` was ruled out (no evidence it exists anywhere in this toolkit's usage) — field reports, issue #127 and a macOS custom-theme project - learn(skills/learned-file-upload-widget.md): **a file upload mxcli can author, with proof that it uploads.** The Mendix File Uploader 2.5.0 bound to a `System.FileDocument` specialisation: the MDL shape (entities, grants, create/delete microflows, advanced formats), which upload widgets mxcli cannot author (classic FileManager, PDS uploader), the two traps with workarounds (a simple-mode `allowedfileformat` passes exec and fails `mx check` with CE0463; an uploader DESCRIBE will not re-exec, `exposes 2 datasources`), and the six-step upload instrument. Field run on stock v0.24.0: the section-4 MDL taken verbatim from the skill gave `mx check` 0 errors, 2/2 files stored, 2/2 downloads sha256-equal, `.csv` rejected with 0 rows; both traps reproduce unchanged on v0.24.0 — a Mendix app-rebuild project - new(skills/mendix-best-practices-index.md): **one row per Mendix best-practice area: the Mendix docs page, the bundled `assess-quality` section, the toolkit skill that applies it before the write, and the `mxcli lint` rule that catches it after exec.** An index, not a copy — the practice text stays on the Mendix pages (17 URLs verified HTTP 200 on 2026-09-25) and in the mxcli-bundled skill; Mendix's own Best Practice Recommender rules (MXP001–016) anchor the performance rows, and four rows say out loud that no lint rule exists and the preflight checklist is the only check. Routed `all` agents, stages 3/5/6, group reference — Maurits Visser - learn(skills/microflow-preflight.md, agents/mdl-agent.md): **a microflow now gets a tier before any MDL — Simple, Guided or Split-first — and Split-first means a posted split plan (thin orchestrator + one `SUB_` per responsibility, with signatures) that the user confirms first.** Prompted by a colleague's session refusing a long microflow as "too difficult" while the same task went through on a stronger model: the piece was too big, not the task. mdl-agent gains two rules — never hand back "too difficult", hand back the split plan; escalate one failing `SUB_` by name after one retry, never the whole script. Also records the mxcli team's answer on positioning: a standalone `mxcli layout` command — which on v0.24.0 and upstream main (2026-09-25) arranges domain models only (`--dry-run` on a scratch copy of a PoC model: 10 entity moves, no microflow), so the no-`@position` rule and the if-branch workaround stand until a microflow mode ships; noted in the bug ledger and `learned-microflow-patterns.md` — Maurits Visser diff --git a/bin/doctor.sh b/bin/doctor.sh index 1875419..65935b2 100755 --- a/bin/doctor.sh +++ b/bin/doctor.sh @@ -924,6 +924,36 @@ fi # The project's own .mpr/mprcontents are never touched — this runs entirely inside a scratch # directory, removed on every exit path via a RETURN trap (bash), not just the happy path. # +# THE SCRATCH COPY IS THE WHOLE MODEL DIRECTORY, NOT JUST .mpr + mprcontents/. Two field +# reports (an Atlas project on Mendix 11.14.0 with mxcli v0.23.0, Windows 11; and a project +# with a custom theme on Mendix 11.12.4, macOS) both saw this self-test FAIL with a dirty +# baseline — 1000+ `Could not find widget ...` / design-property errors, every one of them +# from Atlas_Web_Content page templates or the custom theme — while the real gate (exec.sh / +# verify-model.sh, which run mxbuild --target=deploy IN PLACE against the project as it sits +# on disk) passed with 0 errors on the identical model. A --target=deploy build resolves +# widget/theme/design-property references out of theme/, resources/, widgets/ and javasource/ +# sitting BESIDE the .mpr; a copy of only .mpr + mprcontents/ cannot see any of them, so it +# fails on resources that were never actually broken. Options weighed and why they lost: +# A. mxbuild --target=check (skip resource resolution). No evidence this target exists — +# grepped bin/, project-bin/, skills/, bug-logs/: every real gate in this toolkit runs +# --target=deploy, and nothing here has ever invoked --target=check. +# B. A sibling .mpr copied into the project's OWN directory. Unsafe, not chosen: MPR v2 +# stores its content in mprcontents/ beside the .mpr, so a second .mpr dropped next to +# the real one collides with the project's own mprcontents/ — this would risk corrupting +# the very model it is meant to leave untouched. +# (Symlinks into the project tree instead of copying were also considered and rejected: +# Git Bash on Windows silently COPIES through a symlink instead of linking it, unless +# MSYS=winsymlinks is set on that machine — nothing here can assume it is.) +# C. Copy the whole project tree. Chosen, scoped down: copy the directory that holds the +# .mpr (dirname "$MPR" — the project root on a single-tree checkout, app/ on a two-tree one, +# CLAUDE.md "Shipping an instrument" rule 2), minus .git/, deployment/, node_modules/ and +# .mpr-snapshots/ — none of which mxbuild reads, and the last of which can itself hold +# several full mprcontents/ copies. The model's real basename is preserved automatically, +# because every sibling is copied as-is (a renamed .mpr makes mxbuild bail before it writes +# an error file at all — reported 2026-09-22, the basename fix below). This measures nothing +# it cannot afford: the copied size is printed (du -sh) so a user sees the cost before the +# next run, not after. +# # Skipped under --quick (two extra mxbuild runs); force it with `bin/doctor.sh --gate-selftest # [project-dir]`, which also works stood alone without waiting through the rest of doctor. # Bounded by DOCTOR_GATE_TIMEOUT (default 300s) via mxtk_mxbuild_error_count. exec.sh's own @@ -938,6 +968,7 @@ head_ "Gate self-test (can the mxbuild gate actually see an error?)" gate_selftest() { local scratch scratch_mpr t0 t1 elapsed mdl model_dir base_count bad_count rc timeout_s + local copy_excl copy_ok entry entry_name x copy_size timeout_s="${DOCTOR_GATE_TIMEOUT:-300}" t0=$(date +%s) @@ -964,18 +995,34 @@ gate_selftest() { } trap 'rm -rf "$scratch" 2>/dev/null' RETURN - # Keep the model's REAL basename in the scratch dir: mprcontents/ (copied verbatim below) - # carries an internal record of it, and a renamed copy makes mxbuild bail BEFORE it writes - # any error file — which this self-test would then report as "gate cannot read mxbuild's - # error file", a false FAIL on a healthy gate. (Reported 2026-09-22 by Yvann.) + # Copy the WHOLE model directory — .mpr, mprcontents/, theme/, resources/, widgets/, + # javasource/, everything a --target=deploy build reads off disk beside the .mpr — minus + # what it never reads. See the section header above ("THE SCRATCH COPY IS THE WHOLE MODEL + # DIRECTORY") for the two field reports this fixes and the options it was weighed against. + # The model's REAL basename is preserved automatically because every sibling, the .mpr + # included, is copied as itself: mprcontents/ carries an internal record of the .mpr's own + # name, and a renamed copy makes mxbuild bail BEFORE it writes any error file at all — which + # this self-test would then report as "gate cannot read mxbuild's error file", a false FAIL + # on a healthy gate (reported 2026-09-22 by Yvann). + model_dir="$(dirname "$MPR")" + copy_excl=".git deployment node_modules .mpr-snapshots" + copy_ok=1 + for entry in "$model_dir"/* "$model_dir"/.[!.]*; do + [ -e "$entry" ] || continue + entry_name="$(basename "$entry")" + for x in $copy_excl; do + [ "$entry_name" = "$x" ] && continue 2 + done + cp -R "$entry" "$scratch/" 2>/dev/null || copy_ok=0 + done scratch_mpr="$scratch/$(basename "$MPR")" - if ! cp "$MPR" "$scratch_mpr" 2>/dev/null; then - bad "gate self-test: could not copy the model into the scratch dir" + if [ "$copy_ok" -ne 1 ] || [ ! -e "$scratch_mpr" ]; then + bad "gate self-test: could not copy the model directory into the scratch dir" GATE_SELFTEST_LINE="fail (copy failed)" return 0 fi - model_dir="$(dirname "$MPR")" - [ -d "$model_dir/mprcontents" ] && cp -r "$model_dir/mprcontents" "$scratch/mprcontents" 2>/dev/null + copy_size="$(du -sh "$scratch" 2>/dev/null | awk '{print $1}')" + note "copied ${copy_size:-?} of project resources into the scratch dir (excluding .git, deployment/, node_modules/, .mpr-snapshots/)" # (a) Baseline: the gate must resolve SOME integer off this model, clean or not — "?" here # means the gate cannot read mxbuild's own output, which is the original F-042-class defect.