Sync ako/mxcli: storage-GUID data loss and its write guard, MOVE/RENAME reference sweeps - #1177
Merged
Merged
Conversation
Every ALTER of an existing entity reset each attribute's storage GUID to the attribute's own $ID. The runtime keys mendixsystem$attribute.id on that GUID, so the next deploy against a database holding data made the synchroniser treat every attribute as deleted-and-re-added: it dropped and recreated the columns and lost their values. Reported from production — 28 attributes of 607 rows emptied by one edit (mendixlabs#1119). This is the unclosed half of #657. UpdateEntity rebuilds the target with entityToGen, which rebuilds every attribute and index from the semantic model, so each arrived raw==nil and the codec's EmitGUID default wrote GUID = $ID. #657 carried orig.Raw() onto the rebuilt ENTITY and noted that siblings survive via the list-rebuild raw passthrough — but the target's own children are all rebuilt, and nothing carried their raw. carryChildIdentity does the same carry one level down, for the two children that hold a GUID (attributes and indexes; access rules, validation rules and event handlers have none). The correspondence is the stored $ID, which attributeFromGen and indexFromGen round-trip into the semantic model — not a structural pairing. That is what makes a RENAME carry the GUID forward, as Studio Pro does when it renames the column, while a genuinely new attribute matches nothing and correctly gets a fresh GUID. Every ALTER form routes through UpdateEntity, so all six the report tested are covered, including SET DOCUMENTATION, which touches no attribute at all. Two more paths were not in the report and are fixed with them: CREATE VALIDATION RULE and the OData/contract entity refresh. GRANT was already safe — it mutates the loaded gen tree in place rather than rebuilding, which is the distinction that decides exposure here. Why nothing caught this for so long is worth recording, and the finding says so: the corruption is idempotent. The new GUID is derived from an $ID that TransplantIDs holds stable, so the second identical write produces byte-identical bytes, elision fires, and the run reports "Unchanged". A corruption that does not repeat is invisible to every same-vs-same check, including a re-run of the same script. Tests assert on raw BSON, since the reader never surfaces a GUID. Controls: with the carry stubbed, all six ALTER forms fail with the reported symptom (new GUID == stored $ID) and the rename loses its GUID. The index arm has no Studio Pro-authored fixture, so its stored state is seeded below the writer. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0192fPrkbPzU5gFTKnNYikAo
Changing an element's GUID while keeping its $ID is data loss with no diagnostic behind it. The model stays valid, so mx check passes and the build succeeds; the reader never surfaces a GUID, so DESCRIBE is byte-identical; and it only becomes visible when the package meets a database that already holds data. mendixlabs#1119 lost 28 attributes of 607 rows that way. Three guards already at the write choke point could not see it: - No-op elision compares a document with itself. The bad GUID is derived from an $ID the transplant holds stable, so the second identical write is byte-identical and gets elided — the damage is a one-shot. - TestFreshGUIDFieldsHaveAnIdentityDecision sees only codec FreshGUIDFields. A GUID derived from the $ID is registered as EmitGUID, a different mechanism, so it was never in that guard's view — the same blind spot that let Workflows$*.PersistentId through in mendixlabs#949. - identityFields/CarryIdentity reach only top-level properties of the document root; these GUIDs sit on nested elements. canon.StorageGUIDError pairs the two documents by $ID and reports any element whose GUID would change, in the spirit of DuplicateElementIDError: cheap, at the moment the bytes would land, and phrased as the message the user would otherwise never get. It runs AFTER TransplantIDs, which is what makes it exact — before the transplant a rebuilt element carries a freshly minted $ID that matches nothing, so there would be nothing to compare against. An element present on only one side, or a GUID only one side carries, is not a change, so ADD and DROP ATTRIBUTE do not trip it. It deliberately does not repair. Carrying a GUID on the transplant's structural pairing would trade a dropped column for something worse: a new member silently adopting a removed one's data under a name and type that no longer describe it. The transplant's matching is tolerant by design, because a wrong $ID match only makes a diff bigger. The carry therefore stays in the layer that knows which element is which (carryChildIdentity, previous commit). Wired into reconcileWithStored rather than beside the duplicate-$ID guard in updateUnit, so both choke points get it: WriteTransaction.WriteUnit is how codec.Store reaches storage, and a guard on only one of them is the inconsistency CLAUDE.md's "adding a write path means wiring it to canon.Reconcile" exists to prevent. One legitimate GUID writer exists, and the guard found it on the first full-suite run: marketplace.ApplyIdentities transplants a module's captured GUIDs onto the documents replacing it, which is what stops an update destroying that module's data and is exactly what Studio Pro's own update does. It opts out by name via UpdateRawUnitOwningStorageGUIDs, threaded through the backend interface with a mock stub of its own so a test cannot satisfy it by accident. Verified as a real safety net, not just a function: with the previous commit's carry stubbed out, an ALTER is now refused with a message naming the unit and the consequence, instead of silently corrupting. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0192fPrkbPzU5gFTKnNYikAo
…ive database
The fix and its guard were verified against unit tests and raw BSON. This runs
the whole thing on Mendix 11.13.0 + PostgreSQL 16 and records what the runtime
actually does, because one result changes how anyone should test for this class.
Subject: a blank `mxcli new` app, entity Administration.Account — Studio Pro
authored, so GUID != $ID on all three attributes — seeded with 607 rows.
1. mendixsystem$attribute.id holds the model GUID byte-for-byte once the .NET
field order is undone. Verified on all three attributes; it matches the
GUID and never the $ID. CLAUDE.md asserted this for an entity; it is now
measured for an attribute too.
2. Pre-fix binary (built from the parent commit in a worktree, so it is the
real shipped path and not a stubbed guard), one ALTER ENTITY … SET
DOCUMENTATION: all three GUIDs became equal to their $ID, `mx check`
reported 0 errors on the result, and boot then logged "ConnectionBus:
Executing 14 database synchronization command(s)" with fullname 607 -> 0
and email 607 -> 0.
3. The part nobody had measured: a recreated column with a MODEL DEFAULT is
silently backfilled with that default. The boolean IsLocalUser
(`default true`) read back as 607 non-null and looked untouched — until the
run was repeated with every row seeded false, which came back true. So the
loss can arrive wearing plausible data rather than empty cells, count(col)
proves nothing, and a destructive-rewrite test has to compare values that
could not have been guessed.
4. Fixed binary, same ALTER: GUIDs unchanged, and after redeploy all 607 rows
kept fullname, email and the non-default islocaluser=false.
Also worth knowing before trusting a green test: an entity mxcli CREATED is
immune, because its GUID equals its $ID from birth and a rewrite reproduces the
same value. Only a Studio Pro-authored entity can detect this.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0192fPrkbPzU5gFTKnNYikAo
MOVE ENTITY re-minted the moved entity's GUID, every attribute's GUID, and that of any association the move converted to a cross-association. #657 fixed the first for UpdateEntity and mendixlabs#1119 the second; MoveEntity got neither, and the third was its own defect in crossAssocFromGenAssoc. Since mendixlabs#1119 that third one also made MOVE ENTITY **fail** for any entity in an association — most real entities — because the conversion happens in place in the source unit, keeping the element's $ID, so the write guard could see the GUID move and refused. The other two stayed silent: the entity lands in a different unit, where its $ID pairs with nothing stored, and cross-unit moves are outside that guard's reach by construction. A module move is the most expensive case of this class, not a lesser one. Measured on Mendix 11.13.0 + PostgreSQL 16, same 250-row starting state both ways: GUID re-minted -> old table dropped, empty one created: 250 -> 0 rows GUID preserved -> table RENAMED (myfirstmodule$x -> administration$x, 3 DDL commands): 250 -> 250 rows, values intact The runtime resolves the entity by GUID, not by table name, so a move loses a whole table where an ALTER loses a column. That settles the question the issue left open: no warning is warranted, the carry is the fix. The entity and its children reuse UpdateEntity's two carries. A blanket raw carry is safe across a module change too: the unmodeled Image/ImageData are qualified names for a document that does not move with the entity, so keeping them is correct — without the carry a move dropped an entity's domain-model image as well as its GUID — and everything needing a rewrite is modeled, hence dirty. The cross-association could not reuse either mechanism. SetRaw passes the stored $Type through, and gen's SetDataStorageGuid is unusable twice over: bound under key DataStorageGuid where Studio Pro stores GUID (an existing keyaudit row), and typed string where the property is a 16-byte binary. So the conversion is now a RAW transform of the stored document — $Type rewritten, ChildPointer -> Child, the two on-canvas *Connection waypoints dropped, everything else verbatim — which preserves the GUID by construction rather than by remembering to copy it. The key set is taken from generated/metamodel, the arbiter: exactly the 13 keys DomainModelsCrossAssociation declares, confirmed against the emitted document. The property-by-property build stays as the fallback for an association with no stored bytes, and for a source document with no ChildPointer to rename. Controls: each of the three carries stubbed independently fails the new test with its own symptom, and the child-side move is refused before the fix and succeeds after it, end to end through the CLI. Not this change, and unchanged by it: MOVE ENTITY rewrites no qualified-name references, so moving a referenced entity leaves 33 CE1613s in a blank 11.13 app — the same 33 before and after. Closes #503 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0192fPrkbPzU5gFTKnNYikAo
A cross-module MOVE ENTITY reported success and left the project unbuildable:
33 CE1613s in a blank 11.13 app, every one a reference still naming the source
module — the entity (13), its attribute paths (11), the converted association (9)
— across microflow activities, page widgets, page parameters and access rules.
Nothing warned. Both directions are 33 → 0 now.
Two defects.
(1) No project-wide sweep. execMove returns early for ENTITY (cmd_move.go:42),
before the doctype switch AND before the isCrossModuleMove sweep every other
doctype gets. The comment there is right that an entity is not a top-level unit,
but the sweep is purely name-based and applies just the same. The machinery was
already there and already worked — the control is a sibling doctype:
$ mxcli -c "move microflow Administration.ChangeMyPassword to MyFirstModule"
Updated references in 1 document(s): Administration.… → MyFirstModule.…
The app contains: 0 errors.
so this was a missing call, not a missing capability.
(2) The moved entity's own ACCESS RULES kept Source.Entity.Member. MoveEntity
re-points the view source and each validation rule's attribute; access rules were
the missed sibling. That is worse than a dangling string, and the mechanism
generalises: entityToGen's syncMemberAccesses matches existing entries BY
QUALIFIED NAME, so the stale Source.Entity.Attr never equals the rebuilt
Target.Entity.Attr and it appends the new one while keeping the old. The entity
ends up carrying every member twice, half of them dangling — and DESCRIBE ENTITY
cannot show it, because it renders members bare, so the only visible trace is each
member appearing twice in the grant.
The sweep is driven by the old and new names the backend reports, never derived
from the module names, because the conversion is asymmetric: Mendix stores an
association in the module of its FROM entity, so the parent moving takes the
cross-association to the target (9 of the 33 errors name it) while the child
moving leaves it where it was (0 name it). Deriving would have corrupted the
second case; types.MovedAssociation{Old,New} makes it a no-op instead of a branch
the caller has to know about.
Controls: the access-rule rewrite stubbed leaves four stale refs; the sweep
stubbed reports no rewrite for the entity or the moved association, and a sweep of
an UNmoved association fails the test rather than passing quietly.
Found and NOT fixed, reproduced with a binary predating this and #503, so not
introduced by either: moving both endpoints of one association in sequence leaves
a pre-existing cross-association pointing at an element no longer in its unit, and
the project will not OPEN (AggregateException: the given key was not present in
the dictionary). MoveEntity's conversion loop walks AssociationsItems() and never
CrossAssociationsItems(). It is why the MDL bug-test keeps one endpoint of each
pair put.
Closes #605
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0192fPrkbPzU5gFTKnNYikAo
Backend.UpdateDomainModel removes and rebuilds the entire Entities and Associations lists, so every element arrived raw==nil and the codec's EmitGUID default wrote GUID = $ID across the whole domain-model unit — entities, their attributes and indexes, and every association, including the ones the statement never named. The reporter measured 282 moved GUIDs in one module (37 entities, 224 attributes, 21 associations) and a runtime crash: Cannot invoke "...Table.getTableName()" because "table" is null Its doc comment claimed it preserved "each element's identity", which was true of the $ID it explicitly re-sets and false of the GUID the runtime keys the database on (mendixsystem$entity.id / mendixsystem$attribute.id). This is the same defect as #657 (the ALTER target entity) and mendixlabs#1119 (that entity's children) on the other rebuild SHAPE. UpdateEntity swaps one entity into a list whose other members pass through as stored bytes, so siblings were never at risk; here there are no passthrough siblings at all, which is why one association edit moved every identity in the module. Six statements share the path, and the costliest is not an association statement: ALTER ASSOCIATION ... SET COMMENT ALTER ASSOCIATION ... SET OWNER CREATE OR MODIFY ASSOCIATION (even an identical re-run) RENAME ASSOCIATION RENAME ENTITY view-entity association sync (oql_view_associations.go) RENAME ENTITY is the expensive one: an entity's name is its table name, and per the #503 measurement on 11.13.0 + PostgreSQL 16 a preserved GUID makes the runtime RENAME the table (250 -> 250 rows) while a re-minted one makes it drop the table and create an empty one (250 -> 0). So this loses a whole table where an ALTER loses a column. The CLI's own help for CREATE OR MODIFY promised "preserves UUID. Safe to re-run" throughout — true of the $ID, false of the GUID. The fix indexes the stored entities and associations by $ID before the removal loops and carries each one's raw bytes (and, for an entity, carryChildIdentity) onto its rebuild. It is simpler than #503's cross-module move, which needed a raw transform because the $Type changes: assocToGen is Association -> Association, so a direct SetRaw is enough. The association carry also subsumes the property-by-property patch mendixlabs#872 made for the line anchors, which were lost to this same mechanism. On this branch the symptom was a refusal rather than corruption — the mendixlabs#1119 write guard pairs by $ID within the unit, so it caught all six and left them unavailable for any Studio Pro-authored module. Upstream, with no guard, the GUIDs move. Verified end to end on a Studio Pro-authored Marketplace module (Administration, Mendix 11.13.0): all five MDL statements land, the 16-byte GUID binaries in the rewritten .mxunit are 9 of 9 identical before and after, and mx check reports 0 errors. Before the fix each statement was refused, naming 9 elements with "stored <x>, would write <$ID>". Note the trap the MDL bug-test script documents: everything it creates is created by mxcli, whose GUID equals its $ID from birth, so a re-mint reproduces the same value and the script cannot fail on the bug. The assertions live on the raw BSON over a Studio Pro-authored fixture, keyed on $ID rather than name — a name-keyed census reads a RENAME as one element vanishing and another appearing. Closes mendixlabs#1169 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0192fPrkbPzU5gFTKnNYikAo
RENAME ENTITY and RENAME ASSOCIATION reported success and a reference count and
then left the renamed element's own member references stale, as CE1613 at build
time:
Renamed entity: Administration.AccountPasswordData -> Administration.PasswordData
Updated 24 reference(s) in 11 document(s)
[CE1613] "The selected attribute 'Administration.AccountPasswordData.OldPassword'
no longer exists." at Access rule of entity 'Administration.PasswordData'
That coincidence is the diagnosis. It is a clobber, not a missed sweep:
execRenameEntity / execRenameAssociation read the domain model, run the
project-wide RenameReferences pass — which does rewrite these names, in the raw
unit — and then persist the semantic model read BEFORE the sweep, putting the
stale names straight back. Access rules name a member Module.Entity.Attr,
validation rules hold the same string in AttributeID, and an association member
is Module.Association: all three embed a name the rename changes, and all three
live in the unit the persist overwrites.
Leaving them stale is worse than a dangling string, because entityToGen's
syncMemberAccesses matches existing entries BY qualified name: a stale
Module.Old.Attr never equals the rebuilt Module.New.Attr, so it appends the new
entry and keeps the old, and the entity carries every member twice with half the
entries dangling. DESCRIBE renders members bare, so duplication is the only
visible trace.
This is the RENAME sibling of #605, where the same names went stale on the module
prefix during a cross-module MOVE ENTITY. The two re-points here are deliberately
narrow: only names qualified by the entity are rewritten (an association member
carries no entity name), and the association match is exact rather than a prefix,
so a differently-named association that starts with the same text is left alone.
Pre-existing, not introduced by the GUID fix in the preceding commit — a binary
with only that commit stashed out produces an identical error count.
Verified on a Studio Pro-authored Marketplace module (Mendix 11.13.0): 4 errors
before, 0 after, with the GUID census unchanged at 9 of 9. One asymmetry is worth
knowing and is documented in the bug-test script: against an access rule mxcli
wrote, a stale association name raises no error at all, because the MemberAccess
still carries a valid element pointer and the platform resolves the pointer — so
the attribute half is provable synthetically and the association half only shows
against a document Studio Pro authored.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0192fPrkbPzU5gFTKnNYikAo
…ssue The header explained why the file keeps one endpoint of each pair put but had no issue to send a reader to. #628 now carries the reproduction and the three unhandled cases. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0192fPrkbPzU5gFTKnNYikAo
An entity-type (`entity <>`) argument to CALL JAVASCRIPT ACTION or CALL JAVA ACTION was stored under EntityTypeCodeActionParameterValue.Entity as the argument's expression text. The visitor keeps trailing whitespace on call arguments so expressions round-trip, so an argument on its own line was stored as "Mod.Entity\n", and mxbuild then rejects it with CE1613 "The selected entity ... no longer exists". `mxcli check --references` passed. Both builders now resolve the value through one helper that trims before treating the text as a name or a $variable. Measured on mxbuild 11.6.6 (testdata/expr-checker/minimal.mpr): the pre-fix build gives CE1613, the fixed build gives 0 errors. The report's CE0115 came from v0.23.0 predating the mendixlabs#1137 fix; this is the defect left once that fix is in. Fixes mendixlabs#1171 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MbizCPCsY84wd4bWeJzcHN
On Windows, `mxcli test --local` ran the Linux mxbuild from the CDN cache and died with "mxbuild --serve did not become ready", and the reporter found no flag or environment variable to point it at Studio Pro's mxbuild.exe. The platform part was already fixed after the reporter's v0.21.0: mendixlabs#916 resolves Studio Pro before the cache on non-Linux hosts, and mendixlabs#1122 makes the serve step use the resolved binary. What was still missing is the override. `run` gained --mxbuild-path in mendixlabs#1125, but `test --local` boots through the same resolver, prints the same "pass --mxbuild-path" guidance, and answered `unknown flag`. - register --mxbuild-path on `test` and carry it through RunOptions -> localAppOptions -> LocalAppOptions.MxBuildPath (both runners) - honour MXCLI_MXBUILD_PATH in resolveMxBuildForLocalOn, so `run --local` and `test --local` both get it; the flag wins when both are set - extend TestErrorGuidanceNamesAFlagThatExists to every command that reaches the resolver. It checked only `run`, which is how `test` was missed Control: dropping only the MxBuildPath line from localAppOptions fails the plumbing test for both runners; before the fix the CLI test failed with `unknown flag: --mxbuild-path` and the env test with the "Linux binary cannot run natively on windows" refusal. Fixes mendixlabs#1086 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AMutqdLPkBYyEjm84x3fHd
…(CE7033) `call microflow M.F() in queue M.Q` where F returns Boolean passed `mxcli check --references` and failed the build with CE7033 "A microflow used for background execution must have a Microflow return type of 'Nothing'." (mendixlabs#1064). The call and the queue both resolve; nothing compared the binding with the signature of the flow it names. MDL088 reports it without a project when the script creates the target, and under --references for a stored target. Measured on mxbuild 11.12.0: Boolean target -> CE7033, void target -> 0 errors. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017FzoJCMYbcBApUHGV49E3X
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017FzoJCMYbcBApUHGV49E3X
The mendixlabs#1119 write-up took CLAUDE.md to 29,605 bytes against a 28,000 budget (#611), which TestRepoClaudeMDStaysWithinItsContextBudget fails on. The measurement narrative and the testing method have a canonical home now — docs-wiki/bug-patterns/rewrite-drops-unauthored-state.md, which this session synced — so what stays here is the invariant and the routing: - preserve the stored GUID on a rewrite, and the guard that refuses otherwise, including the one opt-out and why abusing it reships mendixlabs#1169 - the carry is per rebuild SHAPE, not per element type, and the call sites are the blast radius rather than the reported statement - a move (and a rename, since the name is the table name) loses a whole table - check the subject before trusting a GUID test Detail that only matters while measuring one of these — the 607-row run, the default-backfill trap, the column-order tell, the transplant mechanics behind the guard's pairing — now lives in the pattern page and the findings, and is linked from here. 27,881 bytes. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0192fPrkbPzU5gFTKnNYikAo
fix(check): report a queued call to a microflow that returns a value (CE7033)
fix(test): accept --mxbuild-path and MXCLI_MXBUILD_PATH for test --local
…ut IDs
CREATE OR MODIFY ENTITY reaches UpdateEntity through
mergeDeclaredOntoStoredEntity, which sets
merged.Attributes = declared.Attributes
merged.Indexes = declared.Indexes
to the lists the STATEMENT declares. Those are built from text by the visitor and
carry no element ID at all. carryChildIdentity keyed entirely on that ID and read
an empty one as "a genuinely new member, so a fresh GUID is right", so
re-declaring an existing entity re-minted every attribute GUID — which drops
every column on the next deploy.
Same data loss as mendixlabs#1119, through a second executor path. The write guard is what
surfaced it, as a refusal on a doctype script that had been passing for months:
failed to update entity: refusing to write unit d82b0484-...: 1 element(s)
kept their $ID but would be written with a different GUID
dff2ced1-... (DomainModels$Attribute): stored 4b52b36b-..., would write dff2ced1-...
Each list is now paired in two passes: the exact key first, the weaker one only
for what it left over, with each stored element claimable once.
attributes by semantic $ID, then by NAME. A name is unique within an entity,
so a stored attribute of that name is the same member — which is
what Studio Pro assumes when a re-declared attribute keeps its
column. Claiming once is what stops a rename-plus-re-add from
handing the newcomer the renamed member's data: the ID match takes
the stored element and the name fallback finds it claimed.
indexes by semantic $ID, then by POSITION, which is all an index has (no
name) and is the correspondence the ID-bearing branch already
trusts. A reordered or dropped index pairs the wrong way round;
that costs an index rebuild and no data, because nothing the
platform keys on rides on an index GUID. What a MISSING carry costs
is the write itself.
Tests reproduce the CI symptom as a backend unit test in half a second (strip the
IDs off a fixture entity's members, call UpdateEntity), with the fixture's
GUID != $ID precondition asserted first — an element mxcli created has them equal
from birth and cannot detect this. The new-attribute control asserts a genuinely
introduced member still gets a fresh GUID and does not inherit a neighbour's.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0192fPrkbPzU5gFTKnNYikAo
The guard's premise was wrong, and its own doc comment stated the error outright: that after TransplantIDs, "an element sharing an $ID with a stored one is an element the transplant judged to be the same element — so a differing GUID on it is unambiguously a rewrite". TransplantIDs pairs STRUCTURALLY — $Type and shape, LCS-anchored — and its correctness bar is low on purpose, because a wrong $ID match only makes a diff bigger. Feed that pairing to a guard and every wrong match becomes a refused write. Measured: `CREATE OR MODIFY PERSISTENT ENTITY BusinessEvents.PublishedBusinessEvent (EventId: long)` against the marketplace module, whose entity carries six attributes and none named EventId. The statement drops all six and adds one; the transplant paired the NEW attribute with one of the REMOVED ones and handed it that stored $ID; the codec had written GUID = $ID and the transplant substitutes over every 16-byte binary, so the GUID followed. The guard saw a "changed" GUID on a "kept" $ID and refused a write that corrupts nothing. Pairing is now $ID plus the member's own identity: same $Type, and the same Name where the element has one (an index has none, so $Type is the whole test there). What this costs, stated plainly: a RENAME that re-minted a GUID is no longer refused, because the name is what changed. That arm is checked directly where it is decidable — TestIssue1119_AlterPreservesAttributeGUIDs has a RenameAttribute case asserting the GUID moves to the new name. A backstop that refuses correct writes is worse than a backstop with a hole: the first makes documented statements unusable, and this one already had. The test asserts both halves together, because either alone is satisfiable by a guard that is simply wrong — dropping the name check makes the false-positive case fire, and never firing at all makes it pass. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0192fPrkbPzU5gFTKnNYikAo
CLAUDE.md said the guard's pairing was made "exact" by running after the
transplant. It is not — that is the claim the false positive disproved — so the
rule now states what it actually pairs on and that a shared $ID alone is not one
member, plus the carry rule the CREATE OR MODIFY case added (key on name as well
as $ID, since a declared member has no ID). Trimmed elsewhere in the same section
to stay inside the context budget, with margin this time.
Three insights added to the bug-pattern page, each of which cost time here:
- A guard built on an approximate pairing promotes that pairing's error rate
into refusals. Anything consuming an approximate correspondence has to add its
own test of identity, and the resulting narrower guard is the right trade.
- The same error message can carry two defects, and fixing one leaves it
byte-identical. An unchanged failure after a genuine fix means the
reproduction exercises a path the diagnosis did not — here, describing the
real stored document settled it, because the declared member shared no name
with anything stored and the pairing itself was spurious.
- A fast local reproduction and a slow realistic one find different defects.
Only the integration run could expose a spurious pairing, which needs a real
drop-six-add-one document. Build the fast one to iterate; keep running the
slow one to decide.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0192fPrkbPzU5gFTKnNYikAo
MPR011 fired on every `while` loop mxcli wrote — "first activity at (50,80)
lies outside the loop box" — single-level loops included. mxcli's own lint
rule was correctly flagging mxcli's own output; `mx check` passes and the app
runs, the flow just renders wrong in Studio Pro.
One missing term:
addLoopStatement (foreach) innerStartX = LoopPadding + iteratorSpace
+ ActivityWidth/2 = 210
addWhileStatement innerStartX = LoopPadding = 50
A microflow object's Position is its CENTRE — the builder says so itself
("Position is the CENTER point (RelativeMiddlePoint in Mendix)") — so with
ActivityWidth 120 a centre at x=50 puts the left edge at -10.
addWhileStatement's doc comment says its layout "matches addLoopStatement but
without iterator icon space". Dropping the iterator space is right; taking
ActivityWidth/2 with it was not, because that term is not iterator space — it
is what converts a centre to a left edge. The next line proves the omission
was accidental: innerStartY adds ActivityHeight/2 for exactly this reason.
The reported (50,80) matches term for term.
Why it survived: the containment invariant WAS tested — loop_containment_test.go
exists from mendixlabs#884 and asserts precisely this — but every fixture in it built a
FOREACH loop. Two loop builders, one covered, and the uncovered one shipped the
violation to every project that writes a while. An invariant is worth what its
coverage is.
Tests written failing first, reproducing the geometry to the pixel:
children span x[-10,110] y[50,110], box is 200x160
children span x[-10,270] y[50,110], box is 320x160
children span x[-10,590] y[50,110], box is 640x160
children span x[-10,1070] y[50,110], box is 1120x160
One asserts containment at 1/2/4/7 activities, the other the first child's
left edge specifically, so a regression cannot hide behind a box that merely
grew wider on the right.
Still uncovered: addManualWhileTrueStatement, the third loop builder.
Closes #645
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MgcYSQrLLbUcnAMaCHpyqQ
Reported from a project that patched its own copy of the rule and asked for the fix upstream: CONV010 allowed MicroflowCallAction but not NanoflowCallAction, so an ACT_ nanoflow could satisfy it in no way at all — delegate and be flagged, or inline the logic and be flagged. microflows() yields nanoflows too (the catalog's microflows table carries a MicroflowType column), so CONV010 lints them, and a nanoflow delegates with a nanoflow call. Third time this one allowlist has been short, and the rule's own comments record the other two: the wrong vocabulary entirely (storage names where the catalog reports SDK names — matched nothing, 11 false positives of 13 findings) and a missing ExclusiveMerge that a permitted ExclusiveSplit necessarily creates (122 hits on one project). The recurring shape is a rule that cannot be satisfied, and its cost is asymmetric: it does not read as a broken rule, it reads as broken code, so users refactor around it or patch it locally and the defect never comes back upstream. The vocabulary pin test existed to stop this class and did not, because its `permitted` list is hand-maintained and was itself incomplete. NanoflowCallAction is added there too, but pinning to a hand-written list only moves the completeness problem — noted in the finding. Proven by revert: removing the entry fails the test with "CONV010 does not allow \"NanoflowCallAction\", which is what the catalog labels this action." Closes #644 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MgcYSQrLLbUcnAMaCHpyqQ
…conversion 8692785 made StorageGUIDChanges pair elements on $ID plus sameMember, which required an equal $Type and an equal Name. The $Type half went blind to the one conversion mxcli performs. MOVE ENTITY of an association's TO side re-types a DomainModels$Association as a DomainModels$CrossAssociation in place, under the same $ID and Name. That is the arm that exposed #503, and after 8692785 a regression there wrote silently: with MoveEntity's carries stubbed on main, the child-side move succeeded where the issue records a refusal. The type clause excluded no error of the pairing it sits on. TransplantIDs never pairs across a $Type (pairDoc stops at the mismatch; TestTransplantIgnoresMismatchedTypes), so an $ID shared by two types can only be one a writer kept on purpose. The mis-pairing 8692785 fixed is same-type, different-name, and that stays excluded. sameMember is now: the Name when both sides have one, whatever the $Type; the $Type when neither does (an index); not a match when only one side is named. The comment above GUIDChange now lists every arm the guard gives up (renames, elements that change unit, a name on one side only) and where each is checked. The table case DifferentType_NotAChange pinned the old decision on the grounds that "nothing authors this today". MoveEntity did. It is replaced by DifferentTypeSameName_IsAChange, plus two cases pinning what stays excluded. Controls: - New TestStorageGUIDChanges_TypeConversionIsStillTheSameMember on the old sameMember: "got 0 change(s), want 1". - MoveEntity carries stubbed, TestIssue503: the child-side move is refused again ("refusing to write unit 119210cd-...: 1 element(s) kept their $ID but would be written with a different GUID"). The parent-side move still passes the guard, because the element changes unit (a documented hole, covered by TestIssue503 itself). - The 8692785 false positive does not return. BusinessEvents 3.12.0 installed with `marketplace install --file` into a copy of testdata/expr-checker, then `create or modify persistent entity BusinessEvents.PublishedBusinessEvent (EventId: long)`: accepted with this change, refused by a build whose guard pairs on $ID alone. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01DEpyWoAjjwovUm9hAEEAH7
The guard no longer compares $Type for a named element (bc4a832); the rule line still said "$ID + $Type + Name". Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01DEpyWoAjjwovUm9hAEEAH7
…tions The proposal reasoned from one session. `diag loop-report` has now run against three complete builds, and the result contradicts two things it assumed. Both corrections are recorded rather than edited away, because how each assumption survived is the reusable part. **`check` is not the dominant call.** The ledger's 83% was read as the shape of an mxcli loop, and "why so many checks?" was queued as the next attack. Two further projects put check at 15% and 22%. The ledger is the outlier, not the archetype — a distribution from one project is an anecdote with a table around it. (All three logs predate #629, so their `-c` and `exec` counts are inflated by mxcli's own child processes, which is the direction that matters: it inflates exactly the two commands that displaced check.) **The wall-time lever is `run`, and the report understates it.** A killed boot writes no summary record, so its duration is not counted at all — 26 of 30 and 27 of 34 run invocations are uncounted, and the reported totals are floors. Boot count times per-boot median gives ~35 min in demo-2 and ~50 min in CapTrack, against sessions of 5 h and 2 h 45. That also corrects CapTrack's own "only ~30 min was spent inside mxcli": the boots it excludes more than double it. **And 11.14 was not the whole story.** An earlier revision measured the mxbuild serve-rebuild defect and let it explain the restart-per-change. mxcli-demo-2 ran 11.13 — where this proposal's own control measured a 3.4 s hot reload — and still took 30 full restarts. The two projects are a natural experiment: on 11.14 the restart is forced, on 11.13 it is chosen. The first draft said "the fast paths exist, the session just didn't take them"; the 11.14 measurement contradicted it and the correction went one step too far, replacing "defaults problem" with "mxbuild problem" when both are true on different versions. A measurement that explains a symptom on one version does not retire the hypothesis on the others. Sequencing gains 2c (make `--watch` the default invocation — the largest wall-time item in the table and the cheapest, a default in files that already exist) and 2d (count a killed `run`), with the dependency stated: a lever that removes uncounted time cannot be shown to have worked until 2d lands. Lever 6 now also records that testing the instrument against real projects found three defects in it (#617, #620, #629), each of which had been skewing the numbers it exists to produce, and names the two things it still cannot see. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MgcYSQrLLbUcnAMaCHpyqQ
39e617e pushed CLAUDE.md to 28166 bytes, past the 28000-byte budget that TestRepoClaudeMDStaysWithinItsContextBudget enforces, and failed build-and-test. The pairing-rule line is shortened to the same content (27988 bytes). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01DEpyWoAjjwovUm9hAEEAH7
Two rules that contradicted themselves, from a third test project
fix: the storage-GUID guard must see a GUID lost in an in-place type conversion
Upstream mendixlabs#1176: `describe microflow` printed `$obj = import from mapping M.IMM($s) all;` for an activity that binds a single object, which reads as a list import. The stored Range really is All (Studio Pro's ConstantRange{SingleObject: false} for an object-rooted mapping), and mendixlabs#881 deliberately always printed it because a missing keyword then stored First. The later runtime fix made the builder write a missing keyword as All explicitly, so the bare form and `all` now build the same activity -- but the formatter was never revisited. It now omits `all` when the result is an object and keeps it for a list. Verified on a fresh 11.12.3 project: both spellings store identical ResultHandling (ConstantRange All + ObjectType), mx check 0 errors, and exec'ing the described bare text reports "Unchanged microflow". Control: the build without this change describes both with `all`. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NSHguJgWtvazhgmaCDmrZg
fix: describe leaves `all` off an import that returns one object
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
22 commits from the fork's
main, on top of the last sync (57259e8). The bulk of it is one class of defect and the guard built to stop it recurring.Storage GUIDs: silent data loss on every rewrite
An element's
GUIDis what the runtime keys the database on —mendixsystem$entity.idandmendixsystem$attribute.idhold it verbatim. Move it and the synchroniser treats the element as deleted-and-re-added: it drops the column (or the whole table) and loses the data. The model stays valid throughout, somx checkpasses, the build succeeds, DESCRIBE is byte-identical, and nothing is visible until the package meets a database that already holds rows. Reported from production: 28 attributes of 607 rows emptied by one edit.a3814c1, Any MDL write to an existing entity regenerates attribute storage GUIDs → Mendix DB sync drops and recreates every column (data loss) #1119) — every ALTER of an existing entity reset each attribute's GUID to its own$ID. This is the unclosed half of ALTER ENTITY ADD ATTRIBUTE drops system members (createdDate) on sibling generalization root — CE1613 across inheriting entities #657: that carried the target entity's raw bytes and relied on siblings surviving through the list-rebuild passthrough, but the target's own children are all rebuilt and nothing carried theirs. The correspondence is the stored$ID, not a structural pairing, which is what makes a RENAME carry the GUID forward — as Studio Pro does when it renames the column — while a genuinely new attribute correctly gets a fresh one. Every ALTER form routes through one function, so all six the report tested are covered, includingSET DOCUMENTATION, which touches no attribute at all. Why nothing caught it for years: the corruption is idempotent. The bad GUID is derived from an$IDthe transplant holds stable, so the second identical write is byte-identical, elision fires, and the run reports "Unchanged" — invisible to every same-vs-same check, including re-running the same script.76d0720) — three guards already at the write choke point could not see this: no-op elision compares a document with itself; the fresh-GUID test sees only one of the two codec mechanisms (the same blind spot that letWorkflows$*.PersistentIdthrough in Workflow activity PersistentId is re-minted on every write (never preserved, never read back) #949); andidentityFieldsreaches only top-level properties, while these GUIDs sit on nested elements. The new guard pairs the two documents and reports any element whose GUID would change — cheap, at the moment the bytes would land, phrased as the message the user would otherwise never get. It deliberately does not repair: carrying a GUID on a tolerant structural pairing would trade a dropped column for something worse, a new member silently adopting a removed one's data under a name and type that no longer describe it. Wired into the shared reconcile step so both write choke points get it. One legitimate GUID writer exists and the guard found it on the first full-suite run — the marketplace module update, which is exactly what Studio Pro's own update does — and it opts out by name through the backend interface, with its own mock stub so a test cannot satisfy it by accident.4f9ba5a) — Mendix 11.13.0 + PostgreSQL 16, a Studio Pro-authored entity seeded with 607 rows, pre-fix binary built from the parent commit so it is the real shipped path. OneALTER ENTITY … SET DOCUMENTATION: all three GUIDs became equal to their$ID,mx checkreported 0 errors, and boot logged 14 synchronization commands with two columns going 607 → 0. The part nobody had measured: a recreated column with a model default is silently backfilled with that default. A boolean withdefault trueread back as 607 non-null and looked untouched — until the run was repeated with every row seeded false, which came back true. So the loss can arrive wearing plausible data,count(col)