Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 12 additions & 0 deletions .changeset/26760-formula-label-eval.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
---
'@objectstack/skills': patch
---

docs(skills): objectstack-formula evals check that a record name built from a select field reaches for `label(record.kind)`, and that a predicate never calls `label()`

Clause-②: no
Metadata-protocol: none

- **Before.** `skills/objectstack-formula/SKILL.md` teaches `label(record.f)`, but no eval in `evals/cel-predicates-formulas.json` exercised it: an agent authoring a record name from a select field could hand-roll a ternary over the stored value with hard-coded English or Chinese strings, and nothing caught it; nor did anything catch `label()` written into a validation rule, where it faults and aborts every write.
- **Now.** Eval 6 asks for a `contract_version` name built from its `kind` select and expects `label(record.kind)` in an `F` formula, refusing a `== 'clean' ?` ternary. Eval 7 asks for a validation rule and a `visibleWhen` on the same object and refuses `label(` in either, expecting predicates over the stored value. Four sentences in evals 1, 2, 3 and 5 that restated their own `must_not_contain` assertions are gone; every fact and assertion stays.
- No schema, key or behaviour changes.
13 changes: 12 additions & 1 deletion scripts/check-skills-token-ratchet.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -863,7 +863,18 @@ export const CEILINGS = new Map([
// Maintainer ruling 2026-10-27 on the decision card 「skills 修复卡的教学和 eval 都撞 token 天
// 花板,怎么付?」, tap A, verbatim and untranslated: 「A:先削重复,余额按实测提高」.
['skills/objectstack-data/evals/objects-fields-relationships.json', 1050],
['skills/objectstack-formula/evals/cel-predicates-formulas.json', 1046],
// 1046 -> 1352 (card #26760): two eval cases for the `label(record.FIELD)`
// teaching — case 6 asks for a record name built from a select field and
// expects `label(record.kind)`, never a hard-coded label ternary; case 7
// asks for a validation and a `visibleWhen` and refuses `label(` there,
// where it faults and aborts the write. Maintainer ruling 2026-10-27 on the
// decision card 「skills 修复卡的教学和 eval 都撞 token 天花板,怎么付?」, tap A,
// verbatim and untranslated: 「A:先削重复,余额按实测提高」. PAID DOWN FIRST:
// four sentences in cases 1, 2, 3 and 5 that restated their own
// `must_not_contain` assertions, 212 bytes = 53 units (1046 -> 993); the two
// cases add 1438 bytes = 359 units (993 -> 1352); pinned AT the measurement,
// zero headroom. The raising PR's body carries the same arithmetic.
['skills/objectstack-formula/evals/cel-predicates-formulas.json', 1352],
// 1004 -> 1451 (2026-10-27, the objectstack-i18n fix card of the skills gap
// audit): two eval prompts for the teaching that card adds — the `reports`,
// `positions`, `permissions` and `picklists` groups; a form view `title`, a
Expand Down
28 changes: 24 additions & 4 deletions skills/objectstack-formula/evals/cel-predicates-formulas.json
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@
{
"id": 1,
"prompt": "Add a `full_name` formula on `contact` built from salutation, first name and last name, and a `margin_pct` formula on `deal` = (revenue - cost) / cost * 100.",
"expected_output": "Two `type: 'formula'` fields whose `expression` is CEL via the `F` tag. The text formula coalesces every nullable operand — `F`coalesce(record.salutation, '') + ' ' + coalesce(record.first_name, '') + ' ' + coalesce(record.last_name, '')`` — because `null + 'foo'` throws. The numeric one guards the divisor: `F`coalesce(record.cost, 0) > 0 ? ((coalesce(record.revenue, 0) - record.cost) * 100) / record.cost : 0.0``. Field references are bare `record.<field>`, never brace-wrapped.",
"expected_output": "Two `type: 'formula'` fields whose `expression` is CEL via the `F` tag. The text formula coalesces every nullable operand — `F`coalesce(record.salutation, '') + ' ' + coalesce(record.first_name, '') + ' ' + coalesce(record.last_name, '')`` — because `null + 'foo'` throws. The numeric one guards the divisor: `F`coalesce(record.cost, 0) > 0 ? ((coalesce(record.revenue, 0) - record.cost) * 100) / record.cost : 0.0``.",
"files": [],
"assertions": {
"must_contain": ["F`", "coalesce(", "record."],
Expand All @@ -14,7 +14,7 @@
{
"id": 2,
"prompt": "Translate this Salesforce validation rule into ObjectStack: AND(ISBLANK(po_number), amount > 10000, NOT(ISPICKVAL(status, 'draft'))).",
"expected_output": "Applies the mechanical legacy → CEL table: `P`isBlank(record.po_number) && record.amount > 10000 && record.status != 'draft'``, noting that a `script` validation is inverted (it FAILS when the expression is true, so keep the failing condition) and that `record.amount > 10000` on a declared-nullable field needs an explicit `record.amount != null &&` guard or the publish gate rejects it. Uses `'draft'` single quotes and bare `record.` references.",
"expected_output": "Applies the mechanical legacy → CEL table: `P`isBlank(record.po_number) && record.amount > 10000 && record.status != 'draft'``, noting that a `script` validation is inverted (it FAILS when the expression is true, so keep the failing condition) and that `record.amount > 10000` on a declared-nullable field needs an explicit `record.amount != null &&` guard or the publish gate rejects it.",
"files": [],
"assertions": {
"must_contain": ["isBlank(", "&&", "record.", "P`"],
Expand All @@ -24,7 +24,7 @@
{
"id": 3,
"prompt": "Run an automation only when a support case has just become `escalated` — on single updates and on bulk updates alike.",
"expected_output": "Writes it once as `P`previous.status != 'escalated' && record.status == 'escalated'``: `record` is the post-write state (true on every later update too), so only `previous` says \"just became\"; `previous` is bound per row on `before*`/`after*` update events, bulk too. On insert `previous` is `null`: the `previous.status` read faults and aborts the write, so to run on inserts guard with `previous != null &&`. No `ISCHANGED()` function.",
"expected_output": "Writes it once as `P`previous.status != 'escalated' && record.status == 'escalated'``: `record` is the post-write state (true on every later update too), so only `previous` says \"just became\"; `previous` is bound per row on `before*`/`after*` update events, bulk too. On insert `previous` is `null`: the `previous.status` read faults and aborts the write, so to run on inserts guard with `previous != null &&`.",
"files": [],
"assertions": {
"must_contain": ["previous.status", "record.status", "!="],
Expand All @@ -44,12 +44,32 @@
{
"id": 5,
"prompt": "Hide `discount_reason` unless a discount amount was entered, and reject any `end_date` earlier than `start_date`.",
"expected_output": "`visibleWhen: P`record.discount != null`` (or `!isBlank(record.discount)`) on the field — `has(record.discount)` is not a null check: every declared field exists, so it is always true. The date rule is a `script` validation whose predicate dominates the comparison with explicit null tests: `record.start_date != null && record.end_date != null && record.end_date < record.start_date`; a `has()` guard does not satisfy the publish-time gate. Canonical property names only — never `conditionalRequired`.",
"expected_output": "`visibleWhen: P`record.discount != null`` (or `!isBlank(record.discount)`) on the field — `has(record.discount)` is not a null check: every declared field exists, so it is always true. The date rule is a `script` validation whose predicate dominates the comparison with explicit null tests: `record.start_date != null && record.end_date != null && record.end_date < record.start_date`; a `has()` guard does not satisfy the publish-time gate.",
"files": [],
"assertions": {
"must_contain": ["visibleWhen", "!= null", "record.end_date"],
"must_not_contain": ["conditionalRequired:", "{record."]
}
},
{
"id": 6,
"prompt": "Name `contract_version` records by their version number and the `kind` select, so a zh-CN reader sees `v2 · 清稿` and an en-US reader `v2 · Clean`.",
"expected_output": "A `type: 'formula'` `nameField`: `F`'v' + string(record.version_no) + ' · ' + label(record.kind)``. `label()` takes exactly `record.<field>` of a select field and reads its option label in the reader's locale (`''` for null). The stored value is the same in every locale, so a ternary over it with hard-coded strings cannot follow the reader.",
"files": [],
"assertions": {
"must_contain": ["label(record.kind)", "F`"],
"must_not_contain": ["== 'clean' ?", "{record."]
}
},
{
"id": 7,
"prompt": "Refuse a `contract_version` save unless its `kind` is the clean copy, and show `signed_at` only once `status` is signed.",
"expected_output": "Both are predicates over the STORED value: a `script` validation `P`record.kind != 'clean'`` (it fails when true) and `visibleWhen: P`record.status == 'signed'``. The `label` macro evaluates only in a formula field; in a validation, `visibleWhen` or hook predicate it faults and aborts every write, so compare the stored value, never its option label.",
"files": [],
"assertions": {
"must_contain": ["record.kind", "visibleWhen", "P`"],
"must_not_contain": ["label(", "{record."]
}
}
]
}
Loading