Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
15 commits
Select commit Hold shift + click to select a range
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
39 changes: 39 additions & 0 deletions .changeset/25779-seed-files.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
---
'@objectstack/spec': minor
'@objectstack/metadata-protocol': minor
'@objectstack/service-storage': minor
'@objectstack/runtime': minor
'@objectstack/cli': minor
---

feat(spec,metadata-protocol,service-storage,runtime,cli)!: a seed record can carry a file the package ships, `seedFile(path)`, and a file field's id that names no stored file is refused

Clause-②: yes (narrowing)
Metadata-protocol: narrowed

<!-- adr-0087: not-required (no-migration-prescription) No key of any metadata schema is removed, renamed or re-shaped. The narrowing is a seed load refusing a value it used to write: an id-shaped string in a file field that names no stored file. There is nothing for `objectstack migrate meta` to rewrite and no tombstone; the fix is the author's choice of the file the field should hold. -->

A `file`, `image`, `avatar`, `video` or `audio` field stores the id of a stored file, and only an upload made one, so a seed could not fill a required file field at all. A seed record now names the file by the path the package ships it at:

```ts
import { defineSeed, seedFile } from '@objectstack/spec/data';

defineSeed(ContractVersion, {
externalId: 'code',
records: [
{ code: 'NDA-001-V1', file: seedFile('fixtures/nda-v1.pdf') },
{ code: 'NDA-002-V1', file: seedFile('fixtures/nda-v2', { name: 'NDA v2.pdf', contentType: 'application/pdf' }) },
],
});
```

- **The path** is relative to the directory holding `objectstack.config.ts`, uses `/`, and stays inside it: an absolute path, a drive letter, a `\`, or an empty, `.` or `..` segment is refused at parse. A `multiple: true` file field takes an array of `seedFile(...)` values. `defineSeed` refuses `seedFile(...)` in a field that does not store a file.
- **`os build` / `os compile`** reads each referenced file and writes its bytes into the dataset's new `files` map (`{ contentType, size, sha256, data }`, base64), so the compiled artifact carries them to every boot that loads it: `os dev`, `os serve`, a published package, an artifact URL. A path that names no file, or a file whose type is neither named with `contentType` nor known by its extension, fails the build, and `os validate` refuses the same paths. `os serve` / `os dev` booting a config rather than an artifact reads the files the same way, so both boots seed the same files.
- **The seed loader** stores the bytes through the new `field-files` service (`FIELD_FILE_SERVICE`, `IFieldFileService`, registered by `@objectstack/service-storage`), writes the file's id into the field, and makes the written record the file's owner, the (object, record, field) ownership an upload gets. Each row gets a file of its own. A replay that finds the row holding a file stored from the same path with the same bytes, name and type keeps it, so a restart does not store a new file for an unchanged fixture. A file stays `pending` until its row lands, so a file whose row is refused is reclaimed by the pending-upload retention.
- **Refused, by row, before the row is written**: a reference whose bytes the dataset does not carry, bytes that do not match their recorded size and SHA-256, a load with no `field-files` service, a field mixing `seedFile(...)` with other values, and several `seedFile(...)` values in a one-file field. Each is counted, reported in the load's errors and logged at `error`.
- **The inline boot seed** of an app whose datasets carry seed files runs at `kernel:bootstrapped` (as one with `demoStaff` already does), once the storage service has read its settings, so the files land in the store the deployment serves from.
- `SeedLoaderService` takes an optional fourth argument, `{ getFieldFiles }`; every platform caller passes the kernel's `field-files` service.

**BREAKING** (a seed-load narrowing). A seed record whose file field holds an id-shaped string (letters, digits, `_` and `-`, at most 64 characters) that names no stored file in the row's organization is now refused, by row, with an error naming the field and the value. Before, it was written as given, and the field then read as holding no file. An external URL or an inline value is not an id and is written as before; so is any value on a deployment without `@objectstack/service-storage`.

**What to do.** If a seed load now reports `… holds '<value>', which names no stored file`, replace the value with the file the field should hold, `seedFile('<path in the package>')`, or remove it.
44 changes: 44 additions & 0 deletions content/docs/data-modeling/seed-data.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -440,6 +440,47 @@ duplicates on every replay boot ([#3434](https://github.com/objectstack-ai/objec

---

## File Fields — `seedFile(path)`

A `file`, `image`, `avatar`, `video` or `audio` field stores the id of a stored
file, and a seed has no upload to make one. Name the file by the path your
package ships it at instead, and the platform stores it for the record:

```typescript
import { defineSeed, seedFile } from '@objectstack/spec/data';

export const contractVersionsSeed = defineSeed(ContractVersion, {
externalId: 'code',
records: [
{ code: 'NDA-001-V1', file: seedFile('fixtures/nda-v1.pdf') },
// `name` is what a download is saved as; `contentType` for an extension
// the build does not know.
{ code: 'MSA-001-V1', file: seedFile('fixtures/msa-v1', { name: 'MSA v1.pdf', contentType: 'application/pdf' }) },
// A `multiple: true` file field takes an array.
{ code: 'SOW-001-V1', file: seedFile('fixtures/sow.pdf'), scans: [seedFile('fixtures/p1.png'), seedFile('fixtures/p2.png')] },
],
});
```

- **The path** is relative to the directory holding `objectstack.config.ts` and
stays inside it — `/` separators, no leading `/`, no `.` or `..` segment.
- **`os build`** reads each referenced file and bundles its bytes into the
compiled artifact, so the seed carries them wherever the artifact goes
(`os dev`, `os serve`, a published package). A path that names no file fails
the build, and `os validate` refuses it too. `os serve` booting the config
itself reads the files the same way.
- **The seed load** stores each file and writes its id into the field; the
record owns the file exactly as if it had been uploaded into that field. Every
row gets a file of its own, even when two rows name the same path. A replay
keeps a file whose bytes did not change, so restarting does not store it
again; changing the file and rebuilding replaces it.
- **Refused, by row, loudly**: a reference whose bytes the build did not bundle
(a dataset written by hand without its `files`), bytes that do not match what
the build recorded, and a file field holding an id-shaped string that names no
stored file — write `seedFile(...)` there instead.

---

## Dynamic Values (CEL)

Any field value may be a **CEL expression** evaluated at install time against a
Expand Down Expand Up @@ -639,6 +680,9 @@ function defineSeed<
records: Array<SeedRecord<TObj['fields']>>; // declared fields + injectable system columns
}
): Seed

// A file the package ships, for a file-class field (see "File Fields").
function seedFile(path: string, options?: { name?: string; contentType?: string }): { $file: string; name?: string; contentType?: string }
```

`SeedRecord` keys are the object's declared fields (a `lookup` / `master_detail`
Expand Down
2 changes: 1 addition & 1 deletion content/docs/getting-started/quick-reference.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@ Categories that have no section here at all are named under
[Categories Without a Section](#categories-without-a-section) — that curation is stated,
not left implicit.

## Data Protocol (15 of 30 schemas)
## Data Protocol (15 of 31 schemas)

Core business logic and data modeling schemas.

Expand Down
2 changes: 2 additions & 0 deletions content/docs/references/api/package-api.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -250,6 +250,7 @@ Package manifest installed through POST /api/v1/packages (AUTHORING stage; scope
| **env** | `Enum<'prod' \| 'dev' \| 'test'>[]` | optional (default: `["prod","dev","test"]`) | Applicable environments |
| **locale** | `string[]` | optional | Applicable locales (BCP-47 tags); omitted applies to every locale. The publish and install paths do not filter by locale — they load every dataset and warn |
| **records** | `Record<string, any>[]` | ✅ | Data records |
| **files** | `Record<string, { contentType: string; size: integer; sha256: string; data: string }>` | optional | Bytes of the package files the records name with `seedFile(path)`, keyed by path. Written by `os build`, which reads each path relative to the package root; the loader stores each one as a file the written record owns. |
| **_lock** | `Enum<'none' \| 'no-overlay' \| 'no-delete' \| 'full'>` | optional | Item-level lock — controls overlay & delete (ADR-0010). |
| **_lockReason** | `string` | optional | Human-readable reason shown when a write is refused by _lock. |
| **_lockSource** | `Enum<'artifact' \| 'package' \| 'env-forced'>` | optional | Layer that set _lock (artifact \| package \| env-forced). |
Expand Down Expand Up @@ -358,6 +359,7 @@ Package manifest installed through POST /api/v1/packages (AUTHORING stage; scope
| **env** | `Enum<'prod' \| 'dev' \| 'test'>[]` | optional (default: `["prod","dev","test"]`) | Applicable environments |
| **locale** | `string[]` | optional | Applicable locales (BCP-47 tags); omitted applies to every locale. The publish and install paths do not filter by locale — they load every dataset and warn |
| **records** | `Record<string, any>[]` | ✅ | Data records |
| **files** | `Record<string, { contentType: string; size: integer; sha256: string; data: string }>` | optional | Bytes of the package files the records name with `seedFile(path)`, keyed by path. Written by `os build`, which reads each path relative to the package root; the loader stores each one as a file the written record owns. |
| **_lock** | `Enum<'none' \| 'no-overlay' \| 'no-delete' \| 'full'>` | optional | Item-level lock — controls overlay & delete (ADR-0010). |
| **_lockReason** | `string` | optional | Human-readable reason shown when a write is refused by _lock. |
| **_lockSource** | `Enum<'artifact' \| 'package' \| 'env-forced'>` | optional | Layer that set _lock (artifact \| package \| env-forced). |
Expand Down
3 changes: 2 additions & 1 deletion content/docs/references/data/index.mdx
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
---
title: Data Protocol — complete schema reference
navTitle: Data Protocol
description: "The ObjectStack Data Protocol in 30 reference pages: every schema in @objectstack/spec with its properties, types, defaults and a TypeScript example."
description: "The ObjectStack Data Protocol in 31 reference pages: every schema in @objectstack/spec with its properties, types, defaults and a TypeScript example."
---

{/* ⚠️ AUTO-GENERATED — DO NOT EDIT. Run build-docs.ts to regenerate. Hand-written docs live in the module folders under content/docs/. */}
Expand Down Expand Up @@ -37,6 +37,7 @@ This section contains all protocol schemas for the data layer of ObjectStack.
<Card href="/docs/references/data/picklist" title="Picklist" description="Source: packages/spec/src/data/picklist.zod.ts" />
<Card href="/docs/references/data/query" title="Query" description="Source: packages/spec/src/data/query.zod.ts" />
<Card href="/docs/references/data/seed" title="Seed" description="Source: packages/spec/src/data/seed.zod.ts" />
<Card href="/docs/references/data/seed-file" title="Seed File" description="Source: packages/spec/src/data/seed-file.zod.ts" />
<Card href="/docs/references/data/seed-loader" title="Seed Loader" description="Source: packages/spec/src/data/seed-loader.zod.ts" />
<Card href="/docs/references/data/validation" title="Validation" description="Source: packages/spec/src/data/validation.zod.ts" />
</Cards>
3 changes: 2 additions & 1 deletion content/docs/references/data/meta.json
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,7 @@
"driver-sqlite",
"driver-turso",
"field-value",
"picklist"
"picklist",
"seed-file"
]
}
94 changes: 94 additions & 0 deletions content/docs/references/data/seed-file.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,94 @@
---
title: Seed File schema — Data Protocol reference
navTitle: Seed File
description: "A file / image / avatar / video / audio field stores an opaque sys_file id, and only an upload mints one."
---

{/* ⚠️ AUTO-GENERATED — DO NOT EDIT. Run build-docs.ts to regenerate. Hand-written docs live in the module folders under content/docs/. */}

## Seed files — a file a package ships, written into a file field by a seed

A `file` / `image` / `avatar` / `video` / `audio` field stores an opaque
`sys_file` id (ADR-0104 D3 wave 2), and only an upload mints one. A seed has
no upload, so a seed record names the file by the PATH the package ships it
at instead, and the platform carries the bytes from there (#25779, ruled on
hotclm#19: "the platform's seed loader should carry bundled fixture files;
the app keeps `file` required"):

1. **Authoring.** A record writes `seedFile('fixtures/nda-v1.pdf')` — the
value `SeedFileReferenceSchema` — into a file-class field. The path
is relative to the package root: the directory holding
`objectstack.config.ts`, the root a declarative connector's file ref
resolves against too.
2. **Bundling.** `os build` / `os compile` reads every referenced file and
writes its bytes into the dataset's `files` map
(`SeedFileContentSchema`), keyed by the same path. The compiled
artifact is what every boot reads — `os dev`, `os serve`, a published
package, an artifact URL — so the bytes travel with the seed wherever it
goes. A path that names no file fails the build.
3. **Loading.** The seed loader stores the bytes as a `sys_file`, writes its
id into the field, and has the written record own it — (object, record,
field), the ownership an upload gets. A replay that finds the row still
holding a file with the same bytes keeps it, so a boot does not mint a
new file for an unchanged fixture. A reference whose bytes the dataset
does not carry is refused, by row, naming the path.

300 rows may name the same path: each row gets a file of its own, because a
file has one owning field (ADR-0104 D3).

<Callout type="info">
**Source:** `packages/spec/src/data/seed-file.zod.ts`
</Callout>

## TypeScript Usage

```typescript
import { SeedFileContentSchema, SeedFilePathSchema, SeedFileReferenceSchema } from '@objectstack/spec/data';
import type { SeedFileContent, SeedFilePath, SeedFileReference } from '@objectstack/spec/data';

// Validate data
const result = SeedFileContentSchema.parse(data);
```

---

## SeedFileContent

One bundled seed file: its bytes and what they must hash to

### Properties

| Property | Type | Required | Description |
| :--- | :--- | :--- | :--- |
| **contentType** | `string` | ✅ | MIME type the stored file carries |
| **size** | `integer` | ✅ | Byte length of the file |
| **sha256** | `string` | ✅ | SHA-256 of the bytes, lowercase hex — the loader refuses bytes that do not match it |
| **data** | `string` | ✅ | The bytes, base64-encoded |


---

## SeedFilePath

Path of a file the package ships, relative to the package root (e.g. `fixtures/nda-v1.pdf`)

**Type:** `string`


---

## SeedFileReference

A file the package ships, by package-relative path, written into a file field by a seed

### Properties

| Property | Type | Required | Description |
| :--- | :--- | :--- | :--- |
| **$file** | `string` | ✅ | Path of a file the package ships, relative to the package root (e.g. `fixtures/nda-v1.pdf`) |
| **name** | `string` | optional | File name the stored file carries (what a download is saved as); defaults to the last segment of the path |
| **contentType** | `string` | optional | MIME type of the file; defaults to the one `os build` infers from the path's extension |


---

1 change: 1 addition & 0 deletions content/docs/references/data/seed-loader.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -234,6 +234,7 @@ Seed loader request with datasets and configuration
| **env** | `Enum<'prod' \| 'dev' \| 'test'>[]` | optional (default: `["prod","dev","test"]`) | Applicable environments |
| **locale** | `string[]` | optional | Applicable locales (BCP-47 tags); omitted applies to every locale. The publish and install paths do not filter by locale — they load every dataset and warn |
| **records** | `Record<string, any>[]` | ✅ | Data records |
| **files** | `Record<string, { contentType: string; size: integer; sha256: string; data: string }>` | optional | Bytes of the package files the records name with `seedFile(path)`, keyed by path. Written by `os build`, which reads each path relative to the package root; the loader stores each one as a file the written record owns. |
| **_lock** | `Enum<'none' \| 'no-overlay' \| 'no-delete' \| 'full'>` | optional | Item-level lock — controls overlay & delete (ADR-0010). |
| **_lockReason** | `string` | optional | Human-readable reason shown when a write is refused by _lock. |
| **_lockSource** | `Enum<'artifact' \| 'package' \| 'env-forced'>` | optional | Layer that set _lock (artifact \| package \| env-forced). |
Expand Down
12 changes: 12 additions & 0 deletions content/docs/references/data/seed.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,7 @@ const result = SeedSchema.parse(data);
| **env** | `Enum<'prod' \| 'dev' \| 'test'>[]` | optional (default: `["prod","dev","test"]`) | Applicable environments |
| **locale** | `string[]` | optional | Applicable locales (BCP-47 tags); omitted applies to every locale. The publish and install paths do not filter by locale — they load every dataset and warn |
| **records** | `Record<string, any>[]` | ✅ | Data records |
| **files** | `Record<string, { contentType: string; size: integer; sha256: string; data: string }>` | optional | Bytes of the package files the records name with `seedFile(path)`, keyed by path. Written by `os build`, which reads each path relative to the package root; the loader stores each one as a file the written record owns. |
| **_lock** | `Enum<'none' \| 'no-overlay' \| 'no-delete' \| 'full'>` | optional | Item-level lock — controls overlay & delete (ADR-0010). |
| **_lockReason** | `string` | optional | Human-readable reason shown when a write is refused by _lock. |
| **_lockSource** | `Enum<'artifact' \| 'package' \| 'env-forced'>` | optional | Layer that set _lock (artifact \| package \| env-forced). |
Expand All @@ -49,6 +50,17 @@ const result = SeedSchema.parse(data);
| :--- | :--- | :--- | :--- |
| **applyOn** | `Enum<'create' \| 'always'>` | optional (default: `"always"`) | When a seed load writes this column. `always` (default) writes it on every row the load creates or updates. `create` writes it only when the load creates the row: an `upsert` replay that matches an existing row leaves the row's value as it is, and an `update` dataset never writes it, so replaying the seed refreshes the `always` columns without reverting state the app wrote since the first load. A column of the dataset's `externalId` is still matched on and stays in the matched row's update. The same values and meaning as an import mapping entry's `applyOn`. |

### Nested Shape: `Seed.files[string]`

One bundled seed file: its bytes and what they must hash to

| Property | Type | Required | Description |
| :--- | :--- | :--- | :--- |
| **contentType** | `string` | ✅ | MIME type the stored file carries |
| **size** | `integer` | ✅ | Byte length of the file |
| **sha256** | `string` | ✅ | SHA-256 of the bytes, lowercase hex — the loader refuses bytes that do not match it |
| **data** | `string` | ✅ | The bytes, base64-encoded |


---

Expand Down
Loading
Loading