feat(mcp): headless data table and row tools

CHANGELOG.md

@@ -4,6 +4,16 @@ All notable changes to Instatic will be documented here.
 
 This project is pre-1.0. Breaking changes may appear in minor or patch releases until a stable release line exists.
 
+## Unreleased
+
+### AI and integrations
+
+- Added a `data_*` tool scope so reusable data tables can be built and filled headlessly, over MCP or from the in-app agent ([#433](https://github.com/CoreBunch/Instatic/issues/433), [#463](https://github.com/CoreBunch/Instatic/issues/463)). Schema setup was the one manual break in an otherwise automatable pipeline: `content_list_collections` listed post types only, `content_create_document` refused a `kind: 'data'` table id, and no tool could create a table or a field at all, so an agent asked to build a training catalogue had to stop and hand the operator a list of columns to type in. Eight tools now cover the whole surface — `data_list_tables`, `data_create_table`, `data_update_table`, `data_add_fields`, `data_create_rows`, `data_update_row`, `data_set_rows_status`, `data_delete_rows` — all server-resolved, so none of them needs a workspace tab open. Row creation is transactional in batches of up to 200, and the system tables still refuse any change to their identity or built-in fields. Publishing a row happens on `main` only, exactly as it does in the Content workspace — a branch reaches the live site by being merged — while drafting, unpublishing, and deleting work on a branch too.
+- Fixed the data schema tools being impossible to grant. `data_create_table`, `data_update_table`, and `data_add_fields` require the "Manage custom tables" capability, which the connector consent screen never offered, so a client could reconnect as often as it liked and still not see them. The screen now has a Data tables section holding it. It stays off by default, like every other write.
+- Added tool annotations and structured output to the MCP surface. `tools/list` now carries the four MCP behaviour hints per tool (`readOnlyHint`, `destructiveHint`, `idempotentHint`, `openWorldHint`) and an `outputSchema` for every tool in the catalog, and a successful `tools/call` returns the payload as `structuredContent` alongside the JSON text block. A client can tell a read from a delete before it calls, and parse a typed result instead of re-deriving the shape from a text blob. The server also reports its real package version instead of a hardcoded `1.0.0`.
+- Fixed `data_create_rows` sending agents to `data_list_tables` for the field ids its cells are keyed by — that tool does not return them, so the first call guessed and failed. The descriptions now name `content_get_collection_schema` for field ids and `content_list_documents` for reading rows back, and the in-app `data` chat scope gained those same three content reads: it could create a table and fill it, then had no way to see what it wrote.
+- Fixed the Content workspace answering "Collection not found" for a reusable data table that exists. The id was right and the table was real; it is simply authored in the Data workspace, not in the Tiptap editor. Agents responded by creating a duplicate post type to stand in for it. The refusal now names the kind of table it is and the tool that can write it.
+
 ## 0.0.20 - 2026-09-13
 
 ### Features

docs/features/data-workspace.md

@@ -209,6 +209,18 @@ Both actions are opened from `DataSidebar`.
 
 ---
 
+## Agent and MCP access
+
+Everything this workspace does by hand is also reachable headlessly, through the `data` tool scope (`server/ai/tools/data/`): `data_list_tables`, `data_create_table`, `data_update_table`, `data_add_fields`, `data_create_rows`, `data_update_row`, `data_set_rows_status`, `data_delete_rows`. Both paths pass a runtime carrying the uploads directory, so publishing bakes a row's static artefact and retracting or deleting unlinks it again; an MCP connection also passes its connector id, which is what attributes its writes to the connection in the audit log instead of to the signed-in user. A headless call runs on `main` (the MCP context pins it); the in-app chat carries the workspace's branch, and publishing is refused off main.
+
+Those handlers reuse this workspace's server side rather than restating it — the same repository calls, the same `slugForTable` derivation, the same `content.entry.cells` plugin filter, and the same access predicates from `server/handlers/cms/data/access.ts`. Keep it that way: a second copy of a rule is a copy free to drift. When a rule changes here, it changes for the tools in the same edit.
+
+The tools are deliberately `execution: 'server'`, not browser-relayed like the `content_*` writes. A data row is a grid of typed cells, not a Tiptap document, so there is nothing an open tab could render that the server cannot do alone — and requiring one would make the toolset useless to a script or a remote agent.
+
+Full tool table and capability requirements: [docs/features/mcp-connectors.md](mcp-connectors.md) → "Reusable data tables".
+
+---
+
 ## Forbidden patterns
 
 | Pattern | Why |
@@ -229,6 +241,7 @@ Both actions are opened from `DataSidebar`.
 ## Related
 
 - [docs/features/content-storage.md](content-storage.md) — `DataField` schema, field types, `data_tables` / `data_rows` structure
+- [docs/features/mcp-connectors.md](mcp-connectors.md) — the `data_*` toolset that exposes this workspace headlessly
 - [docs/reference/ui-primitives.md](../reference/ui-primitives.md) — `Button`, `Input`, `Select`, `Switch` usage
 - [docs/reference/persistence-keys.md](../reference/persistence-keys.md) — `instatic-data-grid-primary-widths-v1`
 - Source-of-truth files:

docs/features/mcp-connectors.md

@@ -144,15 +144,62 @@ executeAiTool(...) / live editor bridge
 | `editorBridge.ts` | Per-user, per-scope live workspace bridge. The stream carries an **idle lease** (120s, re-armed by every relayed tool request) so an active batch is never cut mid-flight; only quiet streams recycle. The workspace's reconnect loop (`useMcpWorkspaceBridge`) reopens a recycled healthy stream immediately off the stream-end network event — deliberately timer-free, because hidden webviews (backgrounded browser tabs) clamp timers to minutes while network events still fire — and a tab becoming visible short-circuits any pending retry delay. |
 | `tools/publishTool.ts` | Explicit canonical full-site publish with MCP audit metadata. |
 | `tools/uploadMediaTool.ts` | Server-resolved image upload (`media_upload`) — inline base64 or SSRF-guarded `sourceUrl` download, through the shared media pipeline. |
+| `../tools/data/` | Server-resolved schema and row tools for reusable data tables (`data_*`). Shared with the in-app `data` chat scope; the MCP registry passes a runtime so writes are attributed to the connection. |
+
+## What `tools/list` advertises
+
+Each tool ships three things beyond its name and description.
+
+`inputSchema` is the tool's TypeBox schema emitted verbatim as JSON Schema. The same schema re-validates the arguments inside `executeAiTool`, so the advertised contract and the enforced one cannot drift.
+
+`outputSchema` describes the result. The server sends the payload twice on success: as JSON text in `content`, for clients that read only that, and as `structuredContent`, a typed object matching the schema. Nothing validates a result against its schema at runtime — a shape drift must fail a test, never break a tool on a live install — so the catalog test in `registry.test.ts` requires every advertised tool to declare one and the per-tool tests assert the match. Schemas live in `src/core/ai/toolOutputSchemas.ts`, in core because browser tools produce their results in the browser and the server advertises them. Where a tool forwards a structure another engine owns (a page-node tree, a module's prop schema) the leaf stays `unknown` with a description rather than a second definition to drift from the first.
+
+`annotations` carry the four MCP behaviour hints, derived in `server.ts`:
+
+| Hint | How it is set |
+|---|---|
+| `readOnlyHint` | true unless the tool is tagged `mutates` |
+| `destructiveHint` | true for the row/document/node/page deletes, `data_update_table` (dropping a field orphans its values), and `site_publish` (overwrites the live slot) |
+| `idempotentHint` | true for reads and for writes that land in the same state when repeated — status changes, deletes, field and token setters |
+| `openWorldHint` | always false; every tool reaches this install's own database and uploads, nothing else |
+
+They are hints for a client's confirmation UI, not a security boundary. Capabilities are the boundary.
 
 ## Tool execution model
 
 MCP exposes the full deduplicated tool catalog, filtered by the connection's capabilities.
 
-Server-resolved tools work without an editor open. They include content reads, `get_context`, `site_list_documents`, `site_read_styles`, `site_list_breakpoints`, `media_upload`, and explicit `site_publish`. Publishing requires `ai.tools.write` plus `pages.publish`, runs the canonical full-site pipeline, swaps the static slot atomically, and records the connection id in the publish audit event.
+Server-resolved tools work without an editor open. They include content reads, the whole `data_*` toolset, `get_context`, `site_list_documents`, `site_read_styles`, `site_list_breakpoints`, `media_upload`, and explicit `site_publish`. Publishing requires `ai.tools.write` plus `pages.publish`, runs the canonical full-site pipeline, swaps the static slot atomically, and records the connection id in the publish audit event.
 
 `media_upload` is the one server-resolved write that mutates outside the live editor draft: it adds an image to the Media library through the same `acceptUploadedMedia` core the HTTP route uses (magic-byte sniffing, SVG sanitisation, storage dispatch, responsive variants). Bytes arrive inline (base64) or via an https `sourceUrl` the host downloads under the plugin network layer's SSRF blocklist — https-only, DNS-resolved, per-redirect-hop re-validation, size-capped. It requires `ai.tools.write` plus `media.write`.
 
+### Reusable data tables
+
+`content_*` covers post types — documents with a Tiptap body the Content workspace renders. Reusable tables (`kind: 'data'`: a training catalogue, a team roster, anything a page loops over) are a different shape: a grid of typed cells with no body, and `content_list_collections` deliberately does not list them.
+
+They get their own headless toolset instead of being folded into `content_*`, because routing a cell write through an open browser tab would buy nothing and would make the toolset unusable from a script or a remote agent:
+
+| Tool | Does | Requires |
+|---|---|---|
+| `data_list_tables` | Lists data tables and post types with slug, kind, route base, row count, and primary field. Page, component, and layout tables stay hidden — those are Site-editor documents. | a table read/manage capability |
+| `data_create_table` | Creates a table with its fields. `kind: 'data'` gets no route base, so its rows have no public URL; `kind: 'postType'` gets `/<slug>`. | `data.custom.tables.manage` |
+| `data_update_table` | Changes identity or replaces the field array. | a table manage capability |
+| `data_add_fields` | Appends fields, leaving stored values untouched — the safe way to evolve a schema. | a table manage capability |
+| `data_create_rows` | Creates up to 200 rows in one transaction. Any rejection writes nothing. | `content.create` |
+| `data_update_row` | Patches one row's cells (merge by default). | a content edit capability |
+| `data_set_rows_status` | Publishes, unpublishes, or drafts rows in bulk, reporting per-row outcomes. | publish for `published`, edit otherwise |
+| `data_delete_rows` | Soft-deletes rows in bulk. | a content edit capability |
+
+Schema writes are granted separately from row writes: `data.custom.tables.manage` is its own **Data tables** section on the consent screen and in the access-token dialog, off by default. A connector that only fills rows never gets it, and an approver who does not hold it never sees the section. `data.system.tables.manage` is not offered at all — the four built-in tables refuse identity and built-in-field changes regardless, so the grant would read wider than it acts.
+
+Reading rows back is `content_list_documents` and `content_get_document`: both accept a reusable data table's id exactly as they accept a post type's, and `content_get_collection_schema` returns the field ids that `data_create_rows` keys its cells by. There is deliberately no `data_list_rows` / `data_get_row` duplicating them; the three descriptions point at each other so an agent finds the path from either side.
+
+Headless calls always run on `main`. The MCP request context pins `MAIN_SCOPE`, so a connector reads and writes the live site's rows and never a site branch's; only the in-app Data chat carries the workspace's branch through the same tools. `data_set_rows_status` refuses a publish off main for the same reason the HTTP route answers 409 — publishing writes main regardless of the branch the row was read from — while retracting and deleting work on a branch and leave main's baked artefact and render cache alone.
+
+Publishing a row in a table with no route base is allowed and normal. No static artefact is baked because there is no route to bake it at, but the row becomes `published`, which is what an `<instatic-loop>` on some other page reads.
+
+The system tables (`pages`, `posts`, `components`, `layouts`) accept new custom fields but refuse any change to their identity or their built-in fields, enforced by the same `assertSystemTableUpdateAllowed` the HTTP route uses.
+
 Browser tools run against the connection owner's live workspace. Site structure, HTML/CSS, page lifecycle, design-token, content mutation, code-asset, and live-DOM tools route to the matching open Site or Content workspace. If that workspace is not open, the tool returns a scope-specific error while headless tools remain available. `tools/list` states that requirement in each browser tool's description, so a client learns the precondition when it picks the tool rather than from a failed call.
 
 There is intentionally no headless page-tree mutation path. The open editor store is the single source of truth for draft edits; a second DB mutation path would desynchronize node state and overwrite the live document. Relayed edits need no post-tool save step: store mutations stream to the collab relay the moment they land, and every headless read (plus `site_publish`) flushes the relay server-side before it touches the DB — so a following read or publish always observes the edit. There is no client-side save flush, and no window in which the MCP caller can see stale data.
@@ -200,4 +247,5 @@ Create and manual revoke actions retain the existing `ai.mcp_connector.created`
 - `src/__tests__/ai/mcpOAuthAuthorizationHandler.test.ts` covers signed-in consent, capability selection, exact callback redirects, denial, and privilege floors.
 - `src/__tests__/ai/mcpConnectorsHandler.test.ts` covers connection listing, personal-token creation, step-up, revoke, and privilege floors.
 - `server/ai/mcp/e2e.test.ts`, `transports/http.test.ts`, and `publishTool.test.ts` cover the real MCP request flow and publish path.
+- `server/ai/tools/data/*.test.ts` cover the data toolset against a migrated SQLite database: system-table refusals, transactional batches, slug-conflict naming, and publishing a row in a non-routable table.
 - `src/__tests__/architecture/ai-mcp-connectors-never-leak.test.ts` gates the token-free connection projection.

server/ai/handlers/chat.ts

@@ -60,7 +60,7 @@ import {
   canonicaliseAiUserContent,
   preflightAiUserContent,
 } from '../inputImages'
-import { selectToolsForScope } from '../tools'
+import { selectToolsForScope, type ToolsetOptions } from '../tools'
 import {
   buildSiteSystemPrompt,
   SiteAgentSnapshotSchema,
@@ -95,17 +95,19 @@ export function tryHandleAiChat(
   req: Request,
   db: DbClient,
   pathname: string,
+  options: ToolsetOptions = {},
 ): Promise<Response> | null {
   if (!pathname.startsWith('/admin/api/ai/chat/')) return null
   const scope = pathname.slice('/admin/api/ai/chat/'.length)
   if (!VALID_SCOPES.includes(scope as ToolScope)) return null
-  return handleAiChat(req, db, scope as ToolScope)
+  return handleAiChat(req, db, scope as ToolScope, options)
 }
 
 async function handleAiChat(
   req: Request,
   db: DbClient,
   scope: ToolScope,
+  options: ToolsetOptions,
 ): Promise<Response> {
   if (req.method !== 'POST') {
     return jsonResponse({ error: 'Method not allowed' }, { status: 405 })
@@ -191,7 +193,7 @@ async function handleAiChat(
     req.signal,
   )
   if (modelCapabilities === REQUEST_ABORTED) return clientClosedRequest()
-  const tools = selectToolsForScope(scope, user.capabilities)
+  const tools = selectToolsForScope(scope, user.capabilities, options)
   if (requestedImage && !modelCapabilities.visionInput) {
     return jsonResponse(
       { error: 'The selected model does not support image input. Choose a vision-capable model.' },

server/ai/handlers/index.ts

@@ -21,10 +21,20 @@ import { tryHandleAiMcpManagement } from '../mcp/handlers/management'
 import { tryHandleMcpOAuthAuthorization } from '../mcp/handlers/oauthAuthorization'
 import { tryHandleAiEditorBridge } from '../mcp/handlers/editorBridge'
 
+export interface AiHandlerOptions {
+  /**
+   * Where baked artefacts live. The chat handler passes it to the data
+   * toolset so a row retracted or deleted from the Data workspace also loses
+   * its published HTML — the same reason the CMS handlers take one.
+   */
+  uploadsDir?: string
+}
+
 export function tryHandleAi(
   req: Request,
   db: DbClient,
   url: URL,
+  options: AiHandlerOptions = {},
 ): Promise<Response> | null {
   const pathname = url.pathname
   if (!pathname.startsWith('/admin/api/ai/')) return null
@@ -44,7 +54,7 @@ export function tryHandleAi(
     tryHandleAiMcpManagement(req, db, pathname) ??
     tryHandleAiEditorBridge(req, db, pathname) ??
     tryHandleAiAudit(req, db, url, pathname) ??
-    tryHandleAiChat(req, db, pathname) ??
+    tryHandleAiChat(req, db, pathname, options) ??
     tryHandleAiToolResult(req, db, pathname) ??
     tryHandleAiCredentials(req, db, pathname) ??
     tryHandleAiConversations(req, db, url, pathname) ??

server/ai/mcp/e2e.test.ts

@@ -91,11 +91,21 @@ describe('MCP end-to-end (2026-07-28 stateless requests, real handler)', () => {
     expect(names).toContain('site_read_styles') // headless design-system read
     expect(names).toContain('site_insert_html') // browser editing tool, relayed to the editor
 
+    // Every listed tool carries its behaviour hints and a result schema.
+    const listed = list.json.result?.tools ?? []
+    const listTables = listed.find((t) => t.name === 'data_list_tables')
+    expect(listTables?.annotations).toMatchObject({ readOnlyHint: true, openWorldHint: false })
+    expect(listTables?.outputSchema?.type).toBe('object')
+    expect(listed.every((t) => t.outputSchema?.type === 'object')).toBe(true)
+
     const read = await rpc('tools/call', { name: 'content_list_collections', arguments: {} })
     expect(read.json.result?.isError).toBeFalsy()
     const content = JSON.stringify(read.json.result?.content)
     expect(content).toContain('posts')
     expect(content).not.toContain('"id":"pages"')
+    // The same payload as typed data, so a client parses it instead of the blob.
+    const structured = read.json.result?.structuredContent as { collections: Array<{ id: string }> }
+    expect(structured.collections.map((c) => c.id)).toContain('posts')
   })
 
   it('a read-only connector sees reads but no write tools', async () => {