[kriszyp]

⊙ Problem

Let models.embed() cut each input at the model's context window with truncate: true adds a truncate option to models.embed() for v5.4, plus a new truncate capability, an OpenAI truncation: vllm config key and a defineBackend flag. None of it is documented.

🤔 Is this the right scope?

Yes. Readers need the call contract, the per-backend behavior with and without the option, and which config enables it. The docs also say plainly that @embed and /v1/embeddings cannot set it yet (HarperFast/harper#3705).

💡 Solution

• API: a truncate option row and a "Truncating long inputs" section (look hardest here). It covers the hard-requirement rule (incapable backends are skipped, else a capability error before any request), what absent/false means per backend, a YAML + JS example, and the limits on @embed, /v1/embeddings and stored vectors.
• Backends: truncation: vllm on OpenAI entries, Ollama's capability, Bedrock Cohere v3/v4 behavior, and defineBackend({ truncate }).
• Routing: truncate in the capability list, and the auto-require rule.
• Schema: @embed cannot pass truncate; and the 5.4 release note.

🤔 The example uses BAAI/bge-m3, not Nomic

The brief's example had inputType: 'document'. On main, an OpenAI entry adds no task prefix, so a Nomic example would store unprefixed vectors. Once Document OpenAI embedding task prefixes for v5.4 lands, the example can switch to Nomic with inputPrefixes.

✅ Verification

• npm run build passes, with no broken links. All new anchors resolve in the built HTML.
• format:check, lint and test:examples pass.
• Every behavior claim was checked against the core PR head (92f279d1): types.ts, Models.ts, DESIGN.md, the three provider backends, configValidator.ts and backendRegistry.ts.
• Not run live: no vLLM, Ollama or Bedrock endpoint on this machine.

🤔 v5.4.0 badges assume the core PR merges before 5.4 is cut

Core has a v5.3 release branch (newest tag v5.3.4) and no v5.4 branch, so core main ships as 5.4.0. If 5.4 is cut first, the badges need to change.

Overlap with open docs PRs

• Document OpenAI embedding task prefixes for v5.4 edits the same embed() options table, the OpenAI field table, and adds ## Models to the 5.4 notes.
• Document model pricing, the agent's per-session spend caps, and final cancels also adds ## Models to the 5.4 notes.

Whichever merges second keeps both rows and one ## Models heading.

Refs HarperFast/harper#3706

PR QA: skipped @ 45a1533 — docs-only repository

_{🤖 Generated by Claude Opus 5.5 (dev-agent); posted via @kriszyp.}

🤖 Generated with Claude Code

Release target: v5.4 — documents a core feature shipping in 5.4.0

Related PRs: #748 overlaps, #727 overlaps
Complexity: easy

Origin — the dispatch brief this PR was written from

Document the truncate option for models.embed() and the OpenAI truncation: vllm setting

Document the new truncate option for embedding calls, added in HarperFast/harper PR HarperFast/harper#3706 (targets v5.4).

What changed (read the PR for exact behavior; resources/models/types.ts and resources/models/DESIGN.md in that PR are the source of truth):

• models.embed(input, { truncate: true }) cuts each input to what the embedding model accepts, keeping the START of the text (and any task prefix such as search_document: ), instead of failing when the input is longer than the model's context window.
• truncate: true requires the new truncate capability. A backend that cannot truncate is skipped (fallback entries included); if none can, the call fails with ModelCapabilityError before any request is sent. Callers can also list 'truncate' in requires.
• truncate absent or false changes nothing: the provider's own handling applies (OpenAI, vLLM and Bedrock Titan reject an over-long input; Ollama and Bedrock Cohere embed v3 cut by default).
• Per backend:
  • OpenAI-compatible entries: new embedding-only config key truncation: vllm (models.embedding.), for an endpoint served by vLLM 0.18 or later. With it, Harper sends truncate_prompt_tokens: -1 and truncation_side: 'right'. Without it the backend does not advertise truncation and never sends those fields (api.openai.com rejects unknown fields). The value is the operator's declaration; Harper does not probe the endpoint.
  • Ollama: sends truncate: true on /api/embed.
  • Bedrock: Cohere embed v4 sends truncate: RIGHT; Cohere embed v3 (english/multilingual, incl. light) sends truncate: END after cutting each text to Bedrock's 2,048-character per-text limit; Titan cannot truncate.
• defineBackend({ ..., truncate: true }) lets a custom embedding backend declare the capability.

Where: the models / embeddings API reference (EmbedOpts, capabilities, requires), the models: configuration reference for OpenAI entries (truncation), and the custom-backend (defineBackend) page. Add a short YAML + JS example (vLLM entry with truncation: vllm, then models.embed(text, { inputType: 'document', truncate: true })).

Note for readers: enabling truncation does not re-embed records already stored. The @embed directive and the /v1/embeddings gateway do not expose the option yet (tracked in HarperFast/harper#3705) — say so rather than documenting it there.

Mark the docs for v5.4. Cross-link the harper PR in your docs PR description.

_{Dispatch: task docs-harper-3706 · queued by task:harper-embed-token-truncation · ran by claude/opus/high · worker kzyp-xps-1}

Deferred findings

• 5f60e96046a71cca nit documentation — reference/models/routing.md — Custom router docs silent on whether auto-added truncate appears in RouteRequest.requires — NOT FILED
• a8025f71b0fa353e nit documentation — reference/models/local-development.md — Parity caveats omit Ollama default truncation vs strict prod backends for @embed — NOT FILED

_{Review-Coverage: authored=gemini; ran=cursor-composer,codex,claude; adjudicated=domain; declined=cursor-grok,muse; rounds=6; full=2 @ 45a1533}

_{Review-Attention: skim ~2m (decisions: truncate-auto-requires-capability, vllm-opt-in-flag, embed-directive-excluded) @ 45a1533}

[gemini-code-assist]

Code Review

This pull request updates the documentation for Harper v5.4.0 to introduce the new truncate: true option for models.embed(), which cuts long inputs to fit within an embedding model's context window instead of failing. The changes detail backend-specific behaviors (such as Ollama, Bedrock Cohere, and OpenAI/vLLM), routing requirements, and limitations. The review feedback suggests improving readability and adhering to documentation guidelines by splitting sentences that combine critical limitations or input refusal conditions with other concepts into separate, distinct sentences.

[github-actions]

🚀 Preview Deployment

Your preview deployment is ready!

🔗 Preview URL: https://preview.harper-documentation.harperfabric.com/pr-771

This preview will update automatically when you push new commits.

[github-actions]

🚀 Preview Deployment

Your preview deployment is ready!

🔗 Preview URL: https://preview.harper-documentation.harperfabric.com/pr-771

This preview will update automatically when you push new commits.

[github-actions]

🧹 Preview Cleanup

The preview deployment for this PR has been removed.