From 8491bdc1ac67fd0f72bbdb1dc4397a17f1f3e468 Mon Sep 17 00:00:00 2001 From: Kris Zyp Date: Thu, 22 Oct 2026 13:51:21 -0600 Subject: [PATCH 1/2] Document OpenAI embedding task prefixes for v5.4 Companion to HarperFast/harper#3450. Dispatch-Task: harper-openai-embed-inputtype-prefixes Co-Authored-By: GPT-5 Codex --- reference/models/backends.md | 52 +++++++++++++++++++++++++++++++----- 1 file changed, 45 insertions(+), 7 deletions(-) diff --git a/reference/models/backends.md b/reference/models/backends.md index 481c88fe1..6966903a9 100644 --- a/reference/models/backends.md +++ b/reference/models/backends.md @@ -80,13 +80,14 @@ models: model: meta-llama/Llama-3.1-8B-Instruct ``` -| Field | Default | Description | -| -------------- | --------------------------- | --------------------------------------------------------------------------------------------------------------- | -| `apiKey` | — (required) | API key, sent as a bearer token. Use `${VAR}` indirection | -| `baseUrl` | `https://api.openai.com/v1` | API root; point at any OpenAI-compatible endpoint | -| `model` | — | Model name, e.g. `gpt-4o`, `text-embedding-3-large` | -| `organization` | — | Sent as the `OpenAI-Organization` header, for keys spanning multiple organizations | -| `truncation` | — | From v5.4.0, `vllm` on an embedding entry served by vLLM 0.18 or later, so `embed()` can honor `truncate: true` | +| Field | Default | Description | +| --------------- | --------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------- | +| `apiKey` | — (required) | API key, sent as a bearer token. Use `${VAR}` indirection | +| `baseUrl` | `https://api.openai.com/v1` | API root; point at any OpenAI-compatible endpoint | +| `model` | — | Model name, e.g. `gpt-4o`, `text-embedding-3-large` | +| `organization` | — | Sent as the `OpenAI-Organization` header, for keys spanning multiple organizations | +| `truncation` | — | From v5.4.0, `vllm` on an embedding entry served by vLLM 0.18 or later, so `embed()` can honor `truncate: true` | +| `inputPrefixes` | unset | Optional `document` and `query` strings on embedding entries only. See [Embedding task prefixes](#embedding-task-prefixes) (Added in: v5.4.0) | `responseFormat: 'json'` maps to OpenAI's JSON mode and `responseFormat: { schema }` to strict structured outputs (`json_schema`); OpenAI-compatible servers vary in their support for these. @@ -105,6 +106,43 @@ models: truncation: vllm ``` +### Embedding task prefixes + + + +OpenAI embedding entries can opt into task prefixes with `inputPrefixes`: + +```yaml +models: + embedding: + default: + backend: openai + apiKey: ${EMBEDDING_API_KEY} + baseUrl: http://localhost:8000/v1 + model: nomic-ai/nomic-embed-text-v1.5 + inputPrefixes: + document: 'search_document: ' + query: 'search_query: ' +``` + +`inputPrefixes` is supported only on `models.embedding` entries with `backend: openai`; it is rejected on OpenAI generative entries and on the other built-in backends. Its `document` and `query` values are optional strings, including empty strings; non-string values and unknown keys are rejected. Unresolved `${VAR}` placeholders cause the entry to fail registration. Quote values to preserve trailing spaces. + +Harper prepends the matching string to each input in the batch when `embed()` specifies `inputType: 'document'` or `inputType: 'query'`. With no `inputType`, no `inputPrefixes`, or no matching key, the input is sent unchanged. Prefixes belong to the selected logical entry. + +The `/v1/embeddings` gateway does not set `inputType`, so its callers must add task prefixes themselves. + +The `@embed` write hook sets `inputType: 'document'`. Set `inputType: 'query'` when embedding search text: + +```js +const [queryVector] = await models.embed('What is Harper?', { inputType: 'query' }); +``` + +Both [nomic-embed-text-v1](https://huggingface.co/nomic-ai/nomic-embed-text-v1#usage) and [nomic-embed-text-v1.5](https://huggingface.co/nomic-ai/nomic-embed-text-v1.5#usage) require `"search_document: "` for retrieval documents and `"search_query: "` for retrieval queries. For other models, follow the model author's instructions and leave prefixes unset unless required. + +Prefixes are concatenated exactly: Harper does not trim text or detect an existing prefix. Remove any application wrapper adding the same prefix when enabling this option, and leave it unset if the endpoint already adds the task instruction. Existing entries retain their previous behavior until configured. A config reload applies prefix changes immediately. Changing the text sent for existing documents requires rewriting their source fields to re-embed those documents; switching from a wrapper that already adds these exact strings preserves input bytes. + +Ollama already applies these two prefixes for model names containing `nomic-embed-text` when `inputType` is present. Its existing behavior is unchanged; `inputPrefixes` is an OpenAI embedding setting. + ## Anthropic Calls the Anthropic Messages API. Generation only — Anthropic does not offer an embeddings API. From 248a6830da53ef8604ae4066a1d83480e329ae34 Mon Sep 17 00:00:00 2001 From: Kris Zyp Date: Thu, 22 Oct 2026 14:00:21 -0600 Subject: [PATCH 2/2] Align model API and migration guidance with task prefixes Address the authorized documentation review findings without another review round. Dispatch-Task: harper-openai-embed-inputtype-prefixes Dispatch-Task: pr-maint-1833edca00ecdcd301860825953253c6 Co-Authored-By: GPT-5 Codex Co-Authored-By: Gemini --- reference/models/api.md | 16 ++++++++-------- reference/models/backends.md | 8 ++++---- reference/models/local-development.md | 2 +- reference/models/overview.md | 2 +- release-notes/v5-lincoln/5.4.md | 4 ++++ 5 files changed, 18 insertions(+), 14 deletions(-) diff --git a/reference/models/api.md b/reference/models/api.md index 1c0d51a72..b9674c6dd 100644 --- a/reference/models/api.md +++ b/reference/models/api.md @@ -21,16 +21,16 @@ Converts one or more strings into embedding vectors. The result is always an arr import { models } from 'harper'; const [single] = await models.embed('What is Harper?', { inputType: 'query' }); -const batch = await models.embed(['first document', 'second document']); +const batch = await models.embed(['first document', 'second document'], { inputType: 'document' }); ``` -| Option | Type | Default | Description | -| ----------- | ------------------------- | ----------- | ------------------------------------------------------------------------------------------------------------------------------------------------------ | -| `model` | `string` | `'default'` | Logical name of a configured embedding model | -| `requires` | `Capability[]` | — | Capabilities the chosen backend must satisfy; used by [routing](./routing#capability-routing) to select a candidate in the group | -| `inputType` | `'document'` \| `'query'` | — | Hint for models that distinguish document embeddings from query embeddings (e.g. `nomic-embed-text`); ignored by models that do not | -| `truncate` | `boolean` | `false` | From v5.4.0, cut each input to what the model accepts instead of failing on an over-long input — see [Truncating long inputs](#truncating-long-inputs) | -| `signal` | `AbortSignal` | — | Cancels the call; composed with the backend's configured `requestTimeoutMs` | +| Option | Type | Default | Description | +| ----------- | ------------------------- | ----------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | +| `model` | `string` | `'default'` | Logical name of a configured embedding model | +| `requires` | `Capability[]` | — | Capabilities the chosen backend must satisfy; used by [routing](./routing#capability-routing) to select a candidate in the group | +| `inputType` | `'document'` \| `'query'` | — | Document/query hint. OpenAI embedding entries require configured [inputPrefixes](./backends#embedding-task-prefixes) (Added in: v5.4.0). Ollama applies Nomic prefixes automatically | +| `truncate` | `boolean` | `false` | From v5.4.0, cut each input to what the model accepts instead of failing on an over-long input — see [Truncating long inputs](#truncating-long-inputs) | +| `signal` | `AbortSignal` | — | Cancels the call; composed with the backend's configured `requestTimeoutMs` | ### Truncating long inputs diff --git a/reference/models/backends.md b/reference/models/backends.md index 6966903a9..4142feb87 100644 --- a/reference/models/backends.md +++ b/reference/models/backends.md @@ -125,11 +125,11 @@ models: query: 'search_query: ' ``` -`inputPrefixes` is supported only on `models.embedding` entries with `backend: openai`; it is rejected on OpenAI generative entries and on the other built-in backends. Its `document` and `query` values are optional strings, including empty strings; non-string values and unknown keys are rejected. Unresolved `${VAR}` placeholders cause the entry to fail registration. Quote values to preserve trailing spaces. +`inputPrefixes` is supported only on `models.embedding` entries with `backend: openai`; it is rejected on OpenAI generative entries and on the other built-in backends. Its `document` and `query` values are optional strings, including empty strings; non-string values and unknown keys are rejected. An unresolved `${VAR}` prefix is a registration-time error: Harper logs it and skips that entry at startup without blocking other entries, or keeps the previous entry on reload. Quote values to preserve trailing spaces. -Harper prepends the matching string to each input in the batch when `embed()` specifies `inputType: 'document'` or `inputType: 'query'`. With no `inputType`, no `inputPrefixes`, or no matching key, the input is sent unchanged. Prefixes belong to the selected logical entry. +Harper prepends the matching string to each input in the batch when `embed()` specifies `inputType: 'document'` or `inputType: 'query'`. With no `inputType`, no `inputPrefixes`, or no matching key, the input is sent unchanged. Each fallback entry applies its own prefixes to the original input. -The `/v1/embeddings` gateway does not set `inputType`, so its callers must add task prefixes themselves. +Harper's OpenAI-compatible HTTP embedding endpoint, `POST /v1/embeddings`, does not set `inputType`, so its callers must add task prefixes themselves. The `@embed` write hook sets `inputType: 'document'`. Set `inputType: 'query'` when embedding search text: @@ -139,7 +139,7 @@ const [queryVector] = await models.embed('What is Harper?', { inputType: 'query' Both [nomic-embed-text-v1](https://huggingface.co/nomic-ai/nomic-embed-text-v1#usage) and [nomic-embed-text-v1.5](https://huggingface.co/nomic-ai/nomic-embed-text-v1.5#usage) require `"search_document: "` for retrieval documents and `"search_query: "` for retrieval queries. For other models, follow the model author's instructions and leave prefixes unset unless required. -Prefixes are concatenated exactly: Harper does not trim text or detect an existing prefix. Remove any application wrapper adding the same prefix when enabling this option, and leave it unset if the endpoint already adds the task instruction. Existing entries retain their previous behavior until configured. A config reload applies prefix changes immediately. Changing the text sent for existing documents requires rewriting their source fields to re-embed those documents; switching from a wrapper that already adds these exact strings preserves input bytes. +Prefixes are concatenated exactly: Harper does not trim text or detect an existing prefix. Remove any application wrapper adding the same prefix when enabling this option, and leave it unset if the endpoint already adds the task instruction. Existing embedding config entries retain their previous behavior until configured. A config reload applies prefix changes immediately. Changing the text sent for existing documents requires rewriting their source fields to re-embed those documents; switching from a wrapper that already adds these exact strings preserves input bytes. Ollama already applies these two prefixes for model names containing `nomic-embed-text` when `inputType` is present. Its existing behavior is unchanged; `inputPrefixes` is an OpenAI embedding setting. diff --git a/reference/models/local-development.md b/reference/models/local-development.md index b6ae07c78..634b4a58c 100644 --- a/reference/models/local-development.md +++ b/reference/models/local-development.md @@ -85,7 +85,7 @@ models: model: Qwen/Qwen2.5-7B-Instruct ``` -**Embedding compatibility.** Embedding vectors are only comparable within a single model's vector space, and models differ in dimensionality. Vectors written locally with one embedding model cannot be meaningfully searched against vectors produced in production by another — which matters whenever embedded data, or an [HNSW index](../database/schema#vector-indexing) built from [`@embed`](../database/schema#embed) vectors, moves between environments. Where embedded data crosses environments, use the same embedding model in both — for example, an open model served by Ollama locally and by an OpenAI-compatible host in production — or re-embed after the move. +**Embedding compatibility.** Embedding vectors are only comparable within a single model's vector space, and models differ in dimensionality. Vectors written locally with one embedding model cannot be meaningfully searched against vectors produced in production by another — which matters whenever embedded data, or an [HNSW index](../database/schema#vector-indexing) built from [`@embed`](../database/schema#embed) vectors, moves between environments. Where embedded data crosses environments, use the same embedding model in both — for example, an open model served by Ollama locally and by an OpenAI-compatible host in production — or re-embed after the move. Task-prefix text must match too: Ollama adds Nomic prefixes for `inputType: 'document'` or `'query'` automatically, while an OpenAI embedding entry needs the matching [`inputPrefixes`](./backends#embedding-task-prefixes) configured from v5.4.0. ## Per-environment configuration diff --git a/reference/models/overview.md b/reference/models/overview.md index e389c3649..6431c0451 100644 --- a/reference/models/overview.md +++ b/reference/models/overview.md @@ -171,7 +171,7 @@ The price is read by [`maxCostUsd`](./tool-calling#options) on `toolMode: 'auto' ### Credentials -String values in model entries support environment-variable indirection with `${VAR_NAME}` syntax, resolved at startup. Use this for API keys rather than placing the literal key in the configuration file — Harper logs a warning at startup when a credential field contains a literal value. If the referenced environment variable is unset, the placeholder is left as-is; for credential fields the backend rejects the unresolved placeholder at startup, while other fields (such as `host` or `model`) carry the literal placeholder into requests — surfacing as per-request failures rather than a startup error. Indirection applies to string-typed fields only; numeric fields such as `requestTimeoutMs` must be literal values. +String values in model entries support environment-variable indirection with `${VAR_NAME}` syntax, resolved at startup. Use this for API keys rather than placing the literal key in the configuration file — Harper logs a warning at startup when a credential field contains a literal value. If the referenced environment variable is unset, the placeholder is left as-is; for credential fields the backend rejects the unresolved placeholder at startup, while fields without such validation (such as `host` or `model`) carry the literal placeholder into requests — surfacing as per-request failures rather than a startup error. The OpenAI embedding [`inputPrefixes`](./backends#embedding-task-prefixes) fields also reject unresolved placeholders at registration; on reload, the previous entry keeps serving. Indirection applies to string-typed fields only; numeric fields such as `requestTimeoutMs` must be literal values. ### Startup behavior diff --git a/release-notes/v5-lincoln/5.4.md b/release-notes/v5-lincoln/5.4.md index 7e48e6a99..08206c75c 100644 --- a/release-notes/v5-lincoln/5.4.md +++ b/release-notes/v5-lincoln/5.4.md @@ -80,6 +80,10 @@ A `models.generative` entry can now declare `pricing: { inputPerMtok, outputPerM `models.embed(input, { truncate: true })` cuts each input to what the embedding model accepts, keeping the start of the text and any task prefix, instead of failing on an input longer than the model's context window. Only a backend that can truncate serves such a call: Ollama, Bedrock Cohere embed v3 and v4, and OpenAI-compatible embedding entries that set the new `truncation: vllm` for a vLLM 0.18 or later endpoint. If no candidate can, the call fails before any request is sent. Calls without the option send the same requests as before. Custom backends declare the capability with `defineBackend({ truncate: true })`. The `@embed` directive and the `/v1/embeddings` endpoint do not expose the option yet. See [Truncating long inputs](/reference/v5/models/api#truncating-long-inputs). +### Configurable Embedding Task Prefixes + +OpenAI embedding entries can opt into `inputPrefixes.document` and `.query` to prepend exact task instructions when a call supplies `inputType`. This supports Nomic v1/v1.5 retrieval prefixes through OpenAI-compatible endpoints. Existing entries keep their input unchanged until configured; changing document input text requires re-embedding stored documents. See [Embedding task prefixes](/reference/v5/models/backends#embedding-task-prefixes). + ## Memory ### Smaller Per-Thread Baseline