Skip to content

Document OpenAI embedding task prefixes for v5.4 - #748

Merged
Ethan-Arrowood merged 2 commits into
mainfrom
docs/openai-embedding-input-prefixes
Nov 2, 2026
Merged

Ethan-Arrowood merged 2 commits into
mainfrom
docs/openai-embedding-input-prefixes

Conversation

@kriszyp

@kriszyp kriszyp commented Oct 22, 2026 •

Copy link
Copy Markdown
Member
Two robot mail clerks distinguish document and query envelopes with colored prefix labels

📝 Summary

Document the opt-in OpenAI embedding inputPrefixes added by harper#3450, with the v5.4 version badge CONTRIBUTING.md requires. The reference gives the exact Nomic v1/v1.5 document and query prefixes, with their trailing spaces. It explains where prefixes stop: inputType, the gateway and stored vectors.

The backend example and migration guidance cover empty or absent values, live reload and avoiding double-prefixing. Related API, environment-variable and local-development guidance now agree with the new option, and the v5.4 release notes link to it.

🔧 Changes

The backend reference defines exact input bytes and the configuration table links to it. The API reference adds the document batch inputType and configuration prerequisite. The overview explains unresolved prefix variables. The local-development guide covers Nomic prefix parity across backends. The v5.4 release note announces the option.

✅ Verification

  • npm run format:write and npm run format:check passed across the full docs worktree.
  • npm run build passed; built HTML preserves the v5.4 badge, both quoted trailing spaces and the gateway guidance.
  • Original implementation verification against final core head 32fcded2c6f2ecc8ad627493ac212124f18f2a9f: 127 focused regression/migration checks and 3 checks using the actual formatted documentation YAML passed. The gateway omits inputType; reload does not re-embed stored documents, while rewriting the same source sends the changed prefix.
  • No live model or retrieval-quality measurement; these checks verify the documented configuration and input bytes.

Review notes

The rebase onto main at 4aea8b1fffc5 integrates the merged truncate documentation (#771) in reference/models/api.md, reference/models/backends.md, and release-notes/v5-lincoln/5.4.md while preserving the existing inputPrefixes documentation wording. Review still flags the API phrase "require configured" as potentially implying validation; the backend section explicitly says calls without prefixes succeed unchanged.

Depends-on: HarperFast/harper#3450

🤖 Generated by OpenAI Codex; posted via @kriszyp.

Release target: v5.4 — companion guidance for the new core configuration.
Related PRs: #771 overlaps
Complexity: easy

Deferred findings

  • bcb5eda4de862d8b nit documentation — reference/models/backends.md — Gateway exception lacks a documented gateway surface — NOT FILED
  • 5ece319d8865291a nit documentation — reference/models/backends.md — Ollama blurb says 'nomic-embed-text' exact while line 144 says substring match — NOT FILED

Review-Coverage: authored=gemini; ran=claude,cursor-composer; adjudicated=domain; blocked=muse(failed); declined=cursor-grok; rounds=4; full=1 @ 248a683

Review-Attention: skim ~2m (decisions: inputprefixes-openai-only, reembed-by-rewriting-source) @ 248a683

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request updates the documentation to support the new configurable embedding task prefixes feature introduced in v5.4.0. This feature allows OpenAI embedding entries to prepend specific task instructions (such as Nomic retrieval prefixes) when an inputType is provided. The updates span the API reference, backend configuration, local development guidelines, overview, and release notes. The review feedback recommends standardizing the inline version references across the documentation to use the format (v5.4.0) for consistency.

Comment thread reference/models/api.md Resolved
Comment thread reference/models/backends.md Resolved
Comment thread reference/models/local-development.md Resolved
@github-actions
github-actions Bot temporarily deployed to pr-748 October 22, 2026 20:05 Inactive
@github-actions

Copy link
Copy Markdown

🚀 Preview Deployment

Your preview deployment is ready!

🔗 Preview URL: https://preview.harper-documentation.harperfabric.com/pr-748

This preview will update automatically when you push new commits.

@kriszyp
kriszyp marked this pull request as ready for review October 26, 2026 12:49
@kriszyp
kriszyp requested a review from a team as a code owner October 26, 2026 12:49
@kriszyp
kriszyp requested a review from dawsontoth October 26, 2026 12:49
@kriszyp
kriszyp force-pushed the docs/openai-embedding-input-prefixes branch from bcd685e to cff0028 Compare October 28, 2026 16:24
@github-actions
github-actions Bot temporarily deployed to pr-748 October 28, 2026 16:28 Inactive
@github-actions

Copy link
Copy Markdown

🚀 Preview Deployment

Your preview deployment is ready!

🔗 Preview URL: https://preview.harper-documentation.harperfabric.com/pr-748

This preview will update automatically when you push new commits.

@kriszyp
kriszyp force-pushed the docs/openai-embedding-input-prefixes branch from cff0028 to 9e06a33 Compare October 28, 2026 19:19
@github-actions
github-actions Bot temporarily deployed to pr-748 October 28, 2026 19:22 Inactive
@github-actions

Copy link
Copy Markdown

🚀 Preview Deployment

Your preview deployment is ready!

🔗 Preview URL: https://preview.harper-documentation.harperfabric.com/pr-748

This preview will update automatically when you push new commits.

kriszyp and others added 2 commits October 29, 2026 07:30
Companion to HarperFast/harper#3450.

Dispatch-Task: harper-openai-embed-inputtype-prefixes

Co-Authored-By: GPT-5 Codex <noreply@openai.com>
Address the authorized documentation review findings without another review round.

Dispatch-Task: harper-openai-embed-inputtype-prefixes
Dispatch-Task: pr-maint-1833edca00ecdcd301860825953253c6
Co-Authored-By: GPT-5 Codex <noreply@openai.com>
Co-Authored-By: Gemini <noreply@google.com>
@kriszyp
kriszyp force-pushed the docs/openai-embedding-input-prefixes branch from 9e06a33 to 248a683 Compare October 29, 2026 13:36
@github-actions
github-actions Bot temporarily deployed to pr-748 October 29, 2026 13:38 Inactive
@github-actions

Copy link
Copy Markdown

🚀 Preview Deployment

Your preview deployment is ready!

🔗 Preview URL: https://preview.harper-documentation.harperfabric.com/pr-748

This preview will update automatically when you push new commits.

@Ethan-Arrowood Ethan-Arrowood left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm!

@Ethan-Arrowood
Ethan-Arrowood merged commit edc6eea into main Nov 2, 2026
14 checks passed
@Ethan-Arrowood
Ethan-Arrowood deleted the docs/openai-embedding-input-prefixes branch November 2, 2026 22:08
@github-actions

github-actions Bot commented Nov 2, 2026

Copy link
Copy Markdown

🧹 Preview Cleanup

The preview deployment for this PR has been removed.

This branch was previously deployed

1 inactive deployment
pr-748 — 248a6830 Deployed Oct 29, 2026 by github-actions[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants