Feature+debug prompt token breakdown - #100
Merged
MegalithOfficial merged 6 commits intoSep 10, 2026
Merged
MegalithOfficial merged 6 commits into
MegalithOfficial merged 6 commits into
Conversation
Add a "Prompt Token Breakdown" section to the developer message debug page that shows how the reconstructed prompt is distributed across categories: System & Persona, Memories, Lorebook, Author's Note, Companion State and Chat History. Categories are derived from the snapshot's system prompt entries via their stable id/name, chat history from the user/assistant request messages. Token counts use the existing tiktoken (o200k) command, so the figures are an approximation of the model's own tokenizer. Shows per-category tokens, share, total, and context-window usage.
Show a "Context Usage" panel on assistant messages: a context-window bar (occupied / reserved for response / free) plus a breakdown bar of what fills the prompt — system, character, persona, memories, lorebook, author's note, companion state and chat history. Per-category shares are estimated with tiktoken and scaled onto the provider's real prompt tokens; for llama.cpp / Ollama the reserved response budget (response + reasoning) is read from the actual request body. Backend: the debug snapshot now surfaces the individual placeholder source contents (character profile, persona, lorebook, memories, author note, companion state, scheduled notes) and a memory entry count via a new prompt_engine::debug_prompt_source_contents helper, so inline- rendered sources are attributed separately. The companion-state instruction preamble stays in the system bucket. Shared PromptTokenBreakdown component reused by the debug page and the message sheet. Strings translated across all 20 locales.
Port the per-message "Context Usage" panel to group chat messages via a
dedicated synchronous `group_chat_message_debug_snapshot` command that
reconstructs the group prompt (build_group_system_prompt + assembled
messages) from the stored message data — speaker character, model,
memory refs and used lorebook entries — without re-running retrieval.
`build_group_system_prompt` now also returns the raw placeholder source
texts, so the breakdown attributes character profile, persona, lorebook,
memories, author note and the `{{group_characters}}` cast block. Adds a
new "Group cast" category (with a "(X characters)" count) for the other
participants' profiles.
The reserved-for-response segment stays provider-gated (llama.cpp /
Ollama only) via a shared RESERVING_PROVIDER_IDS set, so remote providers
never show it. Shared PromptTokenBreakdown component and all 20 locales
extended accordingly.
In dynamic-memory mode the 1:1 chat path filled {{key_memories}} with the
entire hot memory bank on every prompt, in addition to the retrieved
relevant memories injected as a separate block — bloating the prompt with
redundant context.
Now {{key_memories}} is resolved from the retrieved relevant memories
before the shared renderer runs, and the redundant separate block is
dropped. Covers RP characters and companions (completion, regenerate,
continue) and the debug snapshot, which reconstructs the injected memories
from the message's stored refs. The shared render_with_context, group chat,
manual-memory mode, and memory-management prompts are untouched.
…t window The context-usage bars capped Free at 0 when occupied + reserved exceeded the context window, silently hiding the over-reservation (e.g. llama.cpp n_ctx = prompt + completion budget). Add a red alert under the context-window bar showing the overflow in tokens so the user can react (lower max tokens or context content) before the model truncates the prompt or cuts the response short. Translated across all 20 locales.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
as disussed and shown.
adds token breakdown and context-window usage