Capture real token counts from backends and use for compaction

[antoinezambelli]

Capture real token counts from backends and use for compaction

Summary

• LlamafileClient and OllamaClient now record actual token usage (TokenUsage dataclass) from every LLM response into a last_usage dict keyed by slot ID
• ContextManager uses real token counts (via update_token_count()) instead of the chars/4 heuristic when available
• run_inference() syncs token counts from the client to the context manager after every LLM call
• Streaming path requests stream_options: {include_usage: true} and breaks on [DONE] instead of finish_reason to capture the usage chunk that arrives between the two

Why

The chars/4 heuristic significantly underestimates actual token usage — especially for reasoning models where <think> tokens consume KV cache but aren't persisted in message content. In forge-code eval runs, 14B Q8 reasoning models StreamError at 6-9K estimated tokens against a 12K budget because compaction never fires — the heuristic thinks there's headroom when KV cache is actually full.

With real counts, compaction and context warnings trigger based on actual server-reported usage.

Changes

File	Change
clients/base.py	New TokenUsage frozen dataclass
clients/llamafile.py	last_usage dict, _record_usage(), stream_options, break on [DONE]
clients/ollama.py	last_usage dict, _record_usage() (maps prompt_eval_count/eval_count)
context/manager.py	update_token_count(), estimate_tokens() returns real count when available
core/inference.py	_sync_token_count() called after every send
__init__.py	Export TokenUsage

Design decisions

• last_usage is a dict[int, TokenUsage] keyed by slot ID — supports future sub-agent work (issues Sub-agent support: synchronous swap scheduling #27-Sub-agent support: slot pool #29) where concurrent agents share a client with multiple slots. Default slot 0 for single-slot use.
• Client-side state rather than return type change — avoids breaking the LLMResponse union type. _sync_token_count uses getattr so clients without last_usage (e.g. AnthropicClient) are silently skipped.
• Dropped llamafile 0.9.x compat — streaming now breaks on [DONE] instead of finish_reason to capture the usage chunk. Old llamafile binaries that don't send [DONE] are no longer supported.

Test plan

• 677 unit tests pass
• Integration verified against live llama-server (both streaming and non-streaming)
• Integration verified against live Ollama (both streaming and non-streaming)
• forge-code eval_runner smoke test passed on llamaserver

Closes #32