Claude Code token tally overcounts (no dedupe by message.id) and undercounts (ignores subagents/*.jsonl)

[tepxgit]

Summary

tokens.tally() (the Claude Code transcript parser, bmad_loop/tokens.py, v0.12.0) miscounts session usage in two opposite directions:

1. Overcount: no dedupe by message.id. Claude Code writes one JSONL entry per assistant content block (thinking, text, each tool_use), and every entry of the same API response carries the identical message.usage block. tally() sums every entry that has a usage block, so a response with 3 content blocks is counted 3×.
2. Undercount: subagent transcripts are never read. Work done by subagents (Task/Agent tool) is written to <session-dir>/subagents/*.jsonl, which tally() never opens.

Because the two errors partly cancel, the reported figure is neither an upper nor a lower bound. That makes max_tokens_per_story and session_budget_mode unreliable in both directions: a story can blow past the cap without the guard tripping, or trip it early.

Evidence (one real project, Claude Code adapter)

• One dev-session transcript: 246 entries with a usage block, but only 86 unique message.ids (≈2.9× duplication).
• One story: the harness recorded 4.85M weighted tokens against 4.33M from Claude Code's own cost state (overcount dominates: few subagents).
• Another story: the harness recorded 2.03M against 5.07M (undercount dominates: heavy subagent use).

Suggested fix

Mirror what tally_gemini_chat() already does: key usage by message id and keep one usage per id (last write wins), then sum. Also glob <transcript_dir>/<session_id>/subagents/*.jsonl (when present) through the same deduping path and add it to the total.

def tally(transcript_path: Path) -> TokenUsage:
    by_id: dict[str, dict] = {}
    anon: list[dict] = []
    paths = [transcript_path, *sorted((transcript_path.with_suffix("") / "subagents").glob("*.jsonl"))]
    for path in paths:
        for entry in _jsonl_entries(path):
            usage = _usage_block(entry)
            if not usage:
                continue
            mid = (entry.get("message") or {}).get("id")
            if isinstance(mid, str):
                by_id[mid] = usage
            else:
                anon.append(usage)
    ...sum by_id.values() + anon as today...

(The subagent directory layout is ~/.claude/projects/<project>/<session-id>/subagents/, next to <session-id>.jsonl; worth confirming against current Claude Code builds.)

Related

#778 (timed-out sessions journalled as 0 tokens) and #775 (codex usage never captured) are separate adapter-usage gaps. This one affects every completed Claude Code session.