Expose live indexing progress to MCP clients #2031

What problem does this solve?

index_repository can run for minutes or hours on large repositories, but an MCP client currently receives no quantitative progress until the synchronous tool call completes.

index_status reports the already-published graph (nodes, edges, and ready|empty), not the active indexing attempt. The UI job endpoint similarly exposes only indexing|done|error.

The indexer already emits useful structured events such as:

• pipeline.discover files=N
• pass.start pass=... files=N
• parallel.extract.progress done=N total=M

These events are consumed by the one-shot CLI progress sink, but are not exposed through MCP progress notifications or an in-flight status snapshot. This makes a healthy long-running index indistinguishable from a stalled one.

Proposed solution

Expose live per-attempt indexing progress to MCP clients.

• When the client supplies an MCP progress token, publish progress notifications while index_repository is running.
• Include a structured snapshot for the active/latest attempt in index_status, so another client or session can poll it.
• At minimum expose:
  • attempt/run identifier
  • state: queued, running, completed, failed, or cancelled
  • current phase/pass
  • processed and total units for phases that have real counters
  • discovered file count
  • started and last-updated timestamps
• Do not invent a single overall percentage unless the pipeline has a well-defined weighted model. Phase-local processed/total is enough initially.
• Preserve the current final index_repository response and existing index_status fields for compatibility.
• Ensure daemon-supervised jobs propagate progress without mixing it into MCP stdout or JSON-RPC responses.
• Rate-limit or coalesce updates to avoid flooding clients.

Suggested acceptance criteria:

1. A long-running MCP index_repository call emits monotonic progress updates for the extraction phase.
2. index_status shows the same in-flight attempt state and counters.
3. A client that does not request progress sees unchanged behavior.
4. Completion, failure, and cancellation leave a terminal attempt record, and the final notification agrees with the tool result.
5. Tests cover daemon transport, multiple subscribers, and compatibility when no progress token is supplied.

Related work:

• feat(cli): add --progress flag for live indexing progress output #108 and commit dbab4d37 implemented the CLI presentation.
• feat(index): record and report the latest index attempt #1727 records the latest attempt and freshness; this issue adds quantitative live progress.
• Clean linked-worktree full index stalls silently for workspace-scale projects with persisted stored_hashes >= 267k #1300 is an example where a long-running index can appear stalled.

Alternatives considered

• Rely on daemon logs: they are not available through MCP and require client-specific parsing.
• Return only final node and edge counts: this is too late to distinguish progress from a stall.
• Expose only an overall percentage: it would be misleading because pipeline phases have different costs and not every phase has a meaningful total.

Confirmations

• I searched existing issues and this is not a duplicate.