Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 2 additions & 3 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -1,9 +1,8 @@
# Generated artifacts
codeindex.json
blastradius.json
symbolindex.json
repo_graph.json
symbolindex.json
.codeindex/
.blastradius/
benchmark/results.json

# Claude Code
Expand Down
36 changes: 36 additions & 0 deletions AGENTS.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
# AGENTS.md

## Developer Commands & Verification

- **Run tests**: `uv run pytest` (set `$env:PYTHONPATH="."` if running outside an installed venv).
- **Run single test**: `uv run pytest tests/test_phase1.py` or `uv run pytest -k <test_name>`.
- **Run CLI in dev**: `uv run python -m blastradius.cli <command>` or `uv run blastradius <command>`.
- **Environment note**: Avoid system `pip` or system `python` if broken; prefer `uv run` or `.venv\Scripts\` execution.

## Core Architecture & Package Structure

- **Package name**: `blastradius` (source in `blastradius/`). Entry point: `blastradius.cli:main`.
- **Index artifacts**: Default dependency graph is written to `blastradius.json`; SQLite store is populated at `<repo>/.blastradius/index.db`.
- **Layering rule**: `blastradius.store`, `blastradius.temporal`, and core modules must **never** import from `blastradius.semantic` or `blastradius.analyzers`.
- **Backward compatibility**: The `blastradius.json` file export schema is frozen. SQLite DB synchronization is an additive side-effect of `index.build()`.

## Storage & Database Gotchas (`blastradius/store/db.py`)

- **FTS5 table**: Must remain a standalone table without the `content=` option. Using `content='symbols'` triggers `database disk image is malformed` errors during deletes in WAL mode.
- **Soft deletes**: `sync()` must set `last_seen_commit` on soft-delete for temporal `--as-of` queries to filter correctly.
- **SQLite transactions**: `executescript()` auto-commits pending transactions in Python `sqlite3`. Keep regular DDL and FTS DDL in separate `executescript()` calls.
- **`sqlite-vec` optional dependency**: Never import `sqlite_vec` at module top-level. Wrap imports in `try...except` for runtime feature detection and graceful fallback to FTS5 + graph search.

## Temporal Layer (`blastradius/temporal/history.py`)

- **Git history reading**: `backfill()` streams commit history using `git ls-tree` and `git cat-file --batch`. Do not replace this with `git checkout` or individual `git show` calls.

## Coding Conventions

- **Module header**: Every Python file starts with:
```python
# Copyright 2026 David Scheiderman
# Licensed under the Apache License, Version 2.0
from __future__ import annotations
```
- **Python version**: Targets Python 3.9+. Modern union (`X | Y`) and generic types (`list[str]`) are enabled via `from __future__ import annotations`.
78 changes: 39 additions & 39 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ All notable changes to this project will be documented in this file.

### Added

- **`codeindex symbol-blast <file>`** — per-export blast radius for a file. Lists every
- **`blastradius symbol-blast <file>`** — per-export blast radius for a file. Lists every
exported symbol with a count and list of which importer files reference it by name.
Answers "which routes use each export from lib/db/schema.ts" without manual grep.
Supports `--json` for programmatic use.
Expand Down Expand Up @@ -46,10 +46,10 @@ All notable changes to this project will be documented in this file.

### Fixed

- **`lookup` reads SQLite DB** — `codeindex lookup` and the `lookup_symbol` MCP tool
previously read `symbolindex.json`, which is only written by `codeindex symbols`.
- **`lookup` reads SQLite DB** — `blastradius lookup` and the `lookup_symbol` MCP tool
previously read `symbolindex.json`, which is only written by `blastradius symbols`.
Both now query `Store.lookup_by_name()` from the SQLite DB (the same source as
`codeindex search`), falling back to `symbolindex.json` only when no DB is present.
`blastradius search`), falling back to `symbolindex.json` only when no DB is present.
Symbols found via search — including those extracted from destructured re-exports —
are now consistently reachable via lookup.
- **`lookup` output shows symbol name** — plain-mode output was `auth.ts:8 (const)`;
Expand All @@ -69,9 +69,9 @@ All notable changes to this project will be documented in this file.
genuine API surfaces with broad real coupling.
- **`changed-since` edge origin annotation** — added edges now carry `first_seen_commit`;
when N edges share the `last_indexed_commit` value, the CLI prints a count. The message
branches on whether `codeindex history` has been run:
- History not run: `"run codeindex history to date them accurately"`
- History run: `"bootstrap-gap artifacts: existed before the first codeindex analyze
branches on whether `blastradius history` has been run:
- History not run: `"run blastradius history to date them accurately"`
- History run: `"bootstrap-gap artifacts: existed before the first blastradius analyze
and cannot be dated further"` — correctly reflects the inherent limitation rather
than implying a fixable error. MCP response gains `bootstrap_gap` boolean.

Expand Down Expand Up @@ -113,34 +113,34 @@ All notable changes to this project will be documented in this file.
when FTS (or semantic KNN) already returns ≥ k results, preventing
structurally adjacent but semantically unrelated symbols from diluting
high-quality keyword hits.
- **Search file aggregation** — `codeindex search` and the `semantic_search`
- **Search file aggregation** — `blastradius search` and the `semantic_search`
MCP tool now include a `Files` section aggregating results by file (sorted
by symbol hit count). The entry-point file appears even when no single
symbol from it ranks at the top.
- **`db status` FTS row count** — `codeindex db status` now shows
- **`db status` FTS row count** — `blastradius db status` now shows
`fts_symbols` (rows in `symbols_fts`) making it easy to diagnose whether
the FTS index is populated.

## [0.3.1] - 2026-06-07

### Fixed

- **Multi-word FTS search OR fallback** — `codeindex search "auth token"` previously
- **Multi-word FTS search OR fallback** — `blastradius search "auth token"` previously
returned nothing when AND semantics found no single symbol containing all words.
`fts_search()` now retries with `word1 OR word2 OR ...` automatically when the
AND query returns zero results.
- **`changed-since` backfill warning** — when `codeindex history` has never been run,
- **`changed-since` backfill warning** — when `blastradius history` has never been run,
all files share exactly one `first_seen_commit`, making `changed-since` results
inaccurate against any older ref. The command now detects this via
`COUNT(DISTINCT first_seen_commit) <= 1` and prints a clear warning on stderr
(CLI) / includes a `"warning"` key in the response (JSON + MCP tool) directing
the user to run `codeindex history` first.
the user to run `blastradius history` first.

## [0.3.0] - 2026-06-07

### Summary

codeindex evolves from a stateless point-in-time dependency analyzer into a
blastradius evolves from a stateless point-in-time dependency analyzer into a
**temporal code knowledge graph** — persistent, incremental, and semantically
queryable. Three new properties: persistence + incrementality, time, and
meaning (semantic retrieval). All existing CLI commands, JSON schemas, and MCP
Expand All @@ -149,36 +149,36 @@ tools are unchanged.
### Added

#### Persistent SQLite store (Phase 1)
- `.codeindex/index.db` — SQLite graph store created automatically on
`codeindex analyze`; survives across runs, never touches `codeindex.json`
- `.blastradius/index.db` — SQLite graph store created automatically on
`blastradius analyze`; survives across runs, never touches `blastradius.json`
- Incremental indexing: detects changed files via `git diff --name-status`
between index runs; logs changed file count to stderr
- `codeindex db status` — schema version, last indexed commit, file/edge/symbol counts
- `codeindex db migrate` — applies pending schema migrations (runs automatically on open)
- `codeindex symbols` now syncs symbols to DB with FTS5 full-text index
- `blastradius db status` — schema version, last indexed commit, file/edge/symbol counts
- `blastradius db migrate` — applies pending schema migrations (runs automatically on open)
- `blastradius symbols` now syncs symbols to DB with FTS5 full-text index

#### Temporal layer (Phase 2)
- Every file, edge, and symbol carries `first_seen_commit` / `last_seen_commit`
— facts are never hard-deleted, only soft-deleted with temporal stamps
- `codeindex history [--since REF] [--max-commits N]` — backfills temporal
- `blastradius history [--since REF] [--max-commits N]` — backfills temporal
data from git history without any working-tree checkouts (uses
`git ls-tree` + `git cat-file --batch`)
- `codeindex changed-since <ref>` — files and edges added or removed since a
- `blastradius changed-since <ref>` — files and edges added or removed since a
commit, branch, or tag
- `codeindex impact <file> --as-of <ref>` — blast radius at a historical point
- `blastradius impact <file> --as-of <ref>` — blast radius at a historical point
in time, not just HEAD

#### Semantic layer (Phase 3)
- `codeindex search "<query>" [--k N] [--as-of REF] [--json]` — hybrid
- `blastradius search "<query>" [--k N] [--as-of REF] [--json]` — hybrid
semantic + FTS5 keyword + graph expansion search, fused with Reciprocal Rank
Fusion (RRF)
- `codeindex/semantic/provider.py` — `EmbeddingProvider` ABC +
- `blastradius/semantic/provider.py` — `EmbeddingProvider` ABC +
`OpenAIEmbeddingProvider` HTTP client (stdlib `urllib` only, no new runtime deps)
- `sqlite-vec` optional extension for KNN vector search; absent = graceful
fallback to FTS + graph with a clear notice (no crash, no config required)
- Embeddings generated automatically during `codeindex analyze` when
`CODEINDEX_EMBEDDING_ENDPOINT` / `_MODEL` / `_DIMS` env vars are set
- `codeindex[semantic]` extra: `pip install 'codeindex[semantic]'`
- Embeddings generated automatically during `blastradius analyze` when
`BLASTRADIUS_EMBEDDING_ENDPOINT` / `_MODEL` / `_DIMS` env vars are set
- `blastradius[semantic]` extra: `pip install 'blastradius[semantic]'`

#### MCP surface (Phase 4) — 4 new tools, existing 6 unchanged
- `semantic_search` — hybrid search from an MCP client; degrades gracefully
Expand All @@ -190,28 +190,28 @@ tools are unchanged.
- `schema_version` bumped to `"2"` with forward migration from `"1"`
- FTS5 `symbols_fts` rowid now equals `symbols.id` (enables direct FTS → symbol
row mapping without a secondary lookup)
- `codeindex db status` output extended with `embedding_model`, `embedding_dims`,
- `blastradius db status` output extended with `embedding_model`, `embedding_dims`,
`vec_symbols` fields
- README rewritten to document all new commands, the SQLite store, semantic
setup, and the full 10-tool MCP surface

### Internal
- `codeindex/store/db.py`: `Store` class — `init_vectors()`, `upsert_embeddings()`,
- `blastradius/store/db.py`: `Store` class — `init_vectors()`, `upsert_embeddings()`,
`semantic_search()`, `fts_search()`, `graph_expand()`, `neighborhood()`,
`symbol_visible_at()`, `get_symbol()`, `symbols_needing_embeddings()`
- `codeindex/temporal/history.py`: `backfill()` — BFS over git log via plumbing
- `blastradius/temporal/history.py`: `backfill()` — BFS over git log via plumbing
commands; no checkout side-effects
- `codeindex/semantic/search.py`: `hybrid_search()` with RRF fusion
- `blastradius/semantic/search.py`: `hybrid_search()` with RRF fusion
- Dependency direction enforced: `store/` and `temporal/` never import from
`semantic/` or `graph/`
- 7 new Phase 3 tests; 6 Phase 2 tests; 5 Phase 1 tests (18 total, all green)

## [0.2.0] - 2026-05-24

### Added
- `codeindex lookup <symbol>` — find where a symbol is defined (file + line)
- `codeindex dependencies <file>` — show imports and imported-by for a file
- `codeindex high-blast` — list files above a blast score threshold
- `blastradius lookup <symbol>` — find where a symbol is defined (file + line)
- `blastradius dependencies <file>` — show imports and imported-by for a file
- `blastradius high-blast` — list files above a blast score threshold
- All three new commands support `--json` for machine-readable output
- `lookup_symbol` and `build_symbol_index` tools in MCP server
- CLI integration test suite (`benchmark/test_cli.py`) — 37 assertions covering happy path, `--json` output, error cases, and sort-order invariants
Expand All @@ -231,12 +231,12 @@ tools are unchanged.
### Added
- Multi-language dependency analysis: Python, JavaScript/TypeScript, Go, Ruby, Rust, Java/Kotlin, PHP, CSS
- Blast-radius impact scoring — every file gets a score based on direct and transitive dependents
- `codeindex analyze <repo>` — analyze a repo and write `codeindex.json`
- `codeindex impact <file>` — show blast-radius impact report for a file
- `codeindex symbols <repo>` — build `symbolindex.json` with functions, classes, and exports; supports `--inline` and `--claude-md` modes
- `codeindex serve --mcp` — MCP stdio server exposing `analyze_repo`, `get_impact`, `get_dependencies`, `get_high_blast_files`, `build_symbol_index`, `lookup_symbol`
- `codeindex serve --viz` — visualization UI server
- `codeindex install-hook` — pre-commit hook for blast-radius warnings
- `blastradius analyze <repo>` — analyze a repo and write `blastradius.json`
- `blastradius impact <file>` — show blast-radius impact report for a file
- `blastradius symbols <repo>` — build `symbolindex.json` with functions, classes, and exports; supports `--inline` and `--claude-md` modes
- `blastradius serve --mcp` — MCP stdio server exposing `analyze_repo`, `get_impact`, `get_dependencies`, `get_high_blast_files`, `build_symbol_index`, `lookup_symbol`
- `blastradius serve --viz` — visualization UI server
- `blastradius install-hook` — pre-commit hook for blast-radius warnings
- Phase 4: Docker, CI/CD, and schema analyzers
- Phase 5: monorepo and cross-language intelligence
- Apache 2.0 license
Loading