docs: add readthedocs mkdocs scaffold

Author: raoabinav

.github/workflows/docs-build.yml

@@ -0,0 +1,28 @@
+name: Docs Build
+
+on:
+  push:
+    branches: [ main ]
+  pull_request:
+    branches: [ main ]
+  workflow_dispatch:
+
+jobs:
+  mkdocs:
+    name: MkDocs Strict Build
+    runs-on: ubuntu-22.04
+    permissions:
+      contents: read
+    steps:
+      - uses: actions/checkout@v4
+
+      - name: Set up Python
+        uses: actions/setup-python@v5
+        with:
+          python-version: '3.11'
+
+      - name: Install documentation dependencies
+        run: python -m pip install -r docs/requirements.txt
+
+      - name: Build documentation
+        run: python -m mkdocs build --strict --config-file mkdocs.yml

.readthedocs.yaml

@@ -0,0 +1,14 @@
+version: 2
+
+build:
+  os: ubuntu-22.04
+  tools:
+    python: "3.11"
+
+mkdocs:
+  configuration: mkdocs.yml
+  fail_on_warning: true
+
+python:
+  install:
+    - requirements: docs/requirements.txt

README.md

@@ -1110,7 +1110,10 @@ leann ask my-docs "Where are prompts configured?"
 # Detect file changes since last build/watch checkpoint
 leann watch my-docs
 
-# List all your indexes
+# Keep an index current as files are saved
+leann watch my-docs --live
+
+# List discoverable indexes
 leann list
 
 # Remove an index
@@ -1152,23 +1155,27 @@ leann search INDEX_NAME QUERY [OPTIONS]
 Options:
   --top-k N                     Number of results (default: 5)
   --complexity N                Search complexity (default: 64)
+  --max-depth N                 Maximum app-index discovery depth (default: 4)
   --recompute / --no-recompute  Enable/disable embedding recomputation (default: enabled). Should not do a `no-recompute` search in a `recompute` build.
   --pruning-strategy {global,local,proportional}
 ```
 
 **Watch Command:**
 ```bash
-leann watch INDEX_NAME
+leann watch INDEX_NAME [OPTIONS]
 
 # Compares the current file system state against the last checkpoint (Merkle tree snapshot)
-# and reports which files have been added, removed, or modified, along with their chunk IDs.
+# and updates the index when files have been added, removed, or modified.
 #
-# - Automatically saves a new checkpoint after detecting changes
+# - Use --once --dry-run to report changes without updating the index
+# - Use --live to wake updates from filesystem events instead of the polling interval
+# - --debounce-ms controls live event batching (default: 500)
+# - Automatically saves a new checkpoint after successful index updates
 # - Each subsequent run compares against the most recent checkpoint
 # - File change detection uses SHA-256 content hashing via a Merkle tree
 #
 # Example output:
-#   === Changes since last checkpoint ===
+#   === Changes detected ===
 #   modified (1):
 #     - /path/to/file.py
 #       chunks: 42, 43, 44
@@ -1187,24 +1194,34 @@ Options:
 
 **List Command:**
 ```bash
-leann list
+leann list [OPTIONS]
+
+Options:
+  --max-depth N                 Maximum app-index discovery depth (default: 4)
 
-# Lists all indexes across all projects with status indicators:
+# Lists discoverable indexes from the current project and registered project roots:
 # ✅ - Index is complete and ready to use
 # ❌ - Index is incomplete or corrupted
 # 📁 - CLI-created index (in .leann/indexes/)
 # 📄 - App-created index (*.leann.meta.json files)
+#
+# App-created index discovery is bounded and skips heavyweight directories
+# such as .git, node_modules, virtualenvs, build/cache folders, and directory symlinks.
+#
+# Commands that resolve app-created indexes by name also accept --max-depth,
+# including search, watch, rebuild, migrate-ids, warmup, daemon, and react.
 ```
 
 **Remove Command:**
 ```bash
 leann remove INDEX_NAME [OPTIONS]
 
 Options:
-  --force, -f    Force removal without confirmation
+  --force, -f                    Force removal without confirmation
+  --max-depth N                  Maximum app-index discovery depth (default: 4)
 
 # Smart removal: automatically finds and safely removes indexes
-# - Shows all matching indexes across projects
+# - Shows all bounded matching indexes across registered projects
 # - Requires confirmation for cross-project removal
 # - Interactive selection when multiple matches found
 # - Supports both CLI and app-created indexes
@@ -1272,6 +1289,9 @@ results = searcher.search("banana‑crocodile", use_grep=True, top_k=1)
 
 ## Benchmarks
 
+For the current benchmark inventory and reporting template, see the
+[Benchmark Guide](docs/benchmarks.md).
+
 **[DiskANN vs HNSW Performance Comparison →](benchmarks/diskann_vs_hnsw_speed_comparison.py)** - Compare search performance between both backends
 
 **[Simple Example: Compare LEANN vs FAISS →](benchmarks/compare_faiss_vs_leann.py)** - See storage savings in action

benchmarks/README.md

@@ -1,135 +1,131 @@
-# 🧪 LEANN Benchmarks & Testing
+# LEANN Benchmarks
 
-This directory contains performance benchmarks and comprehensive tests for the LEANN system, including backend comparisons and sanity checks across different configurations.
+This directory contains benchmark scripts and data-dependent evaluation suites for LEANN. The
+canonical benchmark inventory now lives in [docs/benchmarks.md](../docs/benchmarks.md).
 
-## 📁 Test Files
+## Quick Starts
 
-### `diskann_vs_hnsw_speed_comparison.py`
-Performance comparison between DiskANN and HNSW backends:
-- ✅ **Search latency** comparison with both backends using recompute
-- ✅ **Index size** and **build time** measurements
-- ✅ **Score validity** testing (ensures no -inf scores)
-- ✅ **Configurable dataset sizes** for different scales
+Run a small backend comparison:
 
 ```bash
-# Quick comparison with 500 docs, 10 queries
-python benchmarks/diskann_vs_hnsw_speed_comparison.py
-
-# Large-scale comparison with 2000 docs, 20 queries
-python benchmarks/diskann_vs_hnsw_speed_comparison.py 2000 20
-```
-
-### `test_distance_functions.py`
-Tests all supported distance functions across DiskANN backend:
-- ✅ **MIPS** (Maximum Inner Product Search)
-- ✅ **L2** (Euclidean Distance)
-- ✅ **Cosine** (Cosine Similarity)
-
-```bash
-uv run python tests/sanity_checks/test_distance_functions.py
+uv run python benchmarks/diskann_vs_hnsw_speed_comparison.py
 ```
 
-### `test_l2_verification.py`
-Specifically verifies that L2 distance is correctly implemented by:
-- Building indices with L2 vs Cosine metrics
-- Comparing search results and score ranges
-- Validating that different metrics produce expected score patterns
+Compare LEANN storage usage against a FAISS baseline:
 
 ```bash
-uv run python tests/sanity_checks/test_l2_verification.py
+uv run python benchmarks/compare_faiss_vs_leann.py
 ```
 
-### `test_sanity_check.py`
-Comprehensive end-to-end verification including:
-- Distance function testing
-- Embedding model compatibility
-- Search result correctness validation
-- Backend integration testing
+Run the synthetic, model-free code-context expansion benchmark:
 
 ```bash
-uv run python tests/sanity_checks/test_sanity_check.py
+uv run python benchmarks/benchmark_code_context.py \
+  --modules 1000 \
+  --warmups 1 \
+  --repeats 5 \
+  --data-source synthetic-code-context \
+  --data-revision 2026-06-03-fixture \
+  --format markdown \
+  --json-output benchmark-results/code-context.json \
+  --markdown-output benchmark-results/code-context.md
 ```
 
-## 🎯 What These Tests Verify
+This benchmark fixture exercises direct related-code matches, direct immediate-base inherited
+calls, and bounded one-hop exact-call expansion. Its JSON and Markdown artifacts report data
+source/revision, local environment provenance, direct related counts, inherited related counts,
+one-hop related counts, expected inherited/one-hop coverage, unique related passages,
+cap/truncation accounting, command provenance, and timing summaries.
 
-### ✅ Distance Function Support
-- All three distance metrics (MIPS, L2, Cosine) work correctly
-- Score ranges are appropriate for each metric type
-- Different metrics can produce different rankings (as expected)
+The curated documentation report for this benchmark lives in
+[docs/code_context_benchmark.md](../docs/code_context_benchmark.md).
 
-### ✅ Backend Integration
-- DiskANN backend properly initializes and builds indices
-- Graph construction completes without errors
-- Search operations return valid results
+Run the main retrieval evaluation driver:
 
-### ✅ Embedding Pipeline
-- Real-time embedding computation works
-- Multiple embedding models are supported
-- ZMQ server communication functions correctly
-
-### ✅ End-to-End Functionality
-- Index building → searching → result retrieval pipeline
-- Metadata preservation through the entire flow
-- Error handling and graceful degradation
-
-## 🔍 Expected Output
-
-When all tests pass, you should see:
-
-```
-📊 测试结果总结:
-  mips      : ✅ 通过
-  l2        : ✅ 通过
-  cosine    : ✅ 通过
-
-🎉 测试完成!
+```bash
+uv run benchmarks/run_evaluation.py
 ```
 
-## 🐛 Troubleshooting
+This command treats an incomplete `benchmarks/data/` tree, including a README-only checkout or an
+`indices/` tree without any `.index` artifact, as missing benchmark data and downloads the public
+data pack. The repository-provided prebuilt indexes are large; check local disk capacity before
+relying on the automatic download path.
 
-### Common Issues
+For larger retrieval runs after data has been downloaded:
 
-**Import Errors**: Ensure you're running from the project root:
 ```bash
-cd /path/to/leann
-uv run python tests/sanity_checks/test_distance_functions.py
+uv run benchmarks/run_evaluation.py benchmarks/data/indices/rpj_wiki/rpj_wiki \
+  --dataset rpj_wiki \
+  --data-source LEANN-RAG/leann-rag-evaluation-data \
+  --data-revision 2026-06-02-download \
+  --num-queries 2000 \
+  --top-k 3 \
+  --complexity 120 \
+  --format markdown \
+  --json-output benchmark-results/retrieval-rpj-wiki.json \
+  --markdown-output benchmark-results/retrieval-rpj-wiki.md
 ```
 
-**Memory Issues**: Reduce graph complexity for resource-constrained systems:
-```python
-builder = LeannBuilder(
-    backend_name="diskann",
-    graph_degree=8,  # Reduced from 16
-    complexity=16    # Reduced from 32
-)
-```
+Retrieval evaluation is search-only by default. Use `--run-llm` only when generation should be
+included as opt-in metadata outside retrieval latency. Retrieval artifacts include per-query result
+IDs, golden IDs, and duplicate-text counters so reviewers can audit text-overlap recall when passage
+ID schemes differ across indexes. CLI-generated artifacts also record SHA256 hashes for the query
+and ground-truth files plus shell-quoted script command arguments.
 
-**ZMQ Port Conflicts**: The tests use different ports to avoid conflicts, but you may need to kill existing processes:
-```bash
-pkill -f "embedding_server"
-```
+Retrieval, query-log, and code-context artifacts share the same timing-statistics helper. p95 is the
+lower nearest-rank observed sample from sorted timings, so it is deterministic and comparable across
+reviewable benchmark reports. The standalone BM25 and DiskANN baseline scripts use the same
+observed-sample percentile helpers for their latency report fields.
 
-## 📊 Performance Expectations
+Run one retrieval manifest across multiple prebuilt backend indexes:
 
-### Typical Timing (3 documents, consumer hardware):
-- **Index Building**: 2-5 seconds per distance function
-- **Search Query**: 50-200ms
-- **Recompute Mode**: 5-15 seconds (higher accuracy)
-
-### Memory Usage:
-- **Index Storage**: ~1-2 MB per distance function
-- **Runtime Memory**: ~500MB (including model loading)
+```bash
+uv run python benchmarks/compare_retrieval_backends.py benchmark-results/retrieval-comparison.json \
+  --format markdown \
+  --json-output benchmark-results/retrieval-comparison-summary.json \
+  --markdown-output benchmark-results/retrieval-comparison-summary.md
+```
 
-## 🔗 Integration with CI/CD
+The comparison manifest keeps dataset, query file, ground truth, `top_k`, complexity, and batch
+size identical across runs. The combined JSON/Markdown artifacts report per-run backend,
+passage-ID scheme, embedding model/mode, recall@k, hit rate@k, latency, storage bytes, query file
+hash, ground-truth file hash, shell-quoted script command arguments, missing result-ID counts, and
+duplicate result/golden text counters.
+See [docs/benchmarks.md](../docs/benchmarks.md) for the manifest fields and a complete example.
 
-These tests are designed to be run in automated environments:
+Summarize query logs with optional ground truth and storage accounting:
 
-```yaml
-# GitHub Actions example
-- name: Run Sanity Checks
-  run: |
-    uv run python tests/sanity_checks/test_distance_functions.py
-    uv run python tests/sanity_checks/test_l2_verification.py
+```bash
+uv run python benchmarks/summarize_query_log.py queries.jsonl \
+  --ground-truth ground_truth.json \
+  --index-path .leann/indexes/my-index/documents.leann \
+  --data-source my-dataset \
+  --data-revision 2026-06-02-download \
+  --format markdown \
+  --json-output benchmark-results/my-index-summary.json \
+  --markdown-output benchmark-results/my-index-summary.md
 ```
 
-The tests are deterministic and should produce consistent results across different platforms.
+`--format` controls stdout. Use `--json-output` and `--markdown-output` to write durable review
+artifacts in the same run. Query-log summaries include latency mean/median/p95/min/max when
+available plus counters for records with missing result IDs, total missing result IDs, missing
+latency, missing search mode, missing backend metadata, and shell-quoted script command arguments.
+
+The `benchmark-results/` directory is ignored by git and is the default scratch location for local
+summary artifacts. Commit only curated reports with full dataset, hardware, and command context.
+
+## Suites
+
+- `bm25_diskann_baselines/`: Natural Questions BM25 and DiskANN search-only baselines that require
+  externally synced artifacts, report latency with the shared observed-sample percentile helpers,
+  and can write JSON reports with query hashes, timing scope, recursive storage bytes, settings,
+  command arguments, and environment provenance.
+- `contextbench/`: trace-driven code-assistant benchmark tooling.
+- `enron_emails/`: retrieval and generation evaluation on the Enron email corpus.
+- `financebench/`: financial document question-answering benchmark.
+- `laion/`: multimodal image/text retrieval benchmark.
+- `update/`: update-latency and sequential-vs-offline update strategy benchmarks.
+
+Benchmark result files committed under subdirectories are reference outputs from their documented
+settings and hardware. Do not treat them as automatically refreshed results for the current branch
+unless the relevant command has been rerun.