Community readiness: public benchmark, real-output docs, MCP 2.x fix, release gates

.claude/skills/codebase-index/references/response-contract.md

@@ -18,7 +18,10 @@ Each result can contain:
 - `elided_lines`
 
 `recommended_reads` is the read plan. Start with its first one to three entries
-and use exact line ranges.
+and use exact line ranges. An entry with `truncated: true` was capped at the
+definition head (`max_read_lines`, default 120); `line_end_full` gives the real
+extent. Read the capped range first and continue only when the head is not
+enough.
 
 `pagination.has_more` and `pagination.next_offset` indicate additional results.
 Prefer a more specific command or a larger token budget before paging.

.codex/skills/codebase-index/references/response-contract.md

@@ -18,7 +18,10 @@ Each result can contain:
 - `elided_lines`
 
 `recommended_reads` is the read plan. Start with its first one to three entries
-and use exact line ranges.
+and use exact line ranges. An entry with `truncated: true` was capped at the
+definition head (`max_read_lines`, default 120); `line_end_full` gives the real
+extent. Read the capped range first and continue only when the head is not
+enough.
 
 `pagination.has_more` and `pagination.next_offset` indicate additional results.
 Prefer a more specific command or a larger token budget before paging.

.github/ISSUE_TEMPLATE/benchmark_report.yml

@@ -0,0 +1,62 @@
+name: Benchmark report
+description: Share a reproduced, failed, or new benchmark run (good or bad numbers both welcome)
+title: "[Benchmark]: "
+labels: ["benchmark"]
+body:
+  - type: markdown
+    attributes:
+      value: |
+        Independent runs are how this project keeps its numbers honest. Unflattering results are as useful as flattering ones.
+
+  - type: dropdown
+    id: kind
+    attributes:
+      label: What did you run?
+      options:
+        - tests/eval/run_eval.py (retrieval eval, own corpus)
+        - tests/eval/run_eval.py (shipped query sets)
+        - tests/benchmark_public.py (synthetic suite)
+        - tests/benchmark_honest.py (index vs rg+window)
+        - Something else (describe below)
+    validations:
+      required: true
+
+  - type: input
+    id: version
+    attributes:
+      label: codebase-index version / commit
+      placeholder: "1.9.0 or git SHA"
+    validations:
+      required: true
+
+  - type: input
+    id: corpus
+    attributes:
+      label: Corpus
+      description: Public repository URL + commit SHA, or language / size if private.
+      placeholder: "https://github.com/pallets/flask @ d318b68 — Python, ~18k LOC"
+    validations:
+      required: true
+
+  - type: textarea
+    id: command
+    attributes:
+      label: Exact commands
+      render: shell
+    validations:
+      required: true
+
+  - type: textarea
+    id: results
+    attributes:
+      label: Results table / raw output
+      description: Paste the pooled table and significance rows as printed. Attach the JSON if large.
+      render: markdown
+    validations:
+      required: true
+
+  - type: textarea
+    id: notes
+    attributes:
+      label: Observations
+      description: Anything surprising, queries that failed, or where the baseline beat the index.

.github/ISSUE_TEMPLATE/bug_report.yml

@@ -1,74 +1,74 @@
-name: Bug Report
-description: Report a bug or unexpected behavior
+name: Bug report
+description: Something is wrong, crashes, or returns the wrong result
 title: "[Bug]: "
 labels: ["bug", "triage"]
 body:
   - type: markdown
     attributes:
       value: |
-        Thanks for taking the time to report a bug. Please fill in as much of the form below as possible.
+        Thanks. A reproducible report gets fixed fastest. Do not paste real secrets — the tool redacts snippets, your logs may not.
 
   - type: input
     id: version
     attributes:
       label: codebase-index version
-      description: What version of codebase-index are you running?
-      placeholder: "0.1.0"
+      description: "`pip show codebase-index` (there is no --version flag)"
+      placeholder: "1.9.0"
     validations:
       required: true
 
   - type: input
-    id: python-version
+    id: env
     attributes:
-      label: Python version
-      description: What Python version are you using?
-      placeholder: "3.12.3"
+      label: Python and OS
+      placeholder: "Python 3.12.3 · macOS 14.4 / Ubuntu 24.04 / Windows 11"
     validations:
       required: true
 
-  - type: input
-    id: os
+  - type: dropdown
+    id: surface
     attributes:
-      label: Operating system
-      description: What OS and version are you using?
-      placeholder: "macOS 14.4, Ubuntu 24.04, Windows 11"
+      label: Surface
+      options:
+        - CLI
+        - Claude Code skill / plugin
+        - Codex CLI
+        - OpenCode
+        - MCP server
+        - Installer scripts
+        - Other
     validations:
       required: true
 
   - type: textarea
     id: what-happened
     attributes:
-      label: What happened?
-      description: Describe the bug and what you expected to happen.
+      label: What happened, and what did you expect?
       placeholder: |
-        1. Ran `codebase-index index` on my project
-        2. Expected: index completes successfully
-        3. Actual: got error "..."
+        Ran `codebase-index refs open_session --json` on <repo>. Expected the call in ctx.py; got an empty list with coverage.partial=false.
     validations:
       required: true
 
   - type: textarea
-    id: reproduction
+    id: repro
     attributes:
       label: Steps to reproduce
-      description: Provide minimal steps to reproduce the issue.
+      description: Ideally against a public repository or a tiny snippet we can index.
       placeholder: |
-        1. `codebase-index init`
-        2. `codebase-index index`
-        3. `codebase-index search "query"`
-        4. See error
+        1. git clone https://github.com/... && cd ...
+        2. codebase-index index
+        3. codebase-index search "..." --json
     validations:
       required: true
 
   - type: textarea
-    id: logs
+    id: doctor
     attributes:
-      label: Relevant log output
-      description: Paste any relevant terminal output or error messages.
+      label: "`codebase-index doctor` output"
       render: shell
 
   - type: textarea
-    id: additional
+    id: logs
     attributes:
-      label: Additional context
-      description: Add any other context, screenshots, or configuration details.
+      label: Relevant output (`--json` where applicable) or traceback
+      render: shell

.github/ISSUE_TEMPLATE/config.yml

@@ -0,0 +1,8 @@
+blank_issues_enabled: false
+contact_links:
+  - name: Questions and ideas
+    url: https://github.com/denfry/codebase-index/discussions
+    about: Ask how to use the tool, discuss design, or share what you built with it.
+  - name: Security vulnerability
+    url: https://github.com/denfry/codebase-index/security/advisories/new
+    about: Report privately. Please do not open a public issue (see SECURITY.md).

.github/ISSUE_TEMPLATE/feature_request.yml

@@ -1,56 +1,50 @@
-name: Feature Request
-description: Suggest an idea or improvement for codebase-index
+name: Feature request
+description: Propose an improvement or a new capability
 title: "[Feature]: "
 labels: ["enhancement", "triage"]
 body:
-  - type: markdown
-    attributes:
-      value: |
-        Thanks for suggesting a feature. Please describe your idea and its use case.
-
   - type: textarea
     id: problem
     attributes:
-      label: Problem statement
-      description: Is your feature request related to a problem? Describe it.
-      placeholder: "I'm frustrated when..."
+      label: Problem
+      description: What are you trying to do with your agent that the tool makes hard today?
+      placeholder: "When I ask Claude Code about ..., codebase-index returns ..., so the agent ends up reading ..."
     validations:
       required: true
 
   - type: textarea
-    id: solution
+    id: proposal
     attributes:
-      label: Proposed solution
-      description: Describe what you want to happen.
-      placeholder: "It would be great if codebase-index could..."
+      label: Proposal
+      description: What should happen instead? A CLI example or a sketch of the JSON is ideal.
     validations:
       required: true
 
-  - type: textarea
-    id: alternatives
-    attributes:
-      label: Alternatives considered
-      description: Describe any alternative solutions or workarounds you've tried.
-
   - type: dropdown
-    id: scope
+    id: area
     attributes:
-      label: Scope
-      description: Which area does this feature primarily affect?
+      label: Area
       options:
-        - Indexing (discovery, parsing, storage)
-        - Retrieval (search, ranking, fusion)
-        - CLI (commands, output format)
-        - Skill (SKILL.md, allowed-tools, workflow)
-        - Configuration (config.json, hooks)
+        - Retrieval quality (ranking, fusion, budget)
+        - Graph extraction (edges, impact, paths)
+        - Language support
+        - Indexing (discovery, parsing, freshness)
+        - CLI / output format
+        - MCP server
+        - Claude Code / Codex / OpenCode integration
+        - Benchmarks / evaluation
         - Documentation
-        - Testing / CI
         - Other
     validations:
       required: true
 
   - type: textarea
-    id: additional
+    id: evidence
+    attributes:
+      label: How would we know it worked?
+      description: Retrieval changes need a measurable effect (see tests/eval). A query and the file you expected is enough to start.
+
+  - type: textarea
+    id: alternatives
     attributes:
-      label: Additional context
-      description: Add any other context, mockups, or examples.
+      label: Alternatives or workarounds you tried

.github/ISSUE_TEMPLATE/language_support.yml

@@ -0,0 +1,48 @@
+name: Language support
+description: Request or offer symbol/graph extraction for a language
+title: "[Language]: "
+labels: ["language support"]
+body:
+  - type: input
+    id: language
+    attributes:
+      label: Language and file extensions
+      placeholder: "Swift — .swift"
+    validations:
+      required: true
+
+  - type: dropdown
+    id: current
+    attributes:
+      label: Current status (see docs/LANGUAGES.md)
+      options:
+        - Not detected at all
+        - Tier C (line chunks + FTS only)
+        - Tier B (generic Tree-sitter, no graph edges)
+        - Tier A but extraction is wrong or incomplete
+    validations:
+      required: true
+
+  - type: input
+    id: grammar
+    attributes:
+      label: tree-sitter-language-pack grammar name
+      description: "`tree_sitter_language_pack.get_language('<name>')` must succeed."
+      placeholder: "swift"
+
+  - type: textarea
+    id: expectations
+    attributes:
+      label: What should be extracted?
+      description: Definition kinds (function/class/...), calls, imports, inheritance — with a 10–20 line sample file.
+      render: text
+    validations:
+      required: true
+
+  - type: checkboxes
+    id: offer
+    attributes:
+      label: Contribution
+      options:
+        - label: I can open a PR adding the `LangSpec` and fixture (see docs/DEVELOPMENT.md → "Add a language")
+        - label: I can provide a public repository in this language for the retrieval eval

.github/PULL_REQUEST_TEMPLATE.md

@@ -1,38 +1,21 @@
-## Description
+## What and why
 
-Describe your changes and their impact.
+<!-- Problem, then the change. Link the issue: Fixes #123 -->
 
-Fixes #(issue)
+## How I verified it
 
-## Type of change
+```bash
+# commands you ran, e.g.
+pytest tests/test_xxx.py -q --no-cov
+python tests/eval/run_eval.py --ablate   # required for any ranking change; paste the table below
+```
 
-- [ ] Bug fix (non-breaking change that fixes an issue)
-- [ ] New feature (non-breaking change that adds functionality)
-- [ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)
-- [ ] Documentation update
-- [ ] Refactoring (no functional change)
-- [ ] CI / build / tooling change
+<!-- For retrieval changes: pooled table + significance row for your flag. -->
 
 ## Checklist
 
-- [ ] I have read the [CONTRIBUTING.md](../CONTRIBUTING.md) guide
-- [ ] My code follows the project's style conventions (`ruff check`, `ruff format`)
-- [ ] I have added tests that prove my fix is effective or that my feature works
-- [ ] All tests pass (`pytest`)
-- [ ] Type checking passes (`mypy`, if applicable)
-- [ ] I have updated the documentation accordingly
-- [ ] I have updated `CHANGELOG.md` under `[Unreleased]`
-- [ ] My commits follow [Conventional Commits](https://www.conventionalcommits.org/)
-
-## Testing
-
-Describe how you tested your changes:
-
-```bash
-# Example:
-pytest tests/test_my_new_feature.py
-codebase-index index --root tests/fixtures/sample_repo
-codebase-index search "test query"
-```
-
-## Screenshots (if applicable)
+- [ ] `pytest`, `ruff check src tests`, `mypy src/codebase_index` pass
+- [ ] `python scripts/sync_skill_copies.py --check` passes (if `skill_template/` changed)
+- [ ] Goldens regenerated intentionally and the diff explained (if `--json` / MCP output changed)
+- [ ] `CHANGELOG.md` updated under `[Unreleased]` (user-visible changes only; no version bump)
+- [ ] No secrets, generated indexes, or local config committed