Skip to content

Replace the doc-style checker with a documentation maintenance skill - #10

Merged
MirzaMerdovic merged 1 commit into
mainfrom
maintain-docs
Oct 9, 2026
Merged

MirzaMerdovic merged 1 commit into
mainfrom
maintain-docs

Conversation

@MirzaMerdovic

Copy link
Copy Markdown
Contributor

Summary

The doc-style skill now maintains documentation. It no longer checks prose against fixed
rules. When Claude changes code, the skill directs it to find the comments, docstrings, and
documents that describe the changed code. Claude edits, consolidates, or deletes that prose
before it adds more. This change removes the Bash checker and its tests.

Changes

  • skills/doc-style/SKILL.md holds instructions only:
    • a six-step workflow scaled to the size of the change;
    • the edit order: delete, correct, consolidate, replace, add;
    • current state versus change history;
    • rules for comments and docstrings;
    • the writing style, with no sentence limit and no word list.
  • skills/doc-style/examples/maintenance.md gives eight maintenance decisions, with the
    text before and after.
  • skills/doc-style/examples/rewrites.md gives sentence rewrites, each judged in context.
  • hooks/hooks.json adds a SessionStart hook that prints snippets/CLAUDE.md into the
    context of each session. In eval runs on Sonnet without the hook, Claude loaded the skill
    in 0 of 7 tasks that asked only for a code change. With the hook, it loaded the skill in
    every run.
  • evals/ holds nine cases for claude plugin eval. Each case copies a small workspace
    with scaffold.sh and grades the files that Claude leaves.
  • The change removes these files and directories:
    • skills/doc-style/scripts/check, rules.md, and deviations.md;
    • .doc-style and commands/doc-style-check.md;
    • tests/ and .github/workflows/mutate.yml;
    • mise.toml and mise.lock.
  • CI runs only claude plugin validate. mise.ci.toml still pins the Claude Code version
    for that step.

Compatibility

  • The /haddock:doc-style-check command is removed. A .doc-style file has no effect.
  • The plugin version stays 0.4.0. An installed copy can keep the old content until the
    version changes.
  • The hook adds about 190 tokens to every session that has the plugin enabled.

Validation

  • claude plugin validate . passes with Claude Code 2.1.292, the version that CI pins.

  • shellcheck and shfmt -d pass on the scaffold.sh files.

  • claude plugin eval . --scaffold --allow-tools Edit Write ran each case with and
    without the plugin. A score is the fraction of graders that passed:

    Model Runs per case and arm Score with plugin Score without plugin Skill loaded
    Opus 2 0.99 0.76 18 of 18
    Sonnet 1 0.94 0.79 9 of 9

    Without the plugin, Claude did not do these updates:

    • It did not correct a README paragraph that a behavior change made wrong.
    • It did not remove change history from a code comment.
    • It did not update a README table that the change affected.
    • It did not consolidate two documents that stated the same fact.
  • In a three-turn claude -p --resume session on Sonnet, Claude loaded the skill once. It
    did not load the skill again for the two later code changes.

Known limitations

  • Sonnet did not consolidate the duplicated documents in any of 9 runs. Opus consolidated
    them in 6 of 6 runs.
  • Sonnet updated a README table that names no changed identifier in 2 of 6 runs.
  • The eval cases do not grant Bash, so the evals do not test the instruction to read the
    commit history.
  • The evals do not run in CI. Each run starts Claude sessions with the credentials of the
    person who runs it.
  • Most results come from 1 or 2 runs per case. The LLM graders use Haiku.
  • No test covers the hook after compaction.

The doc-style skill directs Claude to keep comments, docstrings, and
documents accurate when code changes. Claude finds the prose that a
change affects, then edits, consolidates, or deletes it before it adds
more. Documentation describes the current system. Commit messages and
changelogs hold the change history.

- Rewrite SKILL.md as instructions only. Add maintenance examples and
  revise the rewrite examples.
- Remove the Bash checker, the .doc-style configuration and
  suppressions, the doc-style-check command, the bats and mutation
  tests, and the ASD-STE100 comparison.
- Add a SessionStart hook that prints the primer. Without it, Claude did
  not load the skill for tasks that asked only for a code change.
- Add eval cases for claude plugin eval.
- Reduce CI to the plugin validation.
@MirzaMerdovic
MirzaMerdovic merged commit 70283cd into main Oct 9, 2026
1 check passed
@MirzaMerdovic
MirzaMerdovic deleted the maintain-docs branch October 10, 2026 10:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant