Skip to content

Tool-definition quality pass: raise low Usage-Guidelines and Behavior scores (Glama) #154

Description

@ajdelaguila

Summary

Tool Definition Quality is A overall (avg 4.1/5), but two dimensions drag every low tool down and directly cause agents to pick the wrong tool:

  • Usage Guidelines — avg 3.56 (weakest): descriptions rarely say "call this when X; use kb_Y instead when Z."
  • Behavior — avg 3.89: some descriptions do not disclose side effects (audit-log writes) that the annotations imply.

Purpose (4.89) and Conciseness (4.50) are already strong — preserve them. This issue is a coordinated docstring pass on the low tools (kb_record_event is tracked separately as the sole C-grade tool).

Score page: https://glama.ai/mcp/servers/knaisoma/data-olympus/score

Target tools

Source: docstrings in src/data_olympus/server.py.

Tool Overall Behavior Usage Fix
kb_gate_check 3.5 2 3 Disclose it records the check to the audit log (annotation readOnlyHint=false is intentional; docstring must explain the side effect) + when-to-use vs kb_consult.
kb_onboarding_status 3.6 4 2 Add trigger + alternative.
kb_compliance 3.8 3 3 When-to-use vs kb_audit/kb_gate_check.
kb_cleanup_plan 4.0 4 3 When-to-use vs kb_search/kb_propose_edit.
kb_get 4.1 4 3 When-to-use vs kb_search/kb_list.
kb_list 4.1 4 3 When-to-use vs kb_search/kb_get.
kb_outline 4.4 4 3 When-to-use vs kb_list.

Exemplars to mirror

kb_session_recap and kb_propose_edit score 5/5 on Usage Guidelines because they explicitly name the trigger and the sibling to prefer otherwise. Copy that pattern: one sentence per tool of the form "Use when ; use kb_<sibling> instead when ."

Guardrails

  • Do not inflate descriptions — Conciseness (4.50) and Parameters must not regress. Parameters sits at baseline 3 because schema coverage is already 100%; only add parameter intent where it is genuinely non-obvious, never restate the schema.
  • Behavior fixes are about disclosing real side effects (audit writes), not padding.

Acceptance

  • Usage Guidelines avg >= 4.3 and Behavior avg >= 4.2 on the next Glama re-score.
  • No tool below A; Conciseness avg not lower than current 4.50.

Metadata

Metadata

Assignees

No one assigned

    Labels

    documentationImprovements or additions to documentationenhancementNew feature or requestglamaGlama.ai registry score / discoverability

    Type

    No type

    Projects

    No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions