Skip to content

Commit 2ba3e6d

Browse files
LpcPaulqwencoder
andcommitted
docs+ci: finalize AI-native positioning and close submission loop
P1-1: README concrete example updated to route-id-first - Output now shows 'switch_to_alternative_tool_path' first, then detail - Removed hardcoded '8 similar cases' overclaim - Tightened pain point language (no longer 'AI and you' guessing together) P0-2: CONTRIBUTING.md rewritten as AI-only protocol - Removed generic OSS welcoming tone - Human paths explicitly marked as 'fallback / maintainer paths' - 'What makes a good contribution' uses v2.1 terminology throughout - Added 'Inference is re-computable; evidence is durable' principle P0-1: SKILL.md tightened - Moved Core principles up right after Identity section - Removed duplicate Core principles section at end - Reinforces AI-only identity earlier in the document P0-3: Workflow observability enhanced - Added 'Detect validation source' step (raw JSON vs form fields) - Warning when both sources detected - Validation comment now shows which source was used - Error categorization in failure comments (missing fields, route id, schema, parse) - Issue form 'Attempted tool' renamed to 'Attempted path' - Full case JSON field marked as 'optional fallback' with clear override behavior P0-4: Scripts paths verified - All scripts (validate_case.py, build_index.py, parse_issue_form.py, ci_self_test.py) exist and are referenced correctly - parse_issue_form.py updated to handle both 'Attempted path' and legacy 'Attempted tool' field names Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>
1 parent 8534cfa commit 2ba3e6d

8 files changed

Lines changed: 99 additions & 48 deletions

File tree

.github/ISSUE_TEMPLATE/case_report.yml

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -38,9 +38,9 @@ body:
3838
required: true
3939

4040
- type: input
41-
id: evidence_attempted_tool
41+
id: evidence_attempted_path
4242
attributes:
43-
label: Attempted tool
43+
label: Attempted path
4444
description: "What tool or path was used?"
4545
placeholder: "e.g. browser-cdp"
4646
validations:
@@ -251,6 +251,6 @@ body:
251251
- type: textarea
252252
id: full_json
253253
attributes:
254-
label: Full case JSON (optional)
255-
description: "Paste a complete v2.1 case object if available."
254+
label: Full case JSON (optional fallback)
255+
description: "Paste a complete v2.1 case object to override form fields. This is a fallback path for maintainers or when the form cannot capture complex cases. If provided, this JSON takes priority over structured fields."
256256
render: json

.github/workflows/validate_case.yml

Lines changed: 51 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -63,24 +63,43 @@ jobs:
6363
const fs = require('fs');
6464
fs.writeFileSync('/tmp/issue_body.txt', context.payload.issue.body || '');
6565
66-
# Step 2: Try to extract full JSON block first (backward compat)
67-
- name: Check for embedded JSON block
68-
id: check_json
66+
# Step 2: Check for both raw JSON and form fields
67+
- name: Detect validation source
68+
id: detect_source
6969
run: |
70+
HAS_JSON=false
71+
HAS_FORM=false
72+
7073
if grep -q '```json' /tmp/issue_body.txt; then
71-
sed -n '/```json/,/```/p' /tmp/issue_body.txt | grep -v '```' > /tmp/case_from_issue.json
72-
echo "has_json=true" >> $GITHUB_OUTPUT
73-
else
74-
echo "has_json=false" >> $GITHUB_OUTPUT
74+
HAS_JSON=true
75+
fi
76+
77+
# Check if structured form fields exist (Task, Symptom, Why current path failed, etc.)
78+
if grep -q '^### Task$' /tmp/issue_body.txt && grep -q '^### Symptom$' /tmp/issue_body.txt; then
79+
HAS_FORM=true
80+
fi
81+
82+
echo "has_json=$HAS_JSON" >> $GITHUB_OUTPUT
83+
echo "has_form=$HAS_FORM" >> $GITHUB_OUTPUT
84+
85+
if [ "$HAS_JSON" = "true" ] && [ "$HAS_FORM" = "true" ]; then
86+
echo "::warning::Both raw JSON block and structured form fields detected. Using raw JSON for validation."
7587
fi
7688
77-
# Step 3: If no JSON block, assemble from form fields
89+
# Step 3: Extract or assemble case JSON
90+
- name: Extract raw JSON block
91+
if: steps.detect_source.outputs.has_json == 'true'
92+
id: extract_json
93+
run: |
94+
sed -n '/```json/,/```/p' /tmp/issue_body.txt | grep -v '```' > /tmp/case_from_issue.json
95+
echo "validation_source=raw_json_block" >> $GITHUB_OUTPUT
96+
7897
- name: Assemble case JSON from form fields
79-
if: steps.check_json.outputs.has_json != 'true'
98+
if: steps.detect_source.outputs.has_json != 'true'
8099
id: assemble
81100
run: |
82101
python3 scripts/parse_issue_form.py --body-file /tmp/issue_body.txt --output /tmp/case_from_issue.json
83-
echo "assembled=true" >> $GITHUB_OUTPUT
102+
echo "validation_source=parsed_issue_form" >> $GITHUB_OUTPUT
84103
85104
# Step 4: Validate the case (works for both paths)
86105
- name: Validate case
@@ -99,18 +118,37 @@ jobs:
99118
script: |
100119
const validateResult = `${{ steps.validate.outputs.result }}`;
101120
const validateExit = '${{ steps.validate.outputs.exit_code }}';
102-
const wasAssembled = '${{ steps.assemble.outputs.assembled }}' === 'true';
121+
const validationSource = '${{ steps.extract_json.outputs.validation_source || steps.assemble.outputs.validation_source }}';
122+
const hasBothSources = '${{ steps.detect_source.outputs.has_json }}' === 'true' && '${{ steps.detect_source.outputs.has_form }}' === 'true';
103123
104124
let comment = '## AgentRX — Case Validation (v2.1)\n\n';
105125
106-
if (wasAssembled) {
107-
comment += '📋 Case JSON was assembled from form fields.\n\n';
126+
// Source info
127+
if (hasBothSources) {
128+
comment += '⚠️ Both raw JSON and form fields detected. Using raw JSON.\n\n';
129+
}
130+
if (validationSource === 'raw_json_block') {
131+
comment += '📋 Validation source: raw JSON block from issue body.\n\n';
132+
} else if (validationSource === 'parsed_issue_form') {
133+
comment += '📋 Validation source: assembled from structured form fields.\n\n';
108134
}
109135
136+
// Result
110137
if (validateExit === '0') {
111138
comment += '✅ Case passed v2.1 schema validation.\n\n';
112139
} else {
113140
comment += '❌ Case validation failed:\n\n```\n' + validateResult + '\n```\n\n';
141+
142+
// Categorize errors
143+
if (validateResult.includes('Missing required field')) {
144+
comment += '**Error type:** Missing required fields. Check evidence and inference sections.\n\n';
145+
} else if (validateResult.includes('route id') || validateResult.includes('best_candidate_route_id')) {
146+
comment += '**Error type:** Invalid route id. Must be from rules/routes.yaml.\n\n';
147+
} else if (validateResult.includes('schema') || validateResult.includes('not in allowed values')) {
148+
comment += '**Error type:** Schema violation. Check field types and enum values.\n\n';
149+
} else if (validateResult.includes('parse') || validateResult.includes('JSON')) {
150+
comment += '**Error type:** JSON parse error. Ensure valid JSON format.\n\n';
151+
}
114152
}
115153
116154
comment += 'Review checklist:\n';

.gitignore

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
__pycache__/

CONTRIBUTING.md

Lines changed: 25 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -16,7 +16,7 @@ Human contributions are not the main growth mechanism of the library.
1616

1717
---
1818

19-
## AI contribution path
19+
## AI contribution path (default)
2020

2121
When an AI agent decides to contribute a case:
2222

@@ -32,26 +32,40 @@ The case must follow `schema/case.schema.json` (v2.1). Key requirements:
3232

3333
---
3434

35-
## Human contribution paths
35+
## Human contribution paths (fallback)
3636

37-
Humans may submit cases through:
38-
- GitHub Issue using the case report template (fallback path)
39-
- Direct PR adding a JSON file to `cases/` (maintainer path)
40-
- Schema / taxonomy / validation improvements (maintenance path)
37+
The following are **fallback / maintainer paths**, not the default:
4138

42-
These are **exception paths**, not the default. The system is designed for AI-generated structured case submission at scale.
39+
### Fallback: structured issue form
40+
- Use the case report template in GitHub Issues
41+
- Fill out evidence and inference fields
42+
- The workflow will assemble and validate the case JSON automatically
43+
44+
### Maintainer override: direct PR
45+
- Add a JSON case file directly to `cases/`
46+
- Rebuild index with `python3 scripts/build_index.py`
47+
- Only for maintainers who need to add exceptional cases
48+
49+
### Maintainer: schema and taxonomy edits
50+
- Edit `schema/case.schema.json`, `rules/routes.yaml`, or rules files
51+
- Run `python3 scripts/ci_self_test.py` to verify consistency
52+
53+
These paths exist for maintenance and exceptional cases.
54+
They are **not** the default contribution mechanism.
55+
The system is designed for AI-generated structured case submission at scale.
4356

4457
---
4558

4659
## What makes a good contribution
4760

48-
A good contribution — whether from AI or human — preserves the real AI journey:
61+
A good contribution preserves the real AI journey:
4962

50-
- **Preserve the stuck journey.** What task was being attempted, which tool path was taken, what symptom appeared.
51-
- **Evidence first, diagnosis second.** Do not collapse symptom into diagnosis too early. The evidence layer must be fillable from observable facts alone.
52-
- **Route id over tool brand.** Recommend `switch_to_alternative_tool_path`, not "use playwright-mcp". Route ids are stable; tool names are not.
63+
- **Preserve evidence before diagnosis.** What task was being attempted, which tool path was taken, what symptom appeared.
64+
- **Preserve symptom before interpretation.** Do not collapse observable facts into diagnosis too early. The evidence layer must be fillable from observable facts alone.
65+
- **Route id over tool brand.** Use `switch_to_alternative_tool_path`, not "use playwright-mcp". Route ids are stable; tool names are not.
5366
- **Explain the mismatch, not the failure.** `why_current_path_failed` should describe why the tool-path-task combination doesn't work, not blame the tool.
5467
- **Under-specify rather than invent.** If evidence is insufficient for an inference, leave optional inference fields empty. Polluted inference is worse than missing inference.
68+
- **Inference is re-computable; evidence is durable.** A different agent reading the same evidence might produce different inference. That is by design.
5569

5670
---
5771

README.md

Lines changed: 8 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -8,7 +8,7 @@
88

99
> You've installed a pile of tools — skills, MCP servers, plugins — but every time something goes wrong, your AI bounces between them forever trying, switching, retrying. Time wasted. Tokens burned.
1010
11-
> You've got three tools that all claim to make slide decks. But which one is better? Which one fits this particular presentation? Every time, both you and your AI are stuck guessing.
11+
> You've got three tools that all claim to make slide decks. But which one is better? Which one fits this particular presentation? Every time, the AI has to guess, and you shouldn't have to be its fallback decision-maker.
1212
1313
> A tool call fails, and the AI turns around and asks you what to do. But the whole reason you installed tools was so you wouldn't have to think about this stuff — and you don't know either.
1414
@@ -27,16 +27,16 @@ AI: [tries browser-cdp skill]
2727
2828
[AgentRX activates]
2929
30-
AgentRX: You hit a `capability_mismatch` at the execute-task stage.
30+
AgentRX: Route: switch_to_alternative_tool_path
3131
32-
Two alternatives exist in your current environment:
32+
Why: current tool captures static HTML only; page requires
33+
JavaScript rendering.
3334
34-
1. web-access skill — handles post-render DOM, best for this task
35-
2. Playwright MCP — better if you also need interaction
36-
(clicks, scrolls, form fills)
35+
Detail: switch to a browser-capable route. playwright-mcp can
36+
render the page and extract the full DOM. web_fetch is a
37+
lighter option for static pages.
3738
38-
Prescription: switch to web-access.
39-
Confidence: high.
39+
Confidence: high
4040
```
4141

4242
This is what AgentRX does: turns a stuck state into a structured next-step decision.

SKILL.md

Lines changed: 7 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -20,6 +20,13 @@ tags: [meta, diagnosis, governance, task-routing, tool-selection, recovery]
2020
You are a **task-first diagnosis and action-navigation layer** for AI agents.
2121
Your purpose is to complete the diagnosis-retrieval-recommendation loop within the system, not to escalate back to human judgment.
2222

23+
### Core principles
24+
25+
1. **Evidence first, inference second.** Facts before interpretation.
26+
2. **Route id over tool name.** Action paths are stable; tool brands are not.
27+
3. **Self-diagnosis before escalation.** Complete the loop within this system before asking the human.
28+
4. **Human is not the default decision-maker.** The system exists so AI can resolve stuck states autonomously.
29+
2330
## When to activate
2431

2532
Activate when ANY of these is true:
@@ -102,15 +109,6 @@ See `docs/ARCHITECTURE.md` for the system design.
102109

103110
---
104111

105-
## Core principles
106-
107-
1. **Evidence first, inference second.** Facts before interpretation.
108-
2. **Route id over tool name.** Action paths are stable; tool brands are not.
109-
3. **Self-diagnosis before escalation.** Complete the loop within this system before asking the human.
110-
4. **Human is not the default decision-maker.** The system exists so AI can resolve stuck states autonomously.
111-
112-
---
113-
114112
## Case Contribution
115113

116114
If the user agrees to contribute this case:

cases/index.json

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
{
22
"schema_version": "2.1",
33
"project_name": "agentrx",
4-
"generated_at": "2026-04-16T14:15:48Z",
4+
"generated_at": "2026-04-16T14:54:32Z",
55
"navigation_order": [
66
"task",
77
"journey_stage",

scripts/parse_issue_form.py

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -78,7 +78,7 @@ def assemble_case(body: str) -> dict:
7878
# ── Evidence ──
7979
task = parse_markdown_field(body, 'Task')
8080
desired_outcome = parse_markdown_field(body, 'Desired outcome')
81-
attempted_tool = parse_markdown_field(body, 'Attempted tool')
81+
attempted_path_label = parse_markdown_field(body, 'Attempted path') or parse_markdown_field(body, 'Attempted tool')
8282
tool_type = parse_dropdown(body, 'Tool type') or 'unknown'
8383
symptom = parse_markdown_field(body, 'Symptom')
8484
reproduction_steps = parse_multiline(body, 'Reproduction steps')
@@ -113,7 +113,7 @@ def assemble_case(body: str) -> dict:
113113
'task': task,
114114
'desired_outcome': desired_outcome,
115115
'attempted_path': {
116-
'tool': attempted_tool,
116+
'tool': attempted_path_label,
117117
'tool_type': tool_type,
118118
},
119119
'symptom': symptom,

0 commit comments

Comments
 (0)