Skip to content

Commit 8534cfa

Browse files
LpcPaulqwencoder
andcommitted
docs+ci: close narrative and submission loop
P0-1: README definition layer aligned to v2.1 - Replaced old 'task+journey_stage+problem_family' framing with evidence/inference/route-id-first model - Tightened 'Human role vs AI role' — human = installer/host, AI = default operator and contributor - Removed 'AI and you' / 'give answer to AI and you' language - Added v2.1 two-layer case diagram - Removed overclaim 'Based on 8 similar cases' from concrete example - Added 'What changes after installation?' section P0-2: ARCHITECTURE.md human role tightened - Audience: 'Primary runtime consumer: AI agents' (not 'primary/secondary') - 'Human role: installer, repository host, and occasional maintainer' - Added: 'Installation is a human decision; operation is not.' - Added parse_issue_form.py to Execution Scripts table P0-3: Issue form -> validation loop closed (Scheme A) - New scripts/parse_issue_form.py: assembles v2.1 case JSON from GitHub issue form fields (evidence + inference) - Workflow uses Python parser instead of inline JS for robustness - Dual-path: extracts embedded JSON block first (backward compat), then falls back to form field assembly - End-to-end tested: form -> parse -> validate_case.py -> PASSED Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>
1 parent 2edfe20 commit 8534cfa

5 files changed

Lines changed: 272 additions & 200 deletions

File tree

.github/workflows/validate_case.yml

Lines changed: 23 additions & 133 deletions
Original file line numberDiff line numberDiff line change
@@ -35,9 +35,7 @@ jobs:
3535
run: pip install pyyaml
3636

3737
- name: Run CI self-test
38-
id: self_test
39-
run: |
40-
python3 scripts/ci_self_test.py
38+
run: python3 scripts/ci_self_test.py
4139

4240
validate-issue-case:
4341
if: github.event_name == 'issues' && contains(github.event.issue.labels.*.name, 'case-report')
@@ -57,143 +55,35 @@ jobs:
5755
- name: Install dependencies
5856
run: pip install pyyaml
5957

60-
# Step A: Try to extract full JSON from issue body first (optional field)
61-
- name: Extract JSON from issue body
62-
id: extract_json
58+
# Step 1: Write issue body to file for parsing
59+
- name: Write issue body to file
6360
uses: actions/github-script@v7
6461
with:
6562
script: |
66-
const body = context.payload.issue.body || '';
67-
const jsonMatch = body.match(/```json\s*\n([\s\S]*?)\n```/);
68-
if (jsonMatch) {
69-
const fs = require('fs');
70-
fs.writeFileSync('/tmp/case_from_issue.json', jsonMatch[1]);
71-
core.setOutput('has_json', 'true');
72-
} else {
73-
core.setOutput('has_json', 'false');
74-
}
63+
const fs = require('fs');
64+
fs.writeFileSync('/tmp/issue_body.txt', context.payload.issue.body || '');
7565
76-
# Step B: If no JSON block, assemble v2.1 case JSON from structured form fields
66+
# Step 2: Try to extract full JSON block first (backward compat)
67+
- name: Check for embedded JSON block
68+
id: check_json
69+
run: |
70+
if grep -q '```json' /tmp/issue_body.txt; then
71+
sed -n '/```json/,/```/p' /tmp/issue_body.txt | grep -v '```' > /tmp/case_from_issue.json
72+
echo "has_json=true" >> $GITHUB_OUTPUT
73+
else
74+
echo "has_json=false" >> $GITHUB_OUTPUT
75+
fi
76+
77+
# Step 3: If no JSON block, assemble from form fields
7778
- name: Assemble case JSON from form fields
79+
if: steps.check_json.outputs.has_json != 'true'
7880
id: assemble
79-
if: steps.extract_json.outputs.has_json != 'true'
80-
uses: actions/github-script@v7
81-
with:
82-
script: |
83-
const body = context.payload.issue.body || '';
84-
const fs = require('fs');
85-
86-
function getField(label) {
87-
const regex = new RegExp(`^### ${label.replace(/[.*+?^${}()|[\]\\]/g, '\\$&')}\\s*\\n([\\s\\S]*?)(?=\\n### |$)`, 'm');
88-
const match = body.match(regex);
89-
if (match) {
90-
let val = match[1].trim();
91-
// Remove "_No response_" placeholder
92-
if (val === '_No response_') return '';
93-
return val;
94-
}
95-
return '';
96-
}
97-
98-
function getDropdown(label) {
99-
const val = getField(label);
100-
// Clean up markdown list formatting
101-
return val.replace(/^-\s*/, '').trim();
102-
}
103-
104-
function getMultiLine(label) {
105-
const val = getField(label);
106-
// Split by lines, remove empty lines and "- " prefix
107-
return val.split('\n')
108-
.map(l => l.replace(/^-\s*/, '').trim())
109-
.filter(l => l.length > 0);
110-
}
111-
112-
const evidenceTask = getField('Task') || '';
113-
const evidenceDesiredOutcome = getField('Desired outcome') || '';
114-
const evidenceAttemptedTool = getField('Attempted tool') || '';
115-
const evidenceToolType = getDropdown('Tool type') || 'unknown';
116-
const evidenceSymptom = getField('Symptom') || '';
117-
const evidenceReproductionSteps = getMultiLine('Reproduction steps');
118-
const evidenceContext = getField('Context (optional)') || '';
119-
const evidenceEnvironmentNotes = getField('Environment notes (optional)') || '';
120-
const evidenceFailedStep = getField('Failed step (optional)') || '';
121-
const evidenceArtifactsUsed = getMultiLine('Artifacts used (optional)');
122-
123-
const inferenceJourneyStage = getDropdown('Journey stage') || 'unknown';
124-
const inferenceProblemFamily = getDropdown('Problem family') || 'unknown';
125-
const inferenceWhyCurrentPathFailed = getField('Why current path failed') || '';
126-
const inferenceBestCandidateRouteId = getDropdown('Best candidate route id') || '';
127-
const inferenceBestCandidateRouteDetail = getField('Route detail (optional)') || '';
128-
const inferencePrerequisitesForSwitch = getMultiLine('Prerequisites for switch (optional)');
129-
const inferenceConfidence = getDropdown('Confidence') || 'medium';
130-
131-
const resolutionOutcome = getDropdown('Outcome') || 'unknown';
132-
const resolutionFollowUpNotes = getField('Follow-up notes (optional)') || '';
133-
134-
const metadataSource = getDropdown('Source') || 'user-reported';
135-
const metadataTags = (getField('Tags') || '').split(',').map(t => t.trim()).filter(t => t.length > 0);
136-
137-
// Parse environment notes for platform and constraints
138-
const environment = { platform: 'other' };
139-
if (evidenceEnvironmentNotes) {
140-
const platformMatch = evidenceEnvironmentNotes.match(/[Pp]latform:\s*(\S+)/);
141-
if (platformMatch) environment.platform = platformMatch[1];
142-
if (evidenceEnvironmentNotes.includes('dynamic render')) environment.requires_dynamic_render = true;
143-
if (evidenceEnvironmentNotes.includes('network')) environment.requires_network = true;
144-
if (evidenceEnvironmentNotes.includes('login')) environment.requires_login = true;
145-
if (evidenceEnvironmentNotes.includes('filesystem')) environment.requires_local_filesystem = true;
146-
if (evidenceEnvironmentNotes.includes('deterministic')) environment.requires_deterministic_execution = true;
147-
environment.notes = evidenceEnvironmentNotes;
148-
}
149-
150-
const caseData = {
151-
schema_version: "2.1",
152-
id: `${new Date().toISOString().slice(0, 10)}-${evidenceTask.replace(/\s+/g, '-').toLowerCase().slice(0, 20)}-${String(Math.floor(Math.random() * 9999)).padStart(4, '0')}`,
153-
title: `${evidenceTask} ${inferenceJourneyStage} ${inferenceProblemFamily}`,
154-
summary: inferenceWhyCurrentPathFailed,
155-
created_at: new Date().toISOString(),
156-
tags: metadataTags,
157-
evidence: {
158-
task: evidenceTask,
159-
desired_outcome: evidenceDesiredOutcome,
160-
attempted_path: {
161-
tool: evidenceAttemptedTool,
162-
tool_type: evidenceToolType
163-
},
164-
symptom: evidenceSymptom,
165-
reproduction_steps: evidenceReproductionSteps.length > 0 ? evidenceReproductionSteps : undefined
166-
},
167-
inference: {
168-
journey_stage: inferenceJourneyStage,
169-
problem_family: inferenceProblemFamily,
170-
why_current_path_failed: inferenceWhyCurrentPathFailed,
171-
best_candidate_route_id: inferenceBestCandidateRouteId,
172-
best_candidate_route_detail: inferenceBestCandidateRouteDetail || undefined,
173-
prerequisites_for_switch: inferencePrerequisitesForSwitch.length > 0 ? inferencePrerequisitesForSwitch : undefined,
174-
confidence: inferenceConfidence
175-
},
176-
resolution: {
177-
outcome: resolutionOutcome,
178-
follow_up_notes: resolutionFollowUpNotes || undefined
179-
},
180-
source: metadataSource,
181-
verified: false,
182-
related_cases: []
183-
};
184-
185-
// Add optional fields only if they have content
186-
if (evidenceContext) caseData.evidence.context = evidenceContext;
187-
if (Object.keys(environment).length > 1) caseData.evidence.environment = environment;
188-
if (evidenceFailedStep) caseData.evidence.failed_step = evidenceFailedStep;
189-
if (evidenceArtifactsUsed.length > 0) caseData.evidence.artifacts_used = evidenceArtifactsUsed;
190-
191-
fs.writeFileSync('/tmp/case_from_issue.json', JSON.stringify(caseData, null, 2));
192-
core.setOutput('has_json', 'true');
193-
core.setOutput('assembled', 'true');
81+
run: |
82+
python3 scripts/parse_issue_form.py --body-file /tmp/issue_body.txt --output /tmp/case_from_issue.json
83+
echo "assembled=true" >> $GITHUB_OUTPUT
19484
85+
# Step 4: Validate the case (works for both paths)
19586
- name: Validate case
196-
if: steps.extract_json.outputs.has_json == 'true' || steps.assemble.outputs.has_json == 'true'
19787
id: validate
19888
run: |
19989
RESULT=$(python3 scripts/validate_case.py --input /tmp/case_from_issue.json 2>&1) || EXIT_CODE=$?
@@ -202,8 +92,8 @@ jobs:
20292
echo "EOF" >> $GITHUB_OUTPUT
20393
echo "exit_code=${EXIT_CODE:-0}" >> $GITHUB_OUTPUT
20494
95+
# Step 5: Post comment with results
20596
- name: Post validation comment
206-
if: steps.extract_json.outputs.has_json == 'true' || steps.assemble.outputs.has_json == 'true'
20797
uses: actions/github-script@v7
20898
with:
20999
script: |

README.md

Lines changed: 48 additions & 63 deletions
Original file line numberDiff line numberDiff line change
@@ -47,22 +47,17 @@ This is what AgentRX does: turns a stuck state into a structured next-step decis
4747

4848
**AgentRX is installed by humans, but operated by AI.**
4949

50-
Once installed, the human is not the primary user. The primary user is the AI agent itself:
51-
- it **diagnoses** stuck states
52-
- it **searches** the case library
53-
- it **chooses** a better route
54-
- it **contributes** new cases back
55-
56-
You don't run AgentRX. Your AI does.
50+
Once installed, the human is not the primary user. The primary runtime user is the AI agent itself.
51+
The default contributor is also AI.
5752

5853
### Human role vs AI role
5954

6055
| | What they do |
6156
|---|---|
62-
| **Human** | Install the skill. Expose the repository / index / rules. Optionally review schema or approve maintenance changes. |
63-
| **AI** | Detect stuck state. Structure evidence. Derive inference. Retrieve similar cases. Choose next action. Optionally contribute a new case. |
57+
| **Human** | Install the skill. Host the repository. Optionally review schema or approve maintenance changes. |
58+
| **AI** | Detect stuck state. Collect evidence. Derive inference. Retrieve similar cases. Choose a route. Optionally contribute a new case. |
6459

65-
**The human is the installer and host. The AI is the default operator.**
60+
**The human is the installer and host. The AI is the default operator and contributor.**
6661

6762
### The AI self-evolution loop
6863

@@ -86,7 +81,42 @@ Each case contributed by an AI agent makes the next agent smarter. The library g
8681
AgentRX diagnoses AI tool-chain failures and prescribes the next best action.
8782
It covers **skills, MCP servers, plugins, built-in tools, agents, workflows, and hooks**.
8883

89-
It is a **stuck-state navigation system** — the first responder when any part of your AI agent's tool path breaks down.
84+
It is a **stuck-state navigation system** — the first responder when any part of an AI agent's tool path breaks down.
85+
86+
## What it covers
87+
88+
This project covers failures and decision paths involving:
89+
90+
| Tool type | Examples |
91+
|---|---|
92+
| **skill** | xlsx, pdf, frontend-design, tavily |
93+
| **mcp** | Playwright MCP, Google Search MCP, Filesystem MCP |
94+
| **plugin** | Browser extensions, IDE plugins |
95+
| **builtin** | Claude's built-in web search, file reader |
96+
| **agent** | Multi-agent orchestration frameworks |
97+
| **workflow / hook** | Pre-commit hooks, deterministic pipelines |
98+
99+
This is **not** a universal benchmark site for all AI tools.
100+
It only enters the picture when an AI tool-path did **not** meet expectations and the agent needs help deciding what to do next.
101+
102+
## The v2.1 model
103+
104+
The system is built around a two-layer case structure:
105+
106+
```
107+
evidence (immutable facts) inference (AI-generated diagnosis)
108+
├── task ├── journey_stage
109+
├── desired_outcome ├── problem_family
110+
├── attempted_path ├── why_current_path_failed ← core field
111+
└── symptom └── best_candidate_route_id ← core field
112+
└── confidence
113+
```
114+
115+
**Evidence** is what the agent observes directly — surface symptoms, attempted tool paths, desired outcomes. These are immutable facts.
116+
117+
**Inference** is the agent's interpretation — where the blockage is, what problem family it fits, why the current path won't work, and which route to take next. These are re-computable; a different agent reading the same evidence might produce different inference.
118+
119+
**Route ids are action paths, not tool brands.** `switch_to_alternative_tool_path` is stable; `playwright-mcp` is not. Tools come and go; action patterns persist.
90120

91121
## Why this project changed
92122

@@ -102,40 +132,14 @@ But real failures usually begin from a messier place:
102132
- "I'm trying to browse a page and the content is incomplete."
103133
- "I generated a document, but the output is wrong."
104134
- "I can do this task with a skill, an MCP, a plugin, or a built-in tool — which one should I switch to?"
105-
- "I am not sure whether this is a routing problem, a config problem, an environment problem, or simply the wrong tool for the job."
106135

107136
So the project has been redesigned around a different principle:
108137

109-
> **Start from the task, then locate the stage, then classify the problem family, then choose the next action.**
138+
> **Start from evidence. Derive inference. Choose a route.**
110139
111140
This repository is no longer only about "skill governance."
112141
It is about **AI tool-path diagnosis and next-step recommendation**.
113142

114-
## What it covers
115-
116-
This project covers failures and decision paths involving:
117-
118-
| Tool type | Examples |
119-
|---|---|
120-
| **skill** | xlsx, pdf, frontend-design, tavily |
121-
| **mcp** | Playwright MCP, Google Search MCP, Filesystem MCP |
122-
| **plugin** | Browser extensions, IDE plugins |
123-
| **builtin** | Claude's built-in web search, file reader |
124-
| **agent** | Multi-agent orchestration frameworks |
125-
| **workflow / hook** | Pre-commit hooks, deterministic pipelines |
126-
127-
This is **not** a universal benchmark site for all AI tools.
128-
It only enters the picture when an AI tool-path did **not** meet expectations and the agent needs help deciding what to do next.
129-
130-
## Core positioning
131-
132-
AgentRX does four things:
133-
134-
1. **Intake** — force the agent to describe its own blockage in a structured way.
135-
2. **Navigate** — route the problem through a task-first knowledge architecture.
136-
3. **Recommend** — propose the most suitable next action, not just a label.
137-
4. **Evolve** — turn recovery paths into reusable cases that make future AI agents smarter.
138-
139143
## Why this is not a generic human-facing tool directory
140144

141145
Some projects catalog every AI tool and let humans browse them. AgentRX does not do that.
@@ -172,32 +176,14 @@ git clone https://github.com/LpcPaul/AgentRX.git ~/.codex/skills/agentrx
172176

173177
After you clone AgentRX into your skill directory, nothing changes on your end.
174178
Your AI agent gains a new capability it can activate when stuck.
175-
Over time, as cases are contributed, the library grows and retrieval quality improves.
176-
177-
The case library starts small and grows through accumulated AI experience:
178-
179-
| What you install today | What you get over time |
180-
|---|---|
181-
| 10 golden cases across 10 route types | Growing collection of real stuck-state diagnoses |
182-
| Structured schema + route registry | Richer context for AI-to-AI knowledge transfer |
183-
| Self-evolution loop ready | Future AI agents benefit from past recoveries |
184-
185-
---
186-
187-
## What changed from the old version
188179

189-
### Old model (v1 — Skill Doctor)
190-
- centered on `skill_triggered`
191-
- organized cases by `by-skill/` and `by-type/`
192-
- retrieved mostly by `skill_triggered + failure_type`
193-
- treated many failures as "skill failures"
180+
You don't manually operate this repository. The AI agent:
181+
- diagnoses its own stuck states
182+
- searches the case library for similar patterns
183+
- switches to a better route based on past cases
184+
- optionally contributes new cases back
194185

195-
### New model (v2.1 — AgentRX)
196-
- centered on `task + journey_stage + problem_family`
197-
- **evidence / inference split** — facts vs. interpretation
198-
- **standardized route ids** — action paths, not tool brands
199-
- recommends the **next action** rather than only classifying the cause
200-
- cases are **AI-contributed**, AI-consumed
186+
Over time, as cases are contributed by AI agents, the library grows and retrieval quality improves.
201187

202188
---
203189

@@ -208,7 +194,6 @@ The case library starts small and grows through accumulated AI experience:
208194
| [SKILL.md](SKILL.md) | The runtime prompt that the AI agent reads when activated |
209195
| [docs/ARCHITECTURE.md](docs/ARCHITECTURE.md) | System design — why AI-only, why evidence/inference, why route ids |
210196
| [docs/INTAKE_CARD.md](docs/INTAKE_CARD.md) | The structured format AI uses to translate stuck states into queries |
211-
| [docs/AI_SELF_EVOLUTION.md](docs/AI_SELF_EVOLUTION.md) | The self-evolution loop — how the library grows over time |
212197
| [CONTRIBUTING.md](CONTRIBUTING.md) | How cases enter the system — default contributor is AI |
213198
| [cases/README.md](cases/README.md) | Case library structure and indexing |
214199

cases/index.json

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
{
22
"schema_version": "2.1",
33
"project_name": "agentrx",
4-
"generated_at": "2026-04-16T13:59:16Z",
4+
"generated_at": "2026-04-16T14:15:48Z",
55
"navigation_order": [
66
"task",
77
"journey_stage",

docs/ARCHITECTURE.md

Lines changed: 10 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -9,8 +9,12 @@ It is a machine-to-machine knowledge layer that turns messy failures into struct
99

1010
## Audience
1111

12-
**Primary: AI agents** (runtime consumers and case contributors)
13-
**Human role: installers, repository hosts, and occasional maintainers**
12+
**Primary runtime consumer: AI agents**
13+
14+
The system is designed for AI agents to read, navigate, and contribute cases.
15+
The case library, schema, and route registry are all machine-consumable first.
16+
17+
**Human role: installer, repository host, and occasional maintainer**
1418

1519
This is not "primary user, secondary user." It is:
1620
- AI runs the system.
@@ -20,7 +24,8 @@ The repository may be discovered by humans, but the runtime consumer is AI.
2024

2125
### Why README still starts with human pain
2226

23-
Because installation is a human decision, but operation is not.
27+
Installation is a human decision; operation is not.
28+
2429
The README first screen serves humans deciding whether to install.
2530
The system itself serves AI agents after installation.
2631
These are two different audiences with two different purposes, and the narrative handles both without conflating them.
@@ -163,6 +168,7 @@ v2.0 flat cases are supported through:
163168
## Case Library Growth
164169

165170
The case library is designed to be expanded **primarily by AI-generated cases**.
171+
166172
Human review is optional governance, not the main growth mechanism.
167173

168174
When the system works as intended:
@@ -178,6 +184,7 @@ When the system works as intended:
178184
|---|---|
179185
| `validate_case.py` | Validate a case against schema + route registry |
180186
| `build_index.py` | Rebuild cases/index.json from all case files |
187+
| `parse_issue_form.py` | Assemble v2.1 case JSON from GitHub issue form fields |
181188
| `ci_self_test.py` | Run all validation checks |
182189

183190
## Design Constraint

0 commit comments

Comments
 (0)