Skip to content

Commit 7b22a1e

Browse files
wong-ziyiwong-ziyi
committed
docs: align cognitive assets with SkillOpt loop and update roadmap with milestones
1 parent e899c06 commit 7b22a1e

4 files changed

Lines changed: 94 additions & 0 deletions

File tree

‎.wiki/system/lessons-learned.md‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -48,3 +48,8 @@ This page serves as a historical repository of significant bugs, architectural m
4848
**Context**: Implementing tests for the Stripe Webhook handler and LLM Gateway billing telemetry in the AROS Cloud Federation project.
4949
**The Bug**: `TestClient` dependency overrides (`app.dependency_overrides[get_db]`) correctly patched `SessionLocal` for HTTP endpoints. However, `background_tasks.add_task()` in the LLM Gateway and the standalone webhook handler imported `SessionLocal` directly at the module or function level, bypassing the FastAPI dependency injection. Consequently, tests interacted with the development database instead of the in-memory SQLite test database, leading to assertion failures (`assert 'active' == 'cancelled'`).
5050
**The Fix**: In `conftest.py`, `SessionLocal` must be explicitly patched via `unittest.mock.patch.object` on the specific router modules (`webhooks_router` and `auth_middleware`) AND the root `aros_cloud.db.base` to intercept lazy imports inside background functions.
51+
52+
## 10. SkillOpt Evolution Pipeline Alignment (June 2026)
53+
**Context**: Cognitive assets (skills, policies, workflows, KIs) are optimized autonomously by the local SkillOpt execution loop, leading to divergence between the active runtime and the Factory source tree.
54+
**The Bug**: Manual pulling of mutated files was error-prone and caused merge conflicts, or overwrote human-curated improvements in the Factory.
55+
**The Fix**: Codified the **SkillOpt Evolution Alignment** specification. Mutations accepted by the local validation gate are pulled back to the Factory via `aros-sync pull` and checked into dedicated evolution branches (`evolution/skillopt-*`), preventing direct main branch contamination.

‎README.md‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -326,3 +326,5 @@ This repository is a **Factory for forging AROS assets**. Before modifying any S
326326
| `/visualize-data` | Data_Analysis_Pipeline | Autonomous diagram generation |
327327
| `/research-discovery` | workspace_management | Research planning & brainstorm |
328328
| `/qa-system-audit` | workspace_management | AROS QA health checks |
329+
| `/skillopt-trigger` | workspace_management | Trigger SkillOpt training on-demand |
330+

‎SPEC.md‎

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -86,3 +86,10 @@ User comments generated in the Global Context Hub UI are automatically triaged b
8686

8787
### 5.4 AI Agent Package Synthesis
8888
The `agent_package_builder.py` script automatically synthesizes structured agent definitions (`agent.yaml`, `input_schema.json`, `output_schema.json`, and `system.md`) for all pipelines using the `gemini-3.1-pro-preview` model. This process ensures scalable deployment to the AROS Cloud Federation with full cross-platform compatibility by safely generating and distributing agent context payloads without exposing raw scripts directly.
89+
90+
## 6. SkillOpt Evolution Alignment
91+
92+
All cognitive assets (Skills, KIs, Policies, Workflows) forged within the Pipeline Factory are designed to serve as the baseline states for the **SkillOpt (arXiv:2605.23904)** optimization loop.
93+
1. **Promoted Mutation Synchronization**: Mutations accepted by the local validation gate MUST be periodically pulled back to the Factory source tree using the `aros-sync pull` command to preserve optimization updates.
94+
2. **Version Control Integration**: Pulled changes MUST be checked in under dedicated evolution branches (`evolution/skillopt-*`) for review before merge into main repository branches.
95+

‎docs/SKILLOPT_ROADMAP.md‎

Lines changed: 80 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,80 @@
1+
# SkillOpt Integration Roadmap — AROS Pipeline Factory
2+
3+
> **Reference**: [SkillOpt: Optimizing LLM System Prompts via Text-Space Optimization](https://arxiv.org/abs/2605.23904)
4+
> **Owner**: AROS_Pipeline_Factory
5+
> **Status**: Roadmap Defined (Not Yet Started)
6+
7+
---
8+
9+
## Overview
10+
11+
AROS Pipeline Factory creates domain-specific scientific pipelines (e.g., Visium,
12+
scRNA-seq, KAKENHI). SkillOpt integration enables automatic optimization of the
13+
pipeline templates, domain skills, and configuration recipes based on actual
14+
execution feedback from researchers.
15+
16+
---
17+
18+
## Milestones
19+
20+
### 🔲 Milestone PF.1: Pipeline Template Optimization
21+
**Goal**: Optimize pipeline templates (SKILL.md files) using SkillOpt.
22+
23+
Pipeline templates in `templates/` are structured as Skills. SkillOpt can:
24+
- Analyze past pipeline execution traces (walkthroughs)
25+
- Identify frequently failing steps (e.g., parameter selection, tool versions)
26+
- Generate bounded edits to template instructions
27+
- Validate improvements against historical success rates
28+
29+
### 🔲 Milestone PF.2: Domain-Specific Skill Refinement
30+
**Goal**: Use SkillOpt to refine domain-specific skills generated by the factory.
31+
32+
When a pipeline factory creates a new skill (e.g., `spaceranger-skill`), SkillOpt:
33+
1. Monitors execution traces from users of that skill
34+
2. Identifies recurring errors or suboptimal patterns
35+
3. Proposes edits via the AnalystError → MergeReflection → EditGenerator chain
36+
4. Validates on holdout trajectory data
37+
38+
### 🔲 Milestone PF.3: Configuration Recipe Evolution
39+
**Goal**: Evolve default configuration recipes based on user feedback.
40+
41+
- Parameter defaults that lead to failures are identified
42+
- SkillOpt generates improved default values
43+
- Validation uses the same gate_metric framework
44+
45+
### 🔲 Milestone PF.4: Cross-Pipeline Knowledge Transfer
46+
**Goal**: Transfer optimization insights across related pipelines.
47+
48+
- When SkillOpt optimizes a Visium pipeline skill, check if similar patterns
49+
apply to other spatial transcriptomics pipelines
50+
- Use the Meta-Skill cross-epoch generalization to propagate improvements
51+
- Feed insights back into the Knowledge Distiller for KI generation
52+
53+
---
54+
55+
## Integration Architecture
56+
57+
```
58+
AROS Pipeline Factory
59+
├── templates/ ← SkillOpt can optimize these
60+
│ ├── visium/SKILL.md
61+
│ └── scrnaseq/SKILL.md
62+
├── generated_skills/ ← SkillOpt monitors and refines
63+
│ └── spaceranger/SKILL.md
64+
└── configs/ ← SkillOpt evolves defaults
65+
└── default_params.yaml
66+
↕
67+
AROS Core (antigravity-evolution)
68+
└── SkillOpt Engine
69+
├── TrajectoryDataset (reads walkthroughs)
70+
└── CascadingPriorityScheduler
71+
```
72+
73+
## Dependencies
74+
75+
| Dependency | Status | Notes |
76+
|------------|--------|-------|
77+
| AROS SkillOpt Engine | ✅ Implemented | Core training loop |
78+
| brain.db Schema | ✅ Migrated | SkillOpt tables present |
79+
| Walkthrough Collection | ✅ Active | Walkthroughs from pipeline executions |
80+
| Pipeline Template Format | ✅ Standardized | SKILL.md format |

0 commit comments

Comments
 (0)