Skip to content

Commit cc12da7

Browse files
author
baiqing
committed
feat: add self-calibration closed loop for navigation demo
Implement online self-calibration system that forms a local closed loop within the navigation demo. The system learns from its own constitutional evaluation results and improves decision quality within and across sessions. New modules: - calibrator.py: Session-level self-calibrator with dynamic threshold, weight, and safety hint adjustment based on violation patterns - learning_store.py: Cross-session JSON persistence for calibration baselines, enabling progressive improvement across runs - prompt_calibrator.py: Translates violation patterns into natural language hints injected into VLM prompts Modified: - runner.py: Integrates calibrator into the propose→validate→feedback loop with threshold sync and session finalization - providers.py: Adds prompt_suffix parameter for calibrated hint injection - config.py: Adds calibration config fields (enable_calibration, calibration_interval, violation_count_trigger, learning_store_path) Tests: 54 new tests (40 unit + 14 integration) covering calibration logic, cross-session learning, prompt generation, and full closed-loop flow verification.
1 parent 1a5a27c commit cc12da7

13 files changed

Lines changed: 2087 additions & 11 deletions

README.md

Lines changed: 69 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -375,6 +375,7 @@ While autonomous driving is our primary demonstration, A-YLM's geometric constit
375375
| 7 | Motion Estimation | Tracked Positions | Velocity / Heading | Dynamic Prediction |
376376
| 8 | Constitution Evaluation | Scene + Decision | Safety Score + Violations | Geometric Safety Verification |
377377
| 9 | Feedback Generation | Validation Results | Training Signals | Self-Evolution |
378+
| 10 | Self-Calibration | Training Signals + Trends | Adjusted Threshold/Weights/Hints | Online Closed-Loop Improvement |
378379

379380
---
380381

@@ -623,6 +624,68 @@ proposes a short-horizon command, validates it through A-YLM's `CommandValidator
623624
falls back to `alternative_decision` or a safe stop when needed, and can render an
624625
annotated result video plus JSONL decision logs.
625626

627+
#### Self-Calibration Closed Loop
628+
629+
The navigation demo includes an **online self-calibration system** that forms a local closed loop — the system learns from its own evaluation results and improves within a session and across sessions:
630+
631+
```
632+
┌─────────────────────────────────────────────────────────────────────────────┐
633+
│ Self-Calibration Closed Loop │
634+
├─────────────────────────────────────────────────────────────────────────────┤
635+
│ │
636+
│ Frame N: │
637+
│ ┌──────────┐ ┌──────────┐ ┌──────────┐ ┌──────────────────┐ │
638+
│ │ Proposer │───▶│Validator │───▶│ Training │───▶│ SessionCalibrator│ │
639+
│ │(+safety │ │ │ │ Signal │ │ .record_frame() │ │
640+
│ │ hints) │ │ │ │ │ │ │ │
641+
│ └──────────┘ └──────────┘ └──────────┘ └────────┬─────────┘ │
642+
│ ▲ │ │
643+
│ │ Calibration Context │ │
644+
│ │ ┌─────────────────────────────────────────┘ │
645+
│ │ │ │
646+
│ │ ┌─────▼──────┐ ┌────────────┐ ┌──────────────┐ │
647+
│ │ │ Adjust │ │ Boost │ │ Generate │ │
648+
│ │ │ Threshold │ │ Weights │ │ Safety │ │
649+
│ │ │ (±0.05) │ │ (up to 2x)│ │ Hints │ │
650+
│ │ └────────────┘ └────────────┘ └──────┬───────┘ │
651+
│ │ │ │
652+
│ └───────────────── Frame N+1 ◀──────────────┘ │
653+
│ │
654+
│ Session End → LearningStore (JSON) → Next Run Starts from Best Baseline │
655+
│ │
656+
└─────────────────────────────────────────────────────────────────────────────┘
657+
```
658+
659+
**Key behaviors:**
660+
- **Within a session**: Every 10 frames or 5 violations, the calibrator recalibrates — tightening approval threshold on declining safety trends, boosting scorer weights for frequently violated principles, and injecting safety hints into VLM prompts
661+
- **Across sessions**: `LearningStore` persists the best-performing calibration configuration to a JSON file; the next run automatically loads and starts from this improved baseline
662+
- **Prompt injection**: Violation patterns are translated into natural language hints appended to VLM prompts (e.g., "Principle 'no_collision' violated 7 times — increase caution")
663+
664+
Enable calibration (on by default):
665+
```bash
666+
# With cross-session learning persistence
667+
aylm nav-demo -i outputs/video_demo -o outputs/nav_demo \
668+
--provider mlx-vlm \
669+
--learning-store ~/.aylm/learning_store.json
670+
671+
# Disable calibration
672+
aylm nav-demo -i outputs/video_demo -o outputs/nav_demo --no-calibration
673+
```
674+
675+
Calibration results are included in `run_summary.json`:
676+
```json
677+
{
678+
"calibration": {
679+
"rounds": 6,
680+
"final_threshold": 0.700,
681+
"final_weights": {"collision": 1.749, "ttc": 0.8, "boundary": 0.5},
682+
"active_hints": ["Principle 'no_collision' violated 7 times — increase caution"],
683+
"violation_summary": "no_collision: 7 violations (35%)",
684+
"frames_analyzed": 20
685+
}
686+
}
687+
```
688+
626689
### 6.3.2 MLX-VLM Training Signal Export (Optional Module)
627690

628691
Training signal export is packaged as a **separate optional module** so the main
@@ -1095,7 +1158,10 @@ A.YLM/
10951158
│ │ ├── artifacts.py # Artifact pairing + scene summarization
10961159
│ │ ├── providers.py # Heuristic / MLX-VLM command proposers
10971160
│ │ ├── overlay.py # Annotated result video renderer
1098-
│ │ └── runner.py # End-to-end offline demo orchestration
1161+
│ │ ├── runner.py # End-to-end offline demo orchestration
1162+
│ │ ├── calibrator.py # Session-level self-calibrator (threshold/weight/hint adjustment)
1163+
│ │ ├── learning_store.py # Cross-session learning persistence (JSON)
1164+
│ │ └── prompt_calibrator.py # Violation-to-prompt safety hint translator
10991165
│ ├── constitution/ # Constitutional AI Module
11001166
│ │ ├── base.py # ConstitutionPrinciple abstract base
11011167
│ │ ├── evaluator.py # ConstitutionEvaluator orchestrator
@@ -1148,7 +1214,7 @@ A.YLM/
11481214
│ ├── voxelized/ # Occupancy Grids + Obstacle JSONs
11491215
│ ├── detections/ # Detection Results
11501216
│ └── video_output/ # Video Pipeline Output
1151-
├── tests/ # Test Suite (382+ tests)
1217+
├── tests/ # Test Suite (420+ tests)
11521218
├── run.sh # One-Click Execution Script
11531219
└── pyproject.toml # Project Configuration
11541220
```
@@ -1171,7 +1237,7 @@ Current policy:
11711237
## 10. Future Work
11721238

11731239
### 10.1 Advanced Safety Learning
1174-
Implementation of reinforcement learning for adaptive safety boundary optimization.
1240+
Online self-calibration with threshold/weight/hint adjustment is implemented. Future work includes full reinforcement learning for adaptive safety boundary optimization with model fine-tuning.
11751241

11761242
### 10.2 Multi-Sensor Fusion
11771243
Integration with LiDAR for enhanced geometric accuracy in safety-critical scenarios.

src/aylm/navigation_demo/__init__.py

Lines changed: 20 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -9,28 +9,45 @@
99
from typing import TYPE_CHECKING
1010

1111
if TYPE_CHECKING:
12+
from .calibrator import CalibrationContext, FrameFeedback, SessionCalibrator
1213
from .config import DEFAULT_MLX_MODEL, NavigationDemoConfig
14+
from .learning_store import BaselineSnapshot, LearningStore, SessionRecord
15+
from .prompt_calibrator import PromptCalibrator, PromptPatch
1316
from .runner import NavigationDemoRunner
1417

1518
__all__ = [
19+
"BaselineSnapshot",
20+
"CalibrationContext",
1621
"DEFAULT_MLX_MODEL",
22+
"FrameFeedback",
23+
"LearningStore",
1724
"NavigationDemoConfig",
1825
"NavigationDemoRunner",
26+
"PromptCalibrator",
27+
"PromptPatch",
28+
"SessionCalibrator",
29+
"SessionRecord",
1930
]
2031

2132
_LAZY_ATTRS = {
33+
"BaselineSnapshot": "aylm.navigation_demo.learning_store",
34+
"CalibrationContext": "aylm.navigation_demo.calibrator",
2235
"DEFAULT_MLX_MODEL": "aylm.navigation_demo.config",
36+
"FrameFeedback": "aylm.navigation_demo.calibrator",
37+
"LearningStore": "aylm.navigation_demo.learning_store",
2338
"NavigationDemoConfig": "aylm.navigation_demo.config",
2439
"NavigationDemoRunner": "aylm.navigation_demo.runner",
40+
"PromptCalibrator": "aylm.navigation_demo.prompt_calibrator",
41+
"PromptPatch": "aylm.navigation_demo.prompt_calibrator",
42+
"SessionCalibrator": "aylm.navigation_demo.calibrator",
43+
"SessionRecord": "aylm.navigation_demo.learning_store",
2544
}
2645

2746

2847
def __getattr__(name: str):
2948
module_path = _LAZY_ATTRS.get(name)
3049
if module_path is None:
31-
raise AttributeError(
32-
f"module 'aylm.navigation_demo' has no attribute {name!r}"
33-
)
50+
raise AttributeError(f"module 'aylm.navigation_demo' has no attribute {name!r}")
3451
module = import_module(module_path)
3552
value = getattr(module, name)
3653
globals()[name] = value

0 commit comments

Comments
 (0)