You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Phase 21 documentation on global orchestration and predictive scheduling.
41
+
18.**[Strategy Roadmap](docs/ROADMAP.md)**
40
42
The project roadmap detailing completed and future phases.
41
43
42
44
---
@@ -84,19 +86,28 @@ When running with the interposer or Python API, the Fabric provides real-time te
84
86
85
87
## 🤖 Adaptive Learning (Phase 20)
86
88
87
-
Starting with Phase 20, the Ternary Fabric is a self-tuning co-processor. It automatically optimizes its own internal parameters based on measured performance:
89
+
Starting with Phase 20, the Ternary Fabric is a self-tuning co-processor. It automatically optimizes its own internal parameters based on measured performance.
88
90
89
-
### Self-Tuning Cost Model
90
-
The scheduler uses a **Hill-Climbing** algorithm to adjust its cost projection coefficients. If the actual `fabric_cost` differs from the `projected_cost`, the fabric updates its internal model to ensure more accurate tile selection in the future.
91
+
## 🌐 Multi-Fabric Orchestration (Phase 21)
91
92
92
-
### Dynamic Scheduler Weighting
93
-
Tiles "learn" which kernels they are most efficient at executing. A tile that consistently delivers higher **Economic Efficiency** for a specific kernel (e.g., LSTM) will be favored for that kernel in future scheduling decisions.
93
+
Phase 21 elevates the TFMBS from a single adaptive co-processor to a **multi-fabric orchestration layer**. It provides proactive, system-level efficiency management:
94
94
95
-
### Eviction Policy Optimization
96
-
The eviction policy is no longer fixed LRU. It now dynamically weights **Frequency**, **Recency (Age)**, and **Residency Success Rate**. This ensures that weights critical to maintaining high efficiency are protected from eviction.
95
+
### Global Orchestration
96
+
Workloads are dynamically distributed across multiple isolated fabric instances. The system tracks buffer residency across all fabrics and automatically manages inter-fabric data movement (transfers) to minimize latency.
97
97
98
-
### Temporal Auto-Tuning
99
-
The asynchronous command queue automatically adjusts its **Batch Size** (between 1 and 32) to maximize a composite score of efficiency and throughput. It occasionally "explores" different batch sizes to find the optimal throughput for the current sparsity regime.
98
+
### Predictive Scheduling (Lookahead)
99
+
The orchestrator uses a **lookahead window of 5 kernels** to anticipate future task requirements. It selects the optimal fabric for the current task by considering where the weights will be needed next, effectively implementing **hot-state anticipation**.
100
+
101
+
### Cross-Fabric Fusion
102
+
The scheduler identifies dependent task sequences (e.g., GEMV output feeding an LSTM gate update) and prioritizes keeping them on the same fabric. This virtual "macro-kernel" approach reduces repeated hydration and inter-fabric communication.
103
+
104
+
### Adaptive Multi-Stage Pipeline
105
+
Each fabric manages a three-stage asynchronous pipeline:
106
+
1.**Pre-fetch:** Handles buffer hydration, PT-5 packing, and inter-fabric transfers.
107
+
2.**Execute:** Performs the native ternary kernel computation.
108
+
3.**Commit:** Finalizes results and signals task completion.
109
+
110
+
The pipeline depth automatically adjusts based on workload density—extending for throughput on dense kernels and shortening for low latency on sparse workloads.
This document details the architecture and implementation of the Phase 21 orchestration layer for the Ternary Fabric.
4
+
5
+
## 1. Overview
6
+
Phase 21 transitions the TFMBS from a single-device accelerator to a system-level orchestrated platform. It introduces a central Global Orchestrator that manages workload distribution across multiple Fabric instances.
7
+
8
+
## 2. Global Orchestrator
9
+
The Orchestrator resides in `src/libtfmbs_device.c` and manages a staging queue for all incoming tasks. It runs in a dedicated background thread.
10
+
11
+
### Residency Tracking
12
+
The system maintains a global residency map (`g_residency_map`) that tracks which fabric instance currently holds a valid PT-5 representation of each memory buffer.
13
+
14
+
### Dispatch Heuristics
15
+
1.**Locality First:** If a buffer required by a task is already resident on a specific fabric, that fabric is prioritized.
16
+
2.**Load Balancing:** If residency is not established, tasks are distributed using a round-robin or least-loaded strategy.
17
+
3.**Automated Transfer:** If the orchestrator decides to move a task to a different fabric than where its data resides, it automatically inserts `KERNEL_TRANSFER` operations into the pipeline.
18
+
19
+
## 3. Predictive Scheduling (Lookahead)
20
+
The orchestrator maintains a **Lookahead Window of 5 kernels**. By inspecting future tasks, the scheduler can:
21
+
-**Anticipate Hot-State:** Pre-load weights onto a fabric that will be used by multiple upcoming tasks.
22
+
-**Minimize Transfers:** Avoid moving a buffer if a future task on the same fabric will need it.
23
+
24
+
## 4. Cross-Fabric Kernel Fusion
25
+
The scheduler detects dependencies between kernels (e.g., Task B uses the output of Task A). It prioritizes keeping these tasks on the same fabric to form a "virtual macro-kernel," eliminating the need for expensive inter-fabric memory movement.
26
+
27
+
## 5. Multi-Stage Pipeline
28
+
Each Fabric instance operates a three-stage pipeline:
29
+
30
+
| Stage | Action |
31
+
| :--- | :--- |
32
+
|**Pre-fetch**| Hydrates RAW buffers, packs them into PT-5, or performs inter-fabric copies. |
33
+
|**Execute**| Runs the native kernel (GEMV, LSTM, etc.) on the ternary tiles. |
34
+
|**Commit**| Finalizes the result buffer and signals the host handle. |
35
+
36
+
### Adaptive Pipeline Depth
37
+
The pipeline depth (`pipeline_depth`) dynamically scales:
38
+
-**Depth=3 (Full):** For high-density, compute-heavy workloads to maximize throughput.
39
+
-**Depth=1 (Short):** For sparse or low-latency workloads to minimize completion time.
40
+
41
+
## 6. Telemetry & Metrics
42
+
`economic_metrics.csv` has been extended to include:
43
+
-`fid`: The Fabric ID.
44
+
-`projected_cost`: The predicted cost used for scheduling.
45
+
-`efficiency`: The measured economic efficiency for that instance.
46
+
47
+
## 7. Configuration
48
+
The number of fabrics can be configured at runtime:
0 commit comments