docs/frontend architecture

.gitignore

@@ -4,3 +4,7 @@ __pycache__/
 *.py[cod]
 data/
 .direnv/
+.fake-backend-config.json
+
+# OS files
+.DS_Store

AGENTS.md

@@ -0,0 +1,9 @@
+This is a project designing the compute platform for UTMIST (University of Toronto Machine Intelligence Student Team). The target cluster is two computers, one with tenstorrent gpu and one with nvidia gpus.
+
+## Documentation
+
+Agents are allowed to edit the README and top level markdown files as well as docs/agents, but shouldn't touch docs/humans as those are by humans for humans (though agents may notify humans if they believe human docs are severely outdated).
+
+## Temporary Decisions
+
+Make the frontend lightweight and move logic to backend. We aren't short on bandwidth, so it is fine to send a new qpi request and refresh UI every once in a while.

README.md

@@ -4,15 +4,17 @@ UTMIST's research compute platform. Submit Python scripts or tagged container
 images, choose CPU, NVIDIA GPUs or Tenstorrent boards, upload datasets, and
 collect persistent results through the same website or CLI.
 
-**Private website:** http://100.73.139.66:8088, accessible through authorized
+**Private website:** [http://100.73.139.66:8088](http://100.73.139.66:8088), accessible through authorized
 Tailscale connectivity. A Mist account and team membership are also required.
 
 ## Current installation
 
-| Machine | Role | LAN address | Tailscale address |
-|---|---|---|---|
-| `utmist-z1opa08` | k3s server, API/auth, website, CPU and two RTX A4000 GPUs | `10.0.0.175` | `100.73.139.66` |
-| `utmist-tt` | k3s worker, four n300 boards/eight chips, shared storage | `10.0.0.112` | `100.95.175.37` |
+
+| Machine          | Role                                                      | LAN address  | Tailscale address |
+| ---------------- | --------------------------------------------------------- | ------------ | ----------------- |
+| `utmist-z1opa08` | k3s server, API/auth, website, CPU and two RTX A4000 GPUs | `10.0.0.175` | `100.73.139.66`   |
+| `utmist-tt`      | k3s worker, four n300 boards/eight chips, shared storage  | `10.0.0.112` | `100.95.175.37`   |
+
 
 The cluster and NFS use the existing LAN. Tailscale provides private portal and
 administrator SSH access. Kubernetes/containerd run workload containers.
@@ -60,3 +62,27 @@ chaining, browser image archives/builds, private pull-credential management and
 distributed multi-machine training are outside this release. Automatic backups,
 HTTPS and monitoring/cleanup products were explicitly excluded. This is a private
 HTTP installation with one control plane, one auth instance and one storage host.
+
+### Documentation
+
+Humans are responsible for editing the human facing docs. This is because the human facing docs will be read by humans, and it is rude to give them untested instructions to follow.
+
+Documentation should include explanations of design decisions for later platform maintainers.
+
+## Frontend Testing
+
+how to use fake backend for testing frontend.
+
+(temporary, functionality, to be removed in future commit.)
+
+```bash
+# Terminal 1
+npm --prefix web-interface run dev -- --host 127.0.0.1 --port 3001 --strictPort
+
+# Terminal 2
+npm --prefix web-interface run fake-backend
+```
+
+
+
+ 

docs/README.md

@@ -18,6 +18,7 @@ host-specific settings when provisioning another environment.
 | Authenticate and submit from a terminal | [CLI](../cli/docs/setup.md) |
 | Run automated checks and read recorded live evidence | [Testing](testing.md) |
 | Maintain UI components and interactions | [Design system](design-system.md) |
+| Understand frontend architecture and backend offloading | [Frontend architecture](frontend-architecture.md) |
 | Read completed scope, exclusions and release verification | [Execution record](department-rollout-execution.md) |
 | See what repository cleanup removed and verified | [Cleanup record](repository-cleanup.md) |
 

docs/frontend-architecture.md

@@ -0,0 +1,125 @@
+# Frontend Architecture & Backend Offloading
+
+Documentation of the current Mist frontend ([`web-interface`](../web-interface)), its structural organization, and a roadmap of client-side processing to offload to the backend.
+
+## Architecture overview
+
+```mermaid
+flowchart TD
+    User([User Browser]) --> Shell[Root Shell: __root.tsx]
+    Shell --> AG[AccountGate: auth.tsx]
+    AG --> TG[TeamGate: teams.tsx]
+    TG --> Nav[Navbar.tsx]
+    TG --> RouterOutlet[TanStack Router Outlet]
+
+    RouterOutlet --> DPage[Dashboard: dashboard.tsx]
+    RouterOutlet --> JPage[Jobs Page: jobs.tsx]
+    RouterOutlet --> SPage[Datasets Page: datasets.tsx]
+    RouterOutlet --> MPage[Machines Page: machines.tsx]
+    RouterOutlet --> TPage[Teams Page: teams.tsx]
+    RouterOutlet --> PPage[Profile: profile.tsx]
+
+    JPage --> JSF[JobSubmissionForm.tsx]
+    JPage --> JC[JobCard.tsx]
+    JC --> JL[JobLogs.tsx]
+    JC --> JF[JobFiles.tsx]
+
+    SPage --> TSB[TeamStorageBrowser.tsx]
+```
+
+### Technology stack
+
+- **Vite 7.3**: Build tool and development proxy (`/api` and `/auth` forwards).
+- **React 19.3**: Functional component architecture with custom hooks.
+- **TanStack Router**: File-based routing located in `src/routes/` with code-splitting.
+- **Tailwind CSS 4**: Modern styling via `@tailwindcss/vite` and customized Mist theme classes.
+- **Better Auth Client**: Authentication state and admin member management.
+- **Vitest 5 & Testing Library**: Unit and component tests.
+
+### Source layout
+
+| Location | Purpose |
+|---|---|
+| `src/main.tsx`, `router.tsx` | Router initialization and DOM hydration |
+| `src/routes/__root.tsx` | Shell wrapper containing `AccountGate`, `TeamGate`, and `Navbar` |
+| `src/routes/` | File-based routes (`dashboard.tsx`, `jobs.tsx`, `datasets.tsx`, `machines.tsx`, `teams.tsx`, `profile.tsx`) |
+| `src/auth.tsx` | Better Auth client configuration, `AccountContext`, and `AccountGate` |
+| `src/teams.tsx` | `TeamContext`, active team selection, workspace selector |
+| `src/api.ts` | Typed HTTP client for jobs, datasets, hardware, and teams |
+| `src/hooks/usePolling.ts` | In-flight request deduplication and abortable periodic polling |
+| `src/components/jobs/` | `JobSubmissionForm`, `JobCard`, `JobLogs`, `JobFiles` |
+| `src/components/` | `HardwarePanel`, `TeamStorageBrowser`, `Modal`, `Picker`, `Tabs`, `Card`, `Buttons` |
+
+---
+
+## Backend offloading plan
+
+To keep the frontend as lightweight and simple as possible, the following responsibilities currently performed on the client should be handled by the backend:
+
+1. **Pagination and filtering**:
+   - `JobsPage`: Replace client-side slicing and array filtering with query parameters on `GET /jobs?status=...&q=...&page=...&limit=...`.
+   - `DatasetsPage`: Add `GET /datasets?q=...`.
+   - `AccountPage`: Replace Better Auth `listUsers({ limit: 1000 })` with a paginated endpoint `GET /members?page=...&q=...`.
+2. **Metrics aggregation**:
+   - Provide `GET /metrics/summary` or summary counters in list endpoints to avoid transferring full item arrays just to compute badge totals.
+3. **Image catalog and validation**:
+   - Move registry string parsing and image approval checks from `JobSubmissionForm` to server-side submission validation.
+4. **Quota and limit calculation**:
+   - Replace client-side calculation of `maxDevices` and `maxTimeout` with authoritative constraints provided by the team policy API.
+5. **Form parsing**:
+   - Accept raw multiline environment variable text and shell commands on `POST /jobs`, delegating parsing and validation to the server.
+6. **Error status handling**:
+   - Replace English error string matching (e.g. `isPreparing`) with typed machine-readable status fields (`{ status: "provisioning" }`).
+7. **Permission flags**:
+   - Return explicit capability booleans (`can_cancel`, `can_delete`, `can_manage`) on entities rather than parsing role strings on the client.
+8. **Atomic operations**:
+   - Consolidate multi-call client sequences (e.g. team creation + creator enrollment, password reset + session revocation) into atomic backend operations.
+9. **Event-driven updates**:
+   - Replace high-frequency polling with Server-Sent Events (SSE) or WebSockets for live job status and log streaming.
+
+---
+
+## Testing with the fake backend
+
+A self-contained testing backend is provided in `scripts/fake-backend.mjs`. It simulates all Mist API endpoints and Better Auth handlers in memory.
+
+### Running locally
+
+```bash
+# Terminal 1: Start fake backend (default port 3000)
+npm --prefix web-interface run fake-backend
+```
+
+```bash
+# Terminal 2: Start frontend dev server (port 3001)
+npm --prefix web-interface run dev -- --host 127.0.0.1 --port 3001 --strictPort
+```
+
+Open http://127.0.0.1:3001 in your browser. Pre-seeded users include `ada@utmist.gitlab.io` (admin), `alan@utmist.gitlab.io`, and `grace@utmist.gitlab.io`.
+
+### Configuring hypothetical scenarios and fake information
+
+A companion configure tool is provided at `scripts/configure-fake-backend.mjs` to set up hypothetical cluster topologies, error conditions, and user setups:
+
+```bash
+# Interactive menu (choose scenarios, add fake nodes, switch active user)
+npm --prefix web-interface run configure-fake-backend
+
+# Switch to a preset scenario directly
+node scripts/configure-fake-backend.mjs --scenario high-load
+node scripts/configure-fake-backend.mjs --scenario unhealthy
+node scripts/configure-fake-backend.mjs --scenario empty-slate
+node scripts/configure-fake-backend.mjs --scenario large-cluster
+node scripts/configure-fake-backend.mjs --scenario default
+
+# Inspect current cluster topology & status
+node scripts/configure-fake-backend.mjs --status
+
+# Add custom fake machines, users, or teams
+node scripts/configure-fake-backend.mjs --add-machine name=dgx-custom,ip=10.99.1.5,cpu=64,ram=256Gi,gpu=8
+node scripts/configure-fake-backend.mjs --add-user name="Marie Curie",email=curie@rad.edu,role=admin
+node scripts/configure-fake-backend.mjs --set-user curie@rad.edu
+```
+
+If the fake backend is running, `configure-fake-backend.mjs` updates the live server immediately without requiring a restart, while also saving to `.fake-backend-config.json` for persistence.
+