Experiment: named core memories and bounded recall

docs/harness-sdk.md

@@ -334,9 +334,9 @@ Public methods on `list`, `dict`, `str`, and `tuple` are available, including
 You can also construct these types and assign or delete list/dictionary items:
 
 ```python
-result = query_data(sql='SELECT status, COUNT(*) AS count FROM resources.workorder GROUP BY status')
+result = {'rows': [{'count': 2}, {'count': 3}]}
 counts = []
-for row in result['data'].get('rows', []):
+for row in result.get('rows', []):
     counts.append(row['count'])
 summary = {}
 summary['total'] = sum(counts)
@@ -655,6 +655,86 @@ agent = AgentHarness(
 Memory scope is an application-defined string. A production storage
 implementation is responsible for enforcing tenant and user isolation.
 
+### Named memories: a small working library, not an append-only log
+
+Use short stable names (1–64 lowercase letters, digits, hyphens, or underscores),
+such as `report-checklist`. The existing three tools manage the whole lifecycle:
+
+```json
+{"name": "report-checklist", "scope": "user_project", "text": "Verified report preparation steps..."}
+```
+
+Pass this to `remember`. Calling `remember` again with the **same name and scope**
+replaces that memory's text. Its internal ID and creation time remain stable;
+`updated_datetime` records the latest change. The result includes `operation`:
+`created`, `updated`, or `unchanged`. Identical content is a no-op: no storage write
+or memory-write event. Distinct names are distinct records; naming alone does not
+deduplicate semantically equivalent recipes with different names.
+
+- `recall(name="report-checklist", scope="user_project")` reads that exact key
+  without searching or listing every memory.
+- `recall(query="report checklist", limit=3)` searches relevant names and text.
+  The default limit is **5**, with a maximum of 50. Empty query lists recent scoped
+  memories. The in-memory implementation ranks lexical term overlap, not embeddings.
+- `forget(name="report-checklist", scope="user_project")` deletes that key.
+
+`remember` and `forget` require `name`; there are no ID-based tool inputs or
+anonymous append-only writes. Drop prototype memory data before adopting this
+format: no legacy compatibility or migration is provided. Writes accept up to
+8,000 characters; scope/name/namespace boundaries are checked before correcting
+or deleting a record.
+
+Stable keys use the application's `account_id`, `user_id`, and `project_id` in
+`execution_context` or persisted conversation metadata. `user` scope excludes
+project identity; `project` scope excludes user identity; `user_project` includes
+both. Other application-defined scopes include both. Applications can additionally
+set a stable `memory_namespace` in execution context. Do not use a conversation ID
+as a namespace if memories should survive across conversations. Applications
+without identity metadata share a namespace within their configured store.
+
+### Storage adapter contract
+
+Adapters implement `get_memory(memory_id)` for scoped point reads,
+upsert `remember(memory)` by its deterministic ID, and accept the optional
+`namespace` argument on `recall`. Namespace filtering happens **before** limiting
+results. Preserve `name`, `created_datetime`, `updated_datetime`, and identity
+metadata on round trips. Enforce tenant authorization in the adapter: hashed IDs
+are not access control. Same-key concurrent writes are last-writer-wins; this API
+does not provide compare-and-swap or semantic conflict resolution.
+
+`memory.remembered` events include `operation: "created" | "updated"`. Consumers
+should distinguish replacements from new records rather than counting all events
+as memory growth. `unchanged` produces no write event.
+
+For controlled cold-memory evaluations, set `execution_context["memory_recall_enabled"] = False`.
+Both search and exact-name recall return no records; storage and named upserts remain
+unchanged. Children inherit this execution-context setting.
+
+### Practical policy and current guidance
+
+Keep memory policy small and task-specific. For live-data workflows, store verified
+tool routes, join keys, and parameterized query recipes—not result facts or entire
+transcripts. Retrieve only what helps the current task. Leave good recipes alone;
+replace the same named entry after a verified correction. Do not require a write
+on every turn or inject the entire memory library into every prompt. Treat recalled
+content as untrusted hints, not instructions that override the agent's policy.
+
+Current vendor guidance supports this just-in-time, editable-record approach:
+
+- [Anthropic memory tool](https://platform.claude.com/docs/en/agents-and-tools/tool-use/memory-tool):
+  addressable files, bounded reads, updates/deletion, namespace/path isolation, and
+  size/expiration policies.
+- [Anthropic context engineering](https://www.anthropic.com/engineering/effective-context-engineering-for-ai-agents):
+  keep the smallest useful context and avoid overlapping/bloated tool sets.
+- [LangGraph memory overview](https://docs.langchain.com/oss/python/langgraph/memory):
+  scoped namespace/key stores, continually maintained memories, over-insertion versus
+  over-updating, and hot-path/background-write tradeoffs.
+
+This implementation intentionally does not add an embedding service, automatic
+consolidation/expiration, or a background memory-writing agent. Those are separate
+policies to evaluate against task quality, retrieval usefulness, latency, token
+cost, and memory growth—not prerequisites for a small procedural library.
+
 ## Sub-Agents
 
 The model can call `spawn_agent` with a self-contained prompt, then call

hyperforge/src/hyperforge/harness_sdk/memory.py

@@ -0,0 +1,56 @@
+"""Stable memory keys and small lexical retrieval helpers.
+
+Storage adapters must still enforce their own tenant authorization. These keys
+separate names by scope and stable application identity, never conversation ID.
+"""
+
+import json
+import re
+import uuid
+from typing import Any
+
+_STOP_WORDS = frozenset(
+    {
+        "a",
+        "an",
+        "and",
+        "are",
+        "by",
+        "for",
+        "from",
+        "in",
+        "is",
+        "of",
+        "on",
+        "or",
+        "the",
+        "to",
+        "using",
+        "with",
+    }
+)
+
+
+def memory_namespace(scope: str, metadata: dict[str, Any]) -> str:
+    identity = {
+        "namespace": metadata.get("memory_namespace"),
+        "account_id": metadata.get("account_id"),
+    }
+    if scope != "project":
+        identity["user_id"] = metadata.get("user_id")
+    if scope != "user":
+        identity["project_id"] = metadata.get("project_id")
+    return json.dumps(identity, sort_keys=True, separators=(",", ":"))
+
+
+def named_memory_id(name: str, scope: str, metadata: dict[str, Any]) -> str:
+    key = json.dumps(["harness-memory", memory_namespace(scope, metadata), scope, name])
+    return uuid.uuid5(uuid.NAMESPACE_URL, key).hex
+
+
+def memory_search_terms(query: str) -> set[str]:
+    return {
+        term
+        for term in re.findall(r"\w+", query.casefold().replace("_", " "))
+        if term not in _STOP_WORDS
+    }

hyperforge/src/hyperforge/harness_sdk/models.py

@@ -2,7 +2,7 @@
 
 from datetime import datetime, timezone
 from enum import StrEnum
-from typing import Any, Literal
+from typing import Annotated, Any, Literal
 
 from pydantic import BaseModel, Field
 
@@ -94,12 +94,19 @@ class HarnessEvent(BaseModel):
     payload: dict[str, Any] = Field(default_factory=dict)
 
 
+MemoryName = Annotated[
+    str, Field(min_length=1, max_length=64, pattern=r"^[a-z0-9][a-z0-9_-]*$")
+]
+
+
 class HarnessMemory(BaseModel):
     id: str
     scope: str
+    name: MemoryName
     text: str
     metadata: dict[str, Any] = Field(default_factory=dict)
     created_datetime: datetime = Field(default_factory=utcnow)
+    updated_datetime: datetime = Field(default_factory=utcnow)
 
 
 class HarnessInboxItem(BaseModel):

hyperforge/src/hyperforge/harness_sdk/storage.py

@@ -4,6 +4,7 @@
 from collections.abc import AsyncIterator
 from typing import Protocol
 
+from .memory import memory_namespace, memory_search_terms
 from .models import HarnessConversation, HarnessEvent, HarnessMemory
 
 
@@ -20,14 +21,19 @@ async def append_event(self, event: HarnessEvent) -> None: ...
 
     def iter_events(self, conversation_id: str) -> AsyncIterator[HarnessEvent]: ...
 
-    async def remember(self, memory: HarnessMemory) -> None: ...
+    async def get_memory(self, memory_id: str) -> HarnessMemory | None: ...
+
+    async def remember(self, memory: HarnessMemory) -> None:
+        """Upsert by ID, preserving created_datetime on replacement."""
+        ...
 
     async def recall(
         self,
         *,
         scope: str,
         query: str,
         limit: int = 20,
+        namespace: str | None = None,
     ) -> list[HarnessMemory]: ...
 
     async def forget(self, memory_id: str) -> None: ...
@@ -73,23 +79,47 @@ async def iter_events(self, conversation_id: str) -> AsyncIterator[HarnessEvent]
             yield event.model_copy(deep=True)
 
     async def remember(self, memory: HarnessMemory) -> None:
-        self.memories[memory.id] = memory.model_copy(deep=True)
+        stored = memory.model_copy(deep=True)
+        if existing := self.memories.get(memory.id):
+            stored.created_datetime = existing.created_datetime
+        self.memories[memory.id] = stored
+
+    async def get_memory(self, memory_id: str) -> HarnessMemory | None:
+        memory = self.memories.get(memory_id)
+        return memory.model_copy(deep=True) if memory is not None else None
 
     async def recall(
         self,
         *,
         scope: str,
         query: str,
         limit: int = 20,
+        namespace: str | None = None,
     ) -> list[HarnessMemory]:
-        needle = query.casefold().strip()
+        terms = memory_search_terms(query)
+        if query.strip() and not terms:
+            return []
         values = [
             memory
             for memory in self.memories.values()
             if memory.scope == scope
-            and (not needle or needle in memory.text.casefold())
+            and (
+                namespace is None
+                or memory_namespace(scope, memory.metadata) == namespace
+            )
+            and (
+                not terms or terms & memory_search_terms(f"{memory.name} {memory.text}")
+            )
         ]
-        return [memory.model_copy(deep=True) for memory in values[-limit:]]
+        values.sort(
+            key=lambda memory: (
+                len(terms & memory_search_terms(f"{memory.name} {memory.text}")),
+                memory.updated_datetime,
+                memory.id,
+            ),
+            reverse=True,
+        )
+        return [memory.model_copy(deep=True) for memory in values[: max(0, limit)]]
 
     async def forget(self, memory_id: str) -> None:
         self.memories.pop(memory_id, None)