Authors: Robin Dey (robin@openhubresearch.org) · Panyanon Viradecha
Affiliation: OpenHub Research, Chiang Mai, Thailand
Date: April 2026
Branch: submission-draft — arXiv-ready LaTeX version
📄 Read the PDF →
📝 LaTeX source →
🗂 arXiv submission guide →
MemPalace is an open-source AI memory system that applies the ancient method of loci (memory palace) spatial metaphor to organize long-term memory for large language models. Launched in April 2026, the project accumulated over 47,000 GitHub stars in its first two weeks and claims state-of-the-art retrieval performance on the LongMemEval benchmark (96.6% Recall@5) without requiring any LLM inference at write time.
We present a comprehensive technical analysis of the MemPalace architecture, examining the mapping between its cognitive-science-inspired hierarchical structure (Wings→Rooms→Drawers) and its actual implementation in code. Through independent codebase analysis, benchmark replication, and comparison with competing systems, we find that MemPalace's headline retrieval performance is attributable primarily to its verbatim storage philosophy combined with ChromaDB's default embedding model (all-MiniLM-L6-v2), rather than to its spatial organizational metaphor per se.
Our analysis concludes that MemPalace represents significant architectural insight wrapped in overstated claims — a pattern common in rapidly adopted open-source projects where marketing velocity exceeds scientific rigor.
Keywords: AI memory systems · method of loci · spatial memory · vector databases · LLM memory · retrieval-augmented generation · MCP protocol · ChromaDB · LongMemEval
| Finding | Result |
|---|---|
| LongMemEval Recall@5 (raw, no LLM) | 96.6% |
| Benchmark attributable to palace structure? | ❌ (it's ChromaDB + verbatim text) |
| Benchmark attributable to verbatim storage? | ✅ |
| Wake-up cost (L0+L1) | ~170 tokens |
| Write-time LLM calls required | 0 |
| Runtime dependencies | 2 (chromadb, pyyaml) |
| Mem0 comparison (old / new) | ~49% → 93.4% Recall@5 |
| "+34% from palace" claim valid? | |
| AAAK "lossless" claim valid? | ❌ 12.4pp recall drop |
Since the initial analysis (v3.1.0), MemPalace shipped pluggable storage backends (v3.2.0), a Closet layer with BM25 hybrid search (v3.3.0), fixed a critical hnsw:space=cosine bug, self-corrected their 100% benchmark to 98.4% held-out, and grew to ~48,000 stars. Mem0 also released their token-efficient algorithm, raising their LongMemEval score from ~49% to 93.4%. See §5.8 of the paper for details.
| Section | Topic |
|---|---|
| §1 | Introduction and contributions |
| §2 | Background: AI memory, method of loci neuroscience, hierarchical memory, vector DBs, MCP |
| §3 | System architecture: palace hierarchy, ingestion, search, knowledge graph, AAAK, MCP |
| §4 | Evaluation: LongMemEval, benchmark controversy, honest assessment, competitive comparison |
| §5 | Discussion: novelty, marketing-science gap, cognitive science verdict, verbatim insight, post-analysis developments |
| §6 | Related systems: Supermemory ASMR, Mem0, Zep/Graphiti, Mastra, Hindsight |
| §7 | Framework for evaluating AI memory systems |
| §8 | Conclusion and recommendations |
| App A | Code statistics |
| App B | Benchmark reproduction notes |
| App C | Glossary |
| App D | Disclosure |
@unpublished{dey2026mempalace,
author = {Dey, Robin and Viradecha, Panyanon},
title = {Spatial Metaphors for {LLM} Memory: A Critical Analysis of the {MemPalace} Architecture},
year = {2026},
url = {https://github.com/web3guru888/mempalace-scientific-analysis},
note = {Preprint, OpenHub Research}
}Or see CITATION.cff for the Citation File Format version.
All benchmark scripts, experiment code, and raw data are included in this repository.
pip install chromadb pyyaml numpy pandascd benchmarks/
python -m scripts --config scripts/config.py
# Results written to results/aggregated/summary.json| Script | Description |
|---|---|
benchmarks/scripts/runner.py |
Main benchmark runner (LongMemEval reproduction) |
experiments/scripts/causal_chain_experiment.py |
Causal chain orient experiment |
experiments/scripts/cycle6_experiment.py |
Retrieval profiles (Cycle 6) |
experiments/scripts/cycle7_experiment.py |
Embedding deduplication (Cycle 7) |
cd paper/
make pdf
# → mempalace-paper.pdf (18 pages)Requires: pdflatex, bibtex (standard TeX Live installation).
mempalace-scientific-analysis/
│
├── paper/
│ ├── mempalace-paper.tex ← LaTeX source (JMLR preprint style)
│ ├── mempalace-paper.bib ← BibTeX bibliography (25 references)
│ ├── mempalace-paper.pdf ← Pre-built PDF (18 pages)
│ ├── jmlr2e.sty ← JMLR style file
│ ├── Makefile ← Reproducible PDF build
│ ├── mempalace-scientific-analysis.md ← Original Markdown paper
│ └── arxiv-submission/
│ ├── README.md ← arXiv upload instructions
│ └── jmlr2e.sty ← Style file for arXiv bundle
│
├── benchmarks/
│ ├── scripts/ ← Benchmark runner code
│ │ ├── runner.py ← Main runner (391 lines)
│ │ ├── metrics.py ← Metric computation
│ │ ├── config.py ← Configuration
│ │ ├── mock_data.py ← Mock data generators
│ │ └── __main__.py ← Entry point
│ ├── results/raw/ ← Raw JSON results (25 runs)
│ └── results/aggregated/summary.json ← Aggregated statistics
│
├── experiments/
│ ├── experiment-registry-2026-04-10.md ← Master registry
│ ├── scripts/ ← Experiment scripts (4 files)
│ └── reports/ ← 33 experiment reports (Markdown)
│
├── integration-code/
│ └── src/mempalace_agi/ ← Full integration source (6 modules)
│
├── data/discovery-runs/ ← Experiment data
├── references/ ← Reference documents
│
├── CITATION.cff ← Citation metadata
├── LICENSE ← CC BY 4.0 (paper)
└── LICENSE-CODE ← MIT (code/benchmarks)
- Paper (
.tex,.pdf,.md): CC BY 4.0 — cite us and use freely - Code (benchmarks, experiments, integration): MIT — use freely
Robin Dey — robin@openhubresearch.org
Panyanon Viradecha
OpenHub Research, Chiang Mai, Thailand
GitHub: web3guru888
Web: openhubresearch.org