Skip to content

wenchanghan/SecondBrainMCP

Repository files navigation

Second Brain MCP Server

A read-only MCP server that lets any MCP client (Claude, Cursor, custom agents, etc.) search and read a personal second brain — a plain folder of markdown files.

Motivation

Personal knowledge bases (notes, journals, clipped articles, meeting summaries) are most valuable when they're searchable from wherever you are. This server exposes a markdown folder as a set of read-only MCP tools, so an MCP client can query your notes on your behalf — from your phone, browser, or terminal.

Design principles:

  • Read-only by default — the only write operation is write_inbox_note, which is restricted to a configurable inbox folder to prevent callers from bypassing internal data ingestion workflows. Inbox items are treated as staging data: they are filtered out of search, list, tag, and stats results by default, and only appear in retrieval when explicitly requested by path or folder
  • Plain folder — reads .md files from any directory; no dependency on Obsidian, Notion, or any specific app
  • Multi-user, multi-vault — a single server instance can serve multiple users, each isolated to their own vault directories. Users can have multiple vaults (e.g., personal + work)
  • Provider-agnostic auth — includes an OAuth 2.1 proxy that works with Azure Entra ID (personal or work accounts), with standard JWT/JWKS validation
  • Self-hostable — runs on a home server (Mac Mini, old laptop) or a cloud VM

How It Works

Your notes folder (synced via iCloud, Dropbox, Git, rsync, etc.)
  ↕
Second Brain MCP Server (Node.js, Streamable HTTP)
  ↕ Tailscale Funnel (HTTPS, no public ports)
  ↕
MCP client (Claude, Cursor, or any Streamable HTTP client)

The server indexes all markdown files in-memory on startup, watches for changes (so synced updates appear automatically), and exposes tools, resources, and prompts over MCP's Streamable HTTP transport.

Tools

Tool Description
search_notes Hybrid search with advanced query syntax ("phrases", +required, -excluded, title:, tag:, after:, before:). Excludes inbox items
read_note Read a note's full content by path
list_notes List notes, optionally filtered by folder. Excludes inbox unless folder targets it
get_note_metadata Frontmatter, tags, file size, last modified
list_tags All tags across the vault, sorted by frequency. Excludes inbox items
search_by_tag Find notes with a specific tag. Excludes inbox items
get_backlinks Notes that link to a given note (forward + backlinks). Excludes inbox items
find_related_notes Find related notes by shared links, tags, and semantic similarity
vault_stats Vault analytics: notes by folder, top tags, orphaned notes, coverage. Excludes inbox (reported separately)
write_inbox_note Write a note to the inbox folder (create or append). Requires secondbrain.write scope

All tools accept an optional vault parameter to target a specific vault in multi-vault setups. Tags are extracted from both YAML frontmatter (tags: [foo, bar]) and inline #tag syntax. Links are detected from standard markdown syntax [text](path.md) — no Obsidian [[wikilinks]].

Resources

MCP resources provide browsable, structured data that clients can discover and read:

Resource URI Description
Tags vault://tags All tags with counts, sorted by frequency
Note vault://notes/{path} Read a specific note by path (URI template)
Recent vault://recent Notes modified in the last 7 days

Prompts

Pre-built workflow templates that guide MCP clients through multi-step tasks:

Prompt Arguments Description
summarize_topic topic Find and summarize everything about a topic
recent_activity days? Summarize recent writing activity
find_connections note_path Find notes related to a specific note
knowledge_gaps topic Identify what's missing about a topic

Quick Start

Prerequisites

  • Node.js 22+ (recommended: install via NVM)
  • A folder of markdown files

Install and run

git clone https://github.com/wenchanghan/SecondBrainMCP.git
cd SecondBrainMCP
npm install
npm run build

# Create your config
cp .env.example .env
cp users.example.json users.json

# Edit users.json — set your vault path
#   For local dev without auth, use any placeholder for "sub" (e.g., "dev-local")
#   For production with auth, the server logs the OAuth "sub" claim on first
#   authenticated request — check the server logs after connecting your MCP client
# Edit .env — set PORT (default 3000), leave auth vars blank for local dev

npm start

Verify

# Health check
curl http://localhost:3000/health

# MCP initialize handshake
curl -X POST http://localhost:3000/mcp \
  -H "Content-Type: application/json" \
  -H "Accept: application/json, text/event-stream" \
  -d '{"jsonrpc":"2.0","id":1,"method":"initialize","params":{"protocolVersion":"2025-03-26","capabilities":{},"clientInfo":{"name":"test","version":"1.0"}}}'

Configuration

users.json

Maps authenticated users to vault directories. Each user can only access their own vaults.

{
  "users": [
    {
      "sub": "oauth-subject-id",
      "name": "alice",
      "vaultPath": "/path/to/alice/notes"
    },
    {
      "sub": "multi-vault-user-id",
      "name": "bob",
      "vaults": {
        "personal": "/path/to/personal/vault",
        "work": "/path/to/work/vault"
      },
      "inboxPath": "incoming"
    }
  ]
}
  • sub — the OAuth subject claim from the JWT (unique user identifier from your OAuth provider)
  • name — display name (used in logs and health endpoint)
  • vaultPath — absolute path to the markdown folder (single-vault, backward-compatible)
  • vaults — named vault map for multi-vault users (use instead of vaultPath)
  • inboxPath — subfolder for write_inbox_note (relative to vault root, default: "inbox")

All tools accept an optional vault parameter to target a specific vault (e.g., "personal" or "work"). When omitted, the first vault is used as the default.

.env

PORT=3000
USERS_CONFIG_PATH=./users.json

# JWT validation — validates Bearer tokens on /mcp routes
# Leave all three blank to disable auth (development only)
JWKS_URI=https://login.microsoftonline.com/{tenant-id-or-consumers}/discovery/v2.0/keys
ISSUER=https://login.microsoftonline.com/{tenant-id}/v2.0
AUDIENCE=your-app-client-id

# OAuth proxy — our server acts as OAuth provider to MCP clients,
# delegating actual authentication to Entra ID.
# Leave blank to disable (clients must bring their own tokens).
ENTRA_AUTHORIZE_URL=https://login.microsoftonline.com/{tenant-id-or-consumers}/oauth2/v2.0/authorize
ENTRA_TOKEN_URL=https://login.microsoftonline.com/{tenant-id-or-consumers}/oauth2/v2.0/token
ENTRA_CLIENT_ID=your-app-registration-client-id
ENTRA_CLIENT_SECRET=your-client-secret
MCP_SERVER_URL=https://your-hostname.your-tailnet.ts.net

# Semantic search — two backends available:
# ONNX (self-contained): EMBEDDING_MODEL=onnx-community/Qwen3-Embedding-0.6B-ONNX
# Ollama (recommended for local): EMBEDDING_MODEL=ollama:qwen3-embedding:0.6b
# Set to "disabled" to skip entirely (keyword/BM25 still works).
EMBEDDING_MODEL=onnx-community/Qwen3-Embedding-0.6B-ONNX
OLLAMA_URL=http://localhost:11434

# Hybrid search blend: 0 = pure semantic, 1 = pure BM25, 0.5 = equal blend
SEARCH_HYBRID_ALPHA=0.5

# Verbose logging — detailed diagnostics for local debugging
# Shows: embedding timeouts, file watch events, startup timing, auth config diagnostics
# Never logs: tokens, user identifiers, emails, note content
VERBOSE_LOGGING=false

Entra ID tenant endpoints:

  • Personal Microsoft accounts: use /consumers (e.g., login.microsoftonline.com/consumers/oauth2/v2.0/authorize)
  • Work/school accounts: use your tenant ID
  • The ISSUER for personal accounts is always https://login.microsoftonline.com/9188040d-6c67-4c5b-b112-36a304b66dad/v2.0 (Microsoft's fixed consumer tenant GUID)
  • The AUDIENCE should match the aud claim in the token — for personal accounts this is typically the client ID without the api:// prefix

Auth is provider-agnostic. Any OAuth provider that issues JWTs with a JWKS endpoint will work. The OAuth proxy is specific to Entra ID but can be adapted.

Deployment

The server is deployment-agnostic — same code runs anywhere Node.js runs. Choose the mode that fits your setup:

Local (Mac/Linux) Cloud VM
Vault access Direct filesystem (iCloud, Dropbox, local folder) rsync, git, or file sync to VM
Availability Only when machine is awake 24/7
Sync lag Zero (files are local) Depends on sync interval (e.g., 5 min rsync)
Multi-user Single user (your machine, your vault) Multiple users with separate vaults
Cost Free VM hosting cost
RAM 8GB+ recommended (with embeddings) 8GB VM (Azure B2ms / AWS t3.large), or 4GB without embeddings

Local deployment (Mac, Linux laptop/desktop)

  1. Point vaultPath in users.json at your local notes folder
  2. Run the server via launchd (macOS) or systemd (Linux)
  3. Install Tailscale and enable Funnel to expose over HTTPS: tailscale funnel --bg 3000
  4. Register the Funnel URL as an MCP connector in your client's settings

No sync needed — the server reads directly from your local folder and watches for changes via chokidar.

Note: If you use Node.js via NVM, launchd won't find the node binary (it doesn't source shell profiles). Use a wrapper script — see scripts/start.sh for an example.

Cloud VM deployment (Azure, AWS, etc.)

  1. Provision a VM with enough RAM — 8GB with embeddings (Azure B2ms, AWS t3.large), or 4GB without (Azure B2s, AWS t3.micro)
  2. Sync your notes to the VM (rsync over Tailscale, git, or any file sync tool)
  3. Run the server via systemd
  4. Expose via Tailscale Funnel: sudo tailscale funnel --bg 3000
  5. Lock down networking — no inbound ports needed (Tailscale handles ingress)

Multiple MCPs on one server

If you run multiple MCP servers on the same machine, use one OAuth App Registration with scopes per MCP:

  • Each MCP listens on a different port
  • Each Tailscale Funnel maps to a different port
  • Each MCP client connector points to a different Funnel URL
  • Auth scopes (e.g., secondbrain.read, secondbrain.write, recipes.read) control per-MCP access
  • One user account per person (not per MCP)

See docs/runbook.md for detailed operational guides for both deployment modes.

Client Registration (Claude Example)

These steps show Claude as an example. Other MCP clients have their own integration flow.

  1. Go to claude.ai → Settings → Integrations
  2. Add your Tailscale Funnel URL (e.g., https://hostname.tailnet.ts.net/mcp)
  3. The client auto-discovers OAuth endpoints via /.well-known/oauth-authorization-server
  4. Authenticate when prompted — you'll be redirected to your OAuth provider
  5. The connector syncs across Claude web, desktop, and mobile

HTTP Endpoints

Method Path Auth Description
GET /health No Health check — returns vault stats and active session count
POST /mcp Bearer JWT MCP Streamable HTTP — initialize new session or send requests to existing session
GET /mcp Bearer JWT SSE stream for server-initiated messages (requires Mcp-Session-Id header)
DELETE /mcp Bearer JWT Terminate an MCP session
GET /.well-known/oauth-protected-resource No OAuth Protected Resource Metadata (RFC 9728)
GET /.well-known/oauth-authorization-server No OAuth Authorization Server Metadata (RFC 8414)
POST /register No Dynamic client registration (MCP SDK requirement)
GET /authorize No OAuth authorization — redirects to Entra ID
GET /callback No OAuth callback — receives auth code from Entra ID
POST /token No Token exchange — proxies to Entra ID, supports authorization_code and refresh_token grants

Session lifecycle: A new MCP session is created by sending a POST /mcp without a Mcp-Session-Id header containing an initialize request. The server responds with a session ID in the Mcp-Session-Id response header. Subsequent requests include this header. If a session is not found (e.g., after server restart), the server returns 404, signaling the client to re-initialize.

Rate limits: MCP routes are limited to 60 requests/min per authenticated user. OAuth routes are limited to 10 requests/min per IP.

Architecture

src/
├── index.ts            # HTTP server, MCP session management, middleware wiring
├── config.ts           # Environment-based configuration (Zod schema)
├── users.ts            # User store (OAuth sub → vault(s) mapping, single + multi-vault)
├── auth.ts             # JWT validation via JWKS (provider-agnostic)
├── log.ts              # Verbose logging utility (gated by VERBOSE_LOGGING env var)
├── oauth-proxy.ts      # OAuth 2.1 proxy (presents our server as OAuth provider to MCP clients)
├── tools/
│   ├── search.ts       # search_notes, search_by_tag, get_backlinks, find_related_notes
│   ├── read.ts         # read_note, get_note_metadata
│   ├── list.ts         # list_notes, list_tags, vault_stats
│   └── write.ts        # write_inbox_note (inbox-only writes, requires secondbrain.write scope)
├── resources/
│   └── notes.ts        # MCP resources (vault://tags, vault://notes/{path}, vault://recent)
├── prompts/
│   └── knowledge.ts    # MCP prompts (summarize_topic, recent_activity, find_connections, knowledge_gaps)
├── middleware/
│   ├── rate-limit.ts   # Rate limiting (per-user for MCP, per-IP for OAuth)
│   └── logger.ts       # Structured request logging (metadata only, never content)
└── vault/
    ├── manager.ts      # FileManager — file watching, indexing, search orchestration
    ├── parser.ts       # YAML frontmatter + tag parsing
    ├── search.ts       # BM25 full-text search engine + hybrid RRF blending
    ├── embeddings.ts   # IEmbeddingEngine interface + ONNX backend (Worker thread)
    ├── ollama-embeddings.ts # Ollama backend (HTTP API, Metal acceleration)
    ├── embedding-worker.ts  # Worker thread for ONNX inference (off main thread)
    └── links.ts        # Link index (standard markdown links, backlinks)

Key design choices:

  • One McpServer instance per session (MCP SDK constraint — each server binds to one transport)
  • FileManager instances are shared across sessions (one per vault per user)
  • In-memory BM25 search — no external search infrastructure needed for <10K files
  • Two embedding backends: ONNX (self-contained, via @huggingface/transformers Worker thread) or Ollama (Metal acceleration, native batch, unified model management). Config-driven selection via EMBEDDING_MODEL prefix
  • ONNX inference runs in a Worker thread so it never blocks the main event loop (HTTP requests stay responsive during indexing). Ollama backend uses HTTP API calls (non-blocking by nature)
  • Hybrid search blends BM25 keyword ranking with semantic similarity using Reciprocal Rank Fusion (RRF)
  • Embedding model loads lazily in the background — server starts immediately, semantic search becomes available once the model is ready
  • Embedding worker auto-restarts on crash (max 3 consecutive failures) with per-request timeouts (30s)
  • chokidar file watcher picks up sync changes automatically (both BM25 and embedding indexes updated incrementally)
  • OAuth proxy uses in-memory stores for auth flows (acceptable for single-user, single-process)

OAuth Proxy Flow

The server acts as an OAuth 2.1 provider to MCP clients, while delegating actual authentication to Entra ID:

MCP client → /.well-known/oauth-authorization-server → discovers our endpoints
MCP client → /authorize → redirects to Entra ID → user authenticates
Entra ID → /callback → we issue our own auth code → redirect to MCP client
MCP client → /token → we exchange with Entra ID → return access token
MCP client → /mcp (Bearer token) → JWT validated via JWKS → MCP tools execute

This architecture means the MCP client never talks directly to Entra ID — our server mediates the entire flow.

Security

  • Inbox-only writes — the only write operation (write_inbox_note) is restricted to a configurable inbox subfolder, preventing callers from modifying organized notes or bypassing ingestion workflows. Inbox items are filtered out of all retrieval tools by default (search, list, tags, stats), preserving the inbox-as-staging-area model. Requires a separate secondbrain.write auth scope
  • Path traversal protection — all paths resolved relative to vault root, .. rejected
  • User isolation — each user's JWT sub maps to a dedicated vault directory
  • No public ports — designed for Tailscale Funnel (HTTPS tunnel, no exposed ports)
  • Allowed file types — only .md, .markdown, .txt files are served
  • Local embeddings — semantic search runs entirely in-process; note content never leaves the machine
  • Rate limiting — per-user rate limiting on MCP routes (60 req/min), per-IP on OAuth routes (10 req/min)
  • Secure logging — request logger records only metadata (method, path, user, status, timing); never logs request/response bodies or auth headers
  • Defense-in-depth — Tailscale provides network perimeter; OAuth provides application-layer auth
  • Pre-commit hooks — block accidental commits of secrets, keys, and personal config

License

MIT

About

No description, website, or topics provided.

Resources

License

Contributing

Stars

Watchers

Forks

Releases

No releases published

Packages

 
 
 

Contributors