Skip to content

Commit 5f1802d

Browse files
authored
Merge pull request #4 from vasylenko/multi-platform
Cross-platform savePath + MCP write sandbox (markfetch 0.6.0)
2 parents a1eec20 + b65d09f commit 5f1802d

16 files changed

Lines changed: 621 additions & 58 deletions

.gitattributes

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
# Keep text files LF on all platforms. Windows runners would otherwise
2+
# autocrlf .md fixtures to CRLF and break snapshot tests.
3+
* text=auto eol=lf

.github/workflows/ci.yml

Lines changed: 13 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -9,7 +9,14 @@ on:
99

1010
jobs:
1111
test:
12-
runs-on: ubuntu-latest
12+
# fail-fast: false lets a failure on one OS surface all of the
13+
# others' results in the same run, instead of cancelling siblings.
14+
# Cheaper to read than re-running individually for each.
15+
strategy:
16+
fail-fast: false
17+
matrix:
18+
os: [ubuntu-latest, macos-latest, windows-latest]
19+
runs-on: ${{ matrix.os }}
1320
permissions:
1421
contents: read
1522
steps:
@@ -24,4 +31,9 @@ jobs:
2431
run: npm ci
2532

2633
- name: Run tests
34+
# `npm test` runs `tsx --test tests/*.test.ts`. The glob is
35+
# shell-expanded; cmd.exe (Windows default) doesn't expand it,
36+
# so we pin Git Bash — preinstalled on the windows-latest runner.
37+
# No-op on Linux/macOS where bash is already the default.
38+
shell: bash
2739
run: npm test

CHANGELOG.md

Lines changed: 11 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -7,8 +7,18 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
77

88
## [Unreleased]
99

10+
## [0.6.0] - 2026-05-14
11+
12+
### Added
13+
- Cross-platform absolute-path validation for `savePath` (MCP). Windows-style paths (`C:\foo`, `C:/foo`, `\\server\share`, `\foo`) are now accepted alongside POSIX absolute paths. The schema delegates to `path.isAbsolute()` so it stays correct on whichever platform the process runs.
14+
- Write sandbox restricting MCP `savePath` writes. By default the allowed set is `realpath(os.tmpdir())``realpath(process.cwd())`. Symlinks are resolved via `fs.realpath` before the containment check, so a planted symlink inside the sandbox cannot be used to escape. Violations return the new `save_forbidden` error code; no file is created. The CLI is intentionally unrestricted (human at the shell is the security boundary; the LLM via MCP is the threat surface).
15+
- `MARKFETCH_ALLOWED_WRITE_ROOTS` env var: platform-delimiter-separated list of absolute paths (`:` on POSIX, `;` on Windows). When set, **replaces** the default allowed roots entirely. Validated at startup with fail-fast-on-stderr semantics matching existing env-var conventions (`MARKFETCH_TIMEOUT_MS`, `MARKFETCH_MAX_BYTES`, `MARKFETCH_USER_AGENT`).
16+
- `save_forbidden` error code (8th in the contract): returned when a `savePath` resolves outside the configured allowed write roots.
17+
- CI test job now runs on `ubuntu-latest`, `macos-latest`, and `windows-latest`. `shell: bash` is set on the `npm test` step so the test-glob expands consistently across runners.
18+
1019
### Changed
11-
- Resolved code smell SonarQube findings (S4325 redundant `Document` casts, S6594 `String#match``RegExp#exec`) — no behavior change, all 50 tests pass. ([c993938](https://github.com/vasylenko/markfetch/commit/c9939385edfbe95f7f34a24ba8e33e5a74ac07f4))
20+
- MCP `savePath` schema replaced the literal `startsWith('/')` constraint with `z.string().refine(path.isAbsolute)`. **Breaking change for MCP callers that previously wrote outside `os.tmpdir()` or `process.cwd()`** — they will now receive `save_forbidden` and must either move the target inside the default roots or set `MARKFETCH_ALLOWED_WRITE_ROOTS`. CLI behavior is unchanged (no sandbox there).
21+
- Resolved code smell SonarQube findings (S4325 redundant `Document` casts, S6594 `String#match``RegExp#exec`) — no behavior change, all tests pass. ([c993938](https://github.com/vasylenko/markfetch/commit/c9939385edfbe95f7f34a24ba8e33e5a74ac07f4))
1222
- Documentation and inline comments cleaned up across README, SPEC, source, and test descriptions. Text-only, no runtime change. ([#2](https://github.com/vasylenko/markfetch/pull/2))
1323

1424
## [0.5.0] - 2026-05-12

README.md

Lines changed: 57 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -59,7 +59,7 @@ gemini mcp add -s user markfetch npx -y markfetch
5959
| Generic Playwright / Puppeteer |||||
6060
| `mcp-server-fetch` (Python) || basic |||
6161
| CloudFlare `/markdown` |||| paid |
62-
| **`markfetch`** | **** | **** | **✓ (7 codes)** | **** |
62+
| **`markfetch`** | **** | **** | **✓ (8 codes)** | **** |
6363

6464
- **Real-browser HTTP/2 + Chrome fingerprint.** ALPN-negotiated h2, `User-Agent`, `Sec-CH-UA-*`, `Sec-Fetch-*`, `Accept-*`. A Chrome UA with no client hints is a *stronger* automation signal than curl — `markfetch` sends the full coherent set, derived from the UA at startup so an override stays internally consistent.
6565

@@ -108,6 +108,19 @@ Flags:
108108

109109
Errors go to stderr with the same `[code] message` shape the MCP tool returns (see the table below), and the process exits with a non-zero status. The same env vars (`MARKFETCH_TIMEOUT_MS`, `MARKFETCH_MAX_BYTES`, `MARKFETCH_USER_AGENT`) apply in both modes.
110110

111+
Errors carry one of eight deterministic codes:
112+
113+
| Code | Meaning |
114+
|---|---|
115+
| `network_error` | DNS / TCP / TLS failure, or an unexpected internal error from the fetcher. |
116+
| `http_error` | Upstream returned a non-2xx status. |
117+
| `timeout` | Per-request budget `MARKFETCH_TIMEOUT_MS` exceeded. |
118+
| `unsupported_content_type` | Response was not `text/html` or `application/xhtml+xml`. |
119+
| `extraction_failed` | Readability returned no article content (typical for pure client-rendered SPAs). |
120+
| `too_large` | Response body or extracted markdown exceeded `MARKFETCH_MAX_BYTES`. |
121+
| `save_failed` | `savePath` was given but `writeFile` failed (parent directory missing, permission denied, etc.). |
122+
| `save_forbidden` | `savePath` resolves outside the allowed write roots — see [Write sandbox](#write-sandbox). MCP-only; the CLI has no sandbox. |
123+
111124
## What it is not
112125

113126
- **Not a crawler.** No recursion, no `robots.txt` parsing, no rate-limit orchestration. One URL in, one document out.
@@ -138,9 +151,51 @@ Pass overrides via the `env` block of your MCP client config:
138151
}
139152
```
140153

154+
### Write sandbox
155+
156+
MCP `savePath` writes are confined to a set of allowed root directories. By default the allowed set is `os.tmpdir()``process.cwd()` (each resolved via `fs.realpath` once at startup). A `savePath` outside that set returns `save_forbidden` and no file is created.
157+
158+
Override the default set with `MARKFETCH_ALLOWED_WRITE_ROOTS` — a list of absolute paths separated by the platform's path delimiter (`:` on POSIX, `;` on Windows). When set, the override **replaces** the defaults entirely — it does not merge. To keep `os.tmpdir()` or `process.cwd()` accessible, list them yourself; the example below shows `/tmp` for that reason. A malformed value (non-absolute entry, or a directory that doesn't exist) fails fast on stderr at startup.
159+
160+
```json
161+
{
162+
"mcpServers": {
163+
"markfetch": {
164+
"command": "npx",
165+
"args": ["-y", "markfetch"],
166+
"env": {
167+
"MARKFETCH_ALLOWED_WRITE_ROOTS": "/Users/me/markfetch-out:/tmp"
168+
}
169+
}
170+
}
171+
}
172+
```
173+
174+
On Windows, use backslashes and `;` as the delimiter:
175+
176+
```json
177+
{
178+
"mcpServers": {
179+
"markfetch": {
180+
"command": "npx",
181+
"args": ["-y", "markfetch"],
182+
"env": {
183+
"MARKFETCH_ALLOWED_WRITE_ROOTS": "C:\\Users\\me\\markfetch-out;C:\\Users\\me\\AppData\\Local\\Temp"
184+
}
185+
}
186+
}
187+
}
188+
```
189+
190+
Notes:
191+
192+
- **The sandbox is MCP-only by design.** The CLI is unrestricted — a human at the shell is the security boundary, and the markfetch CLI doesn't run any sandbox check at all. The asymmetry exists because the MCP tool is driven by a language model, which may be steered by content from a page it just fetched.
193+
- **Symlinks pointing outside are blocked.** Each candidate `savePath` is resolved via `fs.realpath` to its real destination before the containment check, so a symlink planted inside the sandbox cannot be used to escape.
194+
- **Containment is case-insensitive on Windows** (`C:\Users\Bob` and `c:\users\bob` are the same path).
195+
141196
## Develop
142197

143-
Requires Node.js ≥ 24.
198+
Requires Node.js ≥ 24. Tested on Linux, macOS, and Windows in CI.
144199

145200
When iterating on CLI changes, `tsx src/index.ts <url>` and `tsx src/index.ts --help` route through the same argv-discriminated dispatcher as the built `dist/index.js` — no rebuild needed between edits.
146201

docs/SPEC.md

Lines changed: 6 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -13,25 +13,27 @@ URL
1313
→ caller markdown body, or "Saved N bytes to /path" confirmation
1414
```
1515

16-
Errors throw `MarkfetchError` uniformly from core; adapters catch once. Codes: `network_error`, `http_error`, `timeout`, `unsupported_content_type`, `extraction_failed`, `too_large`, `save_failed`. CLI emits `[code] message` to stderr and exits 1; MCP emits `{ isError: true, content: [{ text: "[code] message" }] }`.
16+
Errors throw `MarkfetchError` uniformly from core; adapters catch once. Codes: `network_error`, `http_error`, `timeout`, `unsupported_content_type`, `extraction_failed`, `too_large`, `save_failed`; plus `save_forbidden`, emitted by the MCP adapter only (before `fetchMarkdown` runs — see "Asymmetric write sandbox" under Core Decisions). CLI emits `[code] message` to stderr and exits 1; MCP emits `{ isError: true, content: [{ text: "[code] message" }] }`.
1717

1818
## Core Decisions
1919

2020
- **Argv-discriminated dispatch.** `argv.length === 2` (bare invocation) routes to MCP — preserving every existing client config, which all spawn with zero args. Any argument routes to CLI. No `--mcp` flag, no separate `markfetch-mcp` bin, no `isTTY` sniffing.
2121

2222
- **Lazy adapter imports.** The dispatcher uses `await import()` to load exactly one adapter. The only `console.log` in the project lives in `cli.ts`; under MCP, `cli.ts` never loads, so stdout-discipline is enforced by the module graph — not by linter or convention.
2323

24-
- **Core throws, adapters translate.** All 7 error codes surface from `core.ts` — five are thrown explicitly as `MarkfetchError`; `network_error`, `timeout`, and (sometimes) `http_error` are translated by `classifyError` from underlying-API errors (undici TypeErrors, AbortSignal timeouts). New codes need an `ErrorCode` union member + a throw site; adapters don't change.
24+
- **Core throws, adapters translate.** Seven of the eight error codes surface from `core.ts` — five are thrown explicitly as `MarkfetchError`; `network_error`, `timeout`, and (sometimes) `http_error` are translated by `classifyError` from underlying-API errors (undici TypeErrors, AbortSignal timeouts). The eighth code, `save_forbidden`, is the exception — it's emitted by the MCP adapter before `fetchMarkdown` is invoked (see "Asymmetric write sandbox" below). New core codes need an `ErrorCode` union member + a throw site; adapters don't change.
2525

2626
- **HTTP/2 + coherent Chrome fingerprint.** Wire protocol, headers, and UA must agree — a Chrome UA over HTTP/1.1 or without `Sec-CH-UA-*` is *more* suspicious than curl. `Sec-CH-UA-*` is derived from `MARKFETCH_USER_AGENT` at startup so override-coherence is mechanical.
2727

2828
- **Single-channel MCP response.** `content[0].text` only. Several major MCP clients (Claude Code CLI, VS Code/Copilot) forward only `structuredContent` to the model and drop `content[]` when both are present — a single-channel response keeps the markdown reachable from those clients.
2929

3030
- **Whole document or `too_large`.** No pagination. Partial content lets the agent reason over truncated bodies without knowing they're truncated. `savePath` / `-o` is the escape valve for genuinely large documents.
3131

32-
- **Asymmetric `savePath`.** MCP requires absolute paths (zod `startsWith("/")`); CLI accepts relative and resolves against `process.cwd()`. CLI has a stable cwd the user typed `cd` into; MCP servers run in whatever cwd the client picks.
32+
- **Asymmetric `savePath`.** MCP requires absolute paths via zod `refine(path.isAbsolute)` — accepts platform-appropriate shapes on POSIX (`/foo`) and Windows (`C:\foo`, `C:/foo`, `\\server\share`, `\foo`). CLI accepts relative and resolves against `process.cwd()`. CLI has a stable cwd the user typed `cd` into; MCP servers run in whatever cwd the client picks.
3333

34-
- **Stderr is fatal-only.** Per-request MCP errors round-trip through `{ isError }`; only startup misconfig / unrecoverable crashes touch stderr. CLI is its own session, so its per-request errors *are* fatal for that session. Regression guard: `tests/server.test.ts:436`.
34+
- **Asymmetric write sandbox.** MCP `savePath` writes are confined to `realpath(os.tmpdir())``realpath(process.cwd())` by default; the env var `MARKFETCH_ALLOWED_WRITE_ROOTS` (path-delimiter-separated) replaces the defaults. CLI writes anywhere the human's shell permits — no sandbox check. The asymmetry reflects the threat model: an LLM driving the MCP tool may be steered by content from the page it just fetched; a human typing into a shell is the security boundary. Symlinks are resolved via `fs.realpath` before the containment check, so a planted symlink inside the sandbox cannot escape. Containment compare is case-insensitive on `process.platform === "win32"`. Implementation lives in `src/sandbox.ts` (a leaf module — no imports from siblings — so it's unit-testable without spinning up the MCP server). Known limitation: TOCTOU between `realpath` and `writeFile` is not closed — acceptable for a single-user developer tool.
35+
36+
- **Stderr is fatal-only.** Per-request MCP errors round-trip through `{ isError }`; only startup misconfig / unrecoverable crashes touch stderr. CLI is its own session, so its per-request errors *are* fatal for that session. Regression guard: `tests/server.test.ts:336`.
3537

3638
## Ideas for future
3739

@@ -41,4 +43,3 @@ Errors throw `MarkfetchError` uniformly from core; adapters catch once. Codes: `
4143
- **Cookie reuse across redirects within a single fetch.** Currently none. Trigger: a target serves content only after a session-cookie redirect.
4244
- **Proxy support** (`MARKFETCH_PROXY_URL`) and **`Accept-Language` control** (`MARKFETCH_ACCEPT_LANGUAGE`). Trigger: corporate proxy / locale-specific content.
4345
- **Single-binary distribution.** Bun's `build --compile`, Node SEA, or similar. Trigger: `npx` first-run latency feedback, or an offline / airgapped need.
44-
- **Windows-friendly `savePath` schema.** Currently Unix-shaped (`startsWith("/")`). Trigger: someone needs this on Windows.

package-lock.json

Lines changed: 2 additions & 2 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

package.json

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
{
22
"name": "markfetch",
3-
"version": "0.5.0",
3+
"version": "0.6.0",
44
"description": "Fetch a URL, return clean markdown. MCP server and CLI for AI agents.",
55
"license": "MIT",
66
"author": {

src/cli.ts

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -23,7 +23,7 @@ program
2323
"Fetch a URL and return clean markdown.\n" +
2424
"Run with no arguments to start the MCP stdio server.",
2525
)
26-
.version("0.5.0")
26+
.version("0.6.0")
2727
.argument("<url>", "absolute http(s) URL to fetch")
2828
.option(
2929
"-o, --output <path>",

src/core.ts

Lines changed: 5 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -154,7 +154,8 @@ export type ErrorCode =
154154
| "unsupported_content_type"
155155
| "extraction_failed"
156156
| "too_large"
157-
| "save_failed";
157+
| "save_failed"
158+
| "save_forbidden";
158159

159160
export class MarkfetchError extends Error {
160161
constructor(
@@ -405,6 +406,9 @@ function convertToMarkdown(article: {
405406
// extraction_failed, too_large, save_failed
406407
// (The first three may also come from underlying APIs and be translated by
407408
// classifyError — adapters MUST run classifyError(err) in their catch blocks.)
409+
// Note: the MCP adapter additionally emits save_forbidden (the 8th code in
410+
// the contract) before fetchMarkdown is invoked — this function never throws
411+
// it. See src/sandbox.ts and src/mcp.ts.
408412
export async function fetchMarkdown(input: {
409413
url: string;
410414
savePath?: string;

src/mcp.ts

Lines changed: 18 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -13,6 +13,12 @@ import { McpServer } from "@modelcontextprotocol/sdk/server/mcp.js";
1313
import { StdioServerTransport } from "@modelcontextprotocol/sdk/server/stdio.js";
1414
import { z } from "zod";
1515
import { fetchMarkdown, classifyError, type ErrorCode } from "./core.js";
16+
import { isAbsolute } from "node:path";
17+
import { buildAllowedRoots, checkPath } from "./sandbox.js";
18+
19+
// Built once at startup. Bad config throws and surfaces on stderr (same
20+
// fail-fast convention as intEnv() in core.ts).
21+
const ALLOWED_ROOTS = await buildAllowedRoots(process.env);
1622

1723
function errorResult(code: ErrorCode, message: string) {
1824
return {
@@ -21,13 +27,13 @@ function errorResult(code: ErrorCode, message: string) {
2127
};
2228
}
2329

24-
const server = new McpServer({ name: "markfetch", version: "0.5.0" });
30+
const server = new McpServer({ name: "markfetch", version: "0.6.0" });
2531

2632
server.registerTool(
2733
"fetch_markdown",
2834
{
2935
description:
30-
"Fetch a single public HTTP/S URL and return its main article content as clean markdown. Best for articles, documentation, blog posts, news, and reference pages. Non-HTML responses return `unsupported_content_type`. Pure client-rendered SPAs with no extractable static HTML return `extraction_failed`; SPAs that ship server-rendered or SEO-prerendered HTML will extract whatever static content they expose. Also supports saving the markdown to a file, e.g., to bypass client tool-result size limits or to reuse later.",
36+
"Fetch a single public HTTP/S URL and return its main article content as clean markdown. Best for articles, documentation, blog posts, news, and reference pages. Non-HTML responses return `unsupported_content_type`. Pure client-rendered SPAs with no extractable static HTML return `extraction_failed`; SPAs that ship server-rendered or SEO-prerendered HTML will extract whatever static content they expose. Also supports saving the markdown to a file, e.g., to bypass client tool-result size limits or to reuse later. Saved files must land inside the allowed write roots (defaults: system temp dir and the server's working directory; configurable via `MARKFETCH_ALLOWED_WRITE_ROOTS`); paths outside return `save_forbidden`.",
3137
inputSchema: {
3238
url: z
3339
.string()
@@ -37,14 +43,22 @@ server.registerTool(
3743
),
3844
savePath: z
3945
.string()
40-
.startsWith("/")
46+
.refine(isAbsolute, "savePath must be an absolute filesystem path")
4147
.optional()
4248
.describe(
43-
"Optional. When provided, the fetched markdown is written to this absolute filesystem path and the response becomes a small confirmation. Use this when the markdown might exceed your client's tool-result inline cap. Must be an absolute path starting with '/'; relative paths and tilde-paths ('~/...') are rejected by the schema. Existing files are overwritten; the parent directory must exist (caller's responsibility). The file is written only on fetch success — fetch / extraction / size-cap errors return a [code] string and never touch the file.",
49+
"Optional. When provided, the fetched markdown is written to this absolute filesystem path and the response becomes a small confirmation. Use this when the markdown might exceed your client's tool-result inline cap. Must be an absolute path on the host platform (e.g., `/foo/bar.md` on POSIX; `C:\\foo\\bar.md` or `\\\\server\\share\\bar.md` on Windows); relative paths and tilde paths (`~/...`) are rejected by the schema. Writes are confined to an allow-listed sandbox — defaults are the system temp dir (`os.tmpdir()`) and the server's working directory; operators can override with `MARKFETCH_ALLOWED_WRITE_ROOTS` (path-delimiter-separated). A `savePath` outside the allowed roots returns `save_forbidden` and no file is created. Existing files are overwritten; the parent directory must exist (caller's responsibility). The file is written only on fetch success — fetch / extraction / size-cap errors return a `[code]` string and never touch the file.",
4450
),
4551
},
4652
},
4753
async ({ url, savePath }) => {
54+
// Sandbox gate (MCP-only; CLI is intentionally unbounded). Runs before
55+
// fetchMarkdown so a forbidden path short-circuits the fetch.
56+
if (savePath !== undefined) {
57+
const check = await checkPath(savePath, ALLOWED_ROOTS);
58+
if (!check.ok) {
59+
return errorResult("save_forbidden", check.reason);
60+
}
61+
}
4862
try {
4963
const { markdown, bytes, savedTo } = await fetchMarkdown({
5064
url,

0 commit comments

Comments
 (0)