Align with web standards (RFC 9110) and future-proof

- 402 responses: add Cache-Control: no-store to all 402 paths (x402 middleware)
- Content negotiation: add Vary: Accept on /content/fetch 200 responses
- Standardize X-FairFetch-Version header (was X-Fairfetch-Version) in code and docs
- OpenAPI: version 0.2.0, document 402/Cache-Control and Vary, reference RFC 9110
- Tests/docs: update version header references to X-FairFetch-Version

Author: arun-fairfetch

README.md

@@ -601,7 +601,7 @@ Every successful response includes these headers. Think of them as a receipt and
 | `X-PAYMENT-RECEIPT` | Proof that payment was settled. For x402: a transaction hash. For wallets: a ledger transaction ID (`ff_...`). | `0x6d8ce1bf...` or `ff_3a7c9e...` |
 | `X-FairFetch-Payment-Method` | How the agent paid: `wallet` (pre-funded account) or `x402` (one-time payment). | `wallet` |
 | `X-FairFetch-Wallet-Balance` | Remaining wallet balance after this charge (only present for wallet payments). | `99000` |
-| `X-Fairfetch-Version` | Protocol version, so clients know which Fairfetch spec they're talking to. | `0.2` |
+| `X-FairFetch-Version` | Protocol version, so clients know which Fairfetch spec they're talking to. | `0.2` |
 
 > [!TIP]
 > For a plain-language explanation of all Fairfetch concepts, headers, and terminology, see the [Concepts Guide](docs/CONCEPTS.md).

api/routes.py

@@ -191,6 +191,7 @@ async def fetch_content(
             license_id=grant_header,
         )
         _attach_preferred_access(md_response, request)
+        md_response.headers["Vary"] = "Accept"
         return md_response
 
     try:
@@ -238,6 +239,7 @@ async def fetch_content(
         license_id=grant_header,
     )
     _attach_preferred_access(json_response, request)
+    json_response.headers["Vary"] = "Accept"
     return json_response
 
 

compliance/headers.py

@@ -4,6 +4,7 @@
   X-FairFetch-Origin-Signature  — Ed25519 sig of the content body
   X-FairFetch-License-ID        — Usage Grant compact identifier
   X-FairFetch-Usage-Category    — Permitted usage tier (search_engine_indexing, summary, rag, etc.)
+  X-FairFetch-Version           — Protocol version (e.g. 0.2)
   X-Data-Origin-Verified        — EU AI Act origin attestation
   X-AI-License-Type             — Content license terms
 """
@@ -40,7 +41,7 @@ def to_dict(self) -> dict[str, str]:
             "X-AI-License-Type": self.license_type,
             "X-FairFetch-Usage-Category": self.usage_category,
             "X-FairFetch-Compliance-Level": compliance.value,
-            "X-Fairfetch-Version": "0.2",
+            "X-FairFetch-Version": "0.2",
         }
 
         if self.signature:

docs/CONCEPTS.md

@@ -198,7 +198,7 @@ suggestion and get clean, legal content through the API instead.
 | `X-AI-License-Type` | "These are the publisher's terms for AI usage." | `publisher-terms` |
 | `X-FairFetch-Origin-Signature` | "The publisher digitally signed this content. You can verify it hasn't been tampered with." | `GllQLb/V4Vd+Su...` |
 | `X-Content-Hash` | "Here's a fingerprint of the content. Use it to verify integrity." | `sha256:2c449548...` |
-| `X-Fairfetch-Version` | "This server speaks Fairfetch protocol version 0.2." | `0.2` |
+| `X-FairFetch-Version` | "This server speaks Fairfetch protocol version 0.2." | `0.2` |
 
 ### When Payment Is Involved
 

docs/PUBLISHER_GUIDE.md

@@ -452,7 +452,7 @@ X-FairFetch-Origin-Signature: ...
 X-FairFetch-License-ID: ...
 X-Content-Hash: sha256:...
 X-PAYMENT-RECEIPT: ...
-X-Fairfetch-Version: 0.2
+X-FairFetch-Version: 0.2
 ```
 
 If all of these are present, your pipeline is working end-to-end.

openapi.yaml

@@ -6,8 +6,9 @@ info:
     MCP protocol support, and EU AI Act 2026 compliance.
 
     ## Payment Flow (x402)
+    Uses HTTP status 402 Payment Required per RFC 9110 (HTTP Semantics).
     1. Request content without `X-PAYMENT` header → receive `402 Payment Required`
-    2. Response body contains `accepts` object with price, asset, network, and wallet (price may vary by content URL path when publisher uses route-based pricing)
+    2. Response body contains `accepts` object with price, asset, network, and wallet (price may vary by content URL path when publisher uses route-based pricing). Responses include `Cache-Control: no-store` and must not be cached.
     3. Obtain payment proof via the facilitator
     4. Retry with `X-PAYMENT: <proof>` header → receive content
 
@@ -17,7 +18,7 @@ info:
     - `application/ld+json` — JSON-LD article representation
     - `text/markdown` — Clean markdown extraction
     - `application/json` — Standard JSON response
-  version: 0.1.0
+  version: 0.2.0
   license:
     name: Apache 2.0
     url: https://www.apache.org/licenses/LICENSE-2.0
@@ -52,8 +53,13 @@ paths:
         - $ref: '#/components/parameters/PaymentHeader'
       responses:
         '200':
-          description: Content successfully retrieved and processed
+          description: Content successfully retrieved and processed. Response varies by Accept (Vary: Accept).
           headers:
+            Vary:
+              description: Indicates content negotiation by Accept header (RFC 9110)
+              schema:
+                type: string
+                example: Accept
             X-Data-Origin-Verified:
               $ref: '#/components/headers/X-Data-Origin-Verified'
             X-AI-License-Type:
@@ -84,7 +90,7 @@ paths:
               schema:
                 $ref: '#/components/schemas/UrlBlocked'
         '402':
-          description: Payment required — include X-PAYMENT or X-WALLET-TOKEN header
+          description: Payment required (RFC 9110). Include X-PAYMENT or X-WALLET-TOKEN. Response includes Cache-Control no-store.
           content:
             application/json:
               schema:
@@ -187,7 +193,7 @@ paths:
                     example: fairfetch
                   version:
                     type: string
-                    example: 0.1.0
+                    example: 0.2.0
 
   /compliance/optout:
     get:

payments/x402.py

@@ -36,6 +36,9 @@
 RECEIPT_HEADER = "X-PAYMENT-RECEIPT"
 LICENSE_ID_HEADER = "X-FairFetch-License-ID"
 
+# RFC 9110: 402 responses must not be stored by caches (payment state is per-request).
+HTTP_402_HEADERS = {"Cache-Control": "no-store", "X-Payment-Required": "true"}
+
 
 class X402Middleware(BaseHTTPMiddleware):
     """Intercepts requests to paid routes and enforces payment.
@@ -202,6 +205,7 @@ async def dispatch(self, request: Request, call_next: RequestResponseEndpoint) -
                     amount_required=effective_price,
                     shortfall=effective_price - balance,
                 ),
+                headers=dict(HTTP_402_HEADERS),
             )
 
         # --- Standard path: x402 one-time payment ---
@@ -216,7 +220,7 @@ async def dispatch(self, request: Request, call_next: RequestResponseEndpoint) -
             return JSONResponse(
                 status_code=402,
                 content=body,
-                headers={"X-Payment-Required": "true"},
+                headers=dict(HTTP_402_HEADERS),
             )
 
         # When search_engine_indexing but not free, settle at base price (1x)
@@ -237,6 +241,7 @@ async def dispatch(self, request: Request, call_next: RequestResponseEndpoint) -
                     effective_price,
                     verification_error=result.error,
                 ),
+                headers=dict(HTTP_402_HEADERS),
             )
 
         logger.info(

tests/test_compliance.py

@@ -40,7 +40,7 @@ def test_basic_headers(self):
 
         assert d["X-Data-Origin-Verified"] == "true"
         assert d["X-AI-License-Type"] == "publisher-terms"
-        assert d["X-Fairfetch-Version"] == "0.2"
+        assert d["X-FairFetch-Version"] == "0.2"
 
     def test_headers_with_signature(self, signer: Ed25519Signer):
         sig = signer.sign(b"test content")

tests/test_end_to_end.py

@@ -62,7 +62,7 @@ async def test_paid_response_includes_all_three_pillars(self, client: AsyncClien
         assert "X-PAYMENT-RECEIPT" in resp.headers
         assert resp.headers["X-PAYMENT-RECEIPT"].startswith("0x")
 
-        assert "X-Fairfetch-Version" in resp.headers
+        assert "X-FairFetch-Version" in resp.headers
 
     async def test_scraper_interception_count_increments(self, client: AsyncClient):
         """Publisher metric: scraper interception count goes up when bots are steered."""