Skip to content

PROXY.md

Latest commit

 

History

History
281 lines (202 loc) · 8.72 KB

PROXY.md

File metadata and controls

281 lines (202 loc) · 8.72 KB

DTMS Proxy to external services (Node/Python/REST)

This document explains how the DTMS backend exposes a safe gateway (proxy) to call external microservices. It is designed to be explicit so an automated agent can configure, operate, and troubleshoot it.

Goals

  • Provide a single backend endpoint to reach external services (same-origin for the frontend).
  • Keep secrets and upstream details private (no secret is ever sent to the browser).
  • Apply strict timeouts and minimal headers (no blind forwarding of client headers).
  • Start with a health check (GET), and prepare for future methods (POST/PUT/DELETE) with CSRF, allowlists, etc.

File locations and roles

  • Core router: core/api/index.php
  • Proxy controller: core/api/controllers/proxy.php
  • Service registry: core/config/services.json
  • Secrets (private): core/config/secrets.local.php (ignored by git)
  • Main config (public keys): core/config/config.json
  • Logs: data/core/api.log

Endpoint contract (current)

Health check only (GET).

  • Method: GET
  • URL: /core/api/services/:id/health (or /api/services/:id/health if alias configured in Nginx)
  • Behavior:
    • Finds the service by id in services.json.
    • Builds URL = baseUrl + healthPath.
    • Adds optional auth header from secrets if configured.
    • Sends a GET with a short timeout (3s), no redirects.
    • Tries to parse JSON; if not JSON, data is omitted.
  • Response (normalized):
    • Success: { "ok": true, "status": 200, "data": { ...optional JSON... } }
    • Not found service: { "ok": false, "error": "Unknown service" } with HTTP 404
    • Upstream down/timeout: { "ok": false, "status": 0, "error": "Service unreachable" } with HTTP 200

Note: Upstream raw bodies are not exposed unless valid JSON. Secrets are never exposed.

Services configuration (services.json)

File: core/config/services.json

Schema per service:

{
  "id": "example-node",
  "type": "http",
  "baseUrl": "http://host.docker.internal:4000",
  "healthPath": "/health",
  "authHeader": "X-API-KEY",
  "tokenKey": "services_proxy"
}

Fields:

  • id (string): unique identifier used in the proxy URL.
  • type (string): currently only http is supported.
  • baseUrl (string): upstream base URL. In Docker on macOS, use http://host.docker.internal:<port> to reach a process on the host. If the upstream is another container, use its Docker service name.
  • healthPath (string): path for the healthcheck endpoint (e.g., /health, /healthz, /api/health).
  • authHeader (string, optional): name of an HTTP header to send (e.g., X-API-KEY).
  • tokenKey (string, optional): key name in secrets.local.php['internal_tokens'] to get the secret value to put in authHeader.

Example list (array):

[
  {
    "id": "example-node",
    "type": "http",
    "baseUrl": "http://host.docker.internal:4000",
    "healthPath": "/health",
    "authHeader": "X-API-KEY",
    "tokenKey": "services_proxy"
  }
]

Secrets (secrets.local.php)

File: core/config/secrets.local.php (never commit; it is in .gitignore)

Expected structure (excerpt):

<?php
return [
  'csrf_secret' => '...hex...',
  'internal_tokens' => [
    'services_proxy' => '...long-random-hex-or-string...'
  ]
];

If authHeader and tokenKey are provided in a service, the proxy sends:

  • authHeader: secrets['internal_tokens'][tokenKey]

No secret value is ever logged or returned to clients.

Security model

  • No client headers are forwarded blindly. The proxy sets only minimal, safe headers (e.g., Accept: application/json) and optional authHeader from secrets.
  • Short timeouts (3s) to avoid hanging calls.
  • No redirects are followed.
  • Only the health GET endpoint is exposed for now (no write methods yet).
  • CORS and no-cache headers are enforced by the core API before routing.
  • CSRF will be required later for write methods (POST/PUT/PATCH/DELETE) via X-CSRF-Token.

Planned hardening for future methods:

  • Allowlist per service (regex of allowed paths), allowed methods per service.
  • Max body size per service.
  • Sanitized set of forwarded headers for upstream (e.g., Content-Type, optional Authorization when explicitly allowed).

Networking guide (Docker/macOS)

  • When the upstream runs on the host (your Mac), containers cannot reach localhost/127.0.0.1 of the host. Use host.docker.internal instead.
    • Example: "baseUrl": "http://host.docker.internal:4000"
  • When the upstream runs in another container, prefer the Docker service name and ensure both containers share the same network.
  • A status: 0 with errno=6|7|28 usually means:
    • 6: DNS resolution failed (bad hostname)
    • 7: connection refused (nothing listening or firewall)
    • 28: timeout

Logging (debug mode)

  • Enable debug in core/config/config.json by setting: "debug": true.
  • The proxy appends a single-line entry to data/core/logs/api.log per call (resolved via PathManager) with the following format:
<ISO8601>\t<client-ip>\tPROXY\tservice=<id>\tauth=<0|1>\turl=<upstream-url>\tstatus=<code>\terrno=<curl-errno>\tdurMs=<duration>

Examples:

2025-08-18T03:42:06+00:00	192.168.65.1	PROXY	service=example-node	auth=1	url=http://host.docker.internal:4000/health	status=200	errno=0	durMs=5

Additionally, the API router logs each handled route (e.g., /services/example-node/health with its own HTTP status).

Return codes

For resilience, the proxy responds with HTTP 200 in both success and failure cases, and includes { ok: boolean, status: <upstream status or 0>, data? } in the payload. This avoids leaking upstream details in status codes while keeping clients informed. If you prefer propagating upstream failures as 5xx, adapt the controller to set HTTP 502/504 when ok:false.

Tests and troubleshooting

  1. Direct upstream test (optional)
curl -s -i http://dtms.localhost/health

You should see HTTP/1.1 200 OK and a JSON body if the upstream is running and the path is correct.

  1. Proxy test
curl -s http://dtms.localhost/core/api/services/example-id/health | jq .

Expected success:

{ "ok": true, "status": 200, "data": { "status": "ok" } }

If you get ok:false and status:0, inspect data/core/api.log lines with PROXY to find errno and correct:

  • errno=6 → fix baseUrl hostname
  • errno=7 → ensure upstream is running / correct port
  • errno=28 → increase timeout or fix upstream perf
  • status=404 → fix healthPath in services.json
  1. X-API-KEY on the upstream

If authHeader/tokenKey are set, the proxy sends the header automatically. The upstream should validate it. Examples:

Node/Express:

app.get("/health", (req, res) => {
  const key = req.header("X-API-KEY");
  if (
    process.env.SERVICES_PROXY_KEY &&
    key !== process.env.SERVICES_PROXY_KEY
  ) {
    return res.status(401).json({ ok: false, error: "unauthorized" });
  }
  res.json({ status: "ok", service: "node" });
});

Python/Flask:

from flask import Flask, request, jsonify
import os
app = Flask(__name__)

@app.get('/health')
def health():
    key = request.headers.get('X-API-KEY')
    expected = os.environ.get('SERVICES_PROXY_KEY')
    if expected and key != expected:
        return jsonify({ 'ok': False, 'error': 'unauthorized' }), 401
    return jsonify({ 'status': 'ok', 'service': 'py' })

Set the same value on both sides:

  • Upstream: SERVICES_PROXY_KEY=<value> (env var or config)
  • DTMS: core/config/secrets.local.php → internal_tokens.services_proxy = <value>

Add a new service (checklist)

  1. Edit core/config/services.json and add:
{
  "id": "my-service",
  "type": "http",
  "baseUrl": "http://host.docker.internal:5000",
  "healthPath": "/healthz",
  "authHeader": "X-API-KEY",
  "tokenKey": "services_proxy"
}

  1. Ensure the upstream is reachable from the PHP container (use host.docker.internal or a Docker service name).

  2. Test:

curl -s http://dtms.localhost/core/api/services/my-service/health | jq .
  1. If it fails, check data/core/api.log for PROXY lines and adjust baseUrl/healthPath.

Future extensions (not implemented yet)

  • Expose additional proxy routes:
    • GET /services/:id/... for selected read endpoints.
    • POST|PUT|PATCH|DELETE /services/:id/... for write endpoints.
  • Safeguards to add:
    • Per-service allowlists (regex of allowed paths).
    • Allowed methods per service.
    • Enforce CSRF for non-GET requests (X-CSRF-Token).
    • Limit body size per service.
    • Strict header pass-through allowlist (e.g., Content-Type, optional Authorization when explicitly configured).

Notes

  • The proxy never exposes secrets. Only minimal, safe information is returned to clients.
  • Debug logging is controlled by config.json ("debug": true). In production, set it to false.