Skip to content

s3.md

Latest commit

 

History

History
41 lines (30 loc) · 1.52 KB

s3.md

File metadata and controls

41 lines (30 loc) · 1.52 KB

Backend: s3 (planned)

Status: STUB — not yet implemented. Tracked at #962.

Planned design

S3-compatible object store, primarily intended for bulk artifacts (research corpus, media caches) rather than memory artifacts where latency and lack of full-text query are problematic.

When implemented:

  • Built on @aws-sdk/client-s3 v3
  • Authenticates via the AWS default credential chain (env, instance profile, SSO, etc.) — never via storage.config
  • Supports S3-compatible endpoints (Cloudflare R2, MinIO, Backblaze B2) via the endpoint field
  • read via GetObject, write via PutObject, list via ListObjectsV2

Anticipated configuration

{
  "version": "1",
  "backends": {
    "research": {
      "type": "s3",
      "bucket": "my-aiwg-archive",
      "prefix": "aiwg-research/",
      "region": "us-east-1",
      "endpoint": "https://s3.amazonaws.com"  // optional; for non-AWS endpoints
    }
  }
}

Credentials come from the AWS default credential chain. storage.config will never accept accessKeyId / secretAccessKey (the schema rejects them).

Why deferred

S3 doesn't fit the memory-shaped subsystems (no full-text query, latency for many small reads/writes). It's slated for the bulk-artifact subsystems (research, media). Implementation is gated on phase-3 work.

How to track / contribute

  • Tracking issue: #962
  • Design: .aiwg/architecture/storage-design.md (§5.7)