Skip to content

smithery-ai/oxfs

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

73 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

oxfs

oxfs connects any app to any storage backend

FUSE filesystem backed by any object store, written in Rust.

Built on fuser and Apache OpenDAL. S3 keys map directly to file paths with no local metadata database. 50+ storage backends.

Quick start

cargo build --release

# S3 / R2
AWS_ACCESS_KEY_ID=... AWS_SECRET_ACCESS_KEY=... \
./target/release/oxfs mount /mnt/oxfs \
  --backend s3 --bucket my-bucket --endpoint https://acct.r2.cloudflarestorage.com

# Local filesystem
./target/release/oxfs mount /mnt/oxfs --backend fs --root /tmp/oxfs-data

umount /mnt/oxfs

macOS: brew install --cask macfuse first (reboot required).

Storage backends

oxfs uses Apache OpenDAL. Any S3-compatible endpoint works out of the box.

Backend Example
AWS S3 --backend s3 --bucket name --region us-east-1
Cloudflare R2 --backend s3 --endpoint https://acct.r2.cloudflarestorage.com
GCS --backend s3 --endpoint https://storage.googleapis.com
MinIO --backend s3 --endpoint http://localhost:9000
Tigris --backend s3 --endpoint https://fly.storage.tigris.dev
Google Drive --backend gdrive --access-token $TOKEN
Dropbox --backend dropbox --access-token $TOKEN
Local disk --backend fs --root /path

--prefix scopes all keys within the bucket for per-session isolation.

Architecture

oxfs has two modes, selectable via --mode:

Flat (default): S3 keys map 1:1 to file paths. No local database, no chunking. What you see in the bucket is what you see in the mount. Comparable to GeeseFS and TigrisFS. In write-through mode (default), reads are validated via ETag: a HEAD checks if the cached content is still current, avoiding full GETs when nothing changed.

FUSE (fuser) -> FlatFuse -> FlatVfs -> CachedOperator -> OpenDAL
                               |            |
                         dirty buffers   ETag content cache
                         flush policy    stat/dir cache (moka)

Posix: local metadata database with tiered caching, write-ahead log, and background compaction. 99.6% pjdfstest compliance (8755/8789 tests, Linux). Supports hard links, device nodes, symlinks, nanosecond timestamps, and other features that flat object storage cannot represent.

FUSE (fuser) -> OxfsFuse -> Vfs -> TieredCache (L1 mem + L2 disk + WAL) -> OpenDAL
                              |
                           Metadata (redb or SQLite)
Crate Role
oxfs CLI binary, mode selection
oxfs-flat Flat mode: cache.rs (ETag caching), vfs.rs (file semantics), fuse.rs (FUSE impl)
oxfs-posix Posix mode: metadata engines, tiered cache, VFS, prefetch, WAL, FUSE impl
oxfs-backend Storage backend builder (S3, GDrive, Dropbox, local fs via OpenDAL)

Configuration

oxfs mount <mountpoint>
  --backend <fs|s3|gdrive|dropbox>  Storage backend (default: fs)
  --mode <flat|posix>               Filesystem mode (default: flat)
  --root <path>                     Root dir for fs backend
  --bucket <name>                   S3 bucket
  --region <region>                 S3 region
  --endpoint <url>                  S3-compatible endpoint
  --prefix <path>                   Key prefix within bucket
  --access-token <token>            OAuth access token (gdrive, dropbox)
  --refresh-token <token>           OAuth refresh token (gdrive, dropbox)
  --client-id <id>                  OAuth client ID (gdrive, dropbox)
  --client-secret <secret>          OAuth client secret (gdrive, dropbox)
  --writeback                       Buffer writes, flush on close (flat mode)
  --default-permissions             Kernel-enforced permission checks
  -d, --daemonize                   Fork into the background

  # Posix mode only:
  --meta-db <path>                  Metadata database (default: oxfs.db)
  --meta-backend <redb|sqlite>      Metadata engine (default: redb)
  --cache-mem-mb <N>                L1 memory cache MB (default: 256)
  --cache-disk-path <path>          L2 disk cache directory
  --cache-disk-mb <N>               L2 disk cache MB (default: 1024)
  --wal-path <path>                 Write-ahead log (crash recovery)

License

MIT OR Apache-2.0

About

FUSE filesystem backed by any object store, written in rust.

Resources

Stars

Watchers

Forks

Packages

 
 
 

Contributors

Languages