The Core Difference: Object Identity
Amazon S3 Files and ZeroFS both expose a POSIX filesystem backed by an S3 bucket, but that's where similarity ends. The fundamental choice: does every file in the mount correspond to a plain S3 object, or can the bucket be an opaque persistence layer?
S3 Files maintains a one-to-one mapping: images/cat.jpg on the mount is the same key in the bucket. Changes flow both ways, but asynchronously. ZeroFS deliberately breaks that identity. File contents are split into extents, compressed, encrypted, and packed into immutable segment objects. Metadata lives in an LSM tree. An S3 client sees an internal layout, not the mounted files.
Storage Layout and Write Path
S3 Files uses AWS-managed high-performance storage (backed by EFS) as a low-latency tier. Writes go there immediately and are durable. An asynchronous export to S3 starts after 60 seconds of write inactivity. Calling fsync does not trigger export; the object is not visible via S3 API until export completes.
ZeroFS uses a different architecture: a virtual filesystem talks to a segment store (compressed, encrypted file-data frames) and an LSM tree for metadata. Writes over 9P are acknowledged only after object storage confirms both the data segment and metadata flush. fsync guarantees durability in S3 immediately.
Cold Read and Read-Ahead
First access to S3 Files can trigger an import: listing a directory loads every object's metadata and asynchronously copies files below a threshold (128 KiB by default) into high-performance storage. AWS says listing 1,000 objects may take several seconds. Larger files stay in S3; reads >= 1 MiB go directly to S3.
ZeroFS populates a local cache from reads and uploads. A cold miss resolves extents through the LSM tree and coalesces adjacent frames into ranged GETs. It pays the first round trip without importing the rest of the file or directory. Sequential traffic triggers read-ahead within and across segment objects.
Cost Comparison
Both incur S3 storage and request charges. S3 Files adds charges for high-performance storage: $0.30/GB-month, $0.03/GB for reads, $0.06/GB for writes. Files have a 10 KiB minimum billable size.
An illustrative scenario: 10,000 GiB stored, read once.
| Case | Storage/month | Read once | Illustrative subtotal |
|---|---|---|---|
| ZeroFS (2:1 compression) | ~$115 + overhead | S3 GETs | ~$115 + infrastructure |
| ZeroFS (1:1) | ~$230 + overhead | S3 GETs | ~$230 + infrastructure |
| S3 Files (1 MiB direct reads) | ~$230 + metadata | ~$1.17 + S3 GETs | ~$231 + metadata |
| S3 Files (small resident reads) | $3,230 | $300 | $3,530 + $600 if imported |
ZeroFS figures are estimates: compression reduces payload, but metadata and requests add cost. At ideal 8 MiB GET size, reading costs ~$0.26 (2:1) or $0.51 (1:1). ZeroFS also needs one server (2 for HA) with ~2 GB RAM plus local cache. S3 Files may cost less for large-file streaming. Writing 10,000 GiB through S3 Files adds $600 writes + $300 export; a month residency adds $3,000.
Rename Semantics
S3 has no atomic rename for general-purpose buckets. S3 Files renames on the mount, then copies each affected object to a new key and deletes the old one. During a directory rename, both prefixes can be visible. Synchronizing 100,000 renamed files takes minutes.
ZeroFS changes only its LSM directory entries — no copy per descendant, no S3 prefix exposed.
S3 API Visibility
S3 Files eventually makes files visible as normal objects. ZeroFS never exposes mounted files through S3 API. If other apps need immediate S3 access after a write, neither model provides it: S3 Files exports asynchronously; ZeroFS doesn't export at all.
Practical Implications
Choose S3 Files if you need files to remain ordinary S3 objects for other tools (e.g., Athena, batch processing). Accept async export delays and potential cost from high-performance storage.
Choose ZeroFS if you can treat S3 as a private substrate. Gains: compression, client-side encryption, cheaper small-file handling (packed together), atomic renames, and immediate durability on fsync. Cost: no direct S3 access; recovery requires ZeroFS and encryption password.
Bottom Line
Both solve POSIX-on-object-storage, but for different use cases. S3 Files fits ecosystems needing object interoperability; ZeroFS fits cost-sensitive, security-conscious workloads that can live without direct S3 API access.
