Caching

ZeroFS runs two caches, each with a memory tier and a disk tier: a raw-parts cache for raw object bytes and a decoded-block cache for metadata. Two configuration keys budget them; each budget is split between the two caches.

Cache Layers

The raw-parts cache holds 128 KiB parts of object bytes as fetched from the object store. Segment objects (file data) and metadata SST objects share its one budget, keyed by path. It is the only cache serving file reads (the RAM-only tail cache below holds file data too, but feeds only write-path splicing), and parts are stored as they exist in the object store: compressed and encrypted. Its memory tier receives what remains of the memory budget after the decoded-block slice. Its disk tier lives in parts_cache/ and receives what remains of the disk budget after the decoded-block slice.

The decoded-block cache holds metadata blocks after decryption and decompression. The metadata LSM stores only inode and directory entries plus 32-byte extent pointers, so this cache is small and bounded. Its memory tier receives a quarter of the memory budget, its disk tier lives in hybrid_cache/ and receives a tenth of the disk budget; both slices are clamped (see the split sections below).

A third, RAM-only tail cache holds each inode's most-recently-written extent, 32 MiB total, updated only after the write's transaction commits. A sequential append splices into the cached tail instead of re-reading it.

Both disk tiers sit in a per-bucket directory inside cache.dir:

/var/cache/zerofs/
└── bucket_550e8400/
    ├── hybrid_cache/    decoded metadata blocks
    └── parts_cache/     raw 128 KiB parts (compressed + encrypted)

Configuration

[cache]
dir = "/var/cache/zerofs"
disk_size_gb = 10.0
memory_size_gb = 2.0  # optional

Disk Budget Split

The decoded-block disk tier receives 10% of the disk budget, clamped to between 1 GiB and 16 GiB. The raw-parts disk tier receives the remainder, with a 1 GiB floor. A 100 GiB budget splits into 90 GiB raw-parts + 10 GiB decoded-blocks; a 4096 GiB budget into 4080 GiB + 16 GiB. Budgets below 2 GiB overcommit rather than shrink either side: both floors hold, so a 0.5 GiB budget yields 1 GiB + 1 GiB.

Startup logs the resolved split:

Cache allocation - Disk: 10.00GB total (1073 MB decoded-blocks + 8926 MB raw-parts), Memory: 2.00GB total (500 MB decoded-blocks + 1500 MB raw-parts)

With disk_size_gb = 10.0, 10% of the budget is 1 GB (below the 1 GiB floor) so the decoded-block tier is raised to 1 GiB (1073 MB) and the raw-parts tier receives the remaining 8926 MB.

Memory Budget Split

memory_size_gb budgets both memory tiers. The decoded-block memory tier receives a quarter of the budget, clamped to between 32 MiB and 2 GiB. The raw-parts memory tier receives the remainder, with a 32 MiB floor. A 1 GiB budget splits into 768 MiB raw-parts + 256 MiB decoded-blocks. Earlier versions gave the raw-parts cache a fixed 128 MiB of RAM outside the configured budget; it now comes out of memory_size_gb like everything else.

Prefetch

ZeroFS fetches object data in 128 KiB parts. An adaptive readahead layer tracks up to 4 concurrent sequential read streams per object. Each stream's fetch window starts at 128 KiB and doubles on every sequential access, capped at 8 MiB. Once a stream's window has grown past the minimum, up to 4 full windows are kept fetched ahead of the read, up to 4 × 8 MiB outstanding at the maximum window. Fetched parts land in the raw-parts cache.

Three behaviors reduce GET counts further. Concurrent misses inside the same window join a single GET instead of each issuing their own. The leading fetch covers at least the whole requested range, so a large coalesced read is one GET even before the window ramps. Uploads populate the cache at write time: single PUTs write through to the parts cache, and sealed segment objects (uploaded as multipart) are warmed with the bytes in hand, so a read after a write needs no GET. One exception: conditional GETs (if-match, versioned reads) bypass the cache.

Above the per-object prefetcher, a logical read-ahead follows a file's extents across segment objects. After 2 consecutive sequential reads it prefetches an 8 MiB window ahead of the read, with at most 16 concurrent fetches, and it fires only when the window crosses into a different segment object; within one segment, the per-object prefetcher already covers it.

Per-Bucket Cache Isolation

On first open, ZeroFS writes a .zerofs_bucket_id marker containing a UUID at the database path on the object store; later opens read the same marker. The local cache for that volume lives in bucket_<8hex> inside cache.dir, named after the first 8 hex characters of the UUID, so several volumes can share one cache.dir without mixing entries.

Deleting and recreating a bucket produces a new UUID. The volume starts over with an empty cache directory; the old bucket_* directory is orphaned and ZeroFS does not remove it. Delete it manually to reclaim disk space.