Configuration Guide
ZeroFS uses TOML configuration files for all settings. This guide covers all available configuration options.
Getting Started
Create a configuration file using the init command:
zerofs init
This creates a template configuration file that you can customize.
Configuration File Structure
Basic Configuration
# Cache configuration (required)
[cache]
dir = "/var/cache/zerofs" # Directory for caching data
disk_size_gb = 10.0 # Maximum disk cache size in GB
memory_size_gb = 2.0 # Memory cache size in GB (optional)
# Storage configuration (required)
[storage]
url = "s3://bucket/path" # Storage backend URL
encryption_password = "your-secure-password" # Encryption password
# Server configuration (required table, every subsection optional)
[servers.nfs]
addresses = ["127.0.0.1:2049"]
Three tables are required: [cache], [storage], and [servers]. A file missing any of them fails to parse; omitting [servers] reports missing field servers. Each [servers.*] subsection is optional. An empty [servers] table parses, but zerofs run then exits with No servers configured. At least one server (NFS, 9P, NBD, or RPC) must be enabled. At least one subsection must start a listener: for 9P, NBD, and RPC a unix_socket alone counts, and a [servers.webui] section counts in Web-UI-enabled builds.
Two parsing rules apply to the whole file:
- Unknown keys are rejected. A misspelled key aborts startup with a parse error naming the key. The exception is the free-form backend tables
[aws],[azure], and[gcp]; see Backend Option Tables. - Sizes are decimal gigabytes. All
*_gbkeys (cache.disk_size_gb,cache.memory_size_gb,filesystem.max_size_gb) are interpreted as 10⁹ bytes, not GiB. Fractional values are accepted:0.5is 500 MB.
Upgrading from a pre-segment release: the [lsm] keys wal_enabled and max_unflushed_gb are removed. A configuration file that still sets either fails to parse at startup; delete both keys. See LSM Tree Performance Tuning.
Cache
The [cache] table is required. ZeroFS runs two caches, each with a memory and a disk tier: a decoded-block cache for metadata and a raw-parts cache for object bytes. Both disk tiers live under dir, inside a per-bucket subdirectory: bucket_<8hex>/hybrid_cache/ and bucket_<8hex>/parts_cache/; see Caching for the layout. File data in the parts cache is stored as fetched from the object store: compressed and encrypted.
| Key | Required | Default | Description |
|---|---|---|---|
dir | yes | — | Directory holding both disk tiers. Supports environment variable substitution. |
disk_size_gb | yes | — | Total disk budget for both disk tiers, in decimal GB. |
memory_size_gb | no | 0.25 | Total memory budget for both memory tiers, in decimal GB. |
warm_metadata | no | "filters_index" | Startup warm-up: "off", "filters_index" (metadata SST bloom filters and indexes), or "full" (also metadata data blocks). File data is never warmed. |
See Caching for how the two budgets are split between the tiers.
Storage Backends
URL Schemes
The url field under [storage] accepts the following schemes. The same parser handles [wal].url.
| Scheme | Backend | Example |
|---|---|---|
s3://, s3a:// | Amazon S3 and S3-compatible | s3://bucket/path |
gs:// | Google Cloud Storage | gs://bucket/path |
azure:// | Azure Blob Storage | azure://container/path |
abfs://, abfss:// | Azure Blob Storage | abfss://container@account.dfs.core.windows.net/path |
memory:// | In-memory store, testing only | memory:/// |
az:// and adl:// also parse as Azure aliases. adl://container/path and the bare abfs://container/path form behave like azure://container/path. In the az:// form, the host is used as the container name and the first path segment is excluded from the object path; use azure:// instead.
Full https:// URLs are routed by host suffix. Hosts ending in amazonaws.com or r2.cloudflarestorage.com are treated as S3 (https://bucket.s3.region.amazonaws.com, https://ACCOUNT_ID.r2.cloudflarestorage.com/bucket). Hosts ending in dfs.core.windows.net, blob.core.windows.net, dfs.fabric.microsoft.com, or blob.fabric.microsoft.com are treated as Azure (https://account.blob.core.windows.net/container/path). Any other http:// or https:// URL is rejected at startup with feature for Http not enabled; generic HTTP and WebDAV endpoints are not supported. For S3-compatible services on other hosts, such as MinIO, use url = "s3://bucket/path" with endpoint under [aws], plus allow_http = "true" for plain-HTTP endpoints.
A scheme selects the backend only. Credentials come from the [aws], [azure], or [gcp] table regardless of which alias is used.
memory:/// keeps all data in process memory: the store starts empty and everything is lost when the process exits. Use it for throwaway testing only.
AWS S3
[storage]
url = "s3://my-bucket/zerofs-data"
encryption_password = "secure-password-here"
# storage_class = "..." # Optional, provider-specific; see Storage Class below
[aws]
access_key_id = "AKIAIOSFODNN7EXAMPLE"
secret_access_key = "wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY"
region = "us-east-1" # Optional, defaults to us-east-1
endpoint = "https://s3.amazonaws.com" # Optional for S3-compatible services
allow_http = "false" # Set to "true" (quoted) for plain-HTTP endpoints
conditional_put = "redis://localhost:6379" # For stores without conditional put support
For S3-compatible services like MinIO or Cloudflare R2, set the endpoint field to the service endpoint.
Conditional Put Support
ZeroFS requires conditional write (put-if-not-exists) support for fencing. AWS S3 supports this natively, so the conditional_put option is not needed when using AWS S3 directly.
For S3-compatible object stores that do not support conditional puts, set conditional_put to a Redis URL. ZeroFS will use Redis to coordinate conditional write operations:
[aws]
conditional_put = "redis://localhost:6379"
This is required for object stores that return success instead of an error when a put-if-not-exists operation targets an existing object.
Azure Blob Storage
[storage]
url = "azure://container/path"
encryption_password = "secure-password-here"
[azure]
storage_account_name = "myaccount"
storage_account_key = "your-account-key"
Google Cloud Storage (GCS)
Using Application Default Credentials (GCP VMs/GKE):
[storage]
url = "gs://my-bucket/zerofs-data"
encryption_password = "secure-password-here"
# No [gcp] section needed when running on GCP with attached service account
Using Service Account Key File:
[storage]
url = "gs://my-bucket/zerofs-data"
encryption_password = "secure-password-here"
[gcp]
service_account = "/path/to/service-account-key.json"
# Or use application_credentials = "${GOOGLE_APPLICATION_CREDENTIALS}"
GCS credentials resolve in this order:
- Explicit service-account credentials from the
[gcp]section (service_accountpath orservice_account_key) - An Application Default Credentials file: the
application_credentialskey, theGOOGLE_APPLICATION_CREDENTIALSenvironment variable, or the gcloud CLI's~/.config/gcloud/application_default_credentials.json - The VM/GKE metadata service, as the last fallback; this is why no
[gcp]section is needed on GCP with an attached service account
Backend Option Tables
[aws], [azure], and [gcp] are free-form key-value tables, not fixed schemas. At startup, each key is lowercased, prefixed with aws_, azure_, or google_, and forwarded to the object_store builder for the storage URL's backend. Any configuration key that builder accepts works, for example session_token, virtual_hosted_style_request, or skip_signature under [aws], and storage_sas_token under [azure] (forwarded as azure_storage_sas_token). The full key lists are in the object_store documentation: AmazonS3ConfigKey, AzureConfigKey, GoogleConfigKey.
Two rules apply:
- Every value must be a TOML string.
allow_http = "true", notallow_http = true. A non-string value fails at startup withFailed to parse config file. - Unknown keys are dropped without warning. A misspelled key such as
secret_acess_keyproduces no configuration error and surfaces later as an authentication or connection failure. Check key spelling before debugging credentials.
The same rules apply to the [wal.aws], [wal.azure], and [wal.gcp] tables under the legacy [wal] section. Values in all of these tables support environment variable substitution.
Storage Class
The optional storage_class field under [storage] sets the object storage class/tier for every write. The value is sent verbatim as the backend's own tiering header on every PUT and multipart upload: x-amz-storage-class (S3), x-goog-storage-class (GCS), x-ms-access-tier (Azure).
[storage]
url = "s3://my-bucket/zerofs-data"
encryption_password = "secure-password-here"
storage_class = "..." # provider-specific value, e.g. a single-zone hot class
Because the value is verbatim, it must be valid for your backend; a value the backend does not recognize (an S3 class name on Azure, for example) is rejected by the store on the first write. Omit the field to use the account/bucket default. The typical use is selecting a cheaper single-zone / reduced-redundancy hot or higher performance class where your provider offers one.
Server-side copies cannot carry a storage class: objects written via copy land in the bucket or account default class regardless of this setting. The legacy [wal] section has its own independent storage_class; see Separate WAL Object Store.
ZeroFS reads segment objects, SSTs, and the manifest continuously, so storage_class must be a hot, instant-access tier:
- Archive tiers (S3
GLACIER/DEEP_ARCHIVE, AzureArchive) require a restore before objects can be read. ZeroFS cannot read segment objects, SSTs, or the manifest behind a restore step, so the volume becomes unusable. Never set an archive class. - Infrequent-access tiers work but charge a per-GB retrieval fee on every read, so for ZeroFS's constant reads they usually cost more, not less.
Network Services
The [servers] table is required; each subsection below is optional. A server runs only when its section is present. An empty [servers] table is not runnable: zerofs run exits with No servers configured. At least one server (NFS, 9P, NBD, or RPC) must be enabled.
addresses is a list of socket addresses. Each entry binds one listener, so multiple entries serve the same filesystem on multiple addresses. IPv6 addresses use brackets:
[servers.nfs]
addresses = ["0.0.0.0:2049"] # All IPv4 interfaces
# addresses = ["[::]:2049"] # All IPv6 interfaces
# addresses = ["127.0.0.1:2049", "[::1]:2049"] # Dual-stack localhost
Omitting addresses starts no TCP listener; there is no implicit 127.0.0.1 bind. The 127.0.0.1 addresses for NFS (2049), 9P (5564), NBD (10809), and RPC (7000) appear in the template generated by zerofs init, not as built-in defaults. The 9P, NBD, and RPC servers can serve over unix_socket alone.
NFS Server
[servers.nfs]
addresses = ["0.0.0.0:2049"]
NFS is TCP-only: [servers.nfs] accepts only addresses and has no unix_socket key.
9P Server
[servers.ninep]
addresses = ["0.0.0.0:5564"]
unix_socket = "/tmp/zerofs.9p.sock" # Optional
NBD Server
[servers.nbd]
addresses = ["0.0.0.0:10809"]
unix_socket = "/tmp/zerofs.nbd.sock" # Optional
RPC Server
[servers.rpc]
addresses = ["127.0.0.1:7000"]
unix_socket = "/tmp/zerofs.rpc.sock"
The admin endpoint for zerofs checkpoint, zerofs flush, zerofs monitor, zerofs fatrace, and zerofs otrace. These commands read the same configuration file, try the Unix socket first if the socket file exists, then each configured address in turn (the order is not defined). Without a [servers.rpc] section, they exit with RPC server not configured in config file. See Monitoring and Checkpoints.
Web UI
[servers.webui]
addresses = ["127.0.0.1:8080"] # Default when the section is present
uid = 1000 # Required
gid = 1000 # Required
uid and gid set the POSIX identity for all file operations performed from the browser. See Web UI.
Metrics and Telemetry
The [prometheus] and [telemetry] tables sit at the top level, not under [servers]:
[prometheus]
addresses = ["127.0.0.1:9091"] # Default when the section is present
[telemetry]
enabled = true # Default, also when the section is absent
Without a [prometheus] section, no metrics endpoint is started. Telemetry defaults to enabled whether or not the section is present. See Prometheus Metrics and Telemetry.
Filesystem Quotas
ZeroFS supports configurable filesystem size limits:
[filesystem]
max_size_gb = 100.0 # Limit filesystem to 100 GB
When the quota is reached, write operations return ENOSPC (No space left on device). Delete and truncate operations continue to work, allowing you to free space. If not specified, the limit defaults to 16 EiB, the maximum filesystem size.
Compression
ZeroFS compresses each 32 KiB extent of file data before encrypting it; the resulting frames are packed into segment objects on the object store. The default is tuned for object storage, where network and storage cost dominate local CPU:
[filesystem]
compression = "zstd-3" # Zstd at level 3 (default)
# or
compression = "lz4" # Fast compression, lower ratio
Algorithms
-
zstd-{level}(default:zstd-3): Zstandard compression with configurable level from 1 to 22. Lower levels (1-5) are faster, higher levels (15-22) achieve better compression but are slower. -
lz4: Very fast compression and decompression with moderate compression ratio. Prefer for write-throughput-bound workloads where compression CPU is the bottleneck.
Changing Compression On-the-fly
You can change the compression algorithm at any time without migration:
- New writes use the currently configured algorithm
- Existing data remains readable regardless of how it was compressed
- Compression format is auto-detected per-frame when reading
- Existing data is never re-encoded: compaction relocates frames byte-for-byte without recompressing, so changing the algorithm or level does not rewrite or shrink data already stored; old and new encodings coexist indefinitely until the data is overwritten or removed
LSM Tree Performance Tuning
ZeroFS keeps metadata (inodes, directory entries, and the per-extent frame pointers) in an LSM (Log-Structured Merge) tree database that sits in the object store. File contents live in immutable segment objects written directly to the object store; this data path has no user-facing tuning keys, though the [gc] section described below tunes the loop that reclaims segment space. The [lsm] section tunes the metadata store, though flush_interval_secs and sync_writes also govern when buffered file data is sealed and uploaded. See Architecture.
[lsm]
l0_max_ssts = 256 # Max SST files in L0 before compaction
max_concurrent_compactions = 2 # Max concurrent compaction operations
flush_interval_secs = 30 # Interval between periodic flushes (in seconds)
sync_writes = false # Make every write durable on return
Parameters
-
l0_max_ssts(default: 256, min: 4): Maximum number of SST (Sorted String Table) files allowed in level 0 before triggering compaction. The same value applies per key. Lower values reduce read amplification but increase write amplification. Higher values do the opposite. -
max_concurrent_compactions(default: 2, min: 1): Maximum number of metadata compaction operations that can run concurrently. The store holds only metadata (file contents are immutable segment objects), so compactions are small and seldom pile up; a low cap keeps object-store request and CPU use down. Compaction always runs embedded inzerofs run; there is no separate compactor process. -
flush_interval_secs(default: 30, min: 5): Interval in seconds between periodic background flushes to durable storage. Each flush first seals the open segment buffer, uploading buffered file data, then flushes the metadata store. Between flushes, up to 256 MiB of committed writes can sit in the in-memory open segment.fsynccalls from clients (9P fsync, NFS COMMIT, NBD flush) always trigger an immediate flush regardless of this interval, unless[filesystem] ignore_fsync = true, which makesfsynca no-op; see High Availability. -
sync_writes(default:false): When enabled, every write is durably flushed before returning success, eliminating the "in-memory unflushed" window between fsync calls. This does not change POSIX fsync semantics: with the defaultfalse, explicitfsyncfrom clients is still honored and waits for durable persistence. The flag only governs writes between fsync calls. Expensive: each coalesced batch forces a segment seal plus a memtable flush (concurrent writes still merge into a single batch and a single flush). Rejected in combination with[filesystem] ignore_fsync. See the Eliminating the Durability Window section for guidance on when to enable.
Two keys from earlier releases are gone, and setting either fails config parsing:
wal_enabled: removed. The write-ahead log is permanently off and cannot be enabled.max_unflushed_gb: removed. The unflushed-metadata ceiling is fixed internally at 1 GiB.
These are advanced settings. The defaults suit most workloads. Change them only in response to a specific, observed performance problem.
When to Tune LSM Parameters
Consider tuning these parameters if you're experiencing:
- High read latency: Decrease
l0_max_sststo reduce read amplification - CPU / Network underutilization: Increase
max_concurrent_compactionson high-core systems - Too-frequent background flushes: Increase
flush_interval_secs; the crash-loss window for un-fsync'd writes grows accordingly
Segment Garbage Collection
The [gc] section tunes the segment reclamation loop described under Garbage Collection and not the tombstone sweep, the orphan sweep, or the metadata compaction above.
[gc]
interval_secs = 60 # Pass interval while the store is active
idle_interval_secs = 5 # Pass interval while draining backlog on an idle store
busy_backlog_interval_secs = 15 # Pass interval while active with a large dead backlog
busy_backlog_dead_percent = 20 # Dead-space % at/above which the busy-backlog interval applies
min_batches_per_pass = 4 # Compaction batches a busy pass runs before yielding to load
compact_round_max_mib = 256 # Per-round compaction budget (selection cap, reserve, gather RAM)
read_directed = true # Reads steer compaction (nominations, seams, chains)
tail_scrub_min_dead_percent = 5 # Repack write-cold segments above this % dead (0 = off)
Parameters
-
interval_secs(default: 60, min: 5): Seconds between segment GC passes while the filesystem is active. The ceiling of the adaptive cadence: the loop never sleeps longer. Deletion safety does not depend on this value (it rides on the per-segment delete horizon); only reclamation responsiveness does. Values below the ~30-second flush cadence make a busy pass's barrier seal real sub-1-MiB segments, which are themselves future GC work. -
idle_interval_secs(default: 5, min: 1, capped atinterval_secs): Seconds between passes while the previous pass hit a work budget with actionable backlog remaining and the store was completely idle since that pass. A fast pass performs real reclamation (up to 32 back-to-back batches of the per-round compaction budget while the store stays idle mid-pass, plus 1,024 deletes and roughly two small bookkeeping PUTs of fixed overhead), so this interval paces idleness re-checks rather than capping drain throughput; total fast-mode work is bounded by the backlog. Setting it equal tointerval_secsdisables only the idle acceleration tier; the independent busy-backlog drain tier still accelerates an active store. -
busy_backlog_interval_secs(default: 15, min: 5, capped atinterval_secs): Seconds between passes while the store is active but its dead backlog is large, the dead space at or abovebusy_backlog_dead_percent, or reserve-deferred hot seams remain. This is the middle cadence tier: it lets a loaded, dirty store keep draining without waiting the full base interval. Setting it equal tointerval_secsdisables the tier (a busy store then uses the base interval regardless of backlog). -
busy_backlog_dead_percent(default: 20): Store dead-space percent at or above whichbusy_backlog_interval_secsapplies while the store is active. Because the tier accelerates above this level and relaxes below it, it acts as the dead-space ceiling the store settles toward under sustained load. Below it, a busy store uses the base interval. It is scale-invariant (a ratio), so a large store with a low dead fraction is not accelerated on absolute backlog alone. -
min_batches_per_pass(default: 4, min: 1): Compaction batches a pass runs before it starts yielding to foreground load. Below the floor it drains regardless of client activity; past it a client op ends the pass within one batch. This raises reclaim throughput under sustained load. Batches run sequentially, so this does not raise peak RAM. Idle stores always drain to the internal per-pass cap (32) regardless. -
compact_round_max_mib(default: 256, range: 64–4096): Per-round compaction budget in MiB, the heat reserve for nominations and hot seams (half the budget), and the stored-byte gather cap (frames stay compressed through relocation, so gather RAM tracks stored size whatever the data's compression ratio). Raising it packs hot seams whose cheapest pair exceeds the reserve (the over-reserve chains that otherwise never repack; see Read-Directed Compaction) and lifts dead-space throughput per batch. The cost is peak gather RAM of roughly this many MiB per batch (more when the leading seam chain alone exceeds the budget, since a chain is gathered whole), so treat it as a RAM dial. -
read_directed(default:true): Whether reads steer compaction: the nomination priority, seam statistics, and chain repacks described under Read-Directed Compaction. Turning it off stops all read-side recording; counter-driven reclamation and the tail scrub are unaffected. -
tail_scrub_min_dead_percent(default: 5, range: 1–50; 0 disables the tail scrub): The tail-scrub floor. A write-cold segment more than this percent dead, but not dead enough for normal compaction candidacy, is repacked with leftover pass budget, most-dead-first. This caps the space overhead of write-cold data at 1/(1 − floor/100) of its live bytes (about 1.05× at the default), paid with up to (100 − floor)/floor bytes rewritten per byte reclaimed (19× at the default; the ratio improves as segments get deader, and the most-dead-first order takes the cheapest reclaim first). Raise it on request-billed or churn-sensitive backends; 50 empties the candidacy band, and 0 turns the scrub off entirely. On upgrade, a store with an accumulated 5–49%-dead tail works it off once, rate-limited to one round budget of live bytes per batch (256 MiB by default) and batching only while the store is idle, then goes quiet.
Separate WAL Object Store
ZeroFS no longer writes a WAL: it is permanently disabled and cannot be enabled. The [wal] section from earlier releases (url, storage_class, and the [wal.aws], [wal.azure], [wal.gcp] credential blocks) still parses, but with the WAL off it has no effect on writes or fsync latency. Omit it from new configurations.
Multiple Instances
One read-write instance and any number of read-only instances can run on the same storage backend, all using the same configuration file. See Read Replicas for startup, freshness, and restrictions.
High Availability
A [replication] section runs the node as one half of a leader/standby pair over a single object store, with automatic failover that loses no fsync'd data. Each node has its own config file. See High Availability for the design, guarantees, and a full two-node example.
[replication]
node_id = "node-a"
role = "leader" # initial role: "leader" or "standby"
replication_listen = "10.0.0.1:9000" # this node receives ships and heartbeats here
peers = ["10.0.0.2:9000"] # the other node's replication_listen
| Field | Required | Meaning |
|---|---|---|
node_id | yes | Stable identity of this node within the pair. |
role | yes | "leader" or "standby". The running role is decided at startup by asking the peer; this is the bootstrap hint. |
replication_listen | automatic HA | host:port this node receives the leader's ships and heartbeats on. |
peers | automatic HA | The other node's replication_listen. |
force_recovery | no | Break-glass, one-startup replacement of the durable leader marker. Requires role = "leader" and peers = []; never leave it enabled. |
Set both replication_listen and peers on every node so either can lead after a role swap. ZeroFS rejects a configuration that sets only one of them. A standalone node may omit both, and one-shot forced recovery may remove peers as described below. While the pair is Connected, acknowledged writes that have not been fsync'd survive a single-node failure because the leader ships them before replying. In Solo mode, unflushed writes return to the standalone durability window. For a workload that accepts the risk, [filesystem] ignore_fsync = true makes fsync a no-op and supplies neither an object-store barrier nor a lineage check.
force_recovery is only for a handoff that cannot recover normally after every former participant is known dead. A live or partitioned peer may hold the only copy of an acknowledged tail or an in-flight writer-open request. Quiesce that host and its object-store requests first, then remove force_recovery after the recovery startup succeeds.
Complete Examples
Basic S3 Configuration
# /etc/zerofs/zerofs.toml
[cache]
dir = "/var/cache/zerofs"
disk_size_gb = 10.0
[storage]
url = "s3://my-bucket/zerofs-data"
encryption_password = "your-secure-password"
[aws]
access_key_id = "AKIAIOSFODNN7EXAMPLE"
secret_access_key = "wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY"
# conditional_put = "redis://localhost:6379" # Only needed for S3-compatible stores without conditional put support
[servers.nfs]
addresses = ["0.0.0.0:2049"]
Basic GCS Configuration
On GCP VM/GKE (recommended):
# /etc/zerofs/zerofs.toml - Minimal config for GCP VMs
[cache]
dir = "/var/cache/zerofs"
disk_size_gb = 10.0
[storage]
url = "gs://my-bucket/zerofs-data"
encryption_password = "your-secure-password"
[servers.nfs]
addresses = ["0.0.0.0:2049"]
# No [gcp] section needed - uses VM's attached service account automatically
With Service Account Key File:
# /etc/zerofs/zerofs.toml
[cache]
dir = "/var/cache/zerofs"
disk_size_gb = 10.0
[storage]
url = "gs://my-bucket/zerofs-data"
encryption_password = "your-secure-password"
[gcp]
service_account = "/path/to/service-account-key.json"
[servers.nfs]
addresses = ["0.0.0.0:2049"]
Or using GOOGLE_APPLICATION_CREDENTIALS:
export GOOGLE_APPLICATION_CREDENTIALS="/path/to/service-account-key.json"
zerofs run -c zerofs.toml
Performance-Tuned Configuration
# /etc/zerofs/zerofs-performance.toml
[cache]
dir = "/nvme/zerofs-cache" # Use fast NVMe storage
disk_size_gb = 100.0
memory_size_gb = 16.0 # Large memory cache
[storage]
url = "s3://my-bucket/zerofs-data"
encryption_password = "very-secure-password-here"
[filesystem]
max_size_gb = 500.0 # Optional: limit filesystem size
compression = "zstd-3" # Optional: "zstd-{1-22}" (default: "zstd-3") or "lz4"
[lsm]
max_concurrent_compactions = 16 # More parallel metadata compaction
flush_interval_secs = 60 # Fewer, larger periodic flushes
[aws]
access_key_id = "your-key"
secret_access_key = "your-secret"
region = "us-east-1"
[servers.nfs]
addresses = ["0.0.0.0:2049"]
[servers.nbd]
addresses = ["0.0.0.0:10809"]
unix_socket = "/tmp/zerofs.nbd.sock" # Unix socket for local clients
S3-Compatible Services
[cache]
dir = "/var/cache/zerofs"
disk_size_gb = 10.0
[storage]
url = "s3://my-bucket/path"
encryption_password = "secure-password"
[aws]
access_key_id = "minioadmin"
secret_access_key = "minioadmin"
endpoint = "https://minio.example.com"
allow_http = "true" # Quoted string "true", not a bare boolean
[servers.nfs]
addresses = ["0.0.0.0:2049"]
Running ZeroFS
With Configuration File
# Start ZeroFS with a config file
zerofs run --config /etc/zerofs/zerofs.toml
# Or use the shorthand
zerofs run -c zerofs.toml
Password Management
To change the encryption password:
# Reads the new password from stdin: type one line and press Enter, or pipe it
zerofs change-password --config zerofs.toml
echo "newpassword" | zerofs change-password -c zerofs.toml
The command reads exactly one line from stdin and re-wraps the stored encryption key with the new password. File data is not re-encrypted.
After changing the password, update the encryption_password field in your configuration file.
Environment Variable Substitution
Configuration values can reference environment variables using $VAR or ${VAR} syntax:
[storage]
url = "s3://my-bucket/data"
encryption_password = "${ZEROFS_PASSWORD}"
[aws]
access_key_id = "${AWS_ACCESS_KEY_ID}"
secret_access_key = "${AWS_SECRET_ACCESS_KEY}"
Substitution applies to these keys only; all other values are taken literally:
url,encryption_password, andstorage_classunder[storage]dirunder[cache]urlandstorage_classunder[wal]unix_socketunder[servers.ninep],[servers.nbd], and[servers.rpc]addressesunder[servers.nfs],[servers.ninep],[servers.nbd],[servers.rpc],[servers.webui], and[prometheus]- every value in
[aws],[azure], and[gcp], including their[wal.*]variants node_id,replication_listen, andpeersunder[replication]
For array values (peers and the addresses lists) each entry is expanded independently, so peers = ["${PEER_A}", "${PEER_B}"] substitutes each element on its own; a single variable is not split into multiple entries. Expansion runs on the whole string before an addresses entry is parsed as a socket address, so the host, the port, or the entire value may come from a variable: addresses = ["${POD_IP}:2049"] and addresses = ["${BIND_ADDR}"] are both valid.
If a referenced variable is unset, the configuration fails to load with Failed to expand environment variable. A literal $ in an expandable value (an encryption password, for example) must be written $$.
This keeps secrets out of configuration files and lets the same file serve several environments.
System Integration
systemd Service
Create /etc/systemd/system/zerofs.service:
[Unit]
Description=ZeroFS S3 Filesystem
After=network-online.target
Wants=network-online.target
[Service]
Type=simple
ExecStart=/usr/local/bin/zerofs run --config /etc/zerofs/zerofs.toml
Restart=always
RestartSec=5
# Optional: Load environment variables for substitution
EnvironmentFile=-/etc/zerofs/zerofs.env
[Install]
WantedBy=multi-user.target
If using environment variable substitution, create /etc/zerofs/zerofs.env:
ZEROFS_PASSWORD=your-secure-password
AWS_ACCESS_KEY_ID=your-key
AWS_SECRET_ACCESS_KEY=your-secret
Docker
# Create config file
cat > zerofs.toml <<EOF
[cache]
dir = "/cache"
disk_size_gb = 10.0
[storage]
url = "s3://bucket/path"
encryption_password = "\${ZEROFS_PASSWORD}"
[aws]
access_key_id = "\${AWS_ACCESS_KEY_ID}"
secret_access_key = "\${AWS_SECRET_ACCESS_KEY}"
[servers.nfs]
addresses = ["0.0.0.0:2049"]
EOF
# Run container
docker run -d \
-e ZEROFS_PASSWORD='secure-password' \
-e AWS_ACCESS_KEY_ID='your-key' \
-e AWS_SECRET_ACCESS_KEY='your-secret' \
-v $(pwd)/zerofs.toml:/config/zerofs.toml:ro \
-v /tmp/cache:/cache \
-p 2049:2049 \
ghcr.io/barre/zerofs:latest run --config /config/zerofs.toml
Configuration Best Practices
- Restrict permissions:
chmod 600 /etc/zerofs/zerofs.toml - Use environment variables for secrets: Keep passwords out of config files
- Version control: Track config files (without secrets) in git
- Use strong passwords: Generate with
openssl rand -base64 32 - Separate environments: Use different config files for dev/staging/prod
Logging Configuration
Set log levels using the RUST_LOG environment variable. With RUST_LOG unset, zerofs run logs at info for all crates; zerofs mount uses info,fuser::reply=off. Logs are written to stderr.
# Override the default filter
export RUST_LOG='zerofs=debug'
# Common log levels:
# - error: Only errors
# - warn: Warnings and errors
# - info: Informational messages (default)
# - debug: Detailed debugging
# - trace: Very detailed tracing
# Run with custom logging
RUST_LOG=zerofs=debug zerofs run -c zerofs.toml