SeaweedFS is a distributed file and object store written in Go, inspired by Facebook’s Haystack paper. It is designed to store and serve large numbers of small files efficiently by packing them into append-only volume files and locating them through compact in-memory indexes instead of a traditional inode tree.
The cluster has three roles. Master servers track volume-to-server mapping and keep the topology. Volume servers store the actual data in fixed-size volumes (typically 30 GB) replicated according to a configurable replication policy across racks or data centers. Filer servers provide a POSIX-like metadata layer backed by a pluggable database (LevelDB, BoltDB, SQLite, MySQL, Postgres, Redis, Cassandra, TiKV, or etcd) so that directories and permissions sit on top of the volume store. On top of that, SeaweedFS exposes an HTTP API for raw blob access, an S3-compatible gateway, WebDAV, FUSE mounts, a HDFS adapter, and a Kubernetes CSI driver.
Because data is append-only and reads go straight from an in-memory needle map to a byte offset within a volume file, SeaweedFS handles billions of small files with tight latency budgets, which is its differentiator against Ceph, MinIO, and Longhorn. It also supports erasure coding for cold data, tiered storage to S3, and cross-cluster replication. The project is Apache-2.0 licensed.