fbox is an easy to use and maintain distributed filesystem written in Go using Reed Solomon Erasure with FUSE and S3 interfaces and a modern Web UI https://files.mills.io/
You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
James Mills 407f1a6228
Upgrade to bitcask v1.0.0
10 months ago
.chglog Fix import paths 11 months ago
.github Refactor shard mapping (#4) 11 months ago
blob Fix import paths 11 months ago
data Add /status and /cluster endpoints (#15) 11 months ago
docs Update screenshot 11 months ago
store Fix bad log.Error call 10 months ago
tools Add some project infra borrowed from other projects 11 months ago
ui Cursor and Close (#14) 11 months ago
.dockerignore Add Dockerfile to build Docker images 11 months ago
.drone.yml Add CI triggers 10 months ago
.gitignore Refactor shard mapping (#4) 11 months ago
.goreleaser.yml Add some project infra borrowed from other projects 11 months ago
AUTHORS Updated README, Added Roadmap, AUTHORS and updated Contributing guideline 11 months ago
CHANGELOG.md Fix import paths 11 months ago
CODE_OF_CONDUCT.md Add contributing guidelines and code of conduct 11 months ago
CONTRIBUTING.md Updated README, Added Roadmap, AUTHORS and updated Contributing guideline 11 months ago
Dockerfile Use local images for Docker builds 10 months ago
LICENSE Add a README and LICENSE 11 months ago
Makefile Fix embedding of version and commit hash 11 months ago
README.md Fix Homebrew isntructions 10 months ago
Roadmap.md Updated README, Added Roadmap, AUTHORS and updated Contributing guideline 11 months ago
_config.yml Set theme jekyll-theme-architect 11 months ago
commands.go Add endpoint for deleting files (shards and metdata) as well as a sub-command (#8) 11 months ago
docker-compose.yml Add a working docker-compose.yml 11 months ago
fbox.yml Auto update the deploy docker stack services 11 months ago
go.mod Upgrade to bitcask v1.0.0 10 months ago
go.sum Upgrade to bitcask v1.0.0 10 months ago
handlers.go Fix import paths 11 months ago
main.go Use git.mills.io/prologic/useragent fork 10 months ago
metadata.go Fix import paths 11 months ago
node.go Add endpoint for deleting files (shards and metdata) as well as a sub-command (#8) 11 months ago
start-cluster.sh Fixed dev UI dev workflow 11 months ago
test.sh Optimze getShards() 11 months ago
utils.go Optimze getShards() 11 months ago
version.go Initial Commit 12 months ago


fbox - Files in a Box

Build Status Go Report Card GoDoc

fbox is a distributed file system written in Go that has the the following features:

Current Features:

  • Single portable binary
  • Simple to setup and maintain
  • A Web Interface with Drag 'n Drop
  • Data redundancy with erasure coding and sharding

Planned Features:

  • POSIX compatible FUSE interface
  • S3 Object Storage interface
  • Docker Volume Driver

There is also a publicly (freely) available demo instance available at:

NOTE: I, James Mills, run this instance on pretty cheap hardware on a limited budget. Please use it fairly so everyone can enjoy using it equally!

Sponsor this project to support the development of new features, improving existings ones and fix bugs!


Table of Contents

Created by gh-md-toc


fbox was written primarily and firstly as en education exercise in distributed file systems and a lot of time add effort went into understanding how many distributed file systems in the wild are designed.

Most other distributed file systems are either complex and hard to set up and maintain or come with really expensive license feeds. Therefore fbox is also an attempt at designing and implementing a distributed file system that is free and open source with the reliability, scale and durability of other distributed file systems.

Getting Started

Install from Releases

You can install fbox by simply downloading the latest version from the Release page for your platform and placing the binary in your $PATH.

Install from Homebrew

On macOS you can install fbox using Homebrew:

brew tap prologic/fbox https://git.mills.io/prologic/homebrew-fbox.git
brew install fbox

Install from Source

To install fbox from source you can run go get directly if you have a Go environment setup:

go get git.mills.io/prologic/fbox

NOTE: Be sure to have $GOBIN (if not empty) or your $GOPATH/bin in your $PATH. See Compile and install packages and dependencies

Or grab the source code and build:

git clone https://git.mills.io/prologic/fbox.git
cd fbox
make build

And optionally run make install to place the binary fbox in your $GOBIN or $GOPATH/bin (again see note above).


Prepare storage

mkdir data1 data2 data3

Start a master node

fbox -a -b :8000 -d ./data1

Start some data nodes

fbox -a -b :8001 -d ./data2 -m
fbox -a -b :8002 -d ./data3 -m

This will set up a 3-node cluster on your local machine (for testing only).

You can open the Web UI by navigating to:

You can also use the command-line client fbox itself; See usage:

$ ./fbox --help
Usage: fbox [options] [command [arguments]]

fbox is a simple distributed file system...

Valid commands:
 - cat <name> -- Downloads the given file given by <name> to stdout
 - put <name> -- Upload the given file given by <name>

Valid options:
  -a, --advertise-addr string   [interface]:port to advertise
  -b, --bind string             [interface]:port to bind to (default "")
  -d, --data string             path to store data in (default "./data")
  -s, --data-shards int         no. of data shards (default 3)
  -D, --debug                   enable debug logging
  -m, --master string           address:port of master to join (empty if master)
  -p, --parity-shards int       no. of parity shards (default 1)
  -u, --store string            blob store uri (default "disk://data")
  -V, --version                 display version information and exit
pflag: help requested

For example to store a file:

echo "Hello World" > hello.txt
fbox -m put hello.txt

And to retrieve the file:

fbox -m cat hello.txt

Production Deployments

Docker Swarm

You can deploy fbox to a Docker Swarm cluster by utilising the provided fbox.yml Docker Stack. This also depends on and uses the Traefik ingress load balancer so you must also have that configured and running in your cluster appropriately.

export DOMAIN=files.yourdomain.tld
docker stack deploy -c fbox.yml fbox

In the News


Support the ongoing development of fbox!



Interested in contributing to this project? You are welcome! Here are some ways you can contribute:

  • File an Issue -- For a bug, or interesting idea you have for a new feature or just general questions.
  • Submit a Pull-Request or two! We welcome all PR(s) that improve the project!

Please see the Contributing Guidelines and checkout the Roadmap for inspiration, ideas and the general vision and goals.


Thank you to all those that have contributed to this project, battle-tested it, used it in their own projects or products, fixed bugs, improved performance and even fix tiny typos in documentation! Thank you and keep contributing!

You can find an AUTHORS file where we keep a list of contributors to the project. If you contribute a PR please consider adding your name there. There is also Github's own Contributors statistics.

Stargazers over time

Stargazers over time

Attributions and Prior Art

  • nicolagi/dino -- Some code was borrowed and much inspiration for metadata and FUSE interface design with performance in mind.
  • seaweedfs -- Some inspiration taken from design and access interfaces (e.g: S3).
  • GlusterFS, Portworx, Storidge, StorageOS, MooseFS -- All distributed filesystems evalauted, trialed in anger and some used to drive design goals.


fbox is licensed under the terms of the MIT License