my_fs

What this is

A flist-based filesystem toolkit and server with deduplicated, content-addressed block storage.

MyFS packages directories and container images into FungiList (flist) metadata, stores blocks in pluggable backends (local, ZDB, S3, HTTP, server), and serves content over HTTP or a read-only FUSE mount.

Features:

• Flist metadata format: SQLite-backed metadata for files, directories, and block references
• Deduplicated, content-addressed storage: block-based storage with encryption and compression
• Pluggable stores: local directory, ZDB, S3, HTTP, and server-backed stores
• Cache + FUSE mount: on-demand file access with local caching
• Server API + web UI: manage blocks, files, flists, and websites
• Client utilities: pack/unpack, upload/download, sync, and tracking tools
• Container conversion: build flists from container images

What this repository contains

Crate	Binary	Role
myfs-core	—	Core library: fungi, stores, block/cache, server API client
myfs-lib	myfs	CLI: pack, unpack, mount, upload, download, sync, inspect
myfs-hub	myfs-hub	HTTP server: REST API, block/file/flist/website serving
myfs-tool	myfs-tool	Tooling: container conversion, sync, advanced ops
myfs-test-utils	—	Shared test helpers

Architecture:

• Metadata (fungi): flist metadata read/write and traversal
• Store layer: router abstraction over multiple backends
• Block layer: encryption + compression wrapping stores
• Cache: local block cache for reads
• FUSE filesystem: read-only mount from flists
• Server: Axum REST API over Unix socket

CLI usage:

# Pack a directory into a flist (and upload blocks)
myfs pack --meta ./rootfs.fl --store-spec <store-url> ./rootfs

# Unpack a flist
myfs unpack --meta ./rootfs.fl ./output

# Mount a flist (read-only FUSE)
myfs mount --meta ./rootfs.fl ./mnt

# Inspect flist contents
myfs flist tree <flist-path-or-hash>
myfs flist inspect <flist-path-or-hash>

# Convert a container image to flist
myfs-tool container --image-name docker.io/library/nginx:latest --store <store-url>

# Upload or download files
myfs upload ./file.txt --server http://localhost:8080 --token <token>
myfs download <hash> --output ./file.txt --server http://localhost:8080

# Sync between servers
myfs-tool sync --source http://localhost:8080 --destination http://localhost:8081 --token <token>

Server endpoints:

• Health: GET /health
• Service info: GET /.well-known/heroservice.json
• OpenAPI docs: GET /swagger-ui
• Flist management: POST /api/v1/fl, GET /api/v1/fl, GET /api/v1/fl/preview/:path
• Block storage: POST /api/v1/block, GET /api/v1/block/:hash, HEAD /api/v1/block/:hash
• File storage: POST /api/v1/file, GET /api/v1/file/:hash
• Auth: POST /api/v1/signin

Running:

All lifecycle management uses Nu shell service scripts only:

service myfs start --update --reset
service myfs stop
service myfs status

No Makefiles or bash scripts — Nu shell scripts are the only supported way to build and manage this service.

Building from source:

Prerequisites: Rust 1.92+, Linux.

service myfs install
# or with update + release build:
service myfs install --update --release

FUSE mount requirements:

# Ubuntu/Debian
sudo apt install fuse3

# Fedora/RHEL
sudo dnf install fuse3

Enable user_allow_other in /etc/fuse.conf:

user_allow_other

Configuration:

The server reads a TOML config file. Pass it with:

myfs-hub serve --config-path ./server.toml

Key fields: host, port, store_url, flist_dir, sqlite_path, jwt_secret, users.

Debug builds — stack size:

Debug builds require a larger stack due to async + SQLx overhead:

ulimit -s unlimited

Production release builds do not have this requirement.

Role in the stack

my_fs provides filesystem image packaging, storage, and serving for the Mycelium and Hero stack. It is used to build and distribute root filesystems (flists) for VMs and containers, deduplicating blocks across images to save storage and bandwidth.

The server (myfs-hub) listens on a Unix socket at ~/hero/var/sockets/myfs/rpc.sock.

Relation to ThreeFold

This technology is used within the ThreeFold ecosystem and was first deployed on the ThreeFold Grid. The component itself is designed as reusable infrastructure technology and should be understood by its technical function first, independent of any specific deployment.

Ownership

This repository is owned and maintained by TF-Tech NV, a Belgian company responsible for the development and maintenance of this technology.

License

This project is licensed under the Apache License 2.0 — see the LICENSE file for details.