myfs

A modern filesystem server with deduplication and list-based management capabilities. Previously known as RFS (Redundant File System), myfs provides a unified interface for managing container images with support for multiple storage backends including S3 and local filesystems.

Overview

myfs is a Rust-based filesystem server that handles:

• File Deduplication: Efficient storage by detecting and managing duplicate files
• Image Lists: Organize and track image layers and metadata
• Multiple Storage Backends: Support for S3, local filesystems, SSH, and WebDAV
• Server Component: HTTP server that maps image operations to storage backends
• Built-in Metadata Store: Integrated metadata management for images and layers

Architecture

The project consists of three main components:

1. Server Component

The HTTP server that provides APIs for image operations and manages the connection between clients and storage backends (S3 or local filesystem).

2. Client Library

Rust-based client library for interacting with the image server and managing image operations.

3. Storage Backends

Pluggable storage backends supporting:

• Local filesystem
• S3-compatible object storage

Features

Mount Operations

• Mount image layers from various sources
• Support for multiple mount types (Local, SSH, S3, WebDAV)
• Mount options configuration
• List active mounts
• Unmount individual or all mounts

Packing and Unpacking

• Pack directories into compressed image layers
• Unpack image layers to directories
• Verify image integrity
• List image contents

Deduplication

• Automatic detection of duplicate files
• Efficient storage through content-based deduplication
• Metadata tracking for duplicate files

Building

Prerequisites

• Rust 1.92
• Linux (primary supported platform)

Linux Build

# Build the project in release mode with binary features
cargo build --release --features build-binary

# The binary will be available at:
./target/release/myfs

Development Build

# Build in debug mode (includes debug symbols)
cargo build --features build-binary

# Run tests
cargo test

Usage

As a Library

Add to your Cargo.toml:

[dependencies]
myfs = { path = "." }

Then use in your code:

use myfs::{MyImageBuilder, MountType};

// Create a mount
let builder = MyImageBuilder::new(
    "s3://bucket/image",
    "/mnt/image",
    MountType::S3
);

As a Server

# Start the server (default port 8080)
./myfs server

# The server will provide REST APIs for image operations

Command Line Interface

The project provides command-line tools for common operations:

# Pack a directory
myfs pack <directory> <output> --store-spec <spec>

# Unpack an image
myfs unpack <input> <output>

# List image contents
myfs list <image-path>

# Verify image integrity
myfs verify <image-path>

Configuration

Images are configured using StoreSpec which defines:

• Storage type (local, S3, SSH, WebDAV)
• Location and credentials
• Access permissions and options

Future Roadmap

• Integrate RFS codebase from GitHub
• Performance optimizations for large images
• Enhanced deduplication algorithms