mos_storage

What this is

One-shot disk provisioning utility intended for initramfs. It discovers eligible disks, plans a GPT layout based on a chosen topology, creates filesystems, mounts them under a predictable scheme, and emits a machine-readable report. Safe-by-default with a non-destructive preview mode.

Status: apply mode implemented. Partition application (sgdisk), filesystem creation (vfat/btrfs/bcachefs), mount scheme with subvolumes, and optional fstab writing are available. Preview mode remains supported.

Features

• Topology auto-selection with built-in defaults; optional kernel cmdline override via mosstorage.topology= (see ADR-0002)
• Non-destructive preview: --show/--report outputs JSON summary (disks, partition plan, filesystems, planned mountpoints)
• Safe discovery: excludes removable media by default (USB sticks) unless explicitly allowed
• No external YAML configuration; defaults-only per ADR-0002 (sane built-ins, topology may be overridden by kernel cmdline)

Requirements

• Linux with /proc and /sys mounted (initramfs friendly)
• External tools discovered at runtime:
  • blkid (for probing UUIDs and signatures)
  • sgdisk (for GPT application)
  • mkfs.vfat, mkfs.btrfs, bcachefs (for formatting)
  • udevadm (optional; for settle after partitioning)
• Tracing/logging to stderr by default; optional file at /run/mosstorage/mosstorage.log

Install and Build

• System Rust toolchain:

  cargo build --release
  

Binary is target/release/mosstorage.

CLI Usage

• Preview (non-destructive):
  • --show — Print JSON summary to stdout
  • --report PATH — Write JSON summary to a file
• Discovery policy:
  • --allow-removable — Include removable devices (USB) during discovery
• Tracing/logging:
  • -l, --log-level LEVEL — error|warn|info|debug (default: info)
  • -L, --log-to-file — Also write logs to /run/mosstorage/mosstorage.log
• Other:
  • -s, --fstab — Enable writing /etc/fstab entries (when mounts are applied)
  • -a, --apply — Perform partitioning, filesystem creation, and mounts (destructive)
  • -f, --force — Present but not implemented (returns an error)

Deprecated (ignored with warning; see ADR-0002):

• -t, --topology VALUE — Ignored; use kernel cmdline mosstorage.topology= instead
• -c, --config PATH — Ignored; external YAML configuration is not used at runtime

Examples

• Single disk plan with debug logs (defaults to btrfs_single automatically):

  sudo ./mosstorage --show -l debug
  

• Two-disk plan (defaults to dual_independent automatically), write summary:

  sudo ./mosstorage --show --report /run/mosstorage/plan.json -l debug -L
  

• Include removable devices for lab scenarios:

  sudo ./mosstorage --show --allow-removable -l debug
  

• Quiet plan to file:

  sudo ./mosstorage --report /run/mosstorage/plan.json
  

• Apply single-disk plan (DESTRUCTIVE; wipes target disk; defaults select topology automatically):

  sudo ./mosstorage --apply
  

Kernel Cmdline Override (at boot)

• To force a topology, pass one of:

  mosstorage.topology=btrfs-single | bcachefs-single | dual-independent | btrfs-raid1 | ssd-hdd-bcachefs | bcachefs-2copy
  

• The override affects only topology; all other settings use sane built-in defaults.

Preview JSON Shape (examples)

1. Already provisioned (idempotency success):

{
  "version": "v1",
  "timestamp": "2025-09-26T12:34:56Z",
  "status": "already_provisioned",
  "state": { ... full StateReport JSON ... }
}

2. Planned (not yet provisioned):

{
  "version": "v1",
  "timestamp": "2025-09-26T12:34:56Z",
  "status": "planned",
  "topology": "btrfs_raid1",
  "alignment_mib": 1,
  "require_empty_disks": true,
  "disks": [
    { "path": "/dev/nvme0n1", "size_bytes": 1000204886016, "rotational": false, "model": "...", "serial": "..." },
    { "path": "/dev/nvme1n1", "size_bytes": 1000204886016, "rotational": false, "model": "...", "serial": "..." }
  ],
  "partition_plan": [
    {
      "disk": "/dev/nvme0n1",
      "parts": [
        { "role": "bios_boot", "size_mib": 1, "gpt_name": "mosboot" },
        { "role": "esp", "size_mib": 512, "gpt_name": "mosboot" },
        { "role": "data", "size_mib": null, "gpt_name": "mosdata" }
      ]
    },
    {
      "disk": "/dev/nvme1n1",
      "parts": [
        { "role": "data", "size_mib": null, "gpt_name": "mosdata" }
      ]
    }
  ],
  "filesystems_planned": [
    { "kind": "vfat", "from_roles": ["esp"], "label": "MOSBOOT" },
    { "kind": "btrfs", "from_roles": ["data"], "devices_planned": 2, "label": "MOSDATA" }
  ],
  "mount": {
    "scheme": "per_uuid",
    "base_dir": "/var/cache",
    "fstab_enabled": false,
    "root_mount_template": "/var/mounts/{UUID}",
    "final_targets": ["/var/cache/system", "/var/cache/etc", "/var/cache/modules", "/var/cache/vm-meta"]
  }
}

Defaults and Policies

• Device selection defaults (see src/config/loader.rs):
  • include_patterns: ^/dev/sd\w+$, ^/dev/nvme\w+n\d+$, ^/dev/vd\w+$
  • exclude_patterns: ^/dev/ram\d+$, ^/dev/zram\d+$, ^/dev/loop\d+$, ^/dev/fd\d+$
  • allow_removable: false (USB excluded unless --allow-removable)
  • min_size_gib: 10
• Partitioning defaults:
  • alignment_mib: 1
  • bios_boot: enabled (1 MiB), gpt_name: mosboot
  • esp: 512 MiB, label: MOSBOOT, gpt_name: mosboot
  • data: gpt_name: mosdata
  • cache: gpt_name: moscache (only used in SSD+HDD topology)
• Filesystem defaults:
  • vfat (ESP) label: MOSBOOT
  • btrfs (data) label: MOSDATA
  • bcachefs (data/cache) label: MOSDATA
• Mount scheme:
  • Root mounts (runtime only): each data filesystem is mounted at /var/mounts/{UUID}
    • btrfs root options: rw,noatime,subvolid=5
    • bcachefs root options: rw,noatime
  • Subvolume mounts (from the primary data filesystem only) to final targets:
    • /var/cache/system
    • /var/cache/etc
    • /var/cache/modules
    • /var/cache/vm-meta
  • Subvolume mount options:
    • btrfs: -o rw,noatime,subvol={name}
    • bcachefs: -o rw,noatime,X-mount.subdir={name}
  • /etc/fstab generation is disabled by default; when enabled, only the four subvolume mounts are written (UUID= sources, deterministic order)

Tracing and Logs

• stderr logging level controlled by -l/--log-level (info by default)
• optional file logging with -L/--log-to-file at /run/mosstorage/mosstorage.log
• Debug spans provide insight into discovery, idempotency, planning, and tool invocations

Extending and Execution Plan

• Near-term:
  • Implement sgdisk partition apply and udev settle
  • Pass btrfs raid profile flags -m/-d from config (raid1, none) to mkfs
  • Create per-UUID mount targets; persist fstab when enabled
  • Atomic report write via tempfile+rename
• Longer-term:
  • Dry-run mode with richer diffs
  • Richer schema for disks/partitions/filesystems in the report
  • Additional filesystem and tuning support

Safety Model

• Preview mode never mutates block devices
• Emptiness checks guard destructive actions (skipped only in preview)
• Reserved labels and GPT names validated to reduce risk

Troubleshooting

• No eligible disks found: adjust include/exclude patterns or use --allow-removable for test media
• Removable media excluded: either add --allow-removable or exclude it explicitly via config patterns
• blkid missing: install util-linux; preview can still plan but cannot capture UUIDs for mkfs outputs
• bcachefs command missing: install bcachefs-tools or skip the SSD+HDD topology

What this repository contains

Binary

• mosstorage — The disk provisioning utility

Key Modules

• CLI and entrypoint:
  • src/cli/args.rs
  • src/main.rs
• Orchestration and preview JSON:
  • src/orchestrator/run.rs
• Configuration loader, overlays, defaults:
  • src/config/loader.rs
• Device discovery (sysfs + regex filters, removable policy):
  • src/device/discovery.rs
• Idempotency detection and emptiness checks:
  • src/idempotency/mod.rs
• Partition planning (topology-aware):
  • src/partition/plan.rs
• Filesystem planning/creation and mkfs integration:
  • src/fs/plan.rs
• Mount planning and application:
  • src/mount/ops.rs

Project Structure

mos_storage/
├── Cargo.toml
├── Cargo.lock
├── src/
│   ├── main.rs
│   ├── cli/
│   ├── config/
│   ├── device/
│   ├── fs/
│   ├── idempotency/
│   ├── mount/
│   ├── orchestrator/
│   └── partition/
├── docs/
└── TODO.md

Role in the stack

mos_storage runs inside the MyceliumOS initramfs to provision persistent storage before the root filesystem is mounted. It is responsible for discovering disks, planning layouts, creating filesystems, and mounting them so that subsequent boot services have writable storage available.

It relates to:

• my_init — Calls mos_storage as a boot-time oneshot service
• MyceliumOS — The operating system whose storage is provisioned
• mos_volmgr — The runtime volume manager that takes over after boot to manage storage dynamically
• ThreeFold Grid — The decentralized infrastructure platform where this technology was first deployed

Relation to ThreeFold

This technology is used within the ThreeFold ecosystem and was first deployed on the ThreeFold Grid. The component itself is designed as reusable infrastructure technology and should be understood by its technical function first, independent of any specific deployment.

Ownership

This repository is owned and maintained by TF-Tech NV, a Belgian company responsible for the development and maintenance of this technology.

License

This project is licensed under the Apache License 2.0 — see the LICENSE file for details.