mos_bootstrap

What this is

A self-contained Go-based iPXE bootblock generation service for MyceliumOS. This 2MB single iPXE image or a 70MB full MyceliumOS image creates customized boot artifacts (iPXE scripts, UEFI binaries, ISO images, USB images, LKRN kernels) with configurable parameters including farm ID, kernel URLs, run modes, console targets, and kernel flags.

Features

• Fully Self-Contained: Single 20MB binary with all assets embedded — zero external dependencies
• Auto-Extraction: Automatically extracts documentation, configuration, and build tools on first run
• Smart Caching: Reuses build artifacts with per-target locking for 10x faster subsequent builds
• Web UI: Interactive Alpine.js-powered two-tab interface for artifact generation
• Customizable Kernel URLs: Configure custom kernel download servers and filenames
• Multiple Formats: UEFI binaries, ISO images, USB images, LKRN kernels, iPXE scripts, full MyceliumOS images
• Full MyceliumOS: Hybrid BIOS+EFI bootable USB images with embedded kernel for offline boot (use when iPXE doesn't work with recent NICs or UEFI signing issues)
• HTTPS Boot: Automatically includes Let's Encrypt certificates for secure boot
• Always Documented: README and config template auto-extract with binary

Architecture

Self-Contained Design

mosbootstrap (20MB)
├── Embedded Assets
│   ├── boot.ipxe → Auto-extracts to project root
│   ├── README.md → Auto-extracts to project root
│   ├── config.example.toml → Auto-extracts to project root
│   ├── HTML templates → Loaded in memory
│   ├── Static CSS/JS → Served from memory
│   └── iPXE source tree (4.8MB compressed) → Extracts to build/ipxe/
│
└── Self-Extracting Runtime
    ├── Checks for missing files
    ├── Extracts only what's needed
    ├── Preserves existing customizations
    └── Starts web server

Directory Structure (After First Run)

project/
├── mosbootstrap           # 20MB self-contained binary
├── README.md               # This file (auto-extracted)
├── config.example.toml     # Configuration template (auto-extracted)
├── boot.ipxe              # Boot script template (auto-extracted, customizable)
└── build/
    └── ipxe/              # iPXE source tree (auto-extracted, ~50MB)
        └── src/
            ├── Makefile   # Build system
            ├── config/    # iPXE configuration
            └── ...        # 923 source files

Quick Start

One-Line Deployment

# Download, run, and access web UI
wget https://git.ourworld.tf/geomind_code/mos_bootstrap/releases/latest/mosbootstrap
chmod +x mosbootstrap
./mosbootstrap &
open http://localhost:8080

That's it! The binary automatically extracts all required files on first run.

What Happens on First Run

./mosbootstrap

Auto-Extraction (happens once):

✓ README.md → Documentation (you're reading it!)
✓ config.example.toml → Configuration template
✓ boot.ipxe → Customizable boot script template
✓ build/ipxe/ → Complete iPXE source tree (923 files, ~50MB)

Result:

• Web server starts on http://localhost:8080
• All files ready for customization
• No manual setup required

Installation

Option 1: Download Pre-Built Binary (Recommended)

# Download latest release
wget https://git.ourworld.tf/geomind_code/mos_bootstrap/releases/latest/mosbootstrap

# Make executable
chmod +x mosbootstrap

# Run (auto-extracts everything)
./mosbootstrap

# Access web UI
open http://localhost:8080

Option 2: Build from Source

# Clone repository
git clone https://git.ourworld.tf/geomind_code/mos_bootstrap.git
cd mosbootstrap

# Generate iPXE tarball (only if build/ipxe updated)
go generate ./internal/assets/embed

# Build binary
go build -o mosbootstrap ./cmd/mosbootstrap

# Run
./mosbootstrap

Usage

Web Interface

1. Open http://localhost:8080
2. Choose tab:
   • iPXE Image: Network-based boot (requires internet at boot time)
     • Artifacts are designed for iPXE servers but can also be written to USB sticks
   • Full MyceliumOS: Self-contained offline-bootable image with embedded kernel
     • Use when iPXE doesn't work with recent NICs or UEFI signing issues occur
3. Select artifact type:
   • iPXE Tab: UEFI, iPXE Script, ISO, USB, LKRN
   • Full MyceliumOS Tab: Hybrid BIOS+EFI USB image
4. Configure parameters:
   • Farm ID (required): Your farm identifier
   • Version: prod/test/dev (default: prod)
   • Kernel URL: Base URL for kernel downloads (default: https://mos.grid.tf)
   • Advanced Settings (click to expand):
     • Kernel Path (default: kernel)
     • Kernel Filename (default: mos-latest.efi)
   • Console: Serial/video console target
   • Kernel Parameters: Additional boot parameters
5. Click Generate
6. Download artifact or copy iPXE script

Command Line

# Basic usage
./mosbootstrap

# Custom port
./mosbootstrap --port 3000

# With configuration file
./mosbootstrap --config config.toml

# Override iPXE source location
./mosbootstrap --ipxe-template /custom/ipxe

# Show all options
./mosbootstrap --help

API Usage

See complete API documentation: docs/API.md

Quick Example:

# Generate UEFI binary
curl -X POST http://localhost:8080/generate/uefi \
  -H "Content-Type: application/json" \
  -d '{"farmId":"12345"}' \
  -o boot-12345.efi

# Generate with custom kernel URL
curl -X POST http://localhost:8080/generate/uefi \
  -H "Content-Type: application/json" \
  -d '{
    "farmId":"12345",
    "kernelUrl":"https://custom-mirror.com",
    "kernelFilename":"mos-v3.5.0.efi"
  }' \
  -o boot-12345.efi

Configuration

Environment Variables

# HTTP server port
export PORT=8080

# Generator workspace directory
export GENERATOR_WORK_DIR=/tmp/bootstrap-work

# Override make environment
export GENERATOR_MAKE_ENV="CC=gcc,CFLAGS=-O2"

Configuration File

Create config.toml from the auto-extracted template:

cp config.example.toml config.toml
vim config.toml

[server]
port = "8080"

[generator]
ipxe_template_dir = "build/ipxe"
work_dir = "/tmp/bootstrap-work"
make_env = []

Customizing the Boot Script

The boot script is auto-extracted to boot.ipxe:

# Edit boot script
vim boot.ipxe

# Changes take effect immediately (no rebuild needed)
systemctl restart mosbootstrap

API Reference

POST /generate/{kind}

Generate a boot artifact with custom parameters.

Parameters:

Field	Type	Required	Default	Description
farmId	string	Yes	-	Farm ID
version	string	No	"prod"	MyceliumOS version
kernelUrl	string	No	"https://mos.grid.tf"	Kernel server base URL
kernelFilename	string	No	"mos-latest.efi"	Kernel filename
console	string	No	""	Console device
nomodeset	boolean	No	false	Disable KMS
extra	string	No	""	Extra kernel params

Examples: See docs/api-examples.http for complete request templates.

Full Documentation: docs/API.md

Kernel URL Configuration

Default Configuration

• Base URL: https://mos.grid.tf
• Filename: mos-latest.efi
• Full Path: https://mos.grid.tf/kernel/mos-latest.efi

Custom Kernel Server

{
  "farmId": "12345",
  "kernelUrl": "https://mirror.example.com",
  "kernelFilename": "custom-kernel.efi"
}

Downloads from: https://mirror.example.com/kernel/custom-kernel.efi

Specific Kernel Version

{
  "farmId": "12345",
  "kernelFilename": "mos-v3.5.0-rc2.efi"
}

Downloads from: https://mos.grid.tf/kernel/mos-v3.5.0-rc2.efi

Asset Management

Embedded Assets

All assets are embedded in the 20MB binary:

Asset	Size	Auto-Extract	Purpose
boot.ipxe	~3KB	Yes → root	Boot script template
README.md	~12KB	Yes → root	Documentation
config.example.toml	~2.4KB	Yes → root	Config template
HTML templates	~17KB	No (memory)	Web UI rendering
Static CSS/JS	~1KB	No (memory)	Web UI styling
iPXE source	4.8MB	Yes → build/ipxe/	Binary builds

Auto-Extraction Behavior

First Run (files missing):

./mosbootstrap
# Extracts: boot.ipxe, README.md, config.example.toml, build/ipxe/
# Logs: "extracted successfully"

Subsequent Runs (files exist):

./mosbootstrap
# Uses existing files
# Logs: "found at <path>"
# Starts instantly

Customization Preserved:

# Your edits are NEVER overwritten
vim boot.ipxe  # Make custom changes
./mosbootstrap  # Uses your custom boot.ipxe

Performance

Binary Size

• Total: 20MB
• Code: ~15MB
• Embedded Assets: ~5MB (4.8MB iPXE + 35KB other)

Startup Time

• First run: ~3 seconds (includes file extraction)
• Subsequent: <100ms (no extraction)

Build Performance

Artifact	First Build	Cached Build
iPXE script	<1 second	<1 second
UEFI binary	30-90 seconds	5-10 seconds
ISO image	30-90 seconds	5-10 seconds
USB image	30-90 seconds	5-10 seconds
LKRN kernel	30-90 seconds	5-10 seconds

Concurrent Builds

• ✅ Safe: Per-target workspace locking
• ✅ Efficient: Different targets build in parallel
• ✅ Cached: Reuses compiled objects across requests

Customization

Boot Script Template

Edit the auto-extracted boot.ipxe:

vim boot.ipxe

Available Template Variables:

• {{release}} - MyceliumOS release name
• {{farmerid}} - Farm ID from request
• {{parameters}} - All kernel parameters
• {{kernel}} - Full kernel download URL
• {{cmdline}} - Kernel command line

Example Customization:

#!ipxe
echo Custom boot screen
echo Farm: {{farmerid}}
echo Release: {{release}}

imgfetch {{kernel}} {{cmdline}}
imgexec

Kernel URL Configuration

Web UI: Enter custom URL in "Kernel URL Base" field

API: Include in request:

{
  "farmId": "12345",
  "kernelUrl": "https://your-mirror.com",
  "kernelFilename": "custom-kernel.efi"
}

Result: Downloads from https://your-mirror.com/kernel/custom-kernel.efi

File Naming Convention

All artifacts follow consistent naming:

Type	Format	Example
UEFI	boot-{farmid}.efi	boot-12345.efi
ISO	boot-{farmid}.iso	boot-12345.iso
USB	boot-{farmid}.img	boot-12345.img
LKRN	boot-{farmid}.lkrn	boot-12345.lkrn
iPXE	bootstrap-{version}.ipxe	bootstrap-prod.ipxe
Full OS	boot-{farmid}-full.img	boot-12345-full.img

Prerequisites

For Running (Production)

• iPXE Images: None! Binary is completely self-contained
• Full MyceliumOS Images: GRUB tools and partition utilities required:
  • parted - Partition management
  • grub-pc-bin - GRUB for BIOS boot
  • grub-efi-amd64-bin - GRUB for EFI boot
  • dosfstools - FAT32 formatting
  • mtools - FAT filesystem operations

For Building (Development)

• Go 1.22+
• iPXE build dependencies: build-essential, make, gcc, liblzma-dev, xorriso, isolinux

Installing Runtime Dependencies

Ubuntu/Debian (for Full MyceliumOS generation):

sudo apt-get update
sudo apt-get install -y parted grub-pc-bin grub-efi-amd64-bin dosfstools mtools

Alpine Linux (for Full MyceliumOS generation):

apk add --no-cache parted grub grub-efi dosfstools mtools

Installing Build Dependencies

Ubuntu/Debian:

sudo apt-get update
sudo apt-get install -y build-essential make gcc liblzma-dev xorriso isolinux

Alpine Linux:

apk add --no-cache build-base make gcc musl-dev xz-dev xorriso syslinux

Red Hat/CentOS:

sudo yum groupinstall "Development Tools"
sudo yum install syslinux genisoimage

macOS:

brew install make gcc

Security Considerations

Production Deployment

Recommended:

1. Rate Limiting: Limit requests per IP
2. Input Validation: Strictly validate all parameters
3. URL Whitelist: Whitelist allowed kernel URLs
4. HTTPS: Use HTTPS with valid certificates
5. Authentication: Add API key or OAuth
6. Logging: Log all generation requests
7. Monitoring: Monitor for abuse patterns

Example NGINX Reverse Proxy:

upstream mosbootstrap {
    server localhost:8080;
}

server {
    listen 443 ssl http2;
    server_name bootstrap.example.com;

    ssl_certificate /etc/ssl/certs/bootstrap.crt;
    ssl_certificate_key /etc/ssl/private/bootstrap.key;

    location / {
        proxy_pass http://mosbootstrap;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;

        # Rate limiting
        limit_req zone=api burst=10 nodelay;
    }
}

Troubleshooting

Build Issues

Problem: First build is slow (30-90 seconds)

✓ Expected behavior - initial make compiles all iPXE objects
✓ Subsequent builds reuse cache (5-10 seconds)

Problem: "iPXE source tree not found"

# Source auto-extracts on first run
# If extraction failed, check permissions:
ls -la build/ipxe/src/Makefile

# Force re-extraction:
rm -rf build/ipxe
./mosbootstrap

Problem: Certificate download errors

# Generator auto-downloads Let's Encrypt certs
# If blocked, manually place in build/ipxe/src/:
# - isrgrootx1.pem
# - lets-encrypt-r3-cross-signed.pem
# - lets-encrypt-r3.pem

Runtime Issues

Problem: Port already in use

# Use different port
./mosbootstrap --port 8081

# Or set environment
export PORT=8081
./mosbootstrap

Problem: "failed to extract tarball"

# Check write permissions
chmod 755 .
mkdir -p build

# Check disk space
df -h .

Problem: Slow generation requests

# First request per artifact type is slower (compiles iPXE)
# Subsequent requests are fast (reuse workspace)

# Check logs for actual build time:
tail -f /var/log/mosbootstrap.log

Boot Issues

Problem: iPXE not working on recent hardware

# Recent NICs or UEFI firmware may have issues with iPXE
# Solution: Use the "Full MyceliumOS" tab to generate a bootable image with embedded kernel

# The Full MyceliumOS option provides:
# - Offline boot capability (no network required)
# - Better compatibility with recent NICs
# - Works around UEFI signing issues with iPXE binaries

Problem: Need to boot without network connectivity

# iPXE artifacts require network at boot time
# Solution: Use "Full MyceliumOS" tab for offline-bootable USB images
# This embeds the kernel directly in the bootable image

Customization Issues

Problem: Custom boot.ipxe not being used

# Verify file exists and location is correct
ls -la boot.ipxe  # Should be in working directory

# Check logs for which file is loaded
./mosbootstrap 2>&1 | grep "boot template"

Problem: Changes to boot.ipxe not taking effect

# Restart the service
systemctl restart mosbootstrap

# Or if running in foreground
# Stop (Ctrl+C) and restart
./mosbootstrap

What this repository contains

Project Structure

mosbootstrap/
├── cmd/mosbootstrap/          # Main application
├── internal/
│   ├── assets/
│   │   ├── embed/              # All embeddable assets
│   │   │   ├── boot.ipxe       # Boot template
│   │   │   ├── README.md       # This file
│   │   │   ├── config.example.toml
│   │   │   ├── templates/      # HTML templates
│   │   │   ├── static/         # CSS/JS
│   │   │   ├── ipxe_src.tgz    # Compressed iPXE (4.8MB)
│   │   │   ├── assets.go       # Embed directives
│   │   │   └── generate.go     # Tarball generation
│   │   └── provider.go         # Auto-extraction logic
│   ├── generator/              # Build orchestration
│   │   ├── generator.go
│   │   ├── types.go
│   │   └── extra.go
│   └── handlers/               # HTTP handlers
│       └── landing.go
├── docs/
│   ├── API.md                  # Complete API documentation
│   ├── api-examples.http       # REST client templates
│   └── adr/                    # Architecture decision records
├── build/
│   └── ipxe/                   # iPXE source (auto-extracted)
├── README.md                   # This file (auto-extracted)
└── config.example.toml         # Config template (auto-extracted)

Binaries

• mosbootstrap — Self-contained 20MB binary with all assets embedded

APIs

• REST API for artifact generation (POST /generate/{kind})
• Interactive Web UI served on port 8080
• Complete API documentation in docs/API.md

Key Directories and Files

Path	Purpose
cmd/mosbootstrap/	Main Go application entrypoint
internal/assets/	Embedded asset logic and auto-extraction
internal/generator/	Build orchestration for iPXE and boot artifacts
internal/handlers/	HTTP handlers for web UI and API
docs/API.md	Complete API reference
docs/api-examples.http	Ready-to-use REST client templates
docs/adr/	Architecture decision records

Development

Building

# Regular build
go build -o mosbootstrap ./cmd/mosbootstrap

# With updated iPXE source
go generate ./internal/assets/embed
go build -o mosbootstrap ./cmd/mosbootstrap

# Run tests
go test ./...

# Static analysis
go vet ./...

Updating Embedded Assets

Update Boot Template:

vim internal/assets/embed/boot.ipxe
go build -o mosbootstrap ./cmd/mosbootstrap

Update HTML Templates:

vim internal/assets/embed/templates/landing.tmpl
go build -o mosbootstrap ./cmd/mosbootstrap

Update iPXE Source:

cd build/ipxe
git pull  # or manual updates
cd ../..
go generate ./internal/assets/embed
go build -o mosbootstrap ./cmd/mosbootstrap

Deployment

Systemd Service

[Unit]
Description=MyceliumOS Bootstrap Service
After=network.target

[Service]
Type=simple
User=bootstrap
WorkingDirectory=/opt/mosbootstrap
ExecStart=/opt/mosbootstrap/mosbootstrap
Restart=always
RestartSec=10

# Environment variables (optional)
Environment="PORT=8080"
Environment="GENERATOR_WORK_DIR=/var/cache/mosbootstrap"

[Install]
WantedBy=multi-user.target

Docker

FROM alpine:latest

# Install iPXE build dependencies
RUN apk add --no-cache build-base make gcc musl-dev xz-dev xorriso syslinux

# Copy binary
COPY mosbootstrap /app/mosbootstrap
WORKDIR /app

# Expose port
EXPOSE 8080

# Run binary (auto-extracts on first run)
CMD ["./mosbootstrap"]

Kubernetes

apiVersion: apps/v1
kind: Deployment
metadata:
  name: mosbootstrap
spec:
  replicas: 3
  selector:
    matchLabels:
      app: mosbootstrap
  template:
    metadata:
      labels:
        app: mosbootstrap
    spec:
      containers:
        - name: mosbootstrap
          image: mosbootstrap:latest
          ports:
            - containerPort: 8080
          env:
            - name: PORT
              value: "8080"
            - name: GENERATOR_WORK_DIR
              value: "/tmp/bootstrap-work"
          resources:
            requests:
              memory: "256Mi"
              cpu: "500m"
            limits:
              memory: "2Gi"
              cpu: "2000m"

Role in the stack

mos_bootstrap is the entry point for booting MyceliumOS nodes. It generates the boot artifacts that nodes use to:

1. Network boot via iPXE — Download the MyceliumOS kernel from a configured URL at boot time
2. Boot offline — Use full MyceliumOS images with an embedded kernel when network boot is not available

The service is typically deployed as a web-accessible bootstrap generator. Farm operators use it to create boot media for their nodes. The generated artifacts reference the MyceliumOS kernel distribution infrastructure and can be customized per farm.

It relates to:

• MyceliumOS — The operating system whose kernel and boot artifacts are generated
• Mycelium — The overlay network that MyceliumOS nodes join after successful boot
• ThreeFold Grid — The decentralized infrastructure platform where this technology was first deployed

Relation to ThreeFold

This technology is used within the ThreeFold ecosystem and was first deployed on the ThreeFold Grid. The component itself is designed as reusable infrastructure technology and should be understood by its technical function first, independent of any specific deployment.

Ownership

This repository is owned and maintained by TF-Tech NV, a Belgian company responsible for the development and maintenance of this technology.

License

This project is licensed under the Apache License 2.0 — see the LICENSE file for details.