Create install script for Hero Compute and required binaries #63

Open
opened 2026-04-06 10:09:23 +00:00 by mahmoud · 4 comments
Owner

Description

A script to install all required binaries from release pages and move them to the appropriate path. Think of it like:

curl --proto '=https' --tlsv1.2 -sSf https://forge.ourworld.tf/lhumina_code/hero_compute/scripts/install.sh | sh

The script should:

  1. Install the latest stable version of hero_proc from:
    https://forge.ourworld.tf/lhumina_code/hero_proc/releases

  2. Install the latest stable version of mycelium from:
    https://github.com/threefoldtech/mycelium/releases

  3. Install the latest stable version of my_hypervisor from:
    https://forge.ourworld.tf/geomind_code/my_hypervisor/releases

  4. Install the latest stable version of cloud-hypervisor from:
    https://github.com/cloud-hypervisor/cloud-hypervisor/releases/

  5. Download all required binaries for hero_compute from:
    https://forge.ourworld.tf/lhumina_code/hero_compute/releases


Post-installation steps:

  • Make all downloaded files executable
  • Rename them to match their repository names
  • Move them to the Hero binary path

Note:
The binaries should be moved to the Hero path (~/hero/bin) — not the system path.
No changes should be made to the user's system PATH.


System dependencies:

After installing all binaries:

  • Check for required system dependencies
  • If any are missing, install them automatically

Service setup:

After confirming that all binaries and dependencies are installed:

  • Start required services (e.g., hero_proc, mycelium) in the background using the central config peers

  • Print the next steps for the user, for example:

    hero_compute start --mode master
    

User confirmation:

Before starting the installation, the script should:

  • Clearly print all planned actions
  • Ask for user confirmation

Example:

Some system dependencies are missing. You can either allow us to install them, or exit and install them manually.

System packages to install:
  - iproute2
  - net-tools

Binaries to install:
  - mycelium
  - hero_proc
  - my_hypervisor

Services that will run in the background:
  - mycelium
  - hero_proc

All binaries will be moved to: ~/hero/bin

Do you want to continue? (y/n)

Final output:

After installation completes, the script should:

  • Show how to stop the background services
  • Provide clear next steps for the user

Technical requirements:

  • The script should work on Ubuntu, Alpine, and other Linux distributions

  • Keep the script clean and modular (avoid excessive length)

  • Use functions for all major tasks:

    • Install the latest stable release (accepts a URL as a parameter)
    • Move and rename binaries (accepts binary name and target path)
    • Check system dependencies
    • Install system dependencies
    • Print the entry point and handle user confirmation

P.S. After successful implementation, we should:

  • Add a CI workflow to run the script and ensure the installation flow works reliably without issues
  • Update all documentation to reflect the new installers, so we clearly support both build-from-source and single-install approaches
### Description A script to install all required binaries from release pages and move them to the appropriate path. Think of it like: ```sh curl --proto '=https' --tlsv1.2 -sSf https://forge.ourworld.tf/lhumina_code/hero_compute/scripts/install.sh | sh ``` ### The script should: 1. Install the latest stable version of `hero_proc` from: [https://forge.ourworld.tf/lhumina_code/hero_proc/releases](https://forge.ourworld.tf/lhumina_code/hero_proc/releases) 2. Install the latest stable version of `mycelium` from: [https://github.com/threefoldtech/mycelium/releases](https://github.com/threefoldtech/mycelium/releases) 3. Install the latest stable version of `my_hypervisor` from: [https://forge.ourworld.tf/geomind_code/my_hypervisor/releases](https://forge.ourworld.tf/geomind_code/my_hypervisor/releases) 4. Install the latest stable version of `cloud-hypervisor` from: [https://github.com/cloud-hypervisor/cloud-hypervisor/releases/](https://github.com/cloud-hypervisor/cloud-hypervisor/releases/) 5. Download all required binaries for `hero_compute` from: [https://forge.ourworld.tf/lhumina_code/hero_compute/releases](https://forge.ourworld.tf/lhumina_code/hero_compute/releases) --- ### Post-installation steps: * Make all downloaded files executable * Rename them to match their repository names * Move them to the Hero binary path > Note: > The binaries should be moved to the Hero path (`~/hero/bin`) — not the system path. > No changes should be made to the user's system PATH. --- ### System dependencies: After installing all binaries: * Check for required system dependencies * If any are missing, install them automatically --- ### Service setup: After confirming that all binaries and dependencies are installed: * Start required services (e.g., `hero_proc`, `mycelium`) in the background using the central config peers * Print the next steps for the user, for example: ```sh hero_compute start --mode master ``` --- ### User confirmation: Before starting the installation, the script should: * Clearly print all planned actions * Ask for user confirmation Example: ``` Some system dependencies are missing. You can either allow us to install them, or exit and install them manually. System packages to install: - iproute2 - net-tools Binaries to install: - mycelium - hero_proc - my_hypervisor Services that will run in the background: - mycelium - hero_proc All binaries will be moved to: ~/hero/bin Do you want to continue? (y/n) ``` --- ### Final output: After installation completes, the script should: * Show how to stop the background services * Provide clear next steps for the user --- ### Technical requirements: * The script should work on Ubuntu, Alpine, and other Linux distributions * Keep the script clean and modular (avoid excessive length) * Use functions for all major tasks: * Install the latest stable release (accepts a URL as a parameter) * Move and rename binaries (accepts binary name and target path) * Check system dependencies * Install system dependencies * Print the entry point and handle user confirmation --- **P.S.** After successful implementation, we should: * Add a CI workflow to run the script and ensure the installation flow works reliably without issues * Update all documentation to reflect the new installers, so we clearly support both build-from-source and single-install approaches
mahmoud self-assigned this 2026-04-06 10:24:47 +00:00
mahmoud added this to the ACTIVE project 2026-04-06 10:24:50 +00:00
mahmoud added this to the later milestone 2026-04-06 10:24:52 +00:00
Author
Owner

Implementation Spec for Issue #63 — Install Script

Objective

Create a standalone shell install script (scripts/install.sh) that downloads pre-built binaries for the entire Hero Compute stack from their respective release pages, installs them to ~/hero/bin, starts required services (hero_proc, mycelium), and works across Ubuntu, Alpine, and other Linux distributions.

Requirements

  • Download latest stable binaries from release pages:
    • hero_proc (3 binaries) from forge.ourworld.tf/lhumina_code/hero_proc
    • mycelium from github.com/threefoldtech/mycelium
    • my_hypervisor (2 binaries) from forge.ourworld.tf/geomind_code/my_hypervisor
    • cloud-hypervisor from github.com/cloud-hypervisor/cloud-hypervisor
    • hero_compute (4 binaries) from forge.ourworld.tf/lhumina_code/hero_compute
  • Post-install: make all binaries executable, place in ~/hero/bin
  • Detect and install system dependencies (apt, apk, dnf/yum + fallback)
  • Start hero_proc_server and mycelium with central config peers
  • User confirmation before installing (y/n)
  • Display stop commands and next steps
  • Cross-distro: Ubuntu, Alpine, RHEL/Fedora
  • Modular script using functions, matching scripts/configure.sh conventions

Files to Create

File Description
scripts/install.sh Main install script — standalone, downloads pre-built binaries, installs deps, starts services

Implementation Plan

Step 1: Script skeleton with header, output helpers, and version defaults

Files: scripts/install.sh (new)

  • Shebang, set -e, header comment block matching configure.sh style
  • Source buildenv.sh for version defaults with fallbacks for standalone use
  • Output helpers (step, log, ok, warn, fail) from configure.sh
  • download() function, arch_check() function
  • Dependencies: none

Step 2: User confirmation prompt

Files: scripts/install.sh

  • confirm_install() function with banner, system info, version list, install dir, y/N prompt
  • Dependencies: Step 1

Step 3: System dependency detection and installation

Files: scripts/install.sh

  • install_system_deps() — detects package manager (apt/apk/dnf/yum), installs runtime deps
  • Required packages: curl, tar, iptables, iproute2, virtiofsd, e2fsprogs, busybox-static
  • KVM check as non-fatal warning
  • Dependencies: Step 2

Step 4: Binary download functions

Files: scripts/install.sh

  • install_hero_proc() — 3 binaries from Forgejo releases
  • install_mycelium() — tar.gz from GitHub, extract + move
  • install_my_hypervisor() — 2 binaries (init goes to ~/.my_hypervisor/bin/)
  • install_cloud_hypervisor() — static binary from GitHub
  • install_hero_compute() — 4 binaries from Forgejo releases
  • Skip if already installed at correct version
  • Dependencies: Step 1

Step 5: PATH setup

Files: scripts/install.sh

  • setup_path() — add ~/hero/bin to PATH in .bashrc/.profile if not present
  • Dependencies: Step 1

Step 6: Service startup functions

Files: scripts/install.sh

  • start_hero_proc() — start hero_proc_server, verify socket
  • start_mycelium() — start with hardcoded peers, warn if not root
  • Dependencies: Step 4

Step 7: Summary, next steps, and main() wiring

Files: scripts/install.sh

  • print_summary() — versions, stop commands, next steps
  • main() — calls all functions in order
  • --help flag support
  • Dependencies: Steps 1-6

Acceptance Criteria

  • scripts/install.sh exists and is executable
  • Downloads all 12 binaries to ~/hero/bin (3 hero_proc + 1 mycelium + 2 my_hypervisor + 1 cloud-hypervisor + 4 hero_compute + my_hypervisor-init to ~/.my_hypervisor/bin/)
  • All binaries are executable with canonical names
  • System deps installed on Ubuntu (apt), Alpine (apk), RHEL (dnf/yum)
  • Unsupported distros get manual instructions
  • hero_proc_server started and socket verified
  • mycelium started with central config peers
  • User prompted before changes; n cancels cleanly
  • Stop commands and next steps printed
  • Same coding conventions as configure.sh
  • Idempotent (re-run skips existing correct-version binaries)
  • Works standalone via curl | bash

Notes

  • Version pinning: Sources buildenv.sh when in repo, falls back to hardcoded defaults. All overridable via env vars.
  • Mycelium requires root for TUN device — script warns/elevates as needed.
  • my_hypervisor-init goes to ~/.my_hypervisor/bin/ (not ~/hero/bin).
  • Different from configure.sh: This is end-user focused (no Rust, no compilation, no repo needed).
  • Central config peers: Hardcoded tcp://188.40.132.242:9651,tcp://136.243.47.186:9651,tcp://185.69.166.7:9651.
## Implementation Spec for Issue #63 — Install Script ### Objective Create a standalone shell install script (`scripts/install.sh`) that downloads pre-built binaries for the entire Hero Compute stack from their respective release pages, installs them to `~/hero/bin`, starts required services (hero_proc, mycelium), and works across Ubuntu, Alpine, and other Linux distributions. ### Requirements - Download latest stable binaries from release pages: - `hero_proc` (3 binaries) from forge.ourworld.tf/lhumina_code/hero_proc - `mycelium` from github.com/threefoldtech/mycelium - `my_hypervisor` (2 binaries) from forge.ourworld.tf/geomind_code/my_hypervisor - `cloud-hypervisor` from github.com/cloud-hypervisor/cloud-hypervisor - `hero_compute` (4 binaries) from forge.ourworld.tf/lhumina_code/hero_compute - Post-install: make all binaries executable, place in `~/hero/bin` - Detect and install system dependencies (apt, apk, dnf/yum + fallback) - Start hero_proc_server and mycelium with central config peers - User confirmation before installing (y/n) - Display stop commands and next steps - Cross-distro: Ubuntu, Alpine, RHEL/Fedora - Modular script using functions, matching `scripts/configure.sh` conventions ### Files to Create | File | Description | |------|-------------| | `scripts/install.sh` | Main install script — standalone, downloads pre-built binaries, installs deps, starts services | ### Implementation Plan #### Step 1: Script skeleton with header, output helpers, and version defaults **Files:** `scripts/install.sh` (new) - Shebang, `set -e`, header comment block matching configure.sh style - Source `buildenv.sh` for version defaults with fallbacks for standalone use - Output helpers (step, log, ok, warn, fail) from configure.sh - download() function, arch_check() function - **Dependencies:** none #### Step 2: User confirmation prompt **Files:** `scripts/install.sh` - `confirm_install()` function with banner, system info, version list, install dir, y/N prompt - **Dependencies:** Step 1 #### Step 3: System dependency detection and installation **Files:** `scripts/install.sh` - `install_system_deps()` — detects package manager (apt/apk/dnf/yum), installs runtime deps - Required packages: curl, tar, iptables, iproute2, virtiofsd, e2fsprogs, busybox-static - KVM check as non-fatal warning - **Dependencies:** Step 2 #### Step 4: Binary download functions **Files:** `scripts/install.sh` - `install_hero_proc()` — 3 binaries from Forgejo releases - `install_mycelium()` — tar.gz from GitHub, extract + move - `install_my_hypervisor()` — 2 binaries (init goes to ~/.my_hypervisor/bin/) - `install_cloud_hypervisor()` — static binary from GitHub - `install_hero_compute()` — 4 binaries from Forgejo releases - Skip if already installed at correct version - **Dependencies:** Step 1 #### Step 5: PATH setup **Files:** `scripts/install.sh` - `setup_path()` — add ~/hero/bin to PATH in .bashrc/.profile if not present - **Dependencies:** Step 1 #### Step 6: Service startup functions **Files:** `scripts/install.sh` - `start_hero_proc()` — start hero_proc_server, verify socket - `start_mycelium()` — start with hardcoded peers, warn if not root - **Dependencies:** Step 4 #### Step 7: Summary, next steps, and main() wiring **Files:** `scripts/install.sh` - `print_summary()` — versions, stop commands, next steps - `main()` — calls all functions in order - `--help` flag support - **Dependencies:** Steps 1-6 ### Acceptance Criteria - [ ] `scripts/install.sh` exists and is executable - [ ] Downloads all 12 binaries to `~/hero/bin` (3 hero_proc + 1 mycelium + 2 my_hypervisor + 1 cloud-hypervisor + 4 hero_compute + my_hypervisor-init to ~/.my_hypervisor/bin/) - [ ] All binaries are executable with canonical names - [ ] System deps installed on Ubuntu (apt), Alpine (apk), RHEL (dnf/yum) - [ ] Unsupported distros get manual instructions - [ ] hero_proc_server started and socket verified - [ ] mycelium started with central config peers - [ ] User prompted before changes; `n` cancels cleanly - [ ] Stop commands and next steps printed - [ ] Same coding conventions as configure.sh - [ ] Idempotent (re-run skips existing correct-version binaries) - [ ] Works standalone via `curl | bash` ### Notes - **Version pinning:** Sources `buildenv.sh` when in repo, falls back to hardcoded defaults. All overridable via env vars. - **Mycelium requires root** for TUN device — script warns/elevates as needed. - **my_hypervisor-init** goes to `~/.my_hypervisor/bin/` (not ~/hero/bin). - **Different from configure.sh:** This is end-user focused (no Rust, no compilation, no repo needed). - **Central config peers:** Hardcoded `tcp://188.40.132.242:9651,tcp://136.243.47.186:9651,tcp://185.69.166.7:9651`.
Author
Owner

Implementation Summary

Changes Made

1. hero_compute_config (central config repo) — 5 files changed:

  • Added [releases] section to all 4 environment configs (development.toml, production.toml, canary.toml, testing.toml)
  • Updated config.schema.json with the releases schema
  • Each release entry specifies: repo, version, binaries, asset_suffix (for Forgejo releases) or url template with {version} placeholder (for GitHub releases)
  • Components defined: hero_compute (v0.1.3), hero_proc (v0.4.1), my_hypervisor (v0.1.4), cloud_hypervisor (v43.0), mycelium (v0.7.3)

2. hero_cloud — 1 new file:

  • scripts/install.sh (571 lines) — standalone binary installer

Key Design Decisions

  • Zero hardcoded release URLs: The script only knows two base URLs (forge.ourworld.tf and the config repo). All binary versions, download paths, asset names, and mycelium peers are fetched from central config at runtime.
  • TOML parser in pure awk/bash: No external dependencies (no python, no jq). Handles both inline and multi-line TOML arrays.
  • Environment-aware: --env production|development|testing|canary selects which config to fetch.
  • Idempotent: Skips already-installed binaries at the correct version.
  • Cross-distro: Supports apt (Ubuntu/Debian), apk (Alpine), dnf (Fedora/RHEL), yum, with fallback manual instructions.
  • Pipe-safe: Auto-confirms when run via curl | bash, prompts interactively otherwise.

Script Flow

  1. arch_check — verify x86_64
  2. fetch_config — download environment TOML from central config repo
  3. confirm_install — show plan, prompt y/N
  4. install_system_deps — runtime packages via detected package manager
  5. install_cloud_hypervisor — from GitHub release (URL template in config)
  6. install_mycelium — from GitHub release tar.gz (URL template in config)
  7. install_hero_proc — 3 binaries from Forgejo release (repo + suffix in config)
  8. install_my_hypervisor — 2 binaries + kernel check + doctor (repo + suffix in config)
  9. install_hero_compute — 4 binaries from Forgejo release (repo + suffix in config)
  10. setup_path — add ~/hero/bin to PATH
  11. start_services — hero_proc_server + mycelium with config peers
  12. print_summary — versions, stop commands, next steps

Testing

  • Bash syntax check: passed
  • TOML parser verified against all 5 config components + mycelium peers array
  • No hardcoded release URLs confirmed (only FORGE_BASE and CONFIG_BASE constants)

Still needed

  • Config repo changes need to be committed and pushed before the script works end-to-end
  • Real Linux x86_64 test (script requires x86_64 arch)
## Implementation Summary ### Changes Made **1. hero_compute_config** (central config repo) — 5 files changed: - Added `[releases]` section to all 4 environment configs (`development.toml`, `production.toml`, `canary.toml`, `testing.toml`) - Updated `config.schema.json` with the releases schema - Each release entry specifies: `repo`, `version`, `binaries`, `asset_suffix` (for Forgejo releases) or `url` template with `{version}` placeholder (for GitHub releases) - Components defined: `hero_compute` (v0.1.3), `hero_proc` (v0.4.1), `my_hypervisor` (v0.1.4), `cloud_hypervisor` (v43.0), `mycelium` (v0.7.3) **2. hero_cloud** — 1 new file: - `scripts/install.sh` (571 lines) — standalone binary installer ### Key Design Decisions - **Zero hardcoded release URLs**: The script only knows two base URLs (forge.ourworld.tf and the config repo). All binary versions, download paths, asset names, and mycelium peers are fetched from central config at runtime. - **TOML parser in pure awk/bash**: No external dependencies (no python, no jq). Handles both inline and multi-line TOML arrays. - **Environment-aware**: `--env production|development|testing|canary` selects which config to fetch. - **Idempotent**: Skips already-installed binaries at the correct version. - **Cross-distro**: Supports apt (Ubuntu/Debian), apk (Alpine), dnf (Fedora/RHEL), yum, with fallback manual instructions. - **Pipe-safe**: Auto-confirms when run via `curl | bash`, prompts interactively otherwise. ### Script Flow 1. `arch_check` — verify x86_64 2. `fetch_config` — download environment TOML from central config repo 3. `confirm_install` — show plan, prompt y/N 4. `install_system_deps` — runtime packages via detected package manager 5. `install_cloud_hypervisor` — from GitHub release (URL template in config) 6. `install_mycelium` — from GitHub release tar.gz (URL template in config) 7. `install_hero_proc` — 3 binaries from Forgejo release (repo + suffix in config) 8. `install_my_hypervisor` — 2 binaries + kernel check + doctor (repo + suffix in config) 9. `install_hero_compute` — 4 binaries from Forgejo release (repo + suffix in config) 10. `setup_path` — add ~/hero/bin to PATH 11. `start_services` — hero_proc_server + mycelium with config peers 12. `print_summary` — versions, stop commands, next steps ### Testing - Bash syntax check: passed - TOML parser verified against all 5 config components + mycelium peers array - No hardcoded release URLs confirmed (only FORGE_BASE and CONFIG_BASE constants) ### Still needed - Config repo changes need to be committed and pushed before the script works end-to-end - Real Linux x86_64 test (script requires x86_64 arch)
Author
Owner

Implementation committed:

hero_cloud: 3585f98scripts/install.sh

hero_compute_config: 77f26c3[releases] section in all environment configs

Implementation committed: **hero_cloud:** [`3585f98`](https://forge.ourworld.tf/lhumina_code/hero_compute/commit/3585f98) — `scripts/install.sh` **hero_compute_config:** [`77f26c3`](https://forge.ourworld.tf/lhumina_code/hero_compute_config/commit/77f26c3) — `[releases]` section in all environment configs
Author
Owner

Related Issues:

#77

Related Issues: https://forge.ourworld.tf/lhumina_code/hero_compute/issues/77
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
lhumina_code/hero_compute#63
No description provided.