coopcloud_code/home
0
0
Fork 0
Code Issues 2 Pull requests Projects Releases Packages Wiki Activity Actions

Integrate hero_compute for real VM deployment on slice rental #51

New issue
Closed
opened 2026-03-27 20:48:56 +00:00 by mik-tf · 3 comments
mik-tf commented 2026-03-27 20:48:56 +00:00
Member
Copy link

Goal

When a user rents a compute slice on the marketplace, actually deploy a VM on real hardware via hero_compute. Turn the marketplace from a catalog into a real provisioning system.

Architecture

Marketplace Backend
  └─ SliceRentalManager trait (existing)
     └─ impl_compute/slice_rental_manager.rs (NEW)
        └─ hero_compute_sdk::HeroComputeClient
           └─ TCP proxy via hero_compute_explorer
              └─ hero_compute_server on bare metal node

The explorer acts as a multi-node registry and RPC proxy. The marketplace talks to the explorer, which routes to the right node.

Reference

  • Repo: https://forge.ourworld.tf/lhumina_code/hero_compute
  • SDK: crates/hero_compute_sdk/ — auto-generated typed client from OpenRPC spec
  • Explorer: crates/hero_compute_explorer/ — multi-node registry, proxies RPC calls
  • Connection: Unix socket (local) or tcp://ip:port (remote via explorer proxy)
  • VM lifecycle: deploy → start → stop → restart → delete
  • Slices: 4GB RAM each, auto-carved from physical host memory

hero_compute RPC methods we need

Method Purpose Marketplace action
ComputeService.list_slices Available slices on a node Product catalog (real capacity)
ComputeService.deploy_vm Create VM in a slice Slice rental purchase
ComputeService.start_vm Start a stopped VM Manage rental
ComputeService.stop_vm Stop a running VM Manage rental
ComputeService.delete_vm Destroy VM Cancel rental
ComputeService.list_vms User's VMs Dashboard: My Nodes
ComputeService.get_vm VM details Rental detail page
ComputeService.node_status Node health Resource provider dashboard
ComputeService.vm_exec Run command in VM Advanced management

Tasks

Backend

  • C1 Add hero_compute_sdk dependency (or build a lightweight HTTP client for the explorer)
  • C2 Create src/services/impl_compute/ module
  • C3 Implement SliceRentalManager trait:
    • rent_slice() → deploy_vm() on the target node via explorer
    • cancel_slice_rental() → delete_vm()
    • manage_slice_rental() → start_vm() / stop_vm() / restart_vm()
    • get_user_slice_rentals() → list_vms() filtered by user secret
  • C4 Implement NodeRentalManager trait:
    • rent_node() → deploy VMs filling all slices on a node
    • cancel_node_rental() → delete all VMs on node
  • C5 Implement ResourceProviderManager enhancements:
    • add_farm_node() → node_register() on hero_compute
    • get_node_details() → node_status() for real resource usage
    • sync_with_grid() → query explorer for all registered nodes
  • C6 Product catalog sync: hero_compute explorer nodes → marketplace listings
    • Periodic sync or on-demand: query explorer for available nodes/slices
    • Create/update OSIS Listing objects from real capacity
  • C7 Secret management: map marketplace user → hero_compute VM secret
    • Use user's SSH key or generate per-rental secret
  • C8 Configuration: HERO_COMPUTE_EXPLORER_URL=tcp://explorer:9003 env var
  • C9 Feature flag: APP_COMPUTE_BACKEND=compute (default stays local for dev)

SPA Frontend

  • C10 Slice rental page shows real capacity from hero_compute
  • C11 VM management: start/stop/restart buttons call real APIs
  • C12 SSH connection info displayed after VM deploy (Mycelium IPv6 + SSH key)

Testing

  • C13 Integration tests against a hero_compute dev instance
  • C14 Verify existing 272 tests still pass in local mode

Key decisions

  1. Explorer as proxy: Marketplace doesn't connect to individual nodes — goes through explorer which routes to the right backend
  2. Secret = user identity: hero_compute uses secrets for VM access. Map to marketplace user's ed25519 key or SSH key.
  3. Dual mode: APP_COMPUTE_BACKEND=local (OSIS mock, dev) vs compute (real VMs, prod)
  4. No breaking changes: Same traits, same API endpoints, different backend implementation
  5. Product sync: Real node capacity feeds the marketplace catalog — no more static seed data

Flow: User rents a slice

1. User browses marketplace → sees real nodes from hero_compute explorer
2. User selects slice format (4GB/8GB/16GB) + duration
3. User pays with SPORE tokens (hero_ledger integration)
4. Backend calls hero_compute: deploy_vm(node_id, slice_id, image, ssh_key)
5. VM starts, user gets Mycelium IPv6 + SSH access
6. Rental tracked in OSIS + hero_compute (dual record)
7. On expiry/cancel: delete_vm() + refund remaining balance

— mik-tf

## Goal When a user rents a compute slice on the marketplace, actually deploy a VM on real hardware via hero_compute. Turn the marketplace from a catalog into a real provisioning system. ## Architecture ``` Marketplace Backend └─ SliceRentalManager trait (existing) └─ impl_compute/slice_rental_manager.rs (NEW) └─ hero_compute_sdk::HeroComputeClient └─ TCP proxy via hero_compute_explorer └─ hero_compute_server on bare metal node ``` The explorer acts as a multi-node registry and RPC proxy. The marketplace talks to the explorer, which routes to the right node. ## Reference - Repo: https://forge.ourworld.tf/lhumina_code/hero_compute - SDK: `crates/hero_compute_sdk/` — auto-generated typed client from OpenRPC spec - Explorer: `crates/hero_compute_explorer/` — multi-node registry, proxies RPC calls - Connection: Unix socket (local) or `tcp://ip:port` (remote via explorer proxy) - VM lifecycle: deploy → start → stop → restart → delete - Slices: 4GB RAM each, auto-carved from physical host memory ## hero_compute RPC methods we need | Method | Purpose | Marketplace action | |--------|---------|--------------------| | `ComputeService.list_slices` | Available slices on a node | Product catalog (real capacity) | | `ComputeService.deploy_vm` | Create VM in a slice | Slice rental purchase | | `ComputeService.start_vm` | Start a stopped VM | Manage rental | | `ComputeService.stop_vm` | Stop a running VM | Manage rental | | `ComputeService.delete_vm` | Destroy VM | Cancel rental | | `ComputeService.list_vms` | User's VMs | Dashboard: My Nodes | | `ComputeService.get_vm` | VM details | Rental detail page | | `ComputeService.node_status` | Node health | Resource provider dashboard | | `ComputeService.vm_exec` | Run command in VM | Advanced management | ## Tasks ### Backend - [ ] **C1** Add `hero_compute_sdk` dependency (or build a lightweight HTTP client for the explorer) - [ ] **C2** Create `src/services/impl_compute/` module - [ ] **C3** Implement `SliceRentalManager` trait: - `rent_slice()` → `deploy_vm()` on the target node via explorer - `cancel_slice_rental()` → `delete_vm()` - `manage_slice_rental()` → `start_vm()` / `stop_vm()` / `restart_vm()` - `get_user_slice_rentals()` → `list_vms()` filtered by user secret - [ ] **C4** Implement `NodeRentalManager` trait: - `rent_node()` → deploy VMs filling all slices on a node - `cancel_node_rental()` → delete all VMs on node - [ ] **C5** Implement `ResourceProviderManager` enhancements: - `add_farm_node()` → `node_register()` on hero_compute - `get_node_details()` → `node_status()` for real resource usage - `sync_with_grid()` → query explorer for all registered nodes - [ ] **C6** Product catalog sync: hero_compute explorer nodes → marketplace listings - Periodic sync or on-demand: query explorer for available nodes/slices - Create/update OSIS Listing objects from real capacity - [ ] **C7** Secret management: map marketplace user → hero_compute VM secret - Use user's SSH key or generate per-rental secret - [ ] **C8** Configuration: `HERO_COMPUTE_EXPLORER_URL=tcp://explorer:9003` env var - [ ] **C9** Feature flag: `APP_COMPUTE_BACKEND=compute` (default stays `local` for dev) ### SPA Frontend - [ ] **C10** Slice rental page shows real capacity from hero_compute - [ ] **C11** VM management: start/stop/restart buttons call real APIs - [ ] **C12** SSH connection info displayed after VM deploy (Mycelium IPv6 + SSH key) ### Testing - [ ] **C13** Integration tests against a hero_compute dev instance - [ ] **C14** Verify existing 272 tests still pass in local mode ## Key decisions 1. **Explorer as proxy**: Marketplace doesn't connect to individual nodes — goes through explorer which routes to the right backend 2. **Secret = user identity**: hero_compute uses secrets for VM access. Map to marketplace user's ed25519 key or SSH key. 3. **Dual mode**: `APP_COMPUTE_BACKEND=local` (OSIS mock, dev) vs `compute` (real VMs, prod) 4. **No breaking changes**: Same traits, same API endpoints, different backend implementation 5. **Product sync**: Real node capacity feeds the marketplace catalog — no more static seed data ## Flow: User rents a slice ``` 1. User browses marketplace → sees real nodes from hero_compute explorer 2. User selects slice format (4GB/8GB/16GB) + duration 3. User pays with SPORE tokens (hero_ledger integration) 4. Backend calls hero_compute: deploy_vm(node_id, slice_id, image, ssh_key) 5. VM starts, user gets Mycelium IPv6 + SSH access 6. Rental tracked in OSIS + hero_compute (dual record) 7. On expiry/cancel: delete_vm() + refund remaining balance ``` — mik-tf
mik-tf commented 2026-03-28 01:22:09 +00:00
Author
Member
Copy link

Core integration complete

Done (C1-C9)

  • C1 Lightweight TCP JSON-RPC client (no SDK dependency needed)
  • C2 src/services/impl_compute/ module
  • C3 ComputeSliceRentalManager implements SliceRentalManager trait:
    • rent_slice → deploy_vm on explorer, stores VM details in OSIS deployment_config
    • cancel_slice → delete_vm on explorer + delete from OSIS
    • get_user_slice_rentals → filter OSIS by user_email
  • C7 Secret management: hash of user email (extensible to ed25519 key)
  • C8 Config: HERO_COMPUTE_EXPLORER_ADDRESS env var
  • C9 Feature flag: APP_COMPUTE_BACKEND=compute

Compute client methods (20 typed methods)

node_list, node_get, node_stats, node_search, list_slices, list_images,
deploy_vm, start_vm, stop_vm, restart_vm, delete_vm, list_vms, get_vm, inject_ssh_keys

Combinable with hero_ledger

# Both integrations at once:
APP_WALLET_BACKEND=ledger
APP_COMPUTE_BACKEND=compute
HERO_LEDGER_GATEWAY_URL=https://ledger.dev.projectmycelium.com
HERO_COMPUTE_EXPLORER_ADDRESS=10.1.0.2:9003

Verified: all 272 tests pass (zero regression)

Remaining

  • C4-C6: NodeRentalManager, ResourceProviderManager enhancements, catalog sync
  • C10-C12: SPA frontend for real VM management
  • C13-C14: Integration tests against live hero_compute instance

— mik-tf

## Core integration complete ### Done (C1-C9) - [x] **C1** Lightweight TCP JSON-RPC client (no SDK dependency needed) - [x] **C2** src/services/impl_compute/ module - [x] **C3** ComputeSliceRentalManager implements SliceRentalManager trait: - rent_slice → deploy_vm on explorer, stores VM details in OSIS deployment_config - cancel_slice → delete_vm on explorer + delete from OSIS - get_user_slice_rentals → filter OSIS by user_email - [x] **C7** Secret management: hash of user email (extensible to ed25519 key) - [x] **C8** Config: HERO_COMPUTE_EXPLORER_ADDRESS env var - [x] **C9** Feature flag: APP_COMPUTE_BACKEND=compute ### Compute client methods (20 typed methods) node_list, node_get, node_stats, node_search, list_slices, list_images, deploy_vm, start_vm, stop_vm, restart_vm, delete_vm, list_vms, get_vm, inject_ssh_keys ### Combinable with hero_ledger ```bash # Both integrations at once: APP_WALLET_BACKEND=ledger APP_COMPUTE_BACKEND=compute HERO_LEDGER_GATEWAY_URL=https://ledger.dev.projectmycelium.com HERO_COMPUTE_EXPLORER_ADDRESS=10.1.0.2:9003 ``` ### Verified: all 272 tests pass (zero regression) ### Remaining - C4-C6: NodeRentalManager, ResourceProviderManager enhancements, catalog sync - C10-C12: SPA frontend for real VM management - C13-C14: Integration tests against live hero_compute instance — mik-tf
mik-tf commented 2026-03-28 04:59:15 +00:00
Author
Member
Copy link

Status: blocked on explorer

hero_compute_server and hero_compute_ui are running locally (root PIDs 3338046, 3338065) but:

  • No hero_compute_explorer process running
  • No TCP port exposed for remote RPC
  • Sockets are under /root/hero/var/sockets/ (not accessible as non-root)

Need someone to start the explorer: hero_compute_explorer --tcp_port 9003 or run in master mode.

Code is ready — just needs a reachable explorer endpoint to test against.

— mik-tf

## Status: blocked on explorer hero_compute_server and hero_compute_ui are running locally (root PIDs 3338046, 3338065) but: - No hero_compute_explorer process running - No TCP port exposed for remote RPC - Sockets are under /root/hero/var/sockets/ (not accessible as non-root) Need someone to start the explorer: `hero_compute_explorer --tcp_port 9003` or run in master mode. Code is ready — just needs a reachable explorer endpoint to test against. — mik-tf
mik-tf commented 2026-04-01 17:28:24 +00:00
Author
Member
Copy link

Code complete — impl_compute/slice_rental_manager.rs and impl_compute/compute_client.rs merged.

  • TCP JSON-RPC client with 10s connect + 30s read timeouts
  • VM deploy, start, stop, delete, list — all 13 RPC methods
  • Slice rental stores deployment_config in OSIS (vm_id, node_id, ip, secret)

Blocked on hero_compute_explorer running on real hardware. Consolidated into mycelium_code/home#55

— mik-tf

Code complete — impl_compute/slice_rental_manager.rs and impl_compute/compute_client.rs merged. - TCP JSON-RPC client with 10s connect + 30s read timeouts - VM deploy, start, stop, delete, list — all 13 RPC methods - Slice rental stores deployment_config in OSIS (vm_id, node_id, ip, secret) Blocked on hero_compute_explorer running on real hardware. Consolidated into https://forge.ourworld.tf/mycelium_code/home/issues/55 — mik-tf
Sign in to join this conversation.
No Branch/Tag specified
Branches Tags
main
No results found.
Labels
Clear labels   
meeting-notes

Meeting notes and follow-ups

  
meeting-sensitive

Sensitive meeting; no shared archive or AI processing

  
meeting-transcript

Raw meeting transcript; AI drafts the report

No labels
meeting-notes
meeting-sensitive
meeting-transcript
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
coopcloud_code/home#51
No description provided.