VM console via WebSocket in browser #36

New issue

Closed

opened 2026-03-18 10:46:04 +00:00 by mahmoud · 4 comments

mahmoud commented 2026-03-18 10:46:04 +00:00

Owner

Copy link

Description

Implement a browser-based VM console (terminal) so users can interact with their VMs directly from the hero_compute dashboard without needing SSH or Mycelium.

Background

Kristof built a WebSocket-based PTY terminal in hero_proc (branch , now merged to ). We should follow the same pattern and adapt it for VM console access via chvm's vsock.

hero_proc reference implementation

• Backend: Axum WebSocket handler at
• Process attachment: crate (pseudo-terminal)
• Frontend: xterm.js (embedded via rust-embed), FitAddon for auto-sizing
• Protocol: Binary WebSocket frames for raw I/O, JSON text frames for resize {"resize":{"cols":N,"rows":N}}
• Scrollback: 64 KB ring buffer replayed on reconnect
• Resize: ResizeObserver + FitAddon + JSON control message
• Key files:
  • hero_proc_server/src/web.rs — PTY WebSocket handler
  • hero_proc_server/src/supervisor/executor.rs — PTY creation
  • hero_proc_ui/templates/index.html — xterm.js frontend
  • hero_proc_ui/src/routes.rs — UI WebSocket proxy

Requirements

Backend (hero_compute_server or hero_compute_ui)

• WebSocket endpoint: GET /console/{vm_sid} (upgrades to WebSocket)
• Authenticate via secret (query param or first message)
• Look up VM by SID, get hypervisor_id
• Connect to VM vsock socket at ~/.chvm/vms/{hypervisor_id}/vsock.sock
• Send CONNECT 1025 handshake to vsock
• Bridge WebSocket <-> vsock bidirectionally:
  • Binary frames from browser → vsock (user input)
  • vsock output → Binary frames to browser (terminal output)
• Handle JSON text frames for terminal resize: {"resize":{"cols":N,"rows":N}}
• Clean up on WebSocket disconnect (close vsock connection)
• Return appropriate errors if VM is not running or has no hypervisor_id

Frontend (hero_compute_ui)

• Embed xterm.js and xterm-addon-fit as static assets (same approach as hero_proc)
• Add "Console" button to VM detail dialog and VM action buttons
• Open console in a modal (or full-screen overlay) with xterm.js terminal
• WebSocket connection: ws://host/console/{vm_sid}?secret=...
• Set binaryType = 'arraybuffer' on WebSocket
• User typing → binary frame → vsock → VM
• VM output → binary frame → term.write()
• ResizeObserver on terminal container → send JSON resize frame
• Terminal config: cursor blink, monospace font, dark theme, 5000-line scrollback
• Show connection status (connecting/connected/disconnected)
• Handle reconnection gracefully

Protocol

Frame Type	Direction	Content
Binary	Browser → Server	Raw terminal input bytes
Binary	Server → Browser	Raw terminal output bytes
Text	Browser → Server	JSON: {"resize":{"cols":N,"rows":N}}

Differences from hero_proc

Aspect	hero_proc	hero_compute (this issue)
Process attachment	portable_pty (local PTY)	vsock socket (~/.chvm/vms/{id}/vsock.sock)
Handshake	None (direct PTY)	`CONNECT 1025
` to vsock
Scrollback	64 KB ring buffer	Not needed (VM has its own)
Broadcast	tokio::sync::broadcast (multi-client)	1:1 connection (single client per VM)
Proxy layer	hero_proc_ui proxies via UDS	hero_compute_ui can serve directly
Target identification	Job ID	VM SID → hypervisor_id

Implementation Notes

• The vsock socket path is ~/.chvm/vms/{hypervisor_id}/vsock.sock — we already have hypervisor_id on the VM record
• chvm exposes port 1025 on the vsock for console access
• No need for portable_pty — the VM already has a TTY, we just bridge to it
• Simpler than hero_proc: no PTY management, no scrollback buffer, no broadcast channel
• Consider adding a "Console" tab alongside "View Logs" in the VM detail dialog

Open Questions

1. Does chvm vsock support terminal resize? If so, what is the protocol?
2. Should we support multiple concurrent console sessions to the same VM?
3. Should the console auto-reconnect on disconnect?

## Description Implement a browser-based VM console (terminal) so users can interact with their VMs directly from the hero_compute dashboard without needing SSH or Mycelium. ## Background Kristof built a WebSocket-based PTY terminal in **hero_proc** (branch , now merged to ). We should follow the same pattern and adapt it for VM console access via chvm's vsock. ### hero_proc reference implementation - **Backend**: Axum WebSocket handler at - **Process attachment**: crate (pseudo-terminal) - **Frontend**: xterm.js (embedded via rust-embed), FitAddon for auto-sizing - **Protocol**: Binary WebSocket frames for raw I/O, JSON text frames for resize `{"resize":{"cols":N,"rows":N}}` - **Scrollback**: 64 KB ring buffer replayed on reconnect - **Resize**: ResizeObserver + FitAddon + JSON control message - **Key files**: - `hero_proc_server/src/web.rs` — PTY WebSocket handler - `hero_proc_server/src/supervisor/executor.rs` — PTY creation - `hero_proc_ui/templates/index.html` — xterm.js frontend - `hero_proc_ui/src/routes.rs` — UI WebSocket proxy ## Requirements ### Backend (hero_compute_server or hero_compute_ui) - [ ] WebSocket endpoint: `GET /console/{vm_sid}` (upgrades to WebSocket) - [ ] Authenticate via secret (query param or first message) - [ ] Look up VM by SID, get `hypervisor_id` - [ ] Connect to VM vsock socket at `~/.chvm/vms/{hypervisor_id}/vsock.sock` - [ ] Send `CONNECT 1025 ` handshake to vsock - [ ] Bridge WebSocket <-> vsock bidirectionally: - Binary frames from browser → vsock (user input) - vsock output → Binary frames to browser (terminal output) - [ ] Handle JSON text frames for terminal resize: `{"resize":{"cols":N,"rows":N}}` - [ ] Clean up on WebSocket disconnect (close vsock connection) - [ ] Return appropriate errors if VM is not running or has no hypervisor_id ### Frontend (hero_compute_ui) - [ ] Embed xterm.js and xterm-addon-fit as static assets (same approach as hero_proc) - [ ] Add "Console" button to VM detail dialog and VM action buttons - [ ] Open console in a modal (or full-screen overlay) with xterm.js terminal - [ ] WebSocket connection: `ws://host/console/{vm_sid}?secret=...` - [ ] Set `binaryType = 'arraybuffer'` on WebSocket - [ ] User typing → binary frame → vsock → VM - [ ] VM output → binary frame → `term.write()` - [ ] ResizeObserver on terminal container → send JSON resize frame - [ ] Terminal config: cursor blink, monospace font, dark theme, 5000-line scrollback - [ ] Show connection status (connecting/connected/disconnected) - [ ] Handle reconnection gracefully ### Protocol | Frame Type | Direction | Content | |-----------|-----------|---------| | Binary | Browser → Server | Raw terminal input bytes | | Binary | Server → Browser | Raw terminal output bytes | | Text | Browser → Server | JSON: `{"resize":{"cols":N,"rows":N}}` | ### Differences from hero_proc | Aspect | hero_proc | hero_compute (this issue) | |--------|-----------|--------------------------| | Process attachment | `portable_pty` (local PTY) | vsock socket (`~/.chvm/vms/{id}/vsock.sock`) | | Handshake | None (direct PTY) | `CONNECT 1025 ` to vsock | | Scrollback | 64 KB ring buffer | Not needed (VM has its own) | | Broadcast | `tokio::sync::broadcast` (multi-client) | 1:1 connection (single client per VM) | | Proxy layer | hero_proc_ui proxies via UDS | hero_compute_ui can serve directly | | Target identification | Job ID | VM SID → hypervisor_id | ## Implementation Notes - The vsock socket path is `~/.chvm/vms/{hypervisor_id}/vsock.sock` — we already have `hypervisor_id` on the VM record - chvm exposes port 1025 on the vsock for console access - No need for `portable_pty` — the VM already has a TTY, we just bridge to it - Simpler than hero_proc: no PTY management, no scrollback buffer, no broadcast channel - Consider adding a "Console" tab alongside "View Logs" in the VM detail dialog ## Open Questions 1. Does chvm vsock support terminal resize? If so, what is the protocol? 2. Should we support multiple concurrent console sessions to the same VM? 3. Should the console auto-reconnect on disconnect?

mahmoud self-assigned this 2026-03-21 14:21:44 +00:00

mahmoud added this to the ACTIVE project 2026-03-21 14:21:46 +00:00

mahmoud added this to the now milestone 2026-03-21 14:21:49 +00:00

mahmoud commented 2026-03-21 14:46:54 +00:00

Author

Owner

Copy link

Implementation Spec for Issue #36: VM Console via WebSocket

Architecture

Browser (xterm.js) → WebSocket → hero_compute_ui → Unix socket → vsock.sock → VM shell

Implementation Steps

#	Step	Files
1	Add WebSocket deps to Cargo.toml	hero_compute_ui/Cargo.toml
2	Enable .with_upgrades() in HTTP server	hero_compute_ui/src/server.rs
3	Add /console/{vm_sid} WebSocket route + vsock bridge	hero_compute_ui/src/server.rs
4	Copy xterm.js static assets from hero_proc	static/js/xterm*.js, static/css/xterm.min.css
5	Add xterm.js includes + console modal to base template	templates/base.html
6	Add Console buttons + JS logic to dashboard.js	static/js/dashboard.js

Backend (Step 3 detail)

• GET /console/{vm_sid}?secret=... → WebSocket upgrade
• Auth: call ComputeService.get_vm via SDK to verify secret + get hypervisor_id
• Connect to ~/.chvm/vms/{hypervisor_id}/vsock.sock
• Send CONNECT 1025\n handshake, verify OK\n response
• Bridge: Binary WS frames ↔ vsock bytes (bidirectional via tokio::select!)
• Text WS frames with {"resize":{...}} silently discarded (no vsock resize protocol)

Frontend (Steps 5-6 detail)

• Console modal (modal-xl, 500px terminal height) with xterm.js
• Connection status badge (Connecting/Connected/Disconnected)
• Console button in VM action buttons + VM detail dialog (only for running VMs)
• ws.binaryType = 'arraybuffer', ResizeObserver for auto-fit

Key Dependencies

• axum with ws feature
• futures 0.3 for StreamExt/SinkExt
• xterm.js + FitAddon (copied from hero_proc)

Notes

• Terminal resize not supported at vsock level (raw byte stream)
• Each console session = independent vsock connection
• .with_upgrades() critical — without it, WebSocket upgrade fails silently

## Implementation Spec for Issue #36: VM Console via WebSocket ### Architecture ``` Browser (xterm.js) → WebSocket → hero_compute_ui → Unix socket → vsock.sock → VM shell ``` ### Implementation Steps | # | Step | Files | |---|------|-------| | 1 | Add WebSocket deps to Cargo.toml | `hero_compute_ui/Cargo.toml` | | 2 | Enable `.with_upgrades()` in HTTP server | `hero_compute_ui/src/server.rs` | | 3 | Add `/console/{vm_sid}` WebSocket route + vsock bridge | `hero_compute_ui/src/server.rs` | | 4 | Copy xterm.js static assets from hero_proc | `static/js/xterm*.js`, `static/css/xterm.min.css` | | 5 | Add xterm.js includes + console modal to base template | `templates/base.html` | | 6 | Add Console buttons + JS logic to dashboard.js | `static/js/dashboard.js` | ### Backend (Step 3 detail) - `GET /console/{vm_sid}?secret=...` → WebSocket upgrade - Auth: call `ComputeService.get_vm` via SDK to verify secret + get `hypervisor_id` - Connect to `~/.chvm/vms/{hypervisor_id}/vsock.sock` - Send `CONNECT 1025\n` handshake, verify `OK\n` response - Bridge: Binary WS frames ↔ vsock bytes (bidirectional via tokio::select!) - Text WS frames with `{"resize":{...}}` silently discarded (no vsock resize protocol) ### Frontend (Steps 5-6 detail) - Console modal (`modal-xl`, 500px terminal height) with xterm.js - Connection status badge (Connecting/Connected/Disconnected) - Console button in VM action buttons + VM detail dialog (only for running VMs) - `ws.binaryType = 'arraybuffer'`, ResizeObserver for auto-fit ### Key Dependencies - `axum` with `ws` feature - `futures` 0.3 for `StreamExt`/`SinkExt` - xterm.js + FitAddon (copied from hero_proc) ### Notes - Terminal resize not supported at vsock level (raw byte stream) - Each console session = independent vsock connection - `.with_upgrades()` critical — without it, WebSocket upgrade fails silently

mahmoud commented 2026-03-21 16:25:41 +00:00

Author

Owner

Copy link

Test Results (2026-03-21)

Command: cargo test -p hero_compute_server -p hero_compute_explorer -p hero_compute_ui
Branch: development (commit b96d904)
Status: ALL PASSED

Package	Tests	Passed	Failed	Ignored
hero_compute_explorer (lib)	2	2	0	0
hero_compute_explorer (bin)	5	5	0	0
hero_compute_server (lib)	9	9	0	0
hero_compute_server (bin)	11	11	0	0
hero_compute_ui (bin)	5	5	0	0
hero_compute_explorer (doc)	0	0	0	3
hero_compute_server (doc)	0	0	0	2
Total	32	32	0	5

All 32 unit tests passed. 5 doc-tests were ignored (expected).

## Test Results (2026-03-21) **Command:** `cargo test -p hero_compute_server -p hero_compute_explorer -p hero_compute_ui` **Branch:** `development` (commit `b96d904`) **Status: ALL PASSED** | Package | Tests | Passed | Failed | Ignored | |---------|-------|--------|--------|---------| | hero_compute_explorer (lib) | 2 | 2 | 0 | 0 | | hero_compute_explorer (bin) | 5 | 5 | 0 | 0 | | hero_compute_server (lib) | 9 | 9 | 0 | 0 | | hero_compute_server (bin) | 11 | 11 | 0 | 0 | | hero_compute_ui (bin) | 5 | 5 | 0 | 0 | | hero_compute_explorer (doc) | 0 | 0 | 0 | 3 | | hero_compute_server (doc) | 0 | 0 | 0 | 2 | | **Total** | **32** | **32** | **0** | **5** | All 32 unit tests passed. 5 doc-tests were ignored (expected).

mahmoud commented 2026-03-21 16:26:27 +00:00

Author

Owner

Copy link

Implementation Summary

New files (3)

• crates/hero_compute_ui/static/js/xterm.min.js — xterm.js terminal emulator (copied from hero_proc)
• crates/hero_compute_ui/static/js/xterm-addon-fit.min.js — FitAddon for auto-sizing (copied from hero_proc)
• crates/hero_compute_ui/static/css/xterm.min.css — xterm.js styles (copied from hero_proc)

Modified files (5)

• Cargo.toml (workspace) — Added ws feature to axum, added futures workspace dep
• crates/hero_compute_ui/Cargo.toml — Added futures dependency
• crates/hero_compute_ui/src/server.rs — Added .with_upgrades(), /console/:vm_sid WebSocket route, console_ws_handler (VM auth + vsock lookup), console_bridge (bidirectional WebSocket↔vsock bridge with CONNECT 1025 handshake)
• crates/hero_compute_ui/templates/base.html — Added xterm.js CSS/JS includes, added vmConsoleModal (dark-themed, modal-xl, 500px terminal)
• crates/hero_compute_ui/static/js/dashboard.js — Added Console button to vmActionButtons + renderVmDetailDialog, added openVmConsole/closeVmConsole/openVmConsoleFromDetail functions

Test Results

• 32 tests passed, 0 failed ✅

## Implementation Summary ### New files (3) - `crates/hero_compute_ui/static/js/xterm.min.js` — xterm.js terminal emulator (copied from hero_proc) - `crates/hero_compute_ui/static/js/xterm-addon-fit.min.js` — FitAddon for auto-sizing (copied from hero_proc) - `crates/hero_compute_ui/static/css/xterm.min.css` — xterm.js styles (copied from hero_proc) ### Modified files (5) - **`Cargo.toml` (workspace)** — Added `ws` feature to axum, added `futures` workspace dep - **`crates/hero_compute_ui/Cargo.toml`** — Added `futures` dependency - **`crates/hero_compute_ui/src/server.rs`** — Added `.with_upgrades()`, `/console/:vm_sid` WebSocket route, `console_ws_handler` (VM auth + vsock lookup), `console_bridge` (bidirectional WebSocket↔vsock bridge with `CONNECT 1025` handshake) - **`crates/hero_compute_ui/templates/base.html`** — Added xterm.js CSS/JS includes, added `vmConsoleModal` (dark-themed, modal-xl, 500px terminal) - **`crates/hero_compute_ui/static/js/dashboard.js`** — Added Console button to `vmActionButtons` + `renderVmDetailDialog`, added `openVmConsole`/`closeVmConsole`/`openVmConsoleFromDetail` functions ### Test Results - **32 tests passed, 0 failed** ✅

mahmoud commented 2026-03-21 16:30:29 +00:00

Author

Owner

Copy link

Implementation committed: cde350b

Browse: cde350b

Implementation committed: `cde350b` Browse: https://forge.ourworld.tf/lhumina_code/hero_compute/commit/cde350b

mahmoud referenced this issue from a commit 2026-03-21 16:30:48 +00:00

feat: add VM console via WebSocket with xterm.js

mahmoud closed this issue 2026-03-23 01:23:56 +00:00

Sign in to join this conversation.

No Branch/Tag specified

Branches Tags

development

main

v0.1.8-rc1

v0.1.8

v0.1.7

v0.1.6

v0.1.5

v0.1.4

v0.1.3

v0.1.2

v0.1.1

v0.1.0

Labels

Clear labels   

prio_critical

  

prio_low

  

type_bug

  

type_contact

  

type_issue

  

type_lead

  

type_question

  

type_story

  

type_task

No labels

prio_critical

prio_low

type_bug

type_contact

type_issue

type_lead

type_question

type_story

type_task

Milestone

Clear milestone

No items

No milestone

now

Projects

Clear projects

No items

No project

ACTIVE

Assignees

Clear assignees

No assignees

mahmoud

1 participant

Notifications

Subscribe

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

lhumina_code/hero_compute#36

No description provided.