hero_collab

A markdown-centric team collaboration platform built as a first-class Hero OS service. Channels and threads, real-time chat with reactions, @mentions, file attachments, full-text search, collaborative canvases (CRDT via Yjs/yrs), and voice huddles via LiveKit.

Service model: hero_collab_server exposes JSON-RPC over a Unix domain socket (~/hero/var/sockets/hero_collab/rpc.sock). hero_collab_web serves the HTML/JS frontend and proxies browser-originated RPC/WS (~/hero/var/sockets/hero_collab/web.sock). Both are run by hero_proc. Typical operator deploys them behind hero_proxy → hero_router for identity injection and TLS.

---

Quick start (local development)

Use the nushell service script (preferred):

service collab start --update --reset   # build, wipe DB, seed 4 test users (dev mode)
service collab start --update           # build + start, keep existing DB
service collab start                    # start without rebuild
service collab stop                     # stop all collab processes
service collab status                   # show running status

The --reset flag wipes ~/hero/var/data/hero_collab/collab.db* and attachment files, then seeds a canonical 4-user fixture (Alice=id 1, Bob=2, Carol=3, Dave=4) + one "General" workspace + one #general channel with all 4 users as members. Alice is the channel admin.

Then open http://localhost:9988/hero_collab/ui — that's hero_router's port, which forwards to hero_collab_web's socket. The user picker appears; pick any of the 4 seeded users. Open another tab (or incognito) and pick a different user to simulate multi-user chat.

Alternatively, use the CLI directly (bypassing nushell scripts):

hero_collab --start --auth-mode=dev --seed-dev-users
hero_collab --stop

--reset / service collab start --reset is destructive — intended for iterative dev where starting from a known clean slate is the point. Do not run it against a DB you care about.

Why --auth-mode=dev on the CLI instead of an env var? hero_proc spawns its children with a clean env — anything you export in the parent shell never reaches hero_collab_server. CLI flags are the supported knob that survives the hero_proc spawn boundary. Env vars still work when you launch the binaries directly (handy for tests), but under hero_proc --start only flags propagate. See deploy/README.md for the full flag ↔ env ↔ precedence table.

Prerequisites

• Rust 1.80+ with the 2024 edition (rustup update stable)
• SQLite 3 installed (bundled via rusqlite feature, so system SQLite isn't strictly required)
• A running hero_proc + hero_router + optional hero_proxy (install from their respective Forge repos)
• Optional: LiveKit server for huddles — see deploy/README.md

---

Auth modes

Mode	Behavior
proxy (default)	Expect X-Hero-User from hero_proxy. Reject unauthenticated RPCs.
dev	User picker fallback, no headers required. Logs a prominent warning at startup. Never ship to production.

Three ways to set the mode, most-specific wins:

1. hero_collab_server --auth-mode=<dev\|proxy> — direct-launch flag.
2. hero_collab --start --auth-mode=<dev\|proxy> — CLI wrapper; forwards to hero_collab_server via the hero_proc action spec AND sets COLLAB_AUTH_MODE on the hero_collab_web action. Both processes agree.
3. COLLAB_AUTH_MODE env var — fallback for tests and direct- binary launches. Ignored under hero_proc supervision (clean env).

Running the two binaries by hand (bypassing hero_proc), export COLLAB_AUTH_MODE=dev once and launch both; clap's env fallback picks it up transparently. If you mix the two paths (CLI flag on server, env on UI) you can drift — hero_collab --start --auth-mode=X keeps them in sync.

---

Architecture (6 crates)

crates/
├── hero_collab_server/   JSON-RPC service on rpc.sock (SQLite-backed)
├── hero_collab_admin/       axum HTTP + WS relay serving static assets on web.sock (binary: hero_collab_web)
├── hero_collab_sdk/      Auto-generated typed Rust client (via openrpc_client!)
├── hero_collab/          CLI (`hero_collab --start`, --stop, --status)
├── hero_collab_app/      Admin dashboard crate (standalone Hero OS app)
└── hero_collab_examples/ Runnable examples: basic_usage, health, load_test

Data layout:

• ~/hero/var/data/hero_collab/collab.db — SQLite DB (WAL mode)
• ~/hero/var/data/hero_collab/files/ — attachment blobs, namespaced by workspace then attachment id
• ~/hero/var/logs/core/YYYY/DOY/logs.sqlite — structured logs via hero_proc's log aggregator

Wire protocol: JSON-RPC 2.0 over HTTP/1.1 over Unix Domain Socket. The openrpc.json spec at crates/hero_collab_server/openrpc.json is the source of truth; hero_collab_sdk is generated from it via the openrpc_client! macro. 80+ methods, stable error codes per src/rpc_error.rs.

---

Key features

• Workspaces / channels / DMs — standard Slack topology, with public + private channels and direct-message kinds.
• Messages — send, edit, delete (soft), pin, full-text search (SQLite FTS5), attachments with per-user ownership (B5), and threaded replies.
• Reactions — atomic message.toggle_react returning {action: "added"|"removed"}; no reactions on tombstoned messages (H6).
• @mentions — parsed on send, delivered via WS push (mention.created) and surfaced as OS-level browser notifications when the tab is backgrounded (H10).
• Canvases — Yjs/yrs CRDT-backed collaborative docs with Tiptap editor; multi-client real-time sync over binary WS; role-gated (owner, editor, viewer); viewer mode enforces read-only in-browser (Phase 4-EXT 1A).
• Voice huddles — LiveKit SFU integration, JWTs signed by livekit.rs::generate_token. deploy/docker-compose.yml stands up LiveKit + Redis for a self-hosted deployment.
• Rate limiting — 60 RPC/min global + 10/min on message.send per authenticated caller, two-phase check-then-commit; bypasses dev-mode unauthenticated calls.
• Observability — rpc.dispatch tracing events on every call, atomic counters exposed via system.metrics, /health shows active WS connection count.
• Federation — collab.users.available unions local users with hero_proxy's users.list; UI invite/DM pickers see system-wide identities (K-4-2).

---

Testing

# Unit tests (rate_limit, rpc_error, validation, activity)
cargo test -p hero_collab_server --bin hero_collab_server

# Integration tests (spawn real server per test, ephemeral sockets)
cargo test -p hero_collab_server --test integration

# Full workspace
cargo test --workspace

As of the post-Plan-A P0/P1 sweep: 40 unit + 30 integration tests, with specific coverage of the ship-blocker regressions surfaced during hardening (B1–B6), the 6.1c typed-error wire codes, pin/unpin

• require_caller + claim_federated from the external-audit P0/P1 batch, and the channel.create auto-member fix.

---

Operations

Operational docs:

• deploy/README.md — LiveKit setup, env vars, ports.
• docs/BACKUP.md — online DB backup + restore; retention hints.
• plan/known-issues.md — tracked issues deferred from Plan A, organized by phase of origin (H-3-, K-4-, K-6-, P-A-, P-B-, P-C-, P-D-*).

Metrics to watch in production:

• /health on web.sock: active_ws_connections (should be >0 when users are connected; rebooting the UI pulls it to 0 and it climbs back as clients reconnect).
• system.metrics RPC: rpc_calls_total, rpc_errors_total, avg_latency_ms, workspaces, users, channels, messages.
• Log stream (hero_proc's logs.sqlite): grep rpc.dispatch for per-call timing + error codes; grep task=huddle_reaper or task=attachment_cleanup for background-task activity; grep trace_id= to correlate user-reported issues with sanitized -32603 Internal responses.

---

Browser support

Minimum: Chrome/Edge 100+, Firefox 100+, Safari 15.4+. See crates/hero_collab_admin/BROWSER_SUPPORT.md for the full API coverage rationale.

---

Plan + roadmap

The per-phase execution history lives in plan/:

• phase-6x-plan-a.md — the consolidated backlog executed most recently (Sprint 1–3 + 6.1c + P2)
• slack-feature-parity.md — the original 7-phase plan
• feature-huddles.md, feature-huddles-v2.md, feature-voice-to-canvas.md — feature-specific design docs
• known-issues.md — deferred items

Commit messages tag their plan step (Phase-6x Plan A / Phase B, etc.) so git log reads as a timeline of what was shipped when.

---

Contributing

• Follow the established commit-message format: one header line describing the change + a body explaining why, with references to the plan step or known-issue ID. Use Co-Authored-By: for pair / AI-assisted work.
• New handlers return RpcResult<Value>, not anyhow::Result — see src/rpc_error.rs for the typed-error contract.
• Input validation uses the typed newtype pattern (Name, Email, ChannelName, MessageContent, etc.) from src/validation.rs. Handlers call parse_input::<T>(params) to deserialize + validate.
• Run cargo test and the Python backend smoke before committing; run the Playwright browser smoke when touching chat-app.js or canvas-app.js.

---

License

See Cargo.toml.