hero_browser_server: oschema-first migration (issue #309 Step 1) #40

Merged
mahmoud merged 17 commits from development_mahmoud into development 2026-06-24 06:40:20 +00:00
Owner

Migrates hero_browser_server from the legacy hand-rolled JSON-RPC stack (service_base!() + hand-built axum router + 47-arm match dispatcher + hand-written openrpc.json) to the oschema-first openrpc_server! pattern, per the platform migration (lhumina_code/home#309, Step 1 / backend). The MCP tool surface (server.rs, rmcp #[tool]s) is a separate surface and is untouched.

What changed

  • Deps → hero_lib development (herolib_macros / herolib_oschema_server / async-trait); existing code compiles unchanged.
  • Schema — conformant oschema/main/ (00_types / 80_inputs / 90_rpc), single main domain (model B). No [rootobject] (the Chrome pool owns browser/page identity). Output types mirror the hero_browser_core structs; dynamic JS/cookie/accessibility results are typed str (no any). Required ids/args are required (the dispatch rejects their absence); only true defaults stay optional.
  • Trait impl — all 47 methods implemented (src/rpc/main_impl.rs), bridging onto the existing dispatch (Step-1 manual mapping, zero behavior drift).
  • Servingserve_domains_with(extra, svc): the runtime serves the typed main domain on rpc.sock + TCP 8884; extra carries only the custom routes (MCP /mcp, REST /api/*, SSE screenshot stream, OAuth/MCP probes). Chrome-pool graceful shutdown preserved.
  • Spec — generated openrpc/openrpc_main.json covers all 47 methods (drops the fake rpc.health/rpc.discover; adds js_evaluate_async + page_wait_for_network_idle the old spec omitted). /mcp/openrpc now serves the generated spec (single source of truth).
  • Consumers — SDK regenerated from the new spec; CLI updated to the new client; legacy dead handlers + openrpc.rs removed. Build is warning-clean; server tests + SDK doctest pass.

Verified live (build #15)

browser_create → "<uuid>", browser_list/destroy, browser_list_sessions return typed shapes on /api/main/rpc over both rpc.sock and TCP 8884; MCP initialize on :8884 = 200; REST /api/sessions = 200; lab infocheck + --info --json clean.

Consumer-visible change

The canonical RPC path is now /api/main/rpc (+ /api/rpc); the bare /rpc is gone (standard model B). The hero_components admin was updated to use domain="main" (separate PR on development).

Independent review

A team-lead review flagged a red workspace (broken CLI/SDK doctest) + an optional-vs-required contract drift + a dual-spec smell — all fixed in this branch (commits 896f26d, bf0d4c5, 44c5581, 8cc0f92).

Not in this PR (follow-ups)

  • Phase E — the integration-test crate over the router (the formal #309 Step-1 gate), to be built via the hero_tests_create pattern.
  • hero_browser_app (dormant Dioxus user UI) still POSTs to /rpc and uses stale method names — separate frontend fix.
  • Doubled log-source names are a hero_proc convention, not this migration — filed as lhumina_code/hero_proc#162.

🤖 Generated with Claude Code

Migrates `hero_browser_server` from the legacy hand-rolled JSON-RPC stack (`service_base!()` + hand-built axum router + 47-arm `match` dispatcher + hand-written `openrpc.json`) to the oschema-first `openrpc_server!` pattern, per the platform migration (lhumina_code/home#309, Step 1 / backend). The MCP tool surface (`server.rs`, rmcp `#[tool]`s) is a separate surface and is untouched. ## What changed - **Deps** → hero_lib `development` (`herolib_macros` / `herolib_oschema_server` / `async-trait`); existing code compiles unchanged. - **Schema** — conformant `oschema/main/` (`00_types` / `80_inputs` / `90_rpc`), single `main` domain (model B). No `[rootobject]` (the Chrome pool owns browser/page identity). Output types mirror the `hero_browser_core` structs; dynamic JS/cookie/accessibility results are typed `str` (no `any`). Required ids/args are required (the dispatch rejects their absence); only true defaults stay optional. - **Trait impl** — all 47 methods implemented (`src/rpc/main_impl.rs`), bridging onto the existing dispatch (Step-1 manual mapping, zero behavior drift). - **Serving** — `serve_domains_with(extra, svc)`: the runtime serves the typed `main` domain on `rpc.sock` + TCP 8884; `extra` carries only the custom routes (MCP `/mcp`, REST `/api/*`, SSE screenshot stream, OAuth/MCP probes). Chrome-pool graceful shutdown preserved. - **Spec** — generated `openrpc/openrpc_main.json` covers all 47 methods (drops the fake `rpc.health`/`rpc.discover`; adds `js_evaluate_async` + `page_wait_for_network_idle` the old spec omitted). `/mcp/openrpc` now serves the generated spec (single source of truth). - **Consumers** — SDK regenerated from the new spec; CLI updated to the new client; legacy dead handlers + `openrpc.rs` removed. Build is warning-clean; server tests + SDK doctest pass. ## Verified live (build #15) `browser_create → "<uuid>"`, `browser_list`/`destroy`, `browser_list_sessions` return typed shapes on `/api/main/rpc` over both `rpc.sock` and TCP 8884; MCP `initialize` on :8884 = 200; REST `/api/sessions` = 200; `lab infocheck` + `--info --json` clean. ## Consumer-visible change The canonical RPC path is now `/api/main/rpc` (+ `/api/rpc`); the bare `/rpc` is gone (standard model B). The hero_components admin was updated to use `domain="main"` (separate PR on `development`). ## Independent review A team-lead review flagged a red workspace (broken CLI/SDK doctest) + an optional-vs-required contract drift + a dual-spec smell — **all fixed in this branch** (commits `896f26d`, `bf0d4c5`, `44c5581`, `8cc0f92`). ## Not in this PR (follow-ups) - **Phase E** — the integration-test crate over the router (the formal #309 Step-1 gate), to be built via the `hero_tests_create` pattern. - `hero_browser_app` (dormant Dioxus user UI) still POSTs to `/rpc` and uses stale method names — separate frontend fix. - Doubled log-source names are a hero_proc convention, not this migration — filed as lhumina_code/hero_proc#162. 🤖 Generated with [Claude Code](https://claude.com/claude-code)
Step 1 of the oschema-first migration (issue 309), schema-first checkpoint.

- Move the workspace's hero_lib deps from branch=main to branch=development and
  add herolib_macros + herolib_oschema_server + async-trait (the oschema server
  stack: openrpc_server! / serve_domains). Existing code compiles unchanged.
- Add a conformant oschema for the single 'main' domain (model B), split into
  00_types / 80_inputs / 90_rpc per oschema_best_practices: output types mirror
  the hero_browser_core structs (no 'any'); every method takes one input struct
  or scalar; no [rootobject] (the Chrome pool owns browser/page identity).
- Wire the openrpc_server! macro (src/api.rs) + build.rs rerun trigger; the
  generated openrpc/openrpc_main.json covers all 47 methods. Drops the two fake
  control methods (rpc.health/rpc.discover); adds js_evaluate_async and
  page_wait_for_network_idle that the old hand-written spec omitted.

Serving is still the legacy path (rpc_handler/openrpc) — the trait impl and
serve_domains_with rewire land in the next commits.
Phase B — typed trait impl for all 47 methods (compiler-enforced completeness),
in src/rpc/main_impl.rs over the shared AppState (Chrome pool + activity log).

Each method bridges onto the existing rpc_handler dispatch via small helpers
(raw/params_of/field/field_str/out): builds a JsonRpcRequest, runs the legacy
dispatch, and adapts its {key: value} wrapper into the typed Output. The
browser-automation logic in hero_browser_core is unchanged (Step-1 manual
mapping, zero behavior drift). element_info converts the core's tuple
attributes [[name,value]] -> [{name,value}]; the dynamic JS/cookie/accessibility
results are surfaced as JSON strings (schema types them str).

Not yet served — serve_domains_with rewiring lands in Phase C, which resolves
the current dead-code warnings.
Phase C — replace the hand-built axum router + manual UDS/TCP accept loops with
the macro-generated launcher. main() now hands the HeroBrowserService impl to
api::serve_domains_with(extra, service): the runtime binds rpc.sock + TCP 8884
(from service.toml), mounts the typed 'main' domain (/api/main/rpc + /api/rpc)
with the generated spec, and owns graceful shutdown. The 'extra' router carries
only the genuinely-custom routes — the MCP StreamableHttpService (/mcp), the
REST /api/sessions|activity views, the SSE screenshot stream, and the OAuth/MCP
discovery probes — pruning everything the runtime now provides to avoid a
duplicate-route merge panic. Chrome-pool graceful teardown is preserved after
serve returns.

Verified live (build #13): service starts, smoke tests pass; browser_create →
"<uuid>" and browser_list/destroy work over /api/main/rpc on both rpc.sock and
TCP 8884; MCP initialize on :8884 = 200; REST /api/sessions = 200 on both.

The 5 legacy HTTP handlers (health/heroservice_manifest/ping/domains_json/
openrpc_spec) are now unrouted dead code still referenced by rpc_handler.rs's
tests — both are removed in Phase D.
Phase D (consumer propagation) — point hero_browser_sdk's openrpc_client! at the
macro-generated openrpc/openrpc_main.json instead of the hand-maintained
openrpc.json, and delete that now-orphaned static spec (the SDK was its only
compile-time reader). SDK builds clean against the new typed surface.

Residual cleanup tracked for a follow-up: the now-dead HTTP handlers
(health/heroservice_manifest/ping/domains_json/openrpc_spec) and src/openrpc.rs
(still used by mcp_openrpc + the rpc.discover arm) can be dropped once
mcp_openrpc serves the generated spec const and the rpc_handler tests are
updated; rpc_handler.rs stays as the bridged business-logic layer (Step-1
manual mapping, mirroring whiteboard's retained handlers).
Complete the consumer propagation the migration's Phase D requires:
- The generated client is HeroBrowserServiceClient (was HeroBrowserMCPClient);
  rpc.health was dropped (health is the runtime's GET /health.json), so the CLI
  'health' command now uses a browser_list round-trip as the liveness probe.
- browser_list now returns a bare [str] -> transparent BrowserListOutput, so
  read r.value (was r.browsers).
- Update the SDK doctest to the new client name, the nested BrowserCreateInput
  { input: BrowserCreate { .. } }, and the transparent .value output.

cargo build (server/sdk/cli/core/admin) + cargo test -p hero_browser_sdk --doc
are green.
Review MAJOR-1: the schema advertised browser_id/page_id (and url/selector/text/
value/name/key/expression/width/height/file_path) as optional, but the dispatch
rejects their absence with -32602 — a contract lie. Per oschema_best_practices
("optionals only where absent != empty"), these are now required across the
input structs (PageRef/ElementRef/JsCall and the per-method inputs) and the two
bare-scalar methods (browser_destroy, browser_get_session_activity).

Only genuinely-defaulted params keep '?': browser_create overrides, create-time
url, full_page, timeout_ms, screenshot path, cookie domain/path, mobile
device_scale_factor, dialog prompt_text. Verified against every 'missing X'
check in rpc_handler. Regenerated spec; server/sdk/cli build green.
Review MAJOR-2: /mcp/openrpc served the hand-written openrpc.rs while the typed
surface served the macro-generated spec — two sources of truth that would drift.
Point /mcp/openrpc at the generated openrpc/openrpc_main.json (include_str), so
the oschema is the single source. The hand-written openrpc.rs is now referenced
only by dead rpc_handler arms (rpc.health/rpc.discover/openrpc_spec, unreachable
via the typed surface) — flagged for removal in the dead-code cleanup.
Review MINOR-1 / finish MAJOR-2 — delete the code the runtime/oschema replaced:
- src/openrpc.rs (hand-written spec) and `mod openrpc;`
- the unrouted HTTP handlers health / heroservice_manifest / ping / domains_json
  (the runtime serves /health.json, /heroservice.json, /api/ping,
  /api/domains.json) and rpc_handler::openrpc_spec
- the dead rpc.health / rpc.discover dispatch arms in rpc_handler
- rewrite the in-file test_router to mirror the live `extra` routes only;
  test_openrpc_returns_doc now asserts the generated 47-method spec

Build is warning-clean; 9 server tests pass; sdk + cli green.
hero_browser_sdk: refresh the saved generated-client artifact
All checks were successful
Build and Test / build (pull_request) Successful in 8m20s
ded1c230cb
The openrpc_client! macro saves a reference dump named after the spec path; it's
not compiled (lib.rs invokes the macro directly). Replace the stale
openrpc.openrpc.client.generated.rs (old spec) with the current
openrpc_main.openrpc.client.generated.rs so the tracked artifact matches.
chore: align deps to development, port CLI to new hero_proc_sdk, fix clippy
Some checks failed
Build and Test / build (pull_request) Failing after 36s
87fa1dfd04
- deps: point all git dependencies at `development` (herolib_* x5,
  hero_proc_sdk, hero_macros_previous, hero_website_framework,
  hero_archipelagos). Removes the transitive hero_lib@main duplicate that
  hero_proc_sdk@main was pulling in, so the whole tree resolves one hero_lib.
- cli: port hero_browser to the new dev hero_proc_sdk API — ActionHealthCheck
  / ActionHealthCheckPolicy / ActionKillOther (now required fields) and
  HeroProcClient_::connect() in place of HeroProcRPCAPIClient::connect_socket.
- clippy: fix never_loops (arg parse -> if let .nth(1)), manual_clamp
  (.min().max() -> .clamp()), and collapsible_if across the server crate.
test(browser): add hero_browser_test integration crate (Phase E)
All checks were successful
Build and Test / build (pull_request) Successful in 26m47s
a120aadaa1
Categorized SDK-driven test runner for hero_browser_server (basic /
functional / extended / perf / admin / ai) with --filter, per-test error
files, and README/SPECS per the hero_tests_create skill. Fixes CI: the crate
was listed as a workspace member but its directory was never committed, so
`cargo check` failed to load the manifest.

Note: the server-backed suites can't pass until the hero_lib openrpc_client!
bundle fix lands and the SDK moves to the domain-aware client; the crate
compiles (cargo check/clippy clean) so CI is unblocked now.
fix(app): point the WASM UI at the migrated /api/main/rpc wire
All checks were successful
Build and Test / build (pull_request) Successful in 8m20s
fe384ca8f3
hero_browser migrated to the oschema stack (single `main` domain). Update the
Dioxus app to the new wire so its live views work again:

- rpc.rs: POST /rpc -> /api/main/rpc; rename the stale method names to the
  server's verbatim ones (browser_list -> browser_list_sessions,
  browser_activity -> browser_get_activity, browser_session_activity ->
  browser_get_session_activity); SessionsResponse.browsers -> .sessions to
  match the server's SessionsResponse {sessions, count, max}.
- sessions.rs/app.rs: follow the .sessions rename; backend health probe ->
  /hero_browser/rpc/health.json (the Model B health endpoint).
- guide.rs: refresh the Integration Guide's endpoint + RPC-method listing and
  load the API Docs spec from /api/main/openrpc.json.
feat(sdk): generate the client with hero_lib's openrpc_client! (verbatim wire)
All checks were successful
Build and Test / build (pull_request) Successful in 19m3s
2b7e7ee4f9
Completes the Step 1 SDK migration. The SDK was still generating its client
with the legacy `herolib_derive` macro (hero_macros_previous), which prefixed
service-block method names (`hero_browser_service.browser_create`) while the
server's `openrpc_server!` (hero_lib) registers them verbatim — so every typed
call 404'd. Switch the SDK to hero_lib's `herolib_macros::openrpc_client!`
(same family as the server) in the directory/bundle form, so client + server
agree on verbatim names served at `/api/main/rpc`.

- sdk: herolib_derive -> herolib_macros; bundle form
  (openrpc_client!("../hero_browser_server/oschema/", service="hero_browser")).
  Drops the legacy herolib_derive workspace dep and the stale generated artifact.
- cli + test: consume the bundle API (HeroBrowserClient::connect().main()).
- core: element get_value unwraps the JSON string literal from evaluate(), so
  element_get_value returns the bare value (`hi`) not its JSON encoding (`"hi"`).
- test/ai: env probes now skip (not fail) when the optional resource is absent.

Phase E suite is green end-to-end against the live server (build #25):
basic/functional/extended/perf/admin all pass; ai passes/skips.
Merge branch 'development' into development_mahmoud
All checks were successful
Build and Test / build (pull_request) Successful in 18m24s
12d4f1e166
- Remove the orphan stale generated client
  crates/hero_browser_server/openrpc.client.generated.rs (old
  HeroBrowserMCPClient, referenced nowhere).
- Pin version = "0.6.0" on the hero_lib git deps + hero_archipelagos_core,
  per herolib_import/rust_versions.
- Add ActivityEntry.page_id to the oschema (the core struct + the app + the
  components pane all rely on it); regenerate openrpc_main.json.
- test: fix the always-true `max >= 0` assertion to a real invariant
  (count <= max); clear all of errors/ at startup so stale per-test failures
  don't linger; correct the README "admin needs server" wording.
refactor(server): de-bridge — port handler logic into the typed trait
All checks were successful
Build and Test / build (pull_request) Successful in 18m15s
375fe5f418
Completes the Step 1 migration. Previously rpc/main_impl.rs was a thin bridge:
every typed method forwarded into the legacy string-dispatch
crate::rpc_handler::rpc_handler, which held the real Chrome-pool/page logic.
The migration skill requires the business logic to live in the typed trait
methods and the old handlers deleted.

- Move each method's pool/page logic into the corresponding
  HeroBrowserServiceApi method in rpc/main_impl.rs (typed Input in, typed
  Output out, calling self.state.pool / page directly).
- Delete src/rpc_handler.rs (1262 lines) and the raw()/params_of()/field()/
  field_str() bridge helpers; drop `mod rpc_handler` from main.rs. rpc/mod.rs
  keeps only the small ie()/out()/as_json_str() Output adapters.

Verified: server unit tests pass, workspace builds + clippy clean, and the
Phase E suite is green end-to-end (basic/functional/extended/perf/admin all
pass) against the rebuilt server (build #26).
docs: align all docs with the oschema migration
All checks were successful
Build and Test / build (pull_request) Successful in 23m35s
485dd06b3d
The server now serves a single typed `main` domain via the macro-generated
launcher (api::serve_domains_with). Update every doc to match the actual wire
contract instead of the pre-migration shape:

- Endpoints: `POST /api/main/rpc` (47 methods, verbatim snake_case names, no
  service prefix, no bare `/rpc`), `GET /api/main/openrpc.json`,
  `/health.json`, `/heroservice.json`, `/api/domains.json`.
- TCP 127.0.0.1:8884 + rpc.sock are bound by the launcher from service.toml;
  HERO_BROWSER_MCP_PORT overrides the bind.
- Param shapes: each method takes at most one param in one of three shapes
  (struct `input`, scalar, or empty); SDK is the typed openrpc_client! bundle.
- Drop stale Rhai/OpenRPC-prefix wording; fix curl examples to /api/main/rpc.

Docs only — no code change.
mahmoud merged commit 01b504d52c into development 2026-06-24 06:40:20 +00:00
mahmoud deleted branch development_mahmoud 2026-06-24 06:40:24 +00:00
Sign in to join this conversation.
No reviewers
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
lhumina_code/hero_browser!40
No description provided.