lhumina_code/hero_rpc

Fork 0

Lifecycle alignment, part 2 — examples, dual-home tests, benches-at-root, --bench #129

New issue

Closed

opened 2026-05-22 12:51:32 +00:00 by timur · 1 comment

timur commented

2026-05-22 12:51:32 +00:00

Owner

Lifecycle alignment, part 2 — examples, dual-home tests, benches-at-root, `--bench`

Sibling follow-up to hero_rpc#124. The three PRs that close #124
(hero_rpc#126 + hero_skills#285 + hero_service#8) deliver the core
of the alignment: MultiDomainBuilder, the lab --ephemeral /
--json / --pid / --test surface, the subprocess-driver
tests/src/lib.rs, and the Layer 2–4 nushell skeletons.

This issue picks up the items we agreed on in conversation after
the agent's last issue refresh. None block #124; all belong with
the same alignment story.

Scope

1. `examples/` scaffolded at workspace root

Cargo's standard examples/ directory was dropped from the
per-service template by hero_rpc#116 and never replaced. The
scaffolder should emit one starter file:

examples/
└── 01_connect.rs        ← opens the SDK against `lab --start --ephemeral`,
                            calls `<root>_list` against one rootobject,
                            exits clean. ~30 lines.

Regen-once: byte-identical on every re-scaffold, never overwritten
after first emit.

2. `lab service <name> --example [name]`

New verb in hero_skills/crates/lab paralleling the existing
--test [layer] shape. Dispatch:

lab service <name> --example [name]
  ├── examples/                                  always
  └── crates/<name>_examples/        if exists   also (fallback)

No name → list every example from both sources, exit 0.
name resolution: root examples/<name> wins on collision; sibling crate is fallback.
Standalone --example spawns its own --start --ephemeral, runs the example with HERO_TEST_SOCKET=<path> (or by parsing the JSON envelope itself — the example chooses), then --stop --pid <N>.

3. Two-home dispatch for `--test`

lab service <name> --test [layer] (landed in hero_skills#285) currently runs the tests/ crate only. Extend it:

lab service <name> --test [layer]
  ├── always:           tests/  (current behavior)
  └── if exists:        crates/<name>_test/  → also `cargo run -p <name>_test`,
                          requires zero exit before reporting pass

Existing _test sibling crates (hero_router_test, lab_test, hero_proc_test, hero_aibroker_test) become discoverable through the canonical verb without any per-service migration.

4. `hero_crates_best_practices_check` skill — two-home convention text

Add the following section to the skill so the two homes for tests
and examples are documented canonically:

## Tests and examples — two valid homes each

Every Hero service has two places it MAY hold tests, and two places
it MAY hold examples. Both are sanctioned. Default to the root form;
reach for the sibling crate only when its specific affordances are
needed.

### Tests

| Home                          | Use it for                                              |
| ----------------------------- | ------------------------------------------------------- |
| `tests/`              (default)   | Cargo integration tests — Layers 1–4 of `tests_pyramid`. Scaffolded by `hero_rpc_generator`. Files become `[[test]]` entries (one process per file, parallel by default). Use `#[serial_test::serial]` for env-mutating fns. |
| `crates/<name>_test/`  (rare)     | Sibling `[[bin]]` crate. Reach for it only when cargo's `#[test]` framework isn't enough: long-running stress / load harness (e.g. `hero_router_test`), multi-binary fixture (fake server + tester), or custom runner shape. |

### Examples

| Home                              | Use it for                                          |
| --------------------------------- | --------------------------------------------------- |
| `examples/`              (default)    | Cargo standard — `cargo run --example <name>`. One `.rs` file or `<name>/main.rs` subdir per example. Scaffolded with a starter `01_connect.rs`. Use for short, one-shot demos that exercise the SDK. |
| `crates/<name>_examples/` (rare)      | Sibling `[[bin]]` crate. Reach for it when an example needs to run for a long time, ship multiple binaries, declare its own `service.toml`, or carry dependencies that don't belong on the parent crate. |

`lab service <name> --test [layer]` and `lab service <name> --example [name]`
discover both homes for their respective verb. Root form wins on
name collisions; sibling form is fallback. Scaffolder does not
auto-emit either sibling crate — they stay opt-in.

5. `benches/` at workspace root (consumer services)

Per the per-service layout: tests/ and benches/ mirrored at
root. The scaffolder should emit benches/ as a workspace member
crate that shares spin_up_service from tests/:

benches/
├── Cargo.toml
│      [package] name = "<name>_benches", publish = false
│      [lib] path = "src/lib.rs"
│      [dev-dependencies]
│          <name>_tests = { path = "../tests" }     ← re-uses subprocess driver
│          criterion = { ..., features = ["async_tokio"] }
│          tokio = { workspace = true }
│      [[bench]] name = "index_perf"  harness = false
├── src/lib.rs                        pub use <name>_tests::spin_up_service;
└── index_perf.rs                     starter criterion bench

benches/ is opt-in at first — emit only benches/Cargo.toml and
benches/src/lib.rs skeleton on initial scaffold. Each .rs bench
file is hand-authored thereafter.

6. `lab service <name> --bench [name]`

lab service <name> --bench           # cargo bench --workspace (every bench in benches/)
lab service <name> --bench <name>    # one bench file
lab service <name> --bench --save base       # passes through to criterion
lab service <name> --bench --baseline base   # passes through to criterion

Implementation: cargo bench -p <name>_benches [<flags>] under the
hood. Mirrors the --test verb's pass-through shape.

Sanction both bench shapes in the skill:

In-process direct-handler benches (hero_rpc#126's current
shape): measures pure OSIS storage perf, no UDS overhead. Use for
storage-layer micro-benches.
Subprocess-via-lab benches: bench setup_group calls
Command::new("lab") --start --ephemeral, runs through the
hero_rpc2::Client wire path. Use when you want to measure the
full wire RTT.

Both are legitimate; the bench author picks based on what they're
measuring. Document this in docs/testing.md.

7. `serial_test` dev-dep in scaffolded `tests/Cargo.toml`

Add serial_test = "3" (or current) to dev-deps so contributors
who write env-mutating tests don't have to amend Cargo.toml first.
~50 KB compile cost, only paid when cargo test runs.

8. `hero_rpc/crates/osis_benches/` → `hero_rpc/benches/` (optional)

For symmetry with the consumer-service layout, move
crates/osis_benches/ to benches/ at hero_rpc root. Keep its
internal-bootstrap shape (no lab subprocess — hero_rpc is the
library that defines MultiDomainBuilder, calling lab from it
would be circular).

This is cosmetic-only. Skip if it materially complicates the diff.

Out of scope

Migrating existing _test sibling crates into tests/. They stay where they are; --test dispatch discovers them.
Migrating existing _examples sibling crates into examples/ (only lab_test/src/examples/ exists; not worth touching).
Adding --bench to hero_proc / hero_router / etc. existing services. Per-service follow-up driven by each owner.

Acceptance

lab service hero_service --example lists 01_connect; lab service hero_service --example 01_connect runs it green against an auto-spawned ephemeral.
lab service hero_service --bench runs the scaffolded criterion benches.
Scaffolded examples/01_connect.rs exists, is byte-identical across re-scaffolds, runs green.
Scaffolded benches/Cargo.toml + benches/src/lib.rs skeleton exist; adding a starter benches/index_perf.rs and running lab --bench index_perf works end-to-end.
lab service hero_router --test discovers AND runs crates/hero_router_test alongside tests/. Same for any other existing _test sibling.
hero_crates_best_practices_check skill includes the two-home convention text.
tests/Cargo.toml in the scaffolded template includes serial_test under [dev-dependencies].
No regression in #124's acceptance — lab service hero_service --test still runs all 5 layers green.

hero_rpc#124 — must merge first (delivers the primitives this builds on).
hero_rpc#126, hero_skills#285, hero_service#8 — the three PRs closing #124.
hero_rpc#116 — the change that removed per-service _examples placeholders (and never replaced them — this issue does).
hero_skills/claude/skills_tocheck/hero_crates_best_practices_check — gets the convention update.
hero_skills/claude/skills_tocheck/tests_pyramid — unchanged, layered under --test.

# Lifecycle alignment, part 2 — examples, dual-home tests, benches-at-root, `--bench` Sibling follow-up to hero_rpc#124. The three PRs that close #124 (hero_rpc#126 + hero_skills#285 + hero_service#8) deliver the core of the alignment: `MultiDomainBuilder`, the `lab --ephemeral` / `--json` / `--pid` / `--test` surface, the subprocess-driver `tests/src/lib.rs`, and the Layer 2–4 nushell skeletons. This issue picks up the items we agreed on in conversation *after* the agent's last issue refresh. None block #124; all belong with the same alignment story. ## Scope ### 1. `examples/` scaffolded at workspace root Cargo's standard `examples/` directory was dropped from the per-service template by hero_rpc#116 and never replaced. The scaffolder should emit one starter file: ``` examples/ └── 01_connect.rs ← opens the SDK against `lab --start --ephemeral`, calls `<root>_list` against one rootobject, exits clean. ~30 lines. ``` Regen-once: byte-identical on every re-scaffold, never overwritten after first emit. ### 2. `lab service <name> --example [name]` New verb in `hero_skills/crates/lab` paralleling the existing `--test [layer]` shape. Dispatch: ``` lab service <name> --example [name] ├── examples/ always └── crates/<name>_examples/ if exists also (fallback) ``` - No `name` → list every example from both sources, exit 0. - `name` resolution: root `examples/<name>` wins on collision; sibling crate is fallback. - Standalone `--example` spawns its own `--start --ephemeral`, runs the example with `HERO_TEST_SOCKET=<path>` (or by parsing the JSON envelope itself — the example chooses), then `--stop --pid <N>`. ### 3. Two-home dispatch for `--test` `lab service <name> --test [layer]` (landed in hero_skills#285) currently runs the `tests/` crate only. Extend it: ``` lab service <name> --test [layer] ├── always: tests/ (current behavior) └── if exists: crates/<name>_test/ → also `cargo run -p <name>_test`, requires zero exit before reporting pass ``` Existing `_test` sibling crates (`hero_router_test`, `lab_test`, `hero_proc_test`, `hero_aibroker_test`) become discoverable through the canonical verb without any per-service migration. ### 4. `hero_crates_best_practices_check` skill — two-home convention text Add the following section to the skill so the two homes for tests *and* examples are documented canonically: ``` ## Tests and examples — two valid homes each Every Hero service has two places it MAY hold tests, and two places it MAY hold examples. Both are sanctioned. Default to the root form; reach for the sibling crate only when its specific affordances are needed. ### Tests | Home | Use it for | | ----------------------------- | ------------------------------------------------------- | | `tests/` (default) | Cargo integration tests — Layers 1–4 of `tests_pyramid`. Scaffolded by `hero_rpc_generator`. Files become `[[test]]` entries (one process per file, parallel by default). Use `#[serial_test::serial]` for env-mutating fns. | | `crates/<name>_test/` (rare) | Sibling `[[bin]]` crate. Reach for it only when cargo's `#[test]` framework isn't enough: long-running stress / load harness (e.g. `hero_router_test`), multi-binary fixture (fake server + tester), or custom runner shape. | ### Examples | Home | Use it for | | --------------------------------- | --------------------------------------------------- | | `examples/` (default) | Cargo standard — `cargo run --example <name>`. One `.rs` file or `<name>/main.rs` subdir per example. Scaffolded with a starter `01_connect.rs`. Use for short, one-shot demos that exercise the SDK. | | `crates/<name>_examples/` (rare) | Sibling `[[bin]]` crate. Reach for it when an example needs to run for a long time, ship multiple binaries, declare its own `service.toml`, or carry dependencies that don't belong on the parent crate. | `lab service <name> --test [layer]` and `lab service <name> --example [name]` discover both homes for their respective verb. Root form wins on name collisions; sibling form is fallback. Scaffolder does not auto-emit either sibling crate — they stay opt-in. ``` ### 5. `benches/` at workspace root (consumer services) Per the per-service layout: `tests/` and `benches/` mirrored at root. The scaffolder should emit `benches/` as a workspace member crate that shares `spin_up_service` from `tests/`: ``` benches/ ├── Cargo.toml │ [package] name = "<name>_benches", publish = false │ [lib] path = "src/lib.rs" │ [dev-dependencies] │ <name>_tests = { path = "../tests" } ← re-uses subprocess driver │ criterion = { ..., features = ["async_tokio"] } │ tokio = { workspace = true } │ [[bench]] name = "index_perf" harness = false ├── src/lib.rs pub use <name>_tests::spin_up_service; └── index_perf.rs starter criterion bench ``` `benches/` is opt-in at first — emit only `benches/Cargo.toml` and `benches/src/lib.rs` skeleton on initial scaffold. Each `.rs` bench file is hand-authored thereafter. ### 6. `lab service <name> --bench [name]` ``` lab service <name> --bench # cargo bench --workspace (every bench in benches/) lab service <name> --bench <name> # one bench file lab service <name> --bench --save base # passes through to criterion lab service <name> --bench --baseline base # passes through to criterion ``` Implementation: `cargo bench -p <name>_benches [<flags>]` under the hood. Mirrors the `--test` verb's pass-through shape. **Sanction both bench shapes** in the skill: - **In-process direct-handler benches** (hero_rpc#126's current shape): measures pure OSIS storage perf, no UDS overhead. Use for storage-layer micro-benches. - **Subprocess-via-lab benches**: bench `setup_group` calls `Command::new("lab") --start --ephemeral`, runs through the `hero_rpc2::Client` wire path. Use when you want to measure the full wire RTT. Both are legitimate; the bench author picks based on what they're measuring. Document this in `docs/testing.md`. ### 7. `serial_test` dev-dep in scaffolded `tests/Cargo.toml` Add `serial_test = "3"` (or current) to dev-deps so contributors who write env-mutating tests don't have to amend Cargo.toml first. ~50 KB compile cost, only paid when cargo test runs. ### 8. `hero_rpc/crates/osis_benches/` → `hero_rpc/benches/` (optional) For symmetry with the consumer-service layout, move `crates/osis_benches/` to `benches/` at hero_rpc root. Keep its internal-bootstrap shape (no `lab` subprocess — hero_rpc is the library that *defines* `MultiDomainBuilder`, calling lab from it would be circular). This is cosmetic-only. Skip if it materially complicates the diff. ## Out of scope - Migrating existing `_test` sibling crates into `tests/`. They stay where they are; `--test` dispatch discovers them. - Migrating existing `_examples` sibling crates into `examples/` (only `lab_test/src/examples/` exists; not worth touching). - Adding `--bench` to hero_proc / hero_router / etc. existing services. Per-service follow-up driven by each owner. ## Acceptance - [ ] `lab service hero_service --example` lists `01_connect`; `lab service hero_service --example 01_connect` runs it green against an auto-spawned ephemeral. - [ ] `lab service hero_service --bench` runs the scaffolded criterion benches. - [ ] Scaffolded `examples/01_connect.rs` exists, is byte-identical across re-scaffolds, runs green. - [ ] Scaffolded `benches/Cargo.toml` + `benches/src/lib.rs` skeleton exist; adding a starter `benches/index_perf.rs` and running `lab --bench index_perf` works end-to-end. - [ ] `lab service hero_router --test` discovers AND runs `crates/hero_router_test` alongside `tests/`. Same for any other existing `_test` sibling. - [ ] `hero_crates_best_practices_check` skill includes the two-home convention text. - [ ] `tests/Cargo.toml` in the scaffolded template includes `serial_test` under `[dev-dependencies]`. - [ ] No regression in #124's acceptance — `lab service hero_service --test` still runs all 5 layers green. ## Related - hero_rpc#124 — must merge first (delivers the primitives this builds on). - hero_rpc#126, hero_skills#285, hero_service#8 — the three PRs closing #124. - hero_rpc#116 — the change that removed per-service `_examples` placeholders (and never replaced them — this issue does). - `hero_skills/claude/skills_tocheck/hero_crates_best_practices_check` — gets the convention update. - `hero_skills/claude/skills_tocheck/tests_pyramid` — unchanged, layered under `--test`.

timur commented

2026-05-25 10:48:26 +00:00

Author

Owner

Closed by three squash-merges into development:

lhumina_code/hero_rpc#139 — scaffolder additions (examples/, benches/, serial_test dev-dep) + osis_benches rename to benches/.
lhumina_code/hero_skills#302 — --example / --bench verbs + dual-home --test Layer-1 dispatch + skill docs (hero_crates_best_practices_check + hero_service_test_complete §0).
lhumina_code/hero_service#13 — re-scaffold against the new generator.

Acceptance recap

§1 examples/01_connect.rs scaffolded at workspace root (regen-once); byte-identical across re-scaffolds.
§2 lab service <name> --example [name] — dual-home dispatch (examples/ always + crates/<name>_examples/ fallback). No-name listing enumerates both homes.
§3 lab service <name> --test Layer 1 is dual-home — runs cargo test --workspace and cargo run -p <name>_test when crates/<name>_test/Cargo.toml exists.
§4 hero_crates_best_practices_check skill carries the verbatim "Tests and examples — two valid homes each" section. Cross-linked from hero_service_test_complete §0.
§5 benches/ workspace member emitted with re-export of <name>_tests::* for shared spin_up_service.
§6 lab service <name> --bench [name] [-- <criterion flags>] — pass-through to cargo bench -p <name>_benches. Both bench shapes (in-process direct-handler vs subprocess-via-lab) sanctioned in the skill text.
§7 serial_test = "3" under [dev-dependencies] in scaffolded tests/Cargo.toml.
§8 (optional) crates/osis_benches/ → benches/ move. Crate keeps hero_rpc_osis_benches package name; in-process bootstrap shape preserved (hero_rpc is the library that defines MultiDomainBuilder, calling lab from it would be circular).

Decisions taken without confirmation

examples/ is a real workspace member (<name>_examples package with [[example]] entries pointing at the root .rs files) rather than a bare directory at workspace root. Cargo can only run examples that belong to a package; the wrapper is the smallest change that lets cargo run -p <name>_examples --example <foo> work, which is what the lab --example verb dispatches through.
benches/ emission gated on generate_tests — the starter shape re-exports <name>_tests::spin_up_service and that dep would dangle without tests/.
--bench does not spawn an ephemeral up-front. The issue body explicitly sanctions both bench shapes and notes that in-process direct-handler benches measure storage perf (no UDS overhead) while subprocess-via-lab benches measure full wire RTT. Forcing an ephemeral would penalize the storage-perf shape without buying anything.
Layer-1 dual-home runs cargo run -p <name>_test (singular, matching the existing hero_router_test / lab_test / hero_proc_test / hero_aibroker_test convention). The check looks for crates/<name>_test/Cargo.toml rather than the directory alone, so a stray empty dir from a half-completed move doesn't fire a broken cargo run.
The new --no-examples / --no-benches flags are surfaced on the standalone hero_rpc_generator binary only — a grep across hero_skills/crates/lab/src/ turned up no scaffold invocation, so wiring them into a lab subcommand would be a future change rather than part of this PR series.
hero_skills #302 includes a chore(fmt) commit bundled into the same PR rather than a separate dev-branch cleanup — dev's CI was failing on the same drift across six files, and splitting the work would have blocked this PR on a chore PR that nobody was going to prioritize.

Smoke-test evidence

$ PATH_ROOT=$HOME/hero lab service hero_service --example
lab service hero_service --example: available examples
  01_connect                (examples/)

Unit tests cover the dispatch logic in all three new modules (bench_verb, example_verb, test_verb::sibling_test_crate). A full end-to-end lab service hero_router --test layer1 run (cargo test --workspace + cargo run -p hero_router_test) was not exercised in this PR series — the dispatch is unit-tested and the manual lab service hero_router --test layer1 invocation starts cargo test --workspace against the correct repo path. Closing.

Closed by three squash-merges into `development`: - lhumina_code/hero_rpc#139 — scaffolder additions (`examples/`, `benches/`, `serial_test` dev-dep) + `osis_benches` rename to `benches/`. - lhumina_code/hero_skills#302 — `--example` / `--bench` verbs + dual-home `--test` Layer-1 dispatch + skill docs (`hero_crates_best_practices_check` + `hero_service_test_complete` §0). - lhumina_code/hero_service#13 — re-scaffold against the new generator. ## Acceptance recap - [x] §1 `examples/01_connect.rs` scaffolded at workspace root (regen-once); byte-identical across re-scaffolds. - [x] §2 `lab service <name> --example [name]` — dual-home dispatch (`examples/` always + `crates/<name>_examples/` fallback). No-name listing enumerates both homes. - [x] §3 `lab service <name> --test` Layer 1 is dual-home — runs `cargo test --workspace` *and* `cargo run -p <name>_test` when `crates/<name>_test/Cargo.toml` exists. - [x] §4 `hero_crates_best_practices_check` skill carries the verbatim "Tests and examples — two valid homes each" section. Cross-linked from `hero_service_test_complete` §0. - [x] §5 `benches/` workspace member emitted with re-export of `<name>_tests::*` for shared `spin_up_service`. - [x] §6 `lab service <name> --bench [name] [-- <criterion flags>]` — pass-through to `cargo bench -p <name>_benches`. Both bench shapes (in-process direct-handler vs subprocess-via-lab) sanctioned in the skill text. - [x] §7 `serial_test = "3"` under `[dev-dependencies]` in scaffolded `tests/Cargo.toml`. - [x] §8 (optional) `crates/osis_benches/` → `benches/` move. Crate keeps `hero_rpc_osis_benches` package name; in-process bootstrap shape preserved (hero_rpc is the library that *defines* `MultiDomainBuilder`, calling lab from it would be circular). ## Decisions taken without confirmation 1. `examples/` is a real workspace member (`<name>_examples` package with `[[example]]` entries pointing at the root `.rs` files) rather than a bare directory at workspace root. Cargo can only run examples that belong to a package; the wrapper is the smallest change that lets `cargo run -p <name>_examples --example <foo>` work, which is what the lab `--example` verb dispatches through. 2. `benches/` emission gated on `generate_tests` — the starter shape re-exports `<name>_tests::spin_up_service` and that dep would dangle without `tests/`. 3. `--bench` does **not** spawn an ephemeral up-front. The issue body explicitly sanctions both bench shapes and notes that in-process direct-handler benches measure storage perf (no UDS overhead) while subprocess-via-lab benches measure full wire RTT. Forcing an ephemeral would penalize the storage-perf shape without buying anything. 4. Layer-1 dual-home runs `cargo run -p <name>_test` (singular, matching the existing `hero_router_test` / `lab_test` / `hero_proc_test` / `hero_aibroker_test` convention). The check looks for `crates/<name>_test/Cargo.toml` rather than the directory alone, so a stray empty dir from a half-completed move doesn't fire a broken cargo run. 5. The new `--no-examples` / `--no-benches` flags are surfaced on the standalone `hero_rpc_generator` binary only — a grep across `hero_skills/crates/lab/src/` turned up no scaffold invocation, so wiring them into a lab subcommand would be a future change rather than part of this PR series. 6. hero_skills #302 includes a `chore(fmt)` commit bundled into the same PR rather than a separate dev-branch cleanup — dev's CI was failing on the same drift across six files, and splitting the work would have blocked this PR on a chore PR that nobody was going to prioritize. ## Smoke-test evidence ``` $ PATH_ROOT=$HOME/hero lab service hero_service --example lab service hero_service --example: available examples 01_connect (examples/) ``` Unit tests cover the dispatch logic in all three new modules (`bench_verb`, `example_verb`, `test_verb::sibling_test_crate`). A full end-to-end `lab service hero_router --test layer1` run (cargo test --workspace + cargo run -p hero_router_test) was not exercised in this PR series — the dispatch is unit-tested and the manual `lab service hero_router --test layer1` invocation starts `cargo test --workspace` against the correct repo path. Closing.