[CI] test_server_rpc_methods fails because hero_db is not running in build.yaml #58

New issue

Open

opened 2026-05-03 17:10:31 +00:00 by mik-tf · 0 comments

mik-tf commented

2026-05-03 17:10:31 +00:00

Owner

Summary

test_server_rpc_methods in crates/hero_aibroker_examples/tests/integration.rs:99 fails on every CI run because it spawns hero_aibroker_server, which requires a running hero_db socket at /root/hero/var/sockets/hero_db/rpc.sock. CI does not bring hero_db up, so the spawned server exits and the test panics with "hero_aibroker_server did not become healthy within 10s".

This breaks build.yaml's make check test fmt-check lint step on every push to development / main and on every PR.

Failure mode (excerpt from CI run 17712)

Running tests/integration.rs (target/debug/deps/integration-...)
running 1 test
Error: hero_db management socket unreachable at /root/hero/var/sockets/hero_db/rpc.sock; start hero_db before running hero_aibroker_server
Caused by:
    0: connect to hero_db socket /root/hero/var/sockets/hero_db/rpc.sock
    1: No such file or directory (os error 2)
test test_server_rpc_methods ... FAILED
---- test_server_rpc_methods stdout ----
thread 'test_server_rpc_methods' (15508) panicked at crates/hero_aibroker_examples/tests/integration.rs:99:12:
Test assertions failed: "hero_aibroker_server did not become healthy within 10s"

failures:
    test_server_rpc_methods

test result: FAILED. 0 passed; 1 failed; 0 ignored; 0 measured; 0 filtered out; finished in 31.38s
error: test failed, to rerun pass `-p hero_aibroker_examples --test integration`
make: *** [Makefile:110: test] Error 101

This blocks build.yaml only — release.yaml does not run make test, so tagged releases are not affected (verified: v0.1.1 was published cleanly while build.yaml was red).

Two ways to fix

Option A — exclude the test from default CI (cheapest, mechanical)

Mark the test as ignored unless an env var is set, so the body still compiles and gets caught by cargo check, but is skipped under the default make test:

#[test]
#[cfg_attr(not(feature = "live-hero-db"), ignore = "requires running hero_db")]
fn test_server_rpc_methods() {
    // ...
}

…or just #[ignore] it unconditionally and run with cargo test -- --ignored in a separate "live integration" job.

Pros: one line, no infra change. Documents the dependency in source.
Cons: loses signal — the test never runs in CI. Anyone wanting to validate it has to remember the ignored flag locally.

Option B — start `hero_db` in the CI job before `make test` (closest to reality)

Add a "Start hero_db" step in build.yaml before make test runs. Two flavours:

Bring up hero_db via the service_db nu module + hero_proc — closest to production, but pulls hero_proc + nu + hero_skills into the CI image.

Run the hero_db_server binary directly in the background:

cargo build --bin hero_db_server -p hero_db
./target/debug/hero_db_server &
sleep 2  # wait for socket
make test

Pros: the test runs for real on every push.
Cons: ~30-60 s added to every CI run, plus the test now depends on hero_db's source repo (build container needs to clone it). The integration test promise is "this is realistic", and that promise is only valuable if it's enforceable.

Recommendation

Start with Option A to unred the build immediately, file Option B as a follow-up if/when the CI image has all Hero deps available (which fits naturally if build.yaml ever standardises to use service_* install scripts — same shape as the --from-ci rollout in hero_demo#54).

Out of scope for this issue

--from-ci consumer wiring on hero_skills (separate PR, blocked on this only insofar as the release pipeline confidence story).
Any test mocking refactor of hero_aibroker_examples.

Found while cutting v0.1.1 for the --from-ci rollout. The release itself was unblocked by the gate fix in #57.

Signed-off-by: mik-tf

## Summary `test_server_rpc_methods` in `crates/hero_aibroker_examples/tests/integration.rs:99` fails on every CI run because it spawns `hero_aibroker_server`, which requires a running `hero_db` socket at `/root/hero/var/sockets/hero_db/rpc.sock`. CI does not bring `hero_db` up, so the spawned server exits and the test panics with `"hero_aibroker_server did not become healthy within 10s"`. This breaks `build.yaml`'s `make check test fmt-check lint` step on every push to `development` / `main` and on every PR. ## Failure mode (excerpt from CI run [17712](https://forge.ourworld.tf/lhumina_code/hero_aibroker/actions/runs/17712)) ``` Running tests/integration.rs (target/debug/deps/integration-...) running 1 test Error: hero_db management socket unreachable at /root/hero/var/sockets/hero_db/rpc.sock; start hero_db before running hero_aibroker_server Caused by: 0: connect to hero_db socket /root/hero/var/sockets/hero_db/rpc.sock 1: No such file or directory (os error 2) test test_server_rpc_methods ... FAILED ---- test_server_rpc_methods stdout ---- thread 'test_server_rpc_methods' (15508) panicked at crates/hero_aibroker_examples/tests/integration.rs:99:12: Test assertions failed: "hero_aibroker_server did not become healthy within 10s" failures: test_server_rpc_methods test result: FAILED. 0 passed; 1 failed; 0 ignored; 0 measured; 0 filtered out; finished in 31.38s error: test failed, to rerun pass `-p hero_aibroker_examples --test integration` make: *** [Makefile:110: test] Error 101 ``` This blocks `build.yaml` only — `release.yaml` does not run `make test`, so tagged releases are not affected (verified: [v0.1.1](https://forge.ourworld.tf/lhumina_code/hero_aibroker/releases/tag/v0.1.1) was published cleanly while `build.yaml` was red). ## Two ways to fix ### Option A — exclude the test from default CI (cheapest, mechanical) Mark the test as ignored unless an env var is set, so the body still compiles and gets caught by `cargo check`, but is skipped under the default `make test`: ```rust #[test] #[cfg_attr(not(feature = "live-hero-db"), ignore = "requires running hero_db")] fn test_server_rpc_methods() { // ... } ``` …or just `#[ignore]` it unconditionally and run with `cargo test -- --ignored` in a separate "live integration" job. **Pros:** one line, no infra change. Documents the dependency in source. **Cons:** loses signal — the test never runs in CI. Anyone wanting to validate it has to remember the ignored flag locally. ### Option B — start `hero_db` in the CI job before `make test` (closest to reality) Add a "Start hero_db" step in `build.yaml` before `make test` runs. Two flavours: 1. Bring up `hero_db` via the `service_db` nu module + hero_proc — closest to production, but pulls hero_proc + nu + hero_skills into the CI image. 2. Run the `hero_db_server` binary directly in the background: ```bash cargo build --bin hero_db_server -p hero_db ./target/debug/hero_db_server & sleep 2 # wait for socket make test ``` **Pros:** the test runs for real on every push. **Cons:** ~30-60 s added to every CI run, plus the test now depends on hero_db's source repo (build container needs to clone it). The integration test promise is "this is realistic", and that promise is only valuable if it's enforceable. ## Recommendation Start with **Option A** to unred the build immediately, file Option B as a follow-up if/when the CI image has all Hero deps available (which fits naturally if `build.yaml` ever standardises to use `service_*` install scripts — same shape as the `--from-ci` rollout in [hero_demo#54](https://forge.ourworld.tf/lhumina_code/hero_demo/issues/54)). ## Out of scope for this issue - `--from-ci` consumer wiring on hero_skills (separate PR, blocked on this only insofar as the release pipeline confidence story). - Any test mocking refactor of `hero_aibroker_examples`. Found while cutting [v0.1.1](https://forge.ourworld.tf/lhumina_code/hero_aibroker/releases/tag/v0.1.1) for the `--from-ci` rollout. The release itself was unblocked by the gate fix in [#57](https://forge.ourworld.tf/lhumina_code/hero_aibroker/pulls/57). Signed-off-by: mik-tf