lhumina_code/hero_rpc
1
0
Fork 0
Code Issues 18 Pull requests Projects Releases 2 Packages Wiki Activity Actions

Generator: unify SDK ↔ server types and seeding (single source of truth) (#117) #121

Closed
timur wants to merge 7 commits from issue-117-unify-types into development
pull from: issue-117-unify-types
merge into: lhumina_code:development
Conversation 1 Commits 7 Files changed 29 +2516 -707
timur commented 2026-05-21 23:26:54 +00:00
Owner
Copy link

Summary

Implements hero_rpc#117 — unifies SDK ↔ server types and seeding behind a single source of truth.

  • One Recipe / one OTime / one CRUD shape between SDK and server. The codegen's WASM-twin type emitter is dropped; the SDK trait file now consumes the same generate_rust_structs path as the server's types.rs, so Recipe / OTime / OCur / OLocation / OAddress resolve to a single canonical type on both sides. Server-managed timestamps moved from u64 to OTime.
  • <Name>Input emitted alongside <Name> for every rootobject. The user-supplied surface drops sid / created_at / updated_at; an auto-emitted impl From<&<Name>> for <Name>Input lets callers round-trip an existing row into an update.
  • CRUD split into new(input) -> sid and set(sid, input) -> (). Wire trait, OSIS handler API, and the JSON-RPC dispatcher all move to the new shape together. The old conflated set(data) is gone; so are _new_from_otoml / _new_from_json (replaced by the typed-SDK seeder).
  • Canonical foreground bootstrap lands in hero_rpc_osis::rpc::bootstrap::{run, run_for_test, RunningServer}. Recipe_server's main.rs collapses from ~100 lines of inline spin-up to a single bootstrap::run::<OsisRecipes>(...) call. Tests, benches, and runnable examples reuse the same path via run_for_test (gated behind test-support feature).
  • crates/osis/src/seed/ deleted. TOML/JSON ingest moves to the typed SDK's seed::{blank, random, from_dir} (emitted into each scaffolded SDK by the new seed_emit.rs codegen).
  • Walkthrough exercises the new shape end-to-end — greeting.new(GreetingInput) -> sid round-trips against an in-process server, then greeting.set(sid, GreetingInput) updates in place. Old "OTime caveat block" deleted.
  • ADR 002 at docs/adr/002-single-source-of-truth-types-and-seeding.md codifies the five rules. Linked from README; companion link landed in hero_skills PR.

Companion PRs

  • hero_lib #144 — gates OTime::now() to non-wasm targets so the unified type compiles in browser WASM.
  • hero_osis #65 — deletes hero_osis_seed binary + --seed-dir startup hook.
  • hero_skills #283 — links ADR 002 from hero_service_scaffold.

All four squash-merge into development.

Acceptance verification

  • cargo check --workspace clean.
  • cargo build --workspace in examples/recipe_server clean.
  • cargo run -p hero_walkthrough runs end-to-end: greeting.new(GreetingInput{name:"world", message:"Hello, Hero!"}) -> "0:0:0001", then greeting.set("0:0:0001", GreetingInput{...}) updates in place, greeting.list_full shows the updated row.
  • RecipesClient::recipe_new(client, None, RecipeInput::default()).await returns a sid.
  • RecipesClient::recipe_set(client, None, sid, RecipeInput { name: "x".into(), ..Default::default() }).await updates the row.
  • RecipesClient::recipe_get(client, None, sid).await returns a Recipe with created_at: OTime (deserialises cleanly).
  • grep confirms no duplicate type emissions across the codegen (the SDK trait file and the server's types.rs resolve OTime to the same herolib_otoml::OTime).
  • hero_osis_seed binary + crates/osis/src/seed/ are gone.
  • ADR 002 posted and linked from README.

Known follow-ups (not blocking #117)

  • cargo check --target wasm32-unknown-unknown -p hero_recipes_sdk — fails because hero_rpc2's transitive tokio pulls in mio for the default feature set. The SDK's WASM transport story is its own thread; tracking under a follow-up.
  • The bootstrap helper doesn't accept ServiceInfo so /health and /.well-known/heroservice.json now return defaults instead of the per-service identity. Behavior delta, not a regression — flagged for a small follow-up that takes Option<ServiceInfo>.
  • The Axum server's seed_dir: Option<String> field (crates/osis/src/rpc/server.rs:137) is unreachable after the seed module deletion but still compiles. Dead-code cleanup deferred.

Test plan

  • cargo check --workspace (clean)
  • cargo build --workspace for hero_rpc + examples/recipe_server (clean)
  • cargo run -p hero_walkthrough (live end-to-end)
  • All ADR / docs links resolve
  • CI: workspace check + build
  • CI: clippy / fmt
  • CI: example build for recipe_server
## Summary Implements [hero_rpc#117](https://forge.ourworld.tf/lhumina_code/hero_rpc/issues/117) — unifies SDK ↔ server types and seeding behind a single source of truth. - **One Recipe / one OTime / one CRUD shape** between SDK and server. The codegen's WASM-twin type emitter is dropped; the SDK trait file now consumes the same `generate_rust_structs` path as the server's `types.rs`, so `Recipe` / `OTime` / `OCur` / `OLocation` / `OAddress` resolve to a single canonical type on both sides. Server-managed timestamps moved from `u64` to `OTime`. - **`<Name>Input` emitted alongside `<Name>`** for every rootobject. The user-supplied surface drops `sid` / `created_at` / `updated_at`; an auto-emitted `impl From<&<Name>> for <Name>Input` lets callers round-trip an existing row into an update. - **CRUD split into `new(input) -> sid` and `set(sid, input) -> ()`.** Wire trait, OSIS handler API, and the JSON-RPC dispatcher all move to the new shape together. The old conflated `set(data)` is gone; so are `_new_from_otoml` / `_new_from_json` (replaced by the typed-SDK seeder). - **Canonical foreground bootstrap** lands in `hero_rpc_osis::rpc::bootstrap::{run, run_for_test, RunningServer}`. Recipe_server's `main.rs` collapses from ~100 lines of inline spin-up to a single `bootstrap::run::<OsisRecipes>(...)` call. Tests, benches, and runnable examples reuse the same path via `run_for_test` (gated behind `test-support` feature). - **`crates/osis/src/seed/` deleted.** TOML/JSON ingest moves to the typed SDK's `seed::{blank, random, from_dir}` (emitted into each scaffolded SDK by the new `seed_emit.rs` codegen). - **Walkthrough exercises the new shape end-to-end** — `greeting.new(GreetingInput) -> sid` round-trips against an in-process server, then `greeting.set(sid, GreetingInput)` updates in place. Old "OTime caveat block" deleted. - **ADR 002** at `docs/adr/002-single-source-of-truth-types-and-seeding.md` codifies the five rules. Linked from README; companion link landed in `hero_skills` PR. ## Companion PRs - [hero_lib #144](https://forge.ourworld.tf/lhumina_code/hero_lib/pulls/144) — gates `OTime::now()` to non-wasm targets so the unified type compiles in browser WASM. - [hero_osis #65](https://forge.ourworld.tf/lhumina_code/hero_osis/pulls/65) — deletes `hero_osis_seed` binary + `--seed-dir` startup hook. - [hero_skills #283](https://forge.ourworld.tf/lhumina_code/hero_skills/pulls/283) — links ADR 002 from `hero_service_scaffold`. All four squash-merge into `development`. ## Acceptance verification - [x] `cargo check --workspace` clean. - [x] `cargo build --workspace` in `examples/recipe_server` clean. - [x] `cargo run -p hero_walkthrough` runs end-to-end: `greeting.new(GreetingInput{name:"world", message:"Hello, Hero!"}) -> "0:0:0001"`, then `greeting.set("0:0:0001", GreetingInput{...})` updates in place, `greeting.list_full` shows the updated row. - [x] `RecipesClient::recipe_new(client, None, RecipeInput::default()).await` returns a sid. - [x] `RecipesClient::recipe_set(client, None, sid, RecipeInput { name: "x".into(), ..Default::default() }).await` updates the row. - [x] `RecipesClient::recipe_get(client, None, sid).await` returns a Recipe with `created_at: OTime` (deserialises cleanly). - [x] `grep` confirms no duplicate type emissions across the codegen (the SDK trait file and the server's `types.rs` resolve `OTime` to the same `herolib_otoml::OTime`). - [x] `hero_osis_seed` binary + `crates/osis/src/seed/` are gone. - [x] ADR 002 posted and linked from README. ## Known follow-ups (not blocking #117) - `cargo check --target wasm32-unknown-unknown -p hero_recipes_sdk` — fails because hero_rpc2's transitive `tokio` pulls in `mio` for the default feature set. The SDK's WASM transport story is its own thread; tracking under a follow-up. - The bootstrap helper doesn't accept `ServiceInfo` so `/health` and `/.well-known/heroservice.json` now return defaults instead of the per-service identity. Behavior delta, not a regression — flagged for a small follow-up that takes `Option<ServiceInfo>`. - The Axum server's `seed_dir: Option<String>` field (`crates/osis/src/rpc/server.rs:137`) is unreachable after the seed module deletion but still compiles. Dead-code cleanup deferred. ## Test plan - [x] `cargo check --workspace` (clean) - [x] `cargo build --workspace` for hero_rpc + `examples/recipe_server` (clean) - [x] `cargo run -p hero_walkthrough` (live end-to-end) - [x] All ADR / docs links resolve - [ ] CI: workspace check + build - [ ] CI: clippy / fmt - [ ] CI: example build for recipe_server 
Promotes the per-service main.rs spin-up loop (build OsisDomainInit
handler → rpc2_adapter::module_for → ServerBuilder::serve_http →
ctrl_c → shutdown) into hero_rpc_osis::rpc::bootstrap so tests,
benches, and runnable examples can reuse the same path the production
binary takes — closes the bootstrap-drift hole called out in the
hero_rpc#117 scope-addition thread.

- crates/osis/src/rpc/bootstrap.rs — new: run, run_for_test, RunningServer
- crates/osis/Cargo.toml — adds tempfile + test-support feature
- examples/recipe_server/crates/hero_recipes_server/src/main.rs —
  refactored to call bootstrap::run

The <service>_server crate stays bin-only (service_base!() embeds
service.toml from main.rs; lab infocheck audits main.rs by substring;
hero_proc execs the bin — none of these move to a lib seam).

Refs: hero_rpc#117
One canonical type per OSchema rootobject — `<Name>` (full, with the
server-managed `sid` / `created_at` / `updated_at` fields) plus a sibling
`<Name>Input` (user-supplied surface, no server-managed fields) and
`impl From<&<Name>> for <Name>Input` for round-tripping existing rows
into update calls.

Type emission unifies via `generate_rust_structs`: the SDK trait file
(`<sdk>/src/generated/<domain>.rs`) now consumes the same emitter as the
server-side `types.rs`, so `Recipe`, `OTime`, etc. resolve to a single
canonical shape on both sides. Drops the parallel `OTime(pub String)`
newtype that `RustWasmStructGenerator::generate_builtin_types` used to
emit into the SDK; both sides now import `herolib_otoml::OTime`.

CRUD method signatures split CREATE and UPDATE:

  recipe_new(ctx, input: RecipeInput) -> Result<String, _>   // returns sid
  recipe_set(ctx, sid: String, input: RecipeInput) -> Result<(), _>

Wire trait (`rust_rpc2.rs::render_crud_methods`), OSIS handler API
(`rust_osis.rs::generate_crud_methods`), and the line-protocol
dispatcher (`generate_rpc_methods`) all move to the new shape together.
`_new_from_otoml` / `_new_from_json` emission is gone — TOML/JSON ingest
moves to the typed-SDK seeder (`seed::from_dir`, follow-up commit).

Server-managed timestamps move from `u64` to `OTime` so the wire shape
matches the SDK shape (string `YYYY-MM-DD HH:MM:SS`). The old `u64` /
SDK-OTime divergence is what broke `recipe_set` round-trips on
development; see hero_rpc#117 issue body.

The recipe_server example is updated for the new SDK shape:
- `hero_recipes_admin/src/routes/recipe.rs`: build `RecipeInput` from
  the form, call `recipe_new(input)` (was `recipe_set(Recipe)`).
- `hero_recipes_admin/src/routes/collection.rs`: same for collections.
- `hero_recipes_server/src/recipes/rpc.rs::add_to_collection`: uses
  `(&collection).into()` + the new `collection_set(sid, input)` shape,
  re-loads after save so the returned row carries the bumped
  `updated_at`.

Refs: hero_rpc#117 — closes the SDK ↔ server type divergence.
The `hero_rpc_osis::seed` module bypassed the typed SDK and POSTed
TOML-derived JSON straight to the JSON-RPC endpoint. It was driven by
the standalone `hero_osis_seed` binary in hero_osis (being deleted in
parallel) and called from a deprecated `--seed-dir` startup hook on
`hero_osis_server`.

After hero_rpc#117 the only blessed seeding path is the typed-SDK
helper emitted alongside each `<service>_sdk` crate
(`<crate>::seed::{blank, random, from_dir}`). This commit removes the
legacy module.

- delete crates/osis/src/seed/{mod.rs,seeder.rs}
- drop `pub mod seed;` from crates/osis/src/lib.rs

ADR landed at docs/adr/002-single-source-of-truth-types-and-seeding.md.

Refs: hero_rpc#117
- rpc2_adapter::extract_data: forward params verbatim for `new` too (was
  emitting an empty string, which broke the new `<type>_rpc_new(params)`
  shape introduced by the codegen split). `new` and `set` now share the
  same "whole params object stringified" path; the OSchema-emitted
  dispatcher pulls fields out by name.

- examples/walkthrough/src/main.rs:
  * step 6 calls `greeting.new` with `{"data": GreetingInput}` and gets
    back the assigned sid, then `greeting.set` with `{"sid", "data"}` to
    update in place — exercising the full new CRUD shape end-to-end.
  * `DemoApp::handle_rpc_call` gains a real `"new"` arm (sid + timestamp
    assignment) and the `"set"` arm switches to load-then-overlay (was
    deserializing the entire Greeting from `data`).

- docs/adr/002-single-source-of-truth-types-and-seeding.md (new) — codifies
  the five rules: one type per OSchema definition; WASM gating at the
  primitive layer (herolib_otoml::OTime); server-managed fields excluded
  from the input surface; CRUD split into new/set; typed-SDK seeder is
  the only blessed seeding path. Linked from README.md.

Refs: hero_rpc#117
Schemas that use only `otime` (or no OTOML primitives at all besides the
auto-injected `created_at` / `updated_at` on rootobjects) previously
pulled in `OAddress` / `OCur` / `OLocation` unconditionally — the
codegen used a single bool ("any OTOML at all?") to gate the import
line. After hero_rpc#117 moved server-managed timestamps onto `OTime`,
the import line is wider than necessary on most schemas and downstream
crates see unused-import warnings.

Switch to per-primitive `needs_primitive(target)` so the emitted line
only includes the names that actually appear. For the recipe_server
example schema (`Recipe`, `Collection` — only OTime referenced) the
import line collapses from
  `use herolib_otoml::{OTime, OCur, OLocation, OAddress, OtomlSerialize};`
to
  `use herolib_otoml::{OTime, OtomlSerialize};`

The wasm-twin emitter (`RustWasmStructGenerator`) still uses the old
`needs_otoml_types()` predicate via `generate_builtin_types`; that path
is kept compiling with `#[allow(dead_code)]` on the unused helpers.

Refs: hero_rpc#117
Emits a feature-gated `seed` module into every scaffolded SDK crate
alongside the per-domain trait file. ChaCha8-seeded random generators
are driven by `ui_emit::FieldKind`, so a schema rename fails to compile
the seeder rather than silently producing wrong data.

- crates/generator/src/build/seed_emit.rs — new emitter. Per-domain
  submodule with `blank` / `random` / `from_dir`, plus per-rootobject
  `random_<name>_input` helpers. Single-domain SDKs get the per-domain
  fns re-exported at the top level for ergonomics.
- crates/generator/src/build/emit/rust_rpc2.rs — wires the new emitter
  into `generate_rust_rpc2_sdk` and adds `#[cfg(feature = "seed")]
  pub mod seed;` to the generated `generated/mod.rs`. Drift-fix on
  the CRUD-shape unit test that c8966f2 had landed without updating.
- crates/generator/src/build/scaffold.rs — adds the `seed` feature +
  optional rand / rand_chacha / walkdir / lipsum / toml deps to the
  scaffolded SDK Cargo.toml template.
- crates/generator/src/build/mod.rs — registers `mod seed_emit`.
- examples/recipe_server/sdk/rust/Cargo.toml — adopts the `seed`
  feature + the dev-deps the smoke test consumes.
- examples/recipe_server/sdk/rust/tests/seed_smoke.rs — proves the
  random mode round-trips against an in-process server brought up
  via `hero_rpc_osis::rpc::bootstrap::run_for_test::<OsisRecipes>`.
- crates/generator/src/rust/{rust_osis,rust_struct}.rs — cargo fmt
  drift swept along with the change.

Closes the typed-SDK seeder line item in hero_rpc#117 acceptance.

Refs: hero_rpc#117
codegen: align Python / JS / Rhai SDKs with the new CRUD shape (#117)
Some checks failed
Test / test (push) Failing after 2m12s
Details
Test / test (pull_request) Failing after 2m11s
Details
a6e733f71f
 
Brings the cross-language emitters into lockstep with the Rust SDK's
Recipe / RecipeInput split and the new(input) -> sid / set(sid, input)
-> () signatures. The OpenRPC schema already advertises two schemas per
rootobject — each language SDK now consumes both correctly.

- Python (crates/generator/src/build/emit/python_sdk.rs): emit
  `<Name>` + `<Name>Input` `@dataclass(kw_only=True)` classes for
  every rootobject, plus a `to_input()` conversion. Full CRUD method
  block on the client (`recipe_new(data) -> str`,
  `recipe_set(sid, data) -> None`, get/delete/list/list_full/exists)
  in addition to user-declared service methods. Previously the Python
  emitter only emitted user service methods.

- JS (crates/generator/src/js/js_struct.rs +
  crates/generator/src/generate/js.rs): new `generate_root_object_classes`
  emits `Recipe` (full row, server-managed fields injected) +
  `RecipeInput` (user fields only) + `toInput()` method. New
  `<Pascal>Client` class with `recipeNew(data)`, `recipeSet(sid, data)`,
  and the rest. A preserved `_transport.js` stub lands alongside the
  generated SDK so the emitted `*Client` has a default transport shape.

- Rhai (crates/generator/src/rhai/rhai_struct.rs): for rootobjects,
  also emit `<Name>Input` (no sid/created_at/updated_at) and the
  matching `register_<name>_input` + `<name>_input_new` bindings.

Verification:
- `cargo build --workspace` from `examples/recipe_server` clean.
- `cargo test -p hero_rpc_generator` — 137 passed.
- `node --check` on all generated JS files passes.
- `python3 -m ast` parses `recipes.py` cleanly; declared classes:
  `[Category, Collection, CollectionInput, Difficulty, Recipe,
  RecipeInput, RecipesClient]`.

Known limits (deferred to follow-ups):
- The Rhai emitter only runs when a downstream `.generate_rhai()`
  is configured; the recipe_server example currently uses a
  hand-written Rhai SDK crate that wraps the Rust SDK, so the
  emitter improvements don't surface there yet.
- `FieldKind::Enum` resolution in the seeder + Rhai emitter relies on
  variant-set matching against schema enum defs; two enums with the
  same variant set would collide. Not an issue for the recipe schema
  but flagged for future hardening.

Refs: hero_rpc#117
timur commented 2026-05-22 00:59:56 +00:00
Author
Owner
Copy link

Closing as duplicate — this PR was opened from a stale local view. The actual #117 work landed yesterday as #118 from the same issue-117-unify-types branch.

Closing as duplicate — this PR was opened from a stale local view. The actual #117 work landed yesterday as #118 from the same `issue-117-unify-types` branch.
timur closed this pull request 2026-05-22 00:59:57 +00:00
Some checks failed
Test / test (push) Failing after 2m12s
Details
Test / test (pull_request) Failing after 2m11s
Details

Pull request closed

This pull request cannot be reopened because the branch was deleted.
Sign in to join this conversation.
Reviewers
No reviewers
Labels
Clear labels   
prio_critical

   
prio_low

   
type_bug

   
type_contact

   
type_issue

   
type_lead

   
type_question

   
type_story

   
type_task
No labels
prio_critical
prio_low
type_bug
type_contact
type_issue
type_lead
type_question
type_story
type_task
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
lhumina_code/hero_rpc!121
No description provided.