Generator: unify SDK ↔ server types and seeding (single source of truth) (#117) #118

Merged

timur merged 7 commits from issue-117-unify-types into development 2026-05-21 16:13:59 +00:00

Conversation 0 Commits 7 Files changed 29 +2516 -707

timur commented 2026-05-21 16:13:33 +00:00

Owner

Copy link

Summary

Implements hero_rpc#117 — unifies SDK ↔ server types and seeding behind a single source of truth.

• One Recipe / one OTime / one CRUD shape between SDK and server. The codegen's WASM-twin type emitter is dropped; the SDK trait file now consumes the same generate_rust_structs path as the server's types.rs, so Recipe / OTime / OCur / OLocation / OAddress resolve to a single canonical type on both sides. Server-managed timestamps moved from u64 to OTime.
• <Name>Input emitted alongside <Name> for every rootobject. The user-supplied surface drops sid / created_at / updated_at; an auto-emitted impl From<&<Name>> for <Name>Input lets callers round-trip an existing row into an update.
• CRUD split into new(input) -> sid and set(sid, input) -> (). Wire trait, OSIS handler API, and the JSON-RPC dispatcher all move to the new shape together. The old conflated set(data) is gone; so are _new_from_otoml / _new_from_json (replaced by the typed-SDK seeder).
• Canonical foreground bootstrap lands in hero_rpc_osis::rpc::bootstrap::{run, run_for_test, RunningServer}. Recipe_server's main.rs collapses from ~100 lines of inline spin-up to a single bootstrap::run::<OsisRecipes>(...) call. Tests, benches, and runnable examples reuse the same path via run_for_test (gated behind test-support feature).
• crates/osis/src/seed/ deleted. TOML/JSON ingest moves to the typed SDK's seed::{blank, random, from_dir} (emitted into each scaffolded SDK by the new seed_emit.rs codegen).
• Walkthrough exercises the new shape end-to-end — greeting.new(GreetingInput) -> sid round-trips against an in-process server, then greeting.set(sid, GreetingInput) updates in place. Old "OTime caveat block" deleted.
• ADR 002 at docs/adr/002-single-source-of-truth-types-and-seeding.md codifies the five rules. Linked from README; companion link landed in hero_skills PR.

Companion PRs

• hero_lib #144 — gates OTime::now() to non-wasm targets so the unified type compiles in browser WASM.
• hero_osis #65 — deletes hero_osis_seed binary + --seed-dir startup hook.
• hero_skills #283 — links ADR 002 from hero_service_scaffold.

All four squash-merge into development.

Acceptance verification

• cargo check --workspace clean.
• cargo build --workspace in examples/recipe_server clean.
• cargo run -p hero_walkthrough runs end-to-end: greeting.new(GreetingInput{name:"world", message:"Hello, Hero!"}) -> "0:0:0001", then greeting.set("0:0:0001", GreetingInput{...}) updates in place, greeting.list_full shows the updated row.
• RecipesClient::recipe_new(client, None, RecipeInput::default()).await returns a sid.
• RecipesClient::recipe_set(client, None, sid, RecipeInput { name: "x".into(), ..Default::default() }).await updates the row.
• RecipesClient::recipe_get(client, None, sid).await returns a Recipe with created_at: OTime (deserialises cleanly).
• grep confirms no duplicate type emissions across the codegen (the SDK trait file and the server's types.rs resolve OTime to the same herolib_otoml::OTime).
• hero_osis_seed binary + crates/osis/src/seed/ are gone.
• ADR 002 posted and linked from README.

Known follow-ups (not blocking #117)

• cargo check --target wasm32-unknown-unknown -p hero_recipes_sdk — fails because hero_rpc2's transitive tokio pulls in mio for the default feature set. The SDK's WASM transport story is its own thread; tracking under a follow-up.
• The bootstrap helper doesn't accept ServiceInfo so /health and /.well-known/heroservice.json now return defaults instead of the per-service identity. Behavior delta, not a regression — flagged for a small follow-up that takes Option<ServiceInfo>.
• The Axum server's seed_dir: Option<String> field (crates/osis/src/rpc/server.rs:137) is unreachable after the seed module deletion but still compiles. Dead-code cleanup deferred.

Test plan

• cargo check --workspace (clean)
• cargo build --workspace for hero_rpc + examples/recipe_server (clean)
• cargo run -p hero_walkthrough (live end-to-end)
• All ADR / docs links resolve
• CI: workspace check + build
• CI: clippy / fmt
• CI: example build for recipe_server

## Summary Implements [hero_rpc#117](https://forge.ourworld.tf/lhumina_code/hero_rpc/issues/117) — unifies SDK ↔ server types and seeding behind a single source of truth. - **One Recipe / one OTime / one CRUD shape** between SDK and server. The codegen's WASM-twin type emitter is dropped; the SDK trait file now consumes the same `generate_rust_structs` path as the server's `types.rs`, so `Recipe` / `OTime` / `OCur` / `OLocation` / `OAddress` resolve to a single canonical type on both sides. Server-managed timestamps moved from `u64` to `OTime`. - **`<Name>Input` emitted alongside `<Name>`** for every rootobject. The user-supplied surface drops `sid` / `created_at` / `updated_at`; an auto-emitted `impl From<&<Name>> for <Name>Input` lets callers round-trip an existing row into an update. - **CRUD split into `new(input) -> sid` and `set(sid, input) -> ()`.** Wire trait, OSIS handler API, and the JSON-RPC dispatcher all move to the new shape together. The old conflated `set(data)` is gone; so are `_new_from_otoml` / `_new_from_json` (replaced by the typed-SDK seeder). - **Canonical foreground bootstrap** lands in `hero_rpc_osis::rpc::bootstrap::{run, run_for_test, RunningServer}`. Recipe_server's `main.rs` collapses from ~100 lines of inline spin-up to a single `bootstrap::run::<OsisRecipes>(...)` call. Tests, benches, and runnable examples reuse the same path via `run_for_test` (gated behind `test-support` feature). - **`crates/osis/src/seed/` deleted.** TOML/JSON ingest moves to the typed SDK's `seed::{blank, random, from_dir}` (emitted into each scaffolded SDK by the new `seed_emit.rs` codegen). - **Walkthrough exercises the new shape end-to-end** — `greeting.new(GreetingInput) -> sid` round-trips against an in-process server, then `greeting.set(sid, GreetingInput)` updates in place. Old "OTime caveat block" deleted. - **ADR 002** at `docs/adr/002-single-source-of-truth-types-and-seeding.md` codifies the five rules. Linked from README; companion link landed in `hero_skills` PR. ## Companion PRs - [hero_lib #144](https://forge.ourworld.tf/lhumina_code/hero_lib/pulls/144) — gates `OTime::now()` to non-wasm targets so the unified type compiles in browser WASM. - [hero_osis #65](https://forge.ourworld.tf/lhumina_code/hero_osis/pulls/65) — deletes `hero_osis_seed` binary + `--seed-dir` startup hook. - [hero_skills #283](https://forge.ourworld.tf/lhumina_code/hero_skills/pulls/283) — links ADR 002 from `hero_service_scaffold`. All four squash-merge into `development`. ## Acceptance verification - [x] `cargo check --workspace` clean. - [x] `cargo build --workspace` in `examples/recipe_server` clean. - [x] `cargo run -p hero_walkthrough` runs end-to-end: `greeting.new(GreetingInput{name:"world", message:"Hello, Hero!"}) -> "0:0:0001"`, then `greeting.set("0:0:0001", GreetingInput{...})` updates in place, `greeting.list_full` shows the updated row. - [x] `RecipesClient::recipe_new(client, None, RecipeInput::default()).await` returns a sid. - [x] `RecipesClient::recipe_set(client, None, sid, RecipeInput { name: "x".into(), ..Default::default() }).await` updates the row. - [x] `RecipesClient::recipe_get(client, None, sid).await` returns a Recipe with `created_at: OTime` (deserialises cleanly). - [x] `grep` confirms no duplicate type emissions across the codegen (the SDK trait file and the server's `types.rs` resolve `OTime` to the same `herolib_otoml::OTime`). - [x] `hero_osis_seed` binary + `crates/osis/src/seed/` are gone. - [x] ADR 002 posted and linked from README. ## Known follow-ups (not blocking #117) - `cargo check --target wasm32-unknown-unknown -p hero_recipes_sdk` — fails because hero_rpc2's transitive `tokio` pulls in `mio` for the default feature set. The SDK's WASM transport story is its own thread; tracking under a follow-up. - The bootstrap helper doesn't accept `ServiceInfo` so `/health` and `/.well-known/heroservice.json` now return defaults instead of the per-service identity. Behavior delta, not a regression — flagged for a small follow-up that takes `Option<ServiceInfo>`. - The Axum server's `seed_dir: Option<String>` field (`crates/osis/src/rpc/server.rs:137`) is unreachable after the seed module deletion but still compiles. Dead-code cleanup deferred. ## Test plan - [x] `cargo check --workspace` (clean) - [x] `cargo build --workspace` for hero_rpc + `examples/recipe_server` (clean) - [x] `cargo run -p hero_walkthrough` (live end-to-end) - [x] All ADR / docs links resolve - [ ] CI: workspace check + build - [ ] CI: clippy / fmt - [ ] CI: example build for recipe_server

timur added 7 commits 2026-05-21 16:13:33 +00:00

osis: canonical foreground bootstrap (run / run_for_test) — hero_rpc#117 ... 13e81732f8

Promotes the per-service main.rs spin-up loop (build OsisDomainInit
handler → rpc2_adapter::module_for → ServerBuilder::serve_http →
ctrl_c → shutdown) into hero_rpc_osis::rpc::bootstrap so tests,
benches, and runnable examples can reuse the same path the production
binary takes — closes the bootstrap-drift hole called out in the
hero_rpc#117 scope-addition thread.

- crates/osis/src/rpc/bootstrap.rs — new: run, run_for_test, RunningServer
- crates/osis/Cargo.toml — adds tempfile + test-support feature
- examples/recipe_server/crates/hero_recipes_server/src/main.rs —
  refactored to call bootstrap::run

The <service>_server crate stays bin-only (service_base!() embeds
service.toml from main.rs; lab infocheck audits main.rs by substring;
hero_proc execs the bin — none of these move to a lib seam).

Refs: hero_rpc#117

codegen: unify Recipe / RecipeInput + split _new / _set CRUD (#117) ... c8966f2a16

One canonical type per OSchema rootobject — `<Name>` (full, with the
server-managed `sid` / `created_at` / `updated_at` fields) plus a sibling
`<Name>Input` (user-supplied surface, no server-managed fields) and
`impl From<&<Name>> for <Name>Input` for round-tripping existing rows
into update calls.

Type emission unifies via `generate_rust_structs`: the SDK trait file
(`<sdk>/src/generated/<domain>.rs`) now consumes the same emitter as the
server-side `types.rs`, so `Recipe`, `OTime`, etc. resolve to a single
canonical shape on both sides. Drops the parallel `OTime(pub String)`
newtype that `RustWasmStructGenerator::generate_builtin_types` used to
emit into the SDK; both sides now import `herolib_otoml::OTime`.

CRUD method signatures split CREATE and UPDATE:

  recipe_new(ctx, input: RecipeInput) -> Result<String, _>   // returns sid
  recipe_set(ctx, sid: String, input: RecipeInput) -> Result<(), _>

Wire trait (`rust_rpc2.rs::render_crud_methods`), OSIS handler API
(`rust_osis.rs::generate_crud_methods`), and the line-protocol
dispatcher (`generate_rpc_methods`) all move to the new shape together.
`_new_from_otoml` / `_new_from_json` emission is gone — TOML/JSON ingest
moves to the typed-SDK seeder (`seed::from_dir`, follow-up commit).

Server-managed timestamps move from `u64` to `OTime` so the wire shape
matches the SDK shape (string `YYYY-MM-DD HH:MM:SS`). The old `u64` /
SDK-OTime divergence is what broke `recipe_set` round-trips on
development; see hero_rpc#117 issue body.

The recipe_server example is updated for the new SDK shape:
- `hero_recipes_admin/src/routes/recipe.rs`: build `RecipeInput` from
  the form, call `recipe_new(input)` (was `recipe_set(Recipe)`).
- `hero_recipes_admin/src/routes/collection.rs`: same for collections.
- `hero_recipes_server/src/recipes/rpc.rs::add_to_collection`: uses
  `(&collection).into()` + the new `collection_set(sid, input)` shape,
  re-loads after save so the returned row carries the bumped
  `updated_at`.

Refs: hero_rpc#117 — closes the SDK ↔ server type divergence.

osis: drop crates/osis/src/seed/ — typed-SDK seeder takes over (#117) ... 95cc6ec31a

The `hero_rpc_osis::seed` module bypassed the typed SDK and POSTed
TOML-derived JSON straight to the JSON-RPC endpoint. It was driven by
the standalone `hero_osis_seed` binary in hero_osis (being deleted in
parallel) and called from a deprecated `--seed-dir` startup hook on
`hero_osis_server`.

After hero_rpc#117 the only blessed seeding path is the typed-SDK
helper emitted alongside each `<service>_sdk` crate
(`<crate>::seed::{blank, random, from_dir}`). This commit removes the
legacy module.

- delete crates/osis/src/seed/{mod.rs,seeder.rs}
- drop `pub mod seed;` from crates/osis/src/lib.rs

ADR landed at docs/adr/002-single-source-of-truth-types-and-seeding.md.

Refs: hero_rpc#117

adapter + walkthrough + ADR: round-trip the new CRUD shape (#117) ... 550e7253bc

- rpc2_adapter::extract_data: forward params verbatim for `new` too (was
  emitting an empty string, which broke the new `<type>_rpc_new(params)`
  shape introduced by the codegen split). `new` and `set` now share the
  same "whole params object stringified" path; the OSchema-emitted
  dispatcher pulls fields out by name.

- examples/walkthrough/src/main.rs:
  * step 6 calls `greeting.new` with `{"data": GreetingInput}` and gets
    back the assigned sid, then `greeting.set` with `{"sid", "data"}` to
    update in place — exercising the full new CRUD shape end-to-end.
  * `DemoApp::handle_rpc_call` gains a real `"new"` arm (sid + timestamp
    assignment) and the `"set"` arm switches to load-then-overlay (was
    deserializing the entire Greeting from `data`).

- docs/adr/002-single-source-of-truth-types-and-seeding.md (new) — codifies
  the five rules: one type per OSchema definition; WASM gating at the
  primitive layer (herolib_otoml::OTime); server-managed fields excluded
  from the input surface; CRUD split into new/set; typed-SDK seeder is
  the only blessed seeding path. Linked from README.md.

Refs: hero_rpc#117

codegen: emit precise herolib_otoml imports (#117) ... 903a494c0a

Schemas that use only `otime` (or no OTOML primitives at all besides the
auto-injected `created_at` / `updated_at` on rootobjects) previously
pulled in `OAddress` / `OCur` / `OLocation` unconditionally — the
codegen used a single bool ("any OTOML at all?") to gate the import
line. After hero_rpc#117 moved server-managed timestamps onto `OTime`,
the import line is wider than necessary on most schemas and downstream
crates see unused-import warnings.

Switch to per-primitive `needs_primitive(target)` so the emitted line
only includes the names that actually appear. For the recipe_server
example schema (`Recipe`, `Collection` — only OTime referenced) the
import line collapses from
  `use herolib_otoml::{OTime, OCur, OLocation, OAddress, OtomlSerialize};`
to
  `use herolib_otoml::{OTime, OtomlSerialize};`

The wasm-twin emitter (`RustWasmStructGenerator`) still uses the old
`needs_otoml_types()` predicate via `generate_builtin_types`; that path
is kept compiling with `#[allow(dead_code)]` on the unused helpers.

Refs: hero_rpc#117

codegen: typed-SDK seeder — seed::{blank, random, from_dir} (#117) ... e0f9463b3c

Emits a feature-gated `seed` module into every scaffolded SDK crate
alongside the per-domain trait file. ChaCha8-seeded random generators
are driven by `ui_emit::FieldKind`, so a schema rename fails to compile
the seeder rather than silently producing wrong data.

- crates/generator/src/build/seed_emit.rs — new emitter. Per-domain
  submodule with `blank` / `random` / `from_dir`, plus per-rootobject
  `random_<name>_input` helpers. Single-domain SDKs get the per-domain
  fns re-exported at the top level for ergonomics.
- crates/generator/src/build/emit/rust_rpc2.rs — wires the new emitter
  into `generate_rust_rpc2_sdk` and adds `#[cfg(feature = "seed")]
  pub mod seed;` to the generated `generated/mod.rs`. Drift-fix on
  the CRUD-shape unit test that c8966f2 had landed without updating.
- crates/generator/src/build/scaffold.rs — adds the `seed` feature +
  optional rand / rand_chacha / walkdir / lipsum / toml deps to the
  scaffolded SDK Cargo.toml template.
- crates/generator/src/build/mod.rs — registers `mod seed_emit`.
- examples/recipe_server/sdk/rust/Cargo.toml — adopts the `seed`
  feature + the dev-deps the smoke test consumes.
- examples/recipe_server/sdk/rust/tests/seed_smoke.rs — proves the
  random mode round-trips against an in-process server brought up
  via `hero_rpc_osis::rpc::bootstrap::run_for_test::<OsisRecipes>`.
- crates/generator/src/rust/{rust_osis,rust_struct}.rs — cargo fmt
  drift swept along with the change.

Closes the typed-SDK seeder line item in hero_rpc#117 acceptance.

Refs: hero_rpc#117

codegen: align Python / JS / Rhai SDKs with the new CRUD shape (#117) ... a6e733f71f

Brings the cross-language emitters into lockstep with the Rust SDK's
Recipe / RecipeInput split and the new(input) -> sid / set(sid, input)
-> () signatures. The OpenRPC schema already advertises two schemas per
rootobject — each language SDK now consumes both correctly.

- Python (crates/generator/src/build/emit/python_sdk.rs): emit
  `<Name>` + `<Name>Input` `@dataclass(kw_only=True)` classes for
  every rootobject, plus a `to_input()` conversion. Full CRUD method
  block on the client (`recipe_new(data) -> str`,
  `recipe_set(sid, data) -> None`, get/delete/list/list_full/exists)
  in addition to user-declared service methods. Previously the Python
  emitter only emitted user service methods.

- JS (crates/generator/src/js/js_struct.rs +
  crates/generator/src/generate/js.rs): new `generate_root_object_classes`
  emits `Recipe` (full row, server-managed fields injected) +
  `RecipeInput` (user fields only) + `toInput()` method. New
  `<Pascal>Client` class with `recipeNew(data)`, `recipeSet(sid, data)`,
  and the rest. A preserved `_transport.js` stub lands alongside the
  generated SDK so the emitted `*Client` has a default transport shape.

- Rhai (crates/generator/src/rhai/rhai_struct.rs): for rootobjects,
  also emit `<Name>Input` (no sid/created_at/updated_at) and the
  matching `register_<name>_input` + `<name>_input_new` bindings.

Verification:
- `cargo build --workspace` from `examples/recipe_server` clean.
- `cargo test -p hero_rpc_generator` — 137 passed.
- `node --check` on all generated JS files passes.
- `python3 -m ast` parses `recipes.py` cleanly; declared classes:
  `[Category, Collection, CollectionInput, Difficulty, Recipe,
  RecipeInput, RecipesClient]`.

Known limits (deferred to follow-ups):
- The Rhai emitter only runs when a downstream `.generate_rhai()`
  is configured; the recipe_server example currently uses a
  hand-written Rhai SDK crate that wraps the Rust SDK, so the
  emitter improvements don't surface there yet.
- `FieldKind::Enum` resolution in the seeder + Rhai emitter relies on
  variant-set matching against schema enum defs; two enums with the
  same variant set would collide. Not an issue for the recipe schema
  but flagged for future hardening.

Refs: hero_rpc#117

timur merged commit 5408d78eea into development 2026-05-21 16:13:59 +00:00

timur deleted branch issue-117-unify-types 2026-05-21 16:13:59 +00:00

timur referenced this pull request from a commit 2026-05-21 16:14:00 +00:00

Generator: unify SDK ↔ server types and seeding (single source of truth) (#117) (#118)

timur referenced this pull request 2026-05-21 16:14:23 +00:00

Generator: unify SDK ↔ server types and seeding (single source of truth) #117

timur referenced this pull request 2026-05-22 00:59:56 +00:00

Generator: unify SDK ↔ server types and seeding (single source of truth) (#117) #121

timur referenced this pull request from lhumina_code/hero_skills 2026-05-22 10:09:13 +00:00

[META] Standardize Hero service architecture — template, hero_rpc alignment, framework hoist, skills #262

Sign in to join this conversation.

Reviewers

No reviewers

Labels

Clear labels   

prio_critical

  

prio_low

  

type_bug

  

type_contact

  

type_issue

  

type_lead

  

type_question

  

type_story

  

type_task

No labels

prio_critical

prio_low

type_bug

type_contact

type_issue

type_lead

type_question

type_story

type_task

Milestone

Clear milestone

No items

No milestone

Projects

Clear projects

No items

No project

Assignees

Clear assignees

No assignees

1 participant

Notifications

Subscribe

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

lhumina_code/hero_rpc!118

No description provided.