lhumina_code/hero_rpc
1
0
Fork 0
Code Issues 17 Pull requests Projects Releases 2 Packages Wiki Activity Actions

Separate generated from handwritten code: generated/ subfolders + gitignore #96

New issue
Closed
opened 2026-05-20 22:51:56 +00:00 by timur · 4 comments
timur commented 2026-05-20 22:51:56 +00:00
Owner
Copy link

Problem

Feedback from a contributor reviewing hero_service: it's hard to tell at a glance what's code-generated vs what a human wrote. Today the generator scatters *_generated.rs files into the same src/<domain>/ directory as handwritten code (rpc.rs, mod.rs, etc.). All of it gets committed to git, so:

  • A casual reader doesn't know which files they're allowed to edit.
  • A rename or restructure regenerates files that get checked in → noisy diffs, merge conflicts on every build.
  • New contributors waste time trying to fix codegen output by hand.

The fix is to physically separate generated artifacts and gitignore them so the working tree only shows what humans own.

Solution

Folder layout

Generated files move into a dedicated generated/ subfolder under each consumer. Example for hero_service:

crates/hero_service/src/catalog/
├── mod.rs                       # handwritten — declares `pub mod generated;`
├── handlers.rs                  # handwritten — business logic stubs (the preserved trait impl)
└── generated/                   # ALL CODEGEN OUTPUT, gitignored
    ├── mod.rs                   # re-exports everything below
    ├── types.rs                 # was: types_generated.rs
    ├── server.rs                # was: osis_server_generated.rs
    └── rpc.rs                   # was: rpc_generated.rs

docs/                            # gitignored at root: openrpc.json + per-domain
└── openrpc.json + <domain>/openrpc.json

sdk/
├── rust/src/                    # mixed — handwritten lib.rs + generated/ subfolder
│   └── generated/               # gitignored
├── rhai/src/generated/          # gitignored
├── js/src/                      # gitignored (the .js sources); package.json stays committed
└── python/<package>/            # gitignored; pyproject.toml stays committed

Note: the _generated.rs suffix becomes redundant once the folder name carries the signal. Drop it — generated/types.rs reads cleanly.

Gitignore strategy

Per-crate .gitignore entries the scaffolder emits + the contributor never edits:

# crates/<name>/src/<domain>/.gitignore
generated/

Repo-root .gitignore (also scaffolded):

# Codegen output — produced by `cargo build` (which runs the generator via build.rs)
docs/openrpc.json
docs/*/openrpc.json
sdk/rust/src/generated/
sdk/rhai/src/generated/
sdk/js/src/
sdk/python/*/             # package sources only — pyproject.toml etc. stay
!sdk/python/pyproject.toml
!sdk/python/README.md

Exact wildcards to be tuned during implementation — the principle is: anything code-emitted is ignored; anything scaffolded once and edited by humans is committed.

What stays committed (do not gitignore)

  • service.toml
  • Cargo.toml, workspace Cargo.lock
  • schemas/**/*.oschema (the input to codegen)
  • crates/<name>_server/src/<domain>/handlers.rs (the preserved trait impl)
  • crates/<name>_admin/** (admin UI is hand-written)
  • crates/<name>/src/<domain>/mod.rs — scaffolded once, declares pub mod generated;, owned by the contributor afterward
  • sdk/js/package.json, sdk/python/pyproject.toml, README files, etc.
  • docs/ other than the codegen-emitted openrpc files (the directory must still exist in tree to satisfy crate include_str! paths if any — verify)

mod.rs strategy

The parent mod.rs (scaffolded once) declares the generated submodule + handles any re-exports:

// crates/hero_service/src/catalog/mod.rs — scaffolded once, then yours
pub mod generated;
pub use generated::*;

mod handlers;  // hand-written trait impl
pub use handlers::*;

The generated/mod.rs is itself codegen-emitted and re-exports its siblings. Scaffolder writes the parent mod.rs once; the generator writes everything inside generated/ on every build.

What to do

  1. Audit current emitter outputs. List every file the generator writes today (crates/generator/src/generate/*.rs modules: rust_types, rust_server, rust_rpc, openrpc, js, rhai, python). Tag each: "generated → moves to generated/" vs "scaffolded once → stays where it is."
  2. Update emit paths. Each generator module redirects its output under a generated/ subfolder. Drop the _generated.rs suffix from the resulting filename since the folder name carries the signal.
  3. Update scaffolder. When scaffolding a new service crate, emit:
    • The parent mod.rs with pub mod generated; pub use generated::*;
    • A per-domain .gitignore ignoring generated/
    • A repo-root .gitignore ignoring docs/openrpc.json + docs/*/openrpc.json + sdk/rust/src/generated/ + sdk/rhai/src/generated/ + sdk/js/src/ + sdk/python/<package>/
  4. Apply to hero_service template. Regenerate it so the contributor can clone it and immediately see the new layout. (git status after cargo build should be clean — every generated file is gitignored.)
  5. Apply to recipe_server example. Same thing — in-tree reference matches the new shape.
  6. Update hero_service_scaffold.md skill so it documents the new layout + gitignore convention.

Acceptance

  • Fresh scaffold produces a repo where git status after cargo build is empty (every generated file is gitignored).
  • crates/<name>/src/<domain>/ contains at most: mod.rs + scaffolded human files + the generated/ subfolder. No mixed-purpose dir.
  • Friend opening the hero_service repo on Forgejo can tell what's code vs cruft within 30 seconds.
  • cargo build --workspace clean on both recipe_server and hero_service after regen.
  • lab infocheck clean.
  • hero_service_scaffold.md reflects the new layout.

Out of scope

  • Compile-time validation that a file in generated/ was actually generated by the generator (cute idea, lots of friction).
  • A cargo gen / standalone codegen step (we're keeping codegen inside cargo build via build.rs).
  • Wholesale rename of _generated.rs files in repos other than hero_service + recipe_server — those services need migration but it's a sweep in hero_rpc#90 phase 3, not this issue.

Related

  • Parent META: hero_skills#262
  • Triggered by friend feedback on the hero_service template's readability.
  • Plays into hero_rpc#90 phase 2/3 — when services migrate to hero_rpc2 dispatch, they pick up the new layout simultaneously.
## Problem Feedback from a contributor reviewing `hero_service`: it's hard to tell at a glance what's code-generated vs what a human wrote. Today the generator scatters `*_generated.rs` files into the same `src/<domain>/` directory as handwritten code (`rpc.rs`, `mod.rs`, etc.). All of it gets committed to git, so: - A casual reader doesn't know which files they're allowed to edit. - A rename or restructure regenerates files that get checked in → noisy diffs, merge conflicts on every build. - New contributors waste time trying to fix codegen output by hand. The fix is to **physically separate** generated artifacts and **gitignore** them so the working tree only shows what humans own. ## Solution ### Folder layout Generated files move into a dedicated `generated/` subfolder under each consumer. Example for `hero_service`: ``` crates/hero_service/src/catalog/ ├── mod.rs # handwritten — declares `pub mod generated;` ├── handlers.rs # handwritten — business logic stubs (the preserved trait impl) └── generated/ # ALL CODEGEN OUTPUT, gitignored ├── mod.rs # re-exports everything below ├── types.rs # was: types_generated.rs ├── server.rs # was: osis_server_generated.rs └── rpc.rs # was: rpc_generated.rs docs/ # gitignored at root: openrpc.json + per-domain └── openrpc.json + <domain>/openrpc.json sdk/ ├── rust/src/ # mixed — handwritten lib.rs + generated/ subfolder │ └── generated/ # gitignored ├── rhai/src/generated/ # gitignored ├── js/src/ # gitignored (the .js sources); package.json stays committed └── python/<package>/ # gitignored; pyproject.toml stays committed ``` Note: the `_generated.rs` suffix becomes redundant once the folder name carries the signal. Drop it — `generated/types.rs` reads cleanly. ### Gitignore strategy **Per-crate `.gitignore`** entries the scaffolder emits + the contributor never edits: ``` # crates/<name>/src/<domain>/.gitignore generated/ ``` **Repo-root `.gitignore`** (also scaffolded): ``` # Codegen output — produced by `cargo build` (which runs the generator via build.rs) docs/openrpc.json docs/*/openrpc.json sdk/rust/src/generated/ sdk/rhai/src/generated/ sdk/js/src/ sdk/python/*/ # package sources only — pyproject.toml etc. stay !sdk/python/pyproject.toml !sdk/python/README.md ``` Exact wildcards to be tuned during implementation — the principle is: anything code-emitted is ignored; anything scaffolded once and edited by humans is committed. ### What stays committed (do not gitignore) - `service.toml` - `Cargo.toml`, workspace `Cargo.lock` - `schemas/**/*.oschema` (the input to codegen) - `crates/<name>_server/src/<domain>/handlers.rs` (the preserved trait impl) - `crates/<name>_admin/**` (admin UI is hand-written) - `crates/<name>/src/<domain>/mod.rs` — scaffolded once, declares `pub mod generated;`, owned by the contributor afterward - `sdk/js/package.json`, `sdk/python/pyproject.toml`, README files, etc. - `docs/` other than the codegen-emitted openrpc files (the directory must still exist in tree to satisfy crate `include_str!` paths if any — verify) ### `mod.rs` strategy The parent `mod.rs` (scaffolded once) declares the generated submodule + handles any re-exports: ```rust // crates/hero_service/src/catalog/mod.rs — scaffolded once, then yours pub mod generated; pub use generated::*; mod handlers; // hand-written trait impl pub use handlers::*; ``` The `generated/mod.rs` is itself codegen-emitted and re-exports its siblings. Scaffolder writes the parent `mod.rs` once; the generator writes everything inside `generated/` on every build. ## What to do 1. **Audit current emitter outputs.** List every file the generator writes today (`crates/generator/src/generate/*.rs` modules: rust_types, rust_server, rust_rpc, openrpc, js, rhai, python). Tag each: "generated → moves to `generated/`" vs "scaffolded once → stays where it is." 2. **Update emit paths.** Each generator module redirects its output under a `generated/` subfolder. Drop the `_generated.rs` suffix from the resulting filename since the folder name carries the signal. 3. **Update scaffolder.** When scaffolding a new service crate, emit: - The parent `mod.rs` with `pub mod generated; pub use generated::*;` - A per-domain `.gitignore` ignoring `generated/` - A repo-root `.gitignore` ignoring `docs/openrpc.json` + `docs/*/openrpc.json` + `sdk/rust/src/generated/` + `sdk/rhai/src/generated/` + `sdk/js/src/` + `sdk/python/<package>/` 4. **Apply to `hero_service` template.** Regenerate it so the contributor can clone it and immediately see the new layout. (`git status` after `cargo build` should be clean — every generated file is gitignored.) 5. **Apply to `recipe_server` example.** Same thing — in-tree reference matches the new shape. 6. **Update `hero_service_scaffold.md` skill** so it documents the new layout + gitignore convention. ## Acceptance - Fresh scaffold produces a repo where `git status` after `cargo build` is empty (every generated file is gitignored). - `crates/<name>/src/<domain>/` contains at most: `mod.rs` + scaffolded human files + the `generated/` subfolder. No mixed-purpose dir. - Friend opening the `hero_service` repo on Forgejo can tell what's code vs cruft within 30 seconds. - `cargo build --workspace` clean on both `recipe_server` and `hero_service` after regen. - `lab infocheck` clean. - `hero_service_scaffold.md` reflects the new layout. ## Out of scope - Compile-time validation that a file in `generated/` was actually generated by the generator (cute idea, lots of friction). - A `cargo gen` / standalone codegen step (we're keeping codegen inside `cargo build` via `build.rs`). - Wholesale rename of `_generated.rs` files in repos other than `hero_service` + `recipe_server` — those services need migration but it's a sweep in [hero_rpc#90](https://forge.ourworld.tf/lhumina_code/hero_rpc/issues/90) phase 3, not this issue. ## Related - Parent META: [hero_skills#262](https://forge.ourworld.tf/lhumina_code/hero_skills/issues/262) - Triggered by friend feedback on the `hero_service` template's readability. - Plays into [hero_rpc#90](https://forge.ourworld.tf/lhumina_code/hero_rpc/issues/90) phase 2/3 — when services migrate to hero_rpc2 dispatch, they pick up the new layout simultaneously.
timur commented 2026-05-20 23:07:52 +00:00
Author
Owner
Copy link

Design proposal — request for sign-off before implementation

Working in worktree issue-96-generated-folders off development. Audited the seven generator modules (crates/generator/src/generate/{rust_types,rust_server,rust_rpc,openrpc,js,rhai,python}.rs) plus the build-script scaffolders (crates/generator/src/build/emit/{domain,rust_server,rust_rpc2,python_sdk}.rs). Below is the exact final shape per emitter and the gitignore wording. Naming follows the issue body: generated/ subfolder + drop _generated suffix everywhere.

1. Folder layout per emitter

A. Consumer crate (core types) — crates/<name>/src/<domain>/

Was (emitter: generate/rust_types.rs):

mod.rs                    # generated every build (cfg-gates types/wasm/rhai)
types_generated.rs        # generated
types_wasm_generated.rs   # generated
types.rs                  # scaffolded-once, include!("types_generated.rs") wrapper
rhai_types_generated.rs   # generated (when rhai enabled)

Now:

mod.rs                    # scaffolded once → `pub mod generated; pub use generated::*;`
                          #   + optional `mod extensions; pub use extensions::*;` if user adds custom impls
generated/                # ALL CODEGEN — gitignored
  mod.rs                  # regenerated barrel (cfg-gates native/wasm/rhai)
  types.rs                # was types_generated.rs
  types_wasm.rs           # was types_wasm_generated.rs
  rhai_types.rs           # was rhai_types_generated.rs (when rhai enabled)

The current types.rs include!() wrapper trick is dropped — custom impls live in a separate handwritten extensions.rs (or inline in parent mod.rs) sitting beside generated/. Recipe_server's existing types.rs only has commented-out example impls, so deleting it loses nothing.

B. Server crate — crates/<name>_server/src/<domain>/

Was (emitters: generate/rust_server.rs + build/emit/rust_server.rs):

mod.rs                       # generated every build (`pub use <core>::<domain> as core; pub mod ...; ...`)
osis_server_generated.rs     # generated (OSIS dispatcher)
rpc_generated.rs             # generated (legacy JSON-RPC trait + handler stub)
rpc.rs                       # scaffolded-once preserved trait impl (handwritten)
tests.rs                     # generated CRUD round-trip tests
tests_error_category.rs      # handwritten test extras (only recipe_server has this)

Now:

mod.rs                       # scaffolded once →
                             #   `pub mod generated; pub use generated::*;`
                             #   `mod handlers; pub use handlers::*;`
                             #   `#[cfg(test)] mod tests_error_category;` (when handwritten extras exist)
handlers.rs                  # was rpc.rs — scaffolded-once handwritten trait impl, renamed for clarity
tests_error_category.rs      # handwritten extras stay top-level
generated/                   # ALL CODEGEN — gitignored
  mod.rs                     # regenerated barrel
                             #   `pub use <core_crate>::<domain> as core;`  (codegen-required re-export)
                             #   `pub mod server; pub use server::*;`
                             #   `pub mod rpc; pub use rpc::*;`
                             #   `#[cfg(test)] pub mod tests;`
  server.rs                  # was osis_server_generated.rs
  rpc.rs                     # was rpc_generated.rs (codegen RPC trait stub — distinct from top-level handlers.rs)
  tests.rs                   # was tests.rs (generated CRUD round-trip)

Rename note: the existing handwritten rpc.rs → handlers.rs to disambiguate it from the codegen generated/rpc.rs. Recipe_server's rpc.rs is the only file that needs hand-migrating; hero_service template doesn't have one yet (scaffolder writes it fresh).

C. OpenRPC specs — <workspace_root>/docs/

Was (emitter: generate/openrpc.rs):

docs/openrpc.json            # aggregate (workspace-level)
docs/<domain>/openrpc.json   # per-domain

Now: same paths, gitignored. No emitter changes — the contributor reads these from the on-disk path the admin UI / <hero-api-docs> widget already loads. The docs/ directory itself stays in tree (placeholder .gitkeep if needed so include_str!("../core/openrpc.json") resolves before first build — verified during impl).

D. Rust SDK trait crate — sdk/rust/src/

Was (emitter: build/emit/rust_rpc2.rs):

lib.rs                # preserved/scaffolded-once with `#[cfg(feature = "<domain>")] pub mod <domain>;`
<domain>.rs           # regenerated every build (hero_rpc2 trait file)

Now:

lib.rs                # scaffolded once →
                      #   `pub use herolib_sid::SmartId;`
                      #   `pub mod generated; pub use generated::*;`
generated/            # gitignored
  mod.rs              # regenerated → `#[cfg(feature = "<domain>")] pub mod <domain>; #[cfg(feature = "<domain>")] pub use <domain>::*;` per domain
  <domain>.rs         # regenerated trait file (was sdk/rust/src/<domain>.rs)

The @sdk-feature: marker auto-discovery loop in emit_rpc2_lib_rs moves into the generated/mod.rs writer (handwritten sibling modules still get picked up — they just sit beside generated/, not inside it).

E. Rhai SDK crate — sdk/rhai/src/

Was: lib.rs is currently a handwritten 3-line stub; no generate/rhai.rs emit targets it (only the per-domain rhai_types.rs in the consumer crate). No-op for this issue beyond adding the gitignore line — once a Rhai emitter for the SDK lands, it'll naturally write into sdk/rhai/src/generated/.

F. JS SDK — sdk/js/

Was (emitter: generate/js.rs):

package.json          # preserved on first write
src/index.js          # regenerated barrel
src/<domain>.js       # regenerated per-domain module

Now: same paths, src/ gitignored. package.json stays committed; everything under src/ is codegen output.

G. Python SDK — sdk/python/

Was (emitter: build/emit/python_sdk.rs):

pyproject.toml                          # generated on first run, would normally be preserved
hero_<service>_sdk/__init__.py          # regenerated
hero_<service>_sdk/<domain>.py          # regenerated
hero_<service>_sdk/_transport.py        # regenerated

Now:

pyproject.toml                          # scaffolded once — stays committed
README.md                               # if scaffolder emits one — stays committed
hero_<service>_sdk/                     # gitignored (whole package dir is codegen output)

The Python package layout (hero_<service>_sdk/...) is the import contract — moving it under a generated/ subdirectory would change every consumer's import path. Per the issue body, easier to gitignore the whole package dir and keep pyproject.toml + README at the parent level.

2. Gitignore entries

Per-domain .gitignore files (emitted by scaffolder)

Each crates/<name>/src/<domain>/.gitignore and crates/<name>_server/src/<domain>/.gitignore:

# Generated by build.rs — never edit, never commit.
generated/

These are scaffolded once alongside the parent mod.rs and never touched again.

Repo-root .gitignore (scaffolder appends; merges with existing entries)

Appended block (idempotent — scaffolder skips if marker is already present):

# ── OSchema codegen output (hero_rpc#96) ─────────────────────────────
# Per-crate sources live under `crates/<name>{,_server}/src/<domain>/generated/`
# and are covered by per-domain `.gitignore` files. The entries below cover
# cross-crate artifacts (docs, SDK trees).
docs/openrpc.json
docs/*/openrpc.json

sdk/rust/src/generated/
sdk/rhai/src/generated/

sdk/js/src/

sdk/python/*/
!sdk/python/pyproject.toml
!sdk/python/README.md
!sdk/python/.gitignore

sdk/python/*/ ignores every direct subdir of sdk/python/ (i.e. the per-service package dirs like hero_recipes_sdk/) while keeping pyproject.toml + README + the gitignore file itself in tree.

3. What stays committed (recap from the issue body, made concrete)

  • service.toml, root Cargo.toml, workspace Cargo.lock
  • All schemas/**/*.oschema (codegen input)
  • crates/<name>{,_server}/src/<domain>/mod.rs — scaffolded once, contributor-owned afterward
  • crates/<name>_server/src/<domain>/handlers.rs — preserved handwritten trait impl
  • crates/<name>_server/src/<domain>/tests_*.rs — handwritten test extras (e.g. tests_error_category.rs)
  • crates/<name>_admin/** — admin UI is hand-written
  • sdk/rust/src/lib.rs — scaffolded once
  • sdk/rhai/src/lib.rs — handwritten (Rhai SDK emitter doesn't exist yet)
  • sdk/js/package.json
  • sdk/python/pyproject.toml, optional sdk/python/README.md
  • docs/.gitkeep (so the directory exists pre-first-build for any include_str! paths — verified during impl)

4. Scope clarifications

  • Scaffolder responsibility: writes (a) parent mod.rs files, (b) per-domain .gitignore, (c) repo-root .gitignore block, (d) handlers.rs stubs (renamed from rpc.rs). All once-only.
  • Generator responsibility: writes everything inside generated/ (and docs/), every build, unconditionally.
  • examples/rust/ and per-domain e2e examples: out of scope here. They're currently committed; leaving them as-is. Could move to examples/generated/ in a follow-up if friction shows up.
  • Other services migration: recipe_server + hero_service only (per issue body); broader sweep handled in #90 phase 3.

5. Open questions / call-outs

  1. types.rs wrapper drop: confirming the include!("types_generated.rs") pattern goes away. Custom impls move to a sibling extensions.rs (handwritten, declared from parent mod.rs). Recipe_server's types.rs only has commented examples, so safe.
  2. rpc.rs → handlers.rs rename: in the server crate, the handwritten preserved file. Only one repo (recipe_server) needs the hand-migration; hero_service scaffolder gets it right from the start.
  3. docs/ placeholder: if include_str!("../core/openrpc.json") (or the per-domain variant in rust_server.rs) needs the file at compile time, we need it generated before rustc sees the consuming file. build.rs ordering should handle this, but I'll verify with a clean clone + cargo build during impl. If broken, fall back to scaffolding a .gitkeep and ensuring the codegen step runs first via cargo:rerun-if-changed.

Requesting sign-off on the layout + gitignore wording before I start changing emit paths. If the rename rpc.rs → handlers.rs is contentious (e.g. you'd rather keep rpc.rs as the handwritten file and name the generated dispatcher something else), happy to swap — the only constraint is one of the two names has to change.

## Design proposal — request for sign-off before implementation Working in worktree `issue-96-generated-folders` off `development`. Audited the seven generator modules (`crates/generator/src/generate/{rust_types,rust_server,rust_rpc,openrpc,js,rhai,python}.rs`) plus the build-script scaffolders (`crates/generator/src/build/emit/{domain,rust_server,rust_rpc2,python_sdk}.rs`). Below is the exact final shape per emitter and the gitignore wording. Naming follows the issue body: `generated/` subfolder + drop `_generated` suffix everywhere. ### 1. Folder layout per emitter #### A. Consumer crate (core types) — `crates/<name>/src/<domain>/` Was (emitter: `generate/rust_types.rs`): ``` mod.rs # generated every build (cfg-gates types/wasm/rhai) types_generated.rs # generated types_wasm_generated.rs # generated types.rs # scaffolded-once, include!("types_generated.rs") wrapper rhai_types_generated.rs # generated (when rhai enabled) ``` Now: ``` mod.rs # scaffolded once → `pub mod generated; pub use generated::*;` # + optional `mod extensions; pub use extensions::*;` if user adds custom impls generated/ # ALL CODEGEN — gitignored mod.rs # regenerated barrel (cfg-gates native/wasm/rhai) types.rs # was types_generated.rs types_wasm.rs # was types_wasm_generated.rs rhai_types.rs # was rhai_types_generated.rs (when rhai enabled) ``` The current `types.rs` `include!()` wrapper trick is dropped — custom impls live in a separate handwritten `extensions.rs` (or inline in parent `mod.rs`) sitting beside `generated/`. Recipe_server's existing `types.rs` only has commented-out example impls, so deleting it loses nothing. #### B. Server crate — `crates/<name>_server/src/<domain>/` Was (emitters: `generate/rust_server.rs` + `build/emit/rust_server.rs`): ``` mod.rs # generated every build (`pub use <core>::<domain> as core; pub mod ...; ...`) osis_server_generated.rs # generated (OSIS dispatcher) rpc_generated.rs # generated (legacy JSON-RPC trait + handler stub) rpc.rs # scaffolded-once preserved trait impl (handwritten) tests.rs # generated CRUD round-trip tests tests_error_category.rs # handwritten test extras (only recipe_server has this) ``` Now: ``` mod.rs # scaffolded once → # `pub mod generated; pub use generated::*;` # `mod handlers; pub use handlers::*;` # `#[cfg(test)] mod tests_error_category;` (when handwritten extras exist) handlers.rs # was rpc.rs — scaffolded-once handwritten trait impl, renamed for clarity tests_error_category.rs # handwritten extras stay top-level generated/ # ALL CODEGEN — gitignored mod.rs # regenerated barrel # `pub use <core_crate>::<domain> as core;` (codegen-required re-export) # `pub mod server; pub use server::*;` # `pub mod rpc; pub use rpc::*;` # `#[cfg(test)] pub mod tests;` server.rs # was osis_server_generated.rs rpc.rs # was rpc_generated.rs (codegen RPC trait stub — distinct from top-level handlers.rs) tests.rs # was tests.rs (generated CRUD round-trip) ``` **Rename note:** the existing handwritten `rpc.rs` → `handlers.rs` to disambiguate it from the codegen `generated/rpc.rs`. Recipe_server's `rpc.rs` is the only file that needs hand-migrating; hero_service template doesn't have one yet (scaffolder writes it fresh). #### C. OpenRPC specs — `<workspace_root>/docs/` Was (emitter: `generate/openrpc.rs`): ``` docs/openrpc.json # aggregate (workspace-level) docs/<domain>/openrpc.json # per-domain ``` Now: **same paths, gitignored.** No emitter changes — the contributor reads these from the on-disk path the admin UI / `<hero-api-docs>` widget already loads. The `docs/` directory itself stays in tree (placeholder `.gitkeep` if needed so `include_str!("../core/openrpc.json")` resolves before first build — verified during impl). #### D. Rust SDK trait crate — `sdk/rust/src/` Was (emitter: `build/emit/rust_rpc2.rs`): ``` lib.rs # preserved/scaffolded-once with `#[cfg(feature = "<domain>")] pub mod <domain>;` <domain>.rs # regenerated every build (hero_rpc2 trait file) ``` Now: ``` lib.rs # scaffolded once → # `pub use herolib_sid::SmartId;` # `pub mod generated; pub use generated::*;` generated/ # gitignored mod.rs # regenerated → `#[cfg(feature = "<domain>")] pub mod <domain>; #[cfg(feature = "<domain>")] pub use <domain>::*;` per domain <domain>.rs # regenerated trait file (was sdk/rust/src/<domain>.rs) ``` The `@sdk-feature:` marker auto-discovery loop in `emit_rpc2_lib_rs` moves into the `generated/mod.rs` writer (handwritten sibling modules still get picked up — they just sit beside `generated/`, not inside it). #### E. Rhai SDK crate — `sdk/rhai/src/` Was: `lib.rs` is currently a handwritten 3-line stub; no `generate/rhai.rs` emit targets it (only the per-domain `rhai_types.rs` in the consumer crate). No-op for this issue beyond adding the gitignore line — once a Rhai emitter for the SDK lands, it'll naturally write into `sdk/rhai/src/generated/`. #### F. JS SDK — `sdk/js/` Was (emitter: `generate/js.rs`): ``` package.json # preserved on first write src/index.js # regenerated barrel src/<domain>.js # regenerated per-domain module ``` Now: **same paths, `src/` gitignored.** `package.json` stays committed; everything under `src/` is codegen output. #### G. Python SDK — `sdk/python/` Was (emitter: `build/emit/python_sdk.rs`): ``` pyproject.toml # generated on first run, would normally be preserved hero_<service>_sdk/__init__.py # regenerated hero_<service>_sdk/<domain>.py # regenerated hero_<service>_sdk/_transport.py # regenerated ``` Now: ``` pyproject.toml # scaffolded once — stays committed README.md # if scaffolder emits one — stays committed hero_<service>_sdk/ # gitignored (whole package dir is codegen output) ``` The Python package layout (`hero_<service>_sdk/...`) is the import contract — moving it under a `generated/` subdirectory would change every consumer's import path. Per the issue body, easier to gitignore the whole package dir and keep `pyproject.toml` + README at the parent level. ### 2. Gitignore entries #### Per-domain `.gitignore` files (emitted by scaffolder) Each `crates/<name>/src/<domain>/.gitignore` and `crates/<name>_server/src/<domain>/.gitignore`: ```gitignore # Generated by build.rs — never edit, never commit. generated/ ``` These are scaffolded **once** alongside the parent `mod.rs` and never touched again. #### Repo-root `.gitignore` (scaffolder appends; merges with existing entries) Appended block (idempotent — scaffolder skips if marker is already present): ```gitignore # ── OSchema codegen output (hero_rpc#96) ───────────────────────────── # Per-crate sources live under `crates/<name>{,_server}/src/<domain>/generated/` # and are covered by per-domain `.gitignore` files. The entries below cover # cross-crate artifacts (docs, SDK trees). docs/openrpc.json docs/*/openrpc.json sdk/rust/src/generated/ sdk/rhai/src/generated/ sdk/js/src/ sdk/python/*/ !sdk/python/pyproject.toml !sdk/python/README.md !sdk/python/.gitignore ``` `sdk/python/*/` ignores every direct subdir of `sdk/python/` (i.e. the per-service package dirs like `hero_recipes_sdk/`) while keeping `pyproject.toml` + README + the gitignore file itself in tree. ### 3. What stays committed (recap from the issue body, made concrete) - `service.toml`, root `Cargo.toml`, workspace `Cargo.lock` - All `schemas/**/*.oschema` (codegen input) - `crates/<name>{,_server}/src/<domain>/mod.rs` — scaffolded once, contributor-owned afterward - `crates/<name>_server/src/<domain>/handlers.rs` — preserved handwritten trait impl - `crates/<name>_server/src/<domain>/tests_*.rs` — handwritten test extras (e.g. `tests_error_category.rs`) - `crates/<name>_admin/**` — admin UI is hand-written - `sdk/rust/src/lib.rs` — scaffolded once - `sdk/rhai/src/lib.rs` — handwritten (Rhai SDK emitter doesn't exist yet) - `sdk/js/package.json` - `sdk/python/pyproject.toml`, optional `sdk/python/README.md` - `docs/.gitkeep` (so the directory exists pre-first-build for any `include_str!` paths — verified during impl) ### 4. Scope clarifications - **Scaffolder responsibility:** writes (a) parent `mod.rs` files, (b) per-domain `.gitignore`, (c) repo-root `.gitignore` block, (d) `handlers.rs` stubs (renamed from `rpc.rs`). All once-only. - **Generator responsibility:** writes everything inside `generated/` (and `docs/`), every build, unconditionally. - **`examples/rust/` and per-domain e2e examples:** out of scope here. They're currently committed; leaving them as-is. Could move to `examples/generated/` in a follow-up if friction shows up. - **Other services migration:** recipe_server + hero_service only (per issue body); broader sweep handled in #90 phase 3. ### 5. Open questions / call-outs 1. **`types.rs` wrapper drop:** confirming the `include!("types_generated.rs")` pattern goes away. Custom impls move to a sibling `extensions.rs` (handwritten, declared from parent `mod.rs`). Recipe_server's `types.rs` only has commented examples, so safe. 2. **`rpc.rs` → `handlers.rs` rename:** in the server crate, the handwritten preserved file. Only one repo (recipe_server) needs the hand-migration; hero_service scaffolder gets it right from the start. 3. **`docs/` placeholder:** if `include_str!("../core/openrpc.json")` (or the per-domain variant in `rust_server.rs`) needs the file at compile time, we need it generated before rustc sees the consuming file. build.rs ordering should handle this, but I'll verify with a clean clone + `cargo build` during impl. If broken, fall back to scaffolding a `.gitkeep` and ensuring the codegen step runs first via `cargo:rerun-if-changed`. Requesting sign-off on the layout + gitignore wording before I start changing emit paths. If the rename `rpc.rs` → `handlers.rs` is contentious (e.g. you'd rather keep `rpc.rs` as the handwritten file and name the generated dispatcher something else), happy to swap — the only constraint is one of the two names has to change.
timur commented 2026-05-20 23:24:18 +00:00
Author
Owner
Copy link

Design finalized — proceeding with implementation

Two adjustments from the previous comment based on offline sign-off:

1. Skip the rpc.rs → handlers.rs rename

The handwritten preserved file stays rpc.rs. The codegen dispatcher moves to generated/rpc.rs — different module paths (recipes::rpc vs recipes::generated::rpc), no collision. This preserves the convention every existing service already uses and avoids a hand-migration on each service when #90 phase 3 sweeps the broader codebase.

Server crate final shape:

crates/<name>_server/src/<domain>/
├── mod.rs                  # scaffolded once → `pub mod generated; pub use generated::*; mod rpc; pub use rpc::*;`
├── rpc.rs                  # scaffolded-once handwritten trait impl (the convention everyone knows)
├── tests_*.rs              # handwritten test extras (optional, only if the service has them)
└── generated/              # gitignored
    ├── mod.rs              # regenerated barrel
    ├── server.rs           # was osis_server_generated.rs
    ├── rpc.rs              # was rpc_generated.rs (the codegen trait stub the handwritten rpc.rs implements)
    └── tests.rs            # was tests.rs (generated CRUD round-trip)

2. Keep the include!() wrapper pattern for types.rs

Better dev ergonomics: contributor adds impl Recipe { … } directly below the include!() and it lives in the same module scope as the generated struct. Less ceremony than a sibling extensions.rs that has to use super::generated::*; first.

Consumer crate final shape:

crates/<name>/src/<domain>/
├── mod.rs                  # scaffolded once
├── types.rs                # scaffolded once → `include!("generated/types.rs"); // ← add impl blocks below`
└── generated/              # gitignored
    ├── mod.rs              # regenerated barrel (does NOT re-export types — wrapper owns that)
    ├── types.rs            # was types_generated.rs (pure struct/enum defs, no impls)
    ├── types_wasm.rs       # was types_wasm_generated.rs (self-contained, used on wasm32 directly)
    └── rhai_types.rs       # was rhai_types_generated.rs (when rhai enabled)

Parent mod.rs (scaffolded once):

//! <domain> domain — core types.

pub mod generated;

// Native: use the include!()-wrapper so contributors add impl blocks in scope.
#[cfg(not(target_arch = "wasm32"))]
mod types;
#[cfg(not(target_arch = "wasm32"))]
pub use types::*;

// WASM: use generated types directly (self-contained, no impls expected).
#[cfg(target_arch = "wasm32")]
pub use generated::types_wasm::*;

#[cfg(all(feature = "rhai", not(target_arch = "wasm32")))]
pub use generated::rhai_types;

The types.rs wrapper (scaffolded once):

//! Custom impl blocks for the <domain> domain.
//!
//! Generated definitions come from `generated/types.rs`. Add your impl
//! blocks below the include!() — they live in the same module scope.

include!("generated/types.rs");

// ──────────────────────────────────────────────────────────────────
// Add custom impl blocks below. Example:
//
// impl Recipe {
//     pub fn display(&self) -> String { self.name.clone() }
// }
// ──────────────────────────────────────────────────────────────────

generated/mod.rs deliberately does NOT re-export types — the wrapper owns the native re-export. Only types_wasm and rhai_types get pub mod … lines from the regenerated barrel.

Everything else from the previous comment stands

  • Per-domain .gitignore ignoring generated/ (scaffolded once per domain).
  • Repo-root .gitignore block covering docs/openrpc.json, docs/*/openrpc.json, sdk/rust/src/generated/, sdk/rhai/src/generated/, sdk/js/src/, sdk/python/*/ with !sdk/python/pyproject.toml / !sdk/python/README.md exceptions.
  • SDK trees: Rust gets sdk/rust/src/generated/; JS gitignores sdk/js/src/ (only package.json stays); Python gitignores the per-service package dir.
  • OpenRPC stays under <workspace>/docs/, gitignored.
  • examples/rust/ and per-domain e2e examples out of scope.
  • Migration target: example/recipe_server + hero_service template only (per the issue body).

Starting implementation now in issue-96-generated-folders. Will commit + push in small steps and report back when cargo build --workspace on recipe_server leaves git status empty.

## Design finalized — proceeding with implementation Two adjustments from the previous comment based on offline sign-off: ### 1. Skip the `rpc.rs` → `handlers.rs` rename The handwritten preserved file stays `rpc.rs`. The codegen dispatcher moves to `generated/rpc.rs` — different module paths (`recipes::rpc` vs `recipes::generated::rpc`), no collision. This preserves the convention every existing service already uses and avoids a hand-migration on each service when #90 phase 3 sweeps the broader codebase. Server crate final shape: ``` crates/<name>_server/src/<domain>/ ├── mod.rs # scaffolded once → `pub mod generated; pub use generated::*; mod rpc; pub use rpc::*;` ├── rpc.rs # scaffolded-once handwritten trait impl (the convention everyone knows) ├── tests_*.rs # handwritten test extras (optional, only if the service has them) └── generated/ # gitignored ├── mod.rs # regenerated barrel ├── server.rs # was osis_server_generated.rs ├── rpc.rs # was rpc_generated.rs (the codegen trait stub the handwritten rpc.rs implements) └── tests.rs # was tests.rs (generated CRUD round-trip) ``` ### 2. Keep the `include!()` wrapper pattern for `types.rs` Better dev ergonomics: contributor adds `impl Recipe { … }` directly below the `include!()` and it lives in the same module scope as the generated struct. Less ceremony than a sibling `extensions.rs` that has to `use super::generated::*;` first. Consumer crate final shape: ``` crates/<name>/src/<domain>/ ├── mod.rs # scaffolded once ├── types.rs # scaffolded once → `include!("generated/types.rs"); // ← add impl blocks below` └── generated/ # gitignored ├── mod.rs # regenerated barrel (does NOT re-export types — wrapper owns that) ├── types.rs # was types_generated.rs (pure struct/enum defs, no impls) ├── types_wasm.rs # was types_wasm_generated.rs (self-contained, used on wasm32 directly) └── rhai_types.rs # was rhai_types_generated.rs (when rhai enabled) ``` Parent `mod.rs` (scaffolded once): ```rust //! <domain> domain — core types. pub mod generated; // Native: use the include!()-wrapper so contributors add impl blocks in scope. #[cfg(not(target_arch = "wasm32"))] mod types; #[cfg(not(target_arch = "wasm32"))] pub use types::*; // WASM: use generated types directly (self-contained, no impls expected). #[cfg(target_arch = "wasm32")] pub use generated::types_wasm::*; #[cfg(all(feature = "rhai", not(target_arch = "wasm32")))] pub use generated::rhai_types; ``` The `types.rs` wrapper (scaffolded once): ```rust //! Custom impl blocks for the <domain> domain. //! //! Generated definitions come from `generated/types.rs`. Add your impl //! blocks below the include!() — they live in the same module scope. include!("generated/types.rs"); // ────────────────────────────────────────────────────────────────── // Add custom impl blocks below. Example: // // impl Recipe { // pub fn display(&self) -> String { self.name.clone() } // } // ────────────────────────────────────────────────────────────────── ``` `generated/mod.rs` deliberately does NOT re-export `types` — the wrapper owns the native re-export. Only `types_wasm` and `rhai_types` get `pub mod …` lines from the regenerated barrel. ### Everything else from the previous comment stands - Per-domain `.gitignore` ignoring `generated/` (scaffolded once per domain). - Repo-root `.gitignore` block covering `docs/openrpc.json`, `docs/*/openrpc.json`, `sdk/rust/src/generated/`, `sdk/rhai/src/generated/`, `sdk/js/src/`, `sdk/python/*/` with `!sdk/python/pyproject.toml` / `!sdk/python/README.md` exceptions. - SDK trees: Rust gets `sdk/rust/src/generated/`; JS gitignores `sdk/js/src/` (only `package.json` stays); Python gitignores the per-service package dir. - OpenRPC stays under `<workspace>/docs/`, gitignored. - `examples/rust/` and per-domain e2e examples out of scope. - Migration target: `example/recipe_server` + `hero_service` template only (per the issue body). Starting implementation now in `issue-96-generated-folders`. Will commit + push in small steps and report back when `cargo build --workspace` on recipe_server leaves `git status` empty.
timur closed this issue 2026-05-20 23:46:53 +00:00
timur commented 2026-05-20 23:47:00 +00:00
Author
Owner
Copy link

PR #101 merged into development. Acceptance criteria met for recipe_server: cargo build clean, git status empty post-build, no _generated.rs filenames remain. Next: hero_service template PR + hero_service_scaffold.md skill update.

PR https://forge.ourworld.tf/lhumina_code/hero_rpc/pulls/101 merged into development. Acceptance criteria met for recipe_server: cargo build clean, git status empty post-build, no _generated.rs filenames remain. Next: hero_service template PR + hero_service_scaffold.md skill update.
timur commented 2026-05-20 23:54:48 +00:00
Author
Owner
Copy link

Done. All acceptance criteria met across three merged PRs:

PR Repo Status
hero_rpc#101 hero_rpc merged → development
hero_service#6 hero_service merged → development
hero_skills#280 hero_skills merged → development

Acceptance check

  • cargo build --workspace clean on both example/recipe_server and hero_service from a fresh clone.
  • git status after cargo build --workspace is empty in both repos.
  • No *_generated.rs filenames remain anywhere outside target/ / crates/osis/examples/ (legacy fixtures, out of scope per the issue body).
  • hero_service_scaffold.md reflects the new layout — the layout tree calls out the generated/ subfolder, the include!() wrapper pattern for types.rs, the build-ordering edges for SDK/admin crates, and the per-domain + repo-root gitignore shape.

What landed (recap of the design)

Per-domain generated/ subfolders under every consumer crate, server crate, and the Rust SDK. The codegen writes everything inside generated/; the contributor-facing parent mod.rs, types.rs (include!() wrapper for custom impls), and rpc.rs (handwritten trait impl) sit beside generated/ and are scaffolded once. _generated.rs suffix dropped everywhere — folder name carries the signal.

docs/ and the JS/Python SDK trees are gitignored at the repo root; per-domain .gitignore files cover the per-crate generated/ dirs. sdk/rust/ and <svc>_admin/ get a two-line build.rs shim plus a [build-dependencies] hero_<name> entry so cargo runs the core crate's build.rs (which produces both crates' codegen) before either compiles.

Out of scope (deferred to other issues per the issue body)

  • Migration of services other than recipe_server + the hero_service template — handled in hero_rpc#90 phase 3 sweep.
  • Compile-time validation that a file in generated/ was generated by the generator — out of scope, lots of friction, low value.
  • A separate cargo gen step — codegen stays inside cargo build via build.rs.
Done. All acceptance criteria met across three merged PRs: | PR | Repo | Status | |---|---|---| | [hero_rpc#101](https://forge.ourworld.tf/lhumina_code/hero_rpc/pulls/101) | hero_rpc | merged → `development` | | [hero_service#6](https://forge.ourworld.tf/lhumina_code/hero_service/pulls/6) | hero_service | merged → `development` | | [hero_skills#280](https://forge.ourworld.tf/lhumina_code/hero_skills/pulls/280) | hero_skills | merged → `development` | ### Acceptance check - `cargo build --workspace` clean on both `example/recipe_server` and `hero_service` from a fresh clone. - `git status` after `cargo build --workspace` is empty in both repos. - No `*_generated.rs` filenames remain anywhere outside `target/` / `crates/osis/examples/` (legacy fixtures, out of scope per the issue body). - `hero_service_scaffold.md` reflects the new layout — the layout tree calls out the `generated/` subfolder, the `include!()` wrapper pattern for `types.rs`, the build-ordering edges for SDK/admin crates, and the per-domain + repo-root gitignore shape. ### What landed (recap of the design) Per-domain `generated/` subfolders under every consumer crate, server crate, and the Rust SDK. The codegen writes everything inside `generated/`; the contributor-facing parent `mod.rs`, `types.rs` (`include!()` wrapper for custom impls), and `rpc.rs` (handwritten trait impl) sit beside `generated/` and are scaffolded once. `_generated.rs` suffix dropped everywhere — folder name carries the signal. `docs/` and the JS/Python SDK trees are gitignored at the repo root; per-domain `.gitignore` files cover the per-crate `generated/` dirs. `sdk/rust/` and `<svc>_admin/` get a two-line `build.rs` shim plus a `[build-dependencies] hero_<name>` entry so cargo runs the core crate's `build.rs` (which produces both crates' codegen) before either compiles. ### Out of scope (deferred to other issues per the issue body) - Migration of services other than `recipe_server` + the `hero_service` template — handled in [hero_rpc#90](https://forge.ourworld.tf/lhumina_code/hero_rpc/issues/90) phase 3 sweep. - Compile-time validation that a file in `generated/` was generated by the generator — out of scope, lots of friction, low value. - A separate `cargo gen` step — codegen stays inside `cargo build` via `build.rs`.
Sign in to join this conversation.
No Branch/Tag specified
Branches Tags
development
issue-98-ui-scaffolds
issue-90-phase4-cleanup
issue-96-generated-folders
issue-90-adapter-lowercase-methods
issue-85-rootobj-sid-validation
lab/74-fold-wasm-types
issue-57-recipe-server-e2e
issue-60-method-translators
issue-54-scaffold-extend
development_refactor_list_full
main
development_jan
latest
v0.1.0
Labels
Clear labels   
prio_critical

   
prio_low

   
type_bug

   
type_contact

   
type_issue

   
type_lead

   
type_question

   
type_story

   
type_task
No labels
prio_critical
prio_low
type_bug
type_contact
type_issue
type_lead
type_question
type_story
type_task
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
lhumina_code/hero_rpc#96
No description provided.