Fold indexer-backed find into the openrpc_server! macro #159

New issue

Open

opened 2026-06-22 15:05:40 +00:00 by mik-tf · 0 comments

mik-tf commented

2026-06-22 15:05:40 +00:00

Owner

On the macro path, generated *_find methods are hand-written stubs that ignore their filter params and full-scan the store with list() (for example company_find in hero_biz, and the same in cm50_app), so the OSIS indexer is never used. The machinery already exists in hero_blueprint (the OsisIndexer client, the typed FindParams filters, and a to_indexer_query lowering), but nothing wires it end to end. The goal is that every service migrated onto openrpc_server! gets real indexed queries with no per-service code, before the wider rollout.

Proposed approach, reusing the existing Hooks seam rather than adding new codegen: when a domain has @index fields, serve_domains auto-registers a generic hook set built from the spec's index metadata. A before-hook on find lowers the request's FindParams to an indexer query, runs OsisIndexer::search, and returns the sids, short-circuiting the stub body. After-hooks on set and delete do the index write-through (index_document and delete_document). The to_indexer_query lowering moves to a runtime helper in the osis crate so there is one source and no per-type emission. OsisIndexer changes from log-and-swallow to a hard error when the indexer is down, both per call and with a startup probe. Once the macro owns find, the generator under hero_blueprint/crates/generator is moved to an archive directory and the hand-written *_find stubs in hero_biz and cm50_app are dropped, since they inherit the macro behavior. This spans hero_lib (the macro and serve_domains) and hero_blueprint (the osis helper, the hard-error change, the generator archive).

@timur, before I start, please confirm the four design calls:

Indexer middleware via auto-registered hooks, versus macro-generated method bodies.
to_indexer_query as a runtime helper in the osis crate, versus emitted per type.
Write-through hard-error semantics. The store commits before indexing, so if indexing then errors, the call returns an error while the store has already changed, and the store and index can diverge. Do you want fail-after-commit (the client retries, so index_document must be idempotent), index-before-commit, or a reconcile path.
Startup behavior. Fail the service if the indexer socket is not up, or degrade with a loud error until it appears.

Context: lhumina_code/home#309

Signed-by: mik-tf mik-tf@noreply.invalid

On the macro path, generated `*_find` methods are hand-written stubs that ignore their filter params and full-scan the store with `list()` (for example `company_find` in hero_biz, and the same in cm50_app), so the OSIS indexer is never used. The machinery already exists in hero_blueprint (the `OsisIndexer` client, the typed `FindParams` filters, and a `to_indexer_query` lowering), but nothing wires it end to end. The goal is that every service migrated onto `openrpc_server!` gets real indexed queries with no per-service code, before the wider rollout. Proposed approach, reusing the existing `Hooks` seam rather than adding new codegen: when a domain has `@index` fields, `serve_domains` auto-registers a generic hook set built from the spec's index metadata. A before-hook on find lowers the request's `FindParams` to an indexer query, runs `OsisIndexer::search`, and returns the sids, short-circuiting the stub body. After-hooks on set and delete do the index write-through (`index_document` and `delete_document`). The `to_indexer_query` lowering moves to a runtime helper in the osis crate so there is one source and no per-type emission. `OsisIndexer` changes from log-and-swallow to a hard error when the indexer is down, both per call and with a startup probe. Once the macro owns find, the generator under hero_blueprint/crates/generator is moved to an archive directory and the hand-written `*_find` stubs in hero_biz and cm50_app are dropped, since they inherit the macro behavior. This spans hero_lib (the macro and `serve_domains`) and hero_blueprint (the osis helper, the hard-error change, the generator archive). @timur, before I start, please confirm the four design calls: 1. Indexer middleware via auto-registered hooks, versus macro-generated method bodies. 2. `to_indexer_query` as a runtime helper in the osis crate, versus emitted per type. 3. Write-through hard-error semantics. The store commits before indexing, so if indexing then errors, the call returns an error while the store has already changed, and the store and index can diverge. Do you want fail-after-commit (the client retries, so `index_document` must be idempotent), index-before-commit, or a reconcile path. 4. Startup behavior. Fail the service if the indexer socket is not up, or degrade with a loud error until it appears. Context: https://forge.ourworld.tf/lhumina_code/home/issues/309 Signed-by: mik-tf <mik-tf@noreply.invalid>