lhumina_code/hero_biz
0
0
Fork 0
Code Issues 5 Pull requests Projects Releases 5 Packages Wiki Activity Actions

Route AiClient through local AI Broker instead of direct provider calls #15

New issue
Open
opened 2026-05-04 09:30:14 +00:00 by casper-stevens · 1 comment
casper-stevens commented 2026-05-04 09:30:14 +00:00
Member
Copy link

Problem

The current AiClient in crates/hero_biz_ui/src/ai/client.rs is an OpenAI-compatible HTTP client that calls external AI providers directly via base_url + api_key on AiProvider. This breaks the intended Hero stack architecture.

Required Architecture

All AI calls must flow through the local AI Broker:

Tool / Service
   ↓
AI Client
   ↓
hero_aibroker (local)
   ↓
hero_aibroker (mother) → External providers

The broker already exists (hero_aibroker) and exposes an OpenAI-compatible REST surface at ~/hero/var/sockets/hero_aibroker/rest.sock.

SDK Gap — Blocker

hero_biz uses non-streaming chat exclusively (analyze_intent, generate_suggestions, all assistant calls go through AiClient.chat()). The current hero_aibroker_sdk only provides:

  • AIBrokerAdminAPIClient — admin RPC only (health, models, info), no chat
  • StreamingClient — forces stream: true, streaming only
  • AIBrokerRawClient — raw JSON-RPC; chat is not exposed over the RPC socket

There is no non-streaming chat client in the SDK. This must be resolved first.

Path Forward

Step 1 (prerequisite — work in hero_aibroker): Add a non-streaming REST client to hero_aibroker_sdk that POSTs to rest.sock at /v1/chat/completions without forcing streaming. Something like a RestClient alongside the existing StreamingClient.

Step 2 (this repo): Replace hero_biz's AiClient with the new SDK client:

  • Remove AiProvider, AiConfig, and all provider/API key config from hero_biz — that becomes the broker's concern
  • Remove the try_chat fallback loop — failover is handled inside the broker
  • Wire chat(), transcribe(), and tts() calls to the broker REST socket via the SDK

References

  • crates/hero_biz_ui/src/ai/client.rs — AiClient, try_chat, try_transcribe, try_tts
  • crates/hero_biz_ui/src/ai/config.rs — AiProvider, AiConfig, AiModel
  • hero_aibroker/crates/hero_aibroker_sdk/src/streaming.rs — existing StreamingClient (streaming only)
  • hero_aibroker/crates/hero_aibroker_sdk/src/lib.rs — AIBrokerAdminAPIClient, AIBrokerRawClient
  • Meeting: Engineering flow / Ubuntu 4 May — Section 8 (AI Client Cleanup)
## Problem The current `AiClient` in `crates/hero_biz_ui/src/ai/client.rs` is an OpenAI-compatible HTTP client that calls external AI providers directly via `base_url` + `api_key` on `AiProvider`. This breaks the intended Hero stack architecture. ## Required Architecture All AI calls must flow through the local AI Broker: ``` Tool / Service ↓ AI Client ↓ hero_aibroker (local) ↓ hero_aibroker (mother) → External providers ``` The broker already exists (`hero_aibroker`) and exposes an OpenAI-compatible REST surface at `~/hero/var/sockets/hero_aibroker/rest.sock`. ## SDK Gap — Blocker hero_biz uses **non-streaming chat exclusively** (`analyze_intent`, `generate_suggestions`, all assistant calls go through `AiClient.chat()`). The current `hero_aibroker_sdk` only provides: - `AIBrokerAdminAPIClient` — admin RPC only (health, models, info), no chat - `StreamingClient` — forces `stream: true`, streaming only - `AIBrokerRawClient` — raw JSON-RPC; chat is not exposed over the RPC socket There is no non-streaming chat client in the SDK. This must be resolved first. ## Path Forward **Step 1 (prerequisite — work in `hero_aibroker`):** Add a non-streaming REST client to `hero_aibroker_sdk` that POSTs to `rest.sock` at `/v1/chat/completions` without forcing streaming. Something like a `RestClient` alongside the existing `StreamingClient`. **Step 2 (this repo):** Replace hero_biz's `AiClient` with the new SDK client: - Remove `AiProvider`, `AiConfig`, and all provider/API key config from hero_biz — that becomes the broker's concern - Remove the `try_chat` fallback loop — failover is handled inside the broker - Wire `chat()`, `transcribe()`, and `tts()` calls to the broker REST socket via the SDK ## References - `crates/hero_biz_ui/src/ai/client.rs` — `AiClient`, `try_chat`, `try_transcribe`, `try_tts` - `crates/hero_biz_ui/src/ai/config.rs` — `AiProvider`, `AiConfig`, `AiModel` - `hero_aibroker/crates/hero_aibroker_sdk/src/streaming.rs` — existing `StreamingClient` (streaming only) - `hero_aibroker/crates/hero_aibroker_sdk/src/lib.rs` — `AIBrokerAdminAPIClient`, `AIBrokerRawClient` - Meeting: Engineering flow / Ubuntu 4 May — Section 8 (AI Client Cleanup)
mik-tf added this to the ACTIVE project 2026-05-06 17:32:00 +00:00
casper-stevens commented 2026-05-08 11:48:41 +00:00
Author
Member
Copy link

Blocked by lhumina_code/hero_aibroker#63

This issue cannot start until hero_aibroker_sdk exposes a non-streaming REST chat client. The SDK currently only has StreamingClient and AIBrokerAdminAPIClient — neither supports a plain POST /v1/chat/completions without forcing streaming.

The prerequisite work is tracked in hero_aibroker#63 — Consolidate herolib_ai into hero_aibroker_sdk. Specifically, step 1 of that issue (add from_default_socket() + non-streaming chat to HeroAibrokerClient) must land before this can proceed.

**Blocked by** lhumina_code/hero_aibroker#63 This issue cannot start until `hero_aibroker_sdk` exposes a non-streaming REST chat client. The SDK currently only has `StreamingClient` and `AIBrokerAdminAPIClient` — neither supports a plain `POST /v1/chat/completions` without forcing streaming. The prerequisite work is tracked in [hero_aibroker#63 — Consolidate herolib_ai into hero_aibroker_sdk](https://forge.ourworld.tf/lhumina_code/hero_aibroker/issues/63). Specifically, step 1 of that issue (add `from_default_socket()` + non-streaming chat to `HeroAibrokerClient`) must land before this can proceed.
Sign in to join this conversation.
No Branch/Tag specified
Branches Tags
development
main
v0.5.0-rc1
v0.1.3-rc4
v0.1.3-rc3
v0.1.3-rc2
development_mik_biz_from_ci
v0.1.3-rc1
v0.1.2
v0.1.1
v0.1.0
v0.0.1-rc1
Labels
Clear labels   
prio_critical

   
prio_low

   
type_bug

   
type_contact

   
type_issue

   
type_lead

   
type_question

   
type_story

   
type_task
No labels
prio_critical
prio_low
type_bug
type_contact
type_issue
type_lead
type_question
type_story
type_task
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
ACTIVE
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Depends on
#63 Consolidate herolib_ai into hero_aibroker_sdk (single AI client)
lhumina_code/hero_aibroker
Reference
lhumina_code/hero_biz#15
No description provided.