[hero_agent][P1] UTF-8 byte-slice panic in agent.rs:853 — crashes mid-stream on tool results containing multi-byte chars #17

New issue

Open

opened 2026-05-01 23:38:39 +00:00 by mik-tf · 0 comments

mik-tf commented

2026-05-01 23:38:39 +00:00

Owner

Summary

hero_agent panics with byte index N is not a char boundary when a tool result (most often file_read) contains multi-byte UTF-8 characters and a downstream byte-offset slice lands mid-codepoint. Observed live on herodemo VM 2026-05-01 during a "who are my contacts" agent flow; the agent crashed silently mid-stream and the UI surfaced as "No response received from AI".

The crash kills the response stream while the UI still believes the request is in flight, so the user sees a blank/aborted answer with no error context.

Stack trace (live capture)

thread 'tokio-rt-worker' (14570) panicked at
/data/home/driver/hero/code0/hero_agent/crates/hero_agent/src/agent.rs:853:50:
byte index 500 is not a char boundary; it is inside '─' (bytes 499..502) of
`   1 | //! Hero Biz — business data management backend.
   2 | //!
   3 | //! # Usage
   ...`

The ─ is a 3-byte UTF-8 sequence (0xE2 0x94 0x80); slicing at byte 500 lands inside it. The source string is the rendered output of file_read on a Rust source file with line-prefix box-drawing characters.

Reproduction (live, 2026-05-01)

Geomind context, AI Assistant.
Ask: who are the persons/contacts in my hero biz crm tool?
Agent enters multi-iteration tool loop: search_hero_docs, shell_run, file_read, …
At iteration 10, file_read returns content containing ─ (typical in Rust doc-comment headers).
Agent panics at agent.rs:853:50 while preparing the tool result for the next LLM turn.
UI shows "No response received from AI".

The crash is not deterministic per query — shorter tool flows that don't read box-drawing-laden files complete normally. A retry of the same query a few minutes later succeeded (different tool-iteration path, didn't hit a ─).

Root cause (likely)

agent.rs:853:50 is doing &s[..N] (or equivalent) where N is computed assuming byte = char. Common patterns that trip this:

"truncate tool result to first N bytes for the LLM context window"
"extract a preview substring for logging / audit / OSIS message storage"
"split at a separator at position N"

The fix is one of:

Use s.char_indices() to find a safe slice boundary.
Use s.chars().take(N).collect::<String>() for a char-count cap (slower, allocates).
Use a safe-truncate helper from a crate like unicode-truncate.

A simple floor_char_boundary(N) (stable since Rust 1.80) would also do, walking back from N until landing on a char boundary.

Blast radius

This bites any tool flow whose results contain non-ASCII UTF-8 — which is most of them on this codebase: every file_read of Rust sources with /// doc tables, every README that has em-dashes (—), every nu module with box-drawing in print headers. So most multi-iteration agent queries are at risk; only single-shot queries that don't read source files are safe.

The crash also burns the conversation: the OSIS-store calls afterward (audit entry, usage entry) fail with 404 because the runtime aborted mid-handler. The downstream WARN spam in agent logs (Failed to create usage entry, Failed to list memories) is a symptom of this same crash, not an independent bug.

Suggested fix (minimal)

In hero_agent/crates/hero_agent/src/agent.rs:853 (and a sweep of other byte-slice sites in the same file):

// Before:
let preview = &content[..MAX_PREVIEW];

// After:
let cap = content.floor_char_boundary(MAX_PREVIEW);
let preview = &content[..cap];

Plus a regression test:

#[test]
fn does_not_panic_on_multibyte_at_cutoff() {
    let s = "x".repeat(498) + "─x"; // '─' lands across byte 499..502
    let _ = truncate_for_llm(&s, 500); // must not panic
}

The intended grounding pattern (tool_choice = Specific("search_hero_docs") at iteration 0, per docs_hero/collections/hero_os_guide/service_agent.md) reduces the iteration count and therefore the probability of hitting a ─. Doesn't fix the bug, but reduces exposure.
hero_agent/crates/hero_agent_server/src/routes.rs already swallows the OSIS-store 404s — those are downstream of this crash, not the trigger.

Severity

P1 for demo-visible behaviour (agent silent fail on common queries); P2 by code-quality classification (single-line UTF-8 slicing bug, stable Rust API available).

Signed-off-by: mik-tf

## Summary `hero_agent` panics with `byte index N is not a char boundary` when a tool result (most often `file_read`) contains multi-byte UTF-8 characters and a downstream byte-offset slice lands mid-codepoint. Observed live on herodemo VM 2026-05-01 during a "who are my contacts" agent flow; the agent crashed silently mid-stream and the UI surfaced as `"No response received from AI"`. The crash kills the response stream while the UI still believes the request is in flight, so the user sees a blank/aborted answer with no error context. ## Stack trace (live capture) ``` thread 'tokio-rt-worker' (14570) panicked at /data/home/driver/hero/code0/hero_agent/crates/hero_agent/src/agent.rs:853:50: byte index 500 is not a char boundary; it is inside '─' (bytes 499..502) of ` 1 | //! Hero Biz — business data management backend. 2 | //! 3 | //! # Usage ...` ``` The `─` is a 3-byte UTF-8 sequence (`0xE2 0x94 0x80`); slicing at byte 500 lands inside it. The source string is the rendered output of `file_read` on a Rust source file with line-prefix box-drawing characters. ## Reproduction (live, 2026-05-01) 1. Geomind context, AI Assistant. 2. Ask: `who are the persons/contacts in my hero biz crm tool?` 3. Agent enters multi-iteration tool loop: `search_hero_docs`, `shell_run`, `file_read`, … 4. At iteration 10, `file_read` returns content containing `─` (typical in Rust doc-comment headers). 5. Agent panics at `agent.rs:853:50` while preparing the tool result for the next LLM turn. 6. UI shows `"No response received from AI"`. The crash is **not** deterministic per query — shorter tool flows that don't read box-drawing-laden files complete normally. A retry of the same query a few minutes later succeeded (different tool-iteration path, didn't hit a `─`). ## Root cause (likely) `agent.rs:853:50` is doing `&s[..N]` (or equivalent) where `N` is computed assuming byte = char. Common patterns that trip this: - "truncate tool result to first N bytes for the LLM context window" - "extract a preview substring for logging / audit / OSIS message storage" - "split at a separator at position N" The fix is one of: - Use `s.char_indices()` to find a safe slice boundary. - Use `s.chars().take(N).collect::<String>()` for a char-count cap (slower, allocates). - Use a safe-truncate helper from a crate like `unicode-truncate`. A simple `floor_char_boundary(N)` (stable since Rust 1.80) would also do, walking back from N until landing on a char boundary. ## Blast radius This bites any tool flow whose results contain non-ASCII UTF-8 — which is **most** of them on this codebase: every `file_read` of Rust sources with `///` doc tables, every README that has em-dashes (`—`), every nu module with box-drawing in `print` headers. So most multi-iteration agent queries are at risk; only single-shot queries that don't read source files are safe. The crash also burns the conversation: the OSIS-store calls afterward (audit entry, usage entry) fail with 404 because the runtime aborted mid-handler. The downstream WARN spam in agent logs (`Failed to create usage entry`, `Failed to list memories`) is a symptom of this same crash, not an independent bug. ## Suggested fix (minimal) In `hero_agent/crates/hero_agent/src/agent.rs:853` (and a sweep of other byte-slice sites in the same file): ```rust // Before: let preview = &content[..MAX_PREVIEW]; // After: let cap = content.floor_char_boundary(MAX_PREVIEW); let preview = &content[..cap]; ``` Plus a regression test: ```rust #[test] fn does_not_panic_on_multibyte_at_cutoff() { let s = "x".repeat(498) + "─x"; // '─' lands across byte 499..502 let _ = truncate_for_llm(&s, 500); // must not panic } ``` ## Related - The intended grounding pattern (`tool_choice = Specific("search_hero_docs")` at iteration 0, per docs_hero/collections/hero_os_guide/service_agent.md) reduces the iteration count and therefore the probability of hitting a `─`. Doesn't fix the bug, but reduces exposure. - `hero_agent/crates/hero_agent_server/src/routes.rs` already swallows the OSIS-store 404s — those are downstream of this crash, not the trigger. ## Severity P1 for demo-visible behaviour (agent silent fail on common queries); P2 by code-quality classification (single-line UTF-8 slicing bug, stable Rust API available). Signed-off-by: mik-tf