`RemoteIndex` client uses newline-delimited JSON; `hero_indexer_server` speaks HTTP/1.1 #37

New issue

Open

opened 2026-05-01 15:07:26 +00:00 by mik-tf · 0 comments

mik-tf commented

2026-05-01 15:07:26 +00:00

Owner

Problem

hero_rpc_osis::index::RemoteIndex (the official async client for hero_indexer) sends raw newline-delimited JSON over Unix sockets:

// crates/osis/src/index/remote.rs:282-292
let mut request_json =
    serde_json::to_string(&request).map_err(|e| IndexError::QueryParse(e.to_string()))?;
request_json.push('\n');

let mut conn = self.connection.lock().await;
conn.writer
    .write_all(request_json.as_bytes())
    .await
    .map_err(IndexError::Io)?;
conn.writer.flush().await.map_err(IndexError::Io)?;

The current hero_indexer_server (lhumina_code/hero_indexer) accepts only HTTP/1.1 at POST /rpc — it does not parse the newline-delimited protocol. Result: every RemoteIndex.index_document(...) call returns "Broken pipe (os error 32)" because the HTTP server can't parse the malformed-by-its-standards request line.

Reproduction

let client = hero_rpc_osis::index::RemoteIndex::connect(
    "/path/to/hero_indexer/rpc.sock",
    "mydb",
    &["title"],
).await?;
client.index_document("doc1", &[("title".into(), "Hello".into())]).await?;
// → Err("Broken pipe (os error 32)")

hero_indexer_server log shows the request being received but rejected with a malformed-HTTP error.

Workaround (downstream)

Bypass the official client and write a small HTTP/1.1 client over the Unix socket directly. The hero_embedder crate has used this pattern since its initial integration; mirroring it works fine. See index_add in any consumer that integrates with hero_indexer_server.

Suggested fixes (any one is sufficient)

Update RemoteIndex to use HTTP/1.1. Mirrors what the embedder client does; matches the server.
Re-add newline-delimited support to hero_indexer_server. If the older protocol is intended to remain canonical.
Document the protocol mismatch + recommended client pattern. At minimum so consumers don't burn a session on the "Broken pipe" diagnosis.

Impact

Currently every consumer that uses the official RemoteIndex client silently fails on first write. The error doesn't point at a protocol mismatch — it looks like a connection problem. Both downstream consumers and any future hero_indexer consumer hit this on day one.

Not blocking — workaround is small. Filing as ecosystem-correctness signal.

## Problem `hero_rpc_osis::index::RemoteIndex` (the official async client for `hero_indexer`) sends raw newline-delimited JSON over Unix sockets: ```rust // crates/osis/src/index/remote.rs:282-292 let mut request_json = serde_json::to_string(&request).map_err(|e| IndexError::QueryParse(e.to_string()))?; request_json.push('\n'); let mut conn = self.connection.lock().await; conn.writer .write_all(request_json.as_bytes()) .await .map_err(IndexError::Io)?; conn.writer.flush().await.map_err(IndexError::Io)?; ``` The current `hero_indexer_server` (`lhumina_code/hero_indexer`) accepts only HTTP/1.1 at `POST /rpc` — it does not parse the newline-delimited protocol. Result: every `RemoteIndex.index_document(...)` call returns `"Broken pipe (os error 32)"` because the HTTP server can't parse the malformed-by-its-standards request line. ## Reproduction ```rust let client = hero_rpc_osis::index::RemoteIndex::connect( "/path/to/hero_indexer/rpc.sock", "mydb", &["title"], ).await?; client.index_document("doc1", &[("title".into(), "Hello".into())]).await?; // → Err("Broken pipe (os error 32)") ``` `hero_indexer_server` log shows the request being received but rejected with a malformed-HTTP error. ## Workaround (downstream) Bypass the official client and write a small HTTP/1.1 client over the Unix socket directly. The `hero_embedder` crate has used this pattern since its initial integration; mirroring it works fine. See `index_add` in any consumer that integrates with `hero_indexer_server`. ## Suggested fixes (any one is sufficient) 1. **Update `RemoteIndex` to use HTTP/1.1.** Mirrors what the embedder client does; matches the server. 2. **Re-add newline-delimited support to `hero_indexer_server`.** If the older protocol is intended to remain canonical. 3. **Document the protocol mismatch + recommended client pattern.** At minimum so consumers don't burn a session on the "Broken pipe" diagnosis. ## Impact Currently every consumer that uses the official `RemoteIndex` client silently fails on first write. The error doesn't point at a protocol mismatch — it looks like a connection problem. Both downstream consumers and any future `hero_indexer` consumer hit this on day one. Not blocking — workaround is small. Filing as ecosystem-correctness signal.