#60 - hero_proxy shows all services as Down — probe protocol mismatch - lhumina_code/home

mik-tf commented

2026-03-20 01:17:20 +00:00

Owner

Problem

hero_proxy_ui shows 31 sockets but almost all as 'Down'. Only hero_indexer_server shows 'OK'.

Root cause

hero_proxy_server probes sockets with raw line-delimited JSON-RPC (rpc.health). But most services are HTTP-based (axum) and don't understand raw JSON-RPC — they expect HTTP requests.

Services that use HeroRpcServer bind HTTP on their socket, so the raw JSON-RPC probe gets a 400 Bad Request.

Fix

hero_proxy_server's probe should:

Try HTTP first: GET /health via HTTP over Unix socket
Fall back to raw JSON-RPC: {"method":"rpc.health"} for non-HTTP services

Steps to reproduce

Run hero_zero Docker container
Open http://localhost:9090/hero_proxy_ui
All services show 'Down' except hero_indexer_server

## Problem hero_proxy_ui shows 31 sockets but almost all as 'Down'. Only hero_indexer_server shows 'OK'. ## Root cause hero_proxy_server probes sockets with raw line-delimited JSON-RPC (`rpc.health`). But most services are HTTP-based (axum) and don't understand raw JSON-RPC — they expect HTTP requests. Services that use HeroRpcServer bind HTTP on their socket, so the raw JSON-RPC probe gets a 400 Bad Request. ## Fix hero_proxy_server's probe should: 1. Try HTTP first: `GET /health` via HTTP over Unix socket 2. Fall back to raw JSON-RPC: `{"method":"rpc.health"}` for non-HTTP services ## Steps to reproduce 1. Run hero_zero Docker container 2. Open http://localhost:9090/hero_proxy_ui 3. All services show 'Down' except hero_indexer_server

mik-tf commented

2026-03-20 03:20:02 +00:00

Author

Owner

Fixed: changed hero_proxy_server to probe HTTP first (GET /health), then raw JSON-RPC, then simple TCP connect as fallback. Now 33/33 services show OK including Redis and hero_services_server. Squash-merged to development.

Signed-off-by: mik-tf

Fixed: changed hero_proxy_server to probe HTTP first (GET /health), then raw JSON-RPC, then simple TCP connect as fallback. Now 33/33 services show OK including Redis and hero_services_server. Squash-merged to development. Signed-off-by: mik-tf