Correctness: KeyPool backoff calculation can overflow on very old failures #100

New issue

Open

opened 2026-05-11 13:49:52 +00:00 by thabeta · 0 comments

thabeta commented

2026-05-11 13:49:52 +00:00

Owner

Severity: Low

Location

crates/hero_aibroker_lib/src/providers/keypool.rs — backoff calculation

Finding

The exponential backoff calculation uses the time since last failure:

let backoff_ms = self.config.backoff_base_ms * 2_u64.saturating_pow(state.consecutive_failures.min(30));

While saturating_pow prevents overflow, a key with 30+ consecutive failures gets a backoff of base * 2^30 which is ~536 billion ms (~17 years) at a 1ms base.

Impact

Keys are effectively permanently disabled after 30 failures
No automatic recovery path
Manual intervention required to reset the key

Recommendation

Cap the maximum backoff at a reasonable value (e.g., 1 hour)
Add a max-failure count that triggers permanent disable
Implement automatic reset after a cool-down period

## Severity: Low ## Location `crates/hero_aibroker_lib/src/providers/keypool.rs` — backoff calculation ## Finding The exponential backoff calculation uses the time since last failure: ```rust let backoff_ms = self.config.backoff_base_ms * 2_u64.saturating_pow(state.consecutive_failures.min(30)); ``` While `saturating_pow` prevents overflow, a key with 30+ consecutive failures gets a backoff of `base * 2^30` which is ~536 billion ms (~17 years) at a 1ms base. ## Impact - Keys are effectively permanently disabled after 30 failures - No automatic recovery path - Manual intervention required to reset the key ## Recommendation - Cap the maximum backoff at a reasonable value (e.g., 1 hour) - Add a max-failure count that triggers permanent disable - Implement automatic reset after a cool-down period