68 lines
2.4 KiB
Markdown
68 lines
2.4 KiB
Markdown
# Provider Key Pool
|
|
|
|
## Purpose
|
|
Route generation traffic through multiple provider API keys while hiding transient failures from end users.
|
|
|
|
## Key selection
|
|
- Only keys in `active` state are eligible for first-pass routing.
|
|
- Requests start from the next active key by round robin.
|
|
- A single request must not attempt the same key twice.
|
|
|
|
## Optional proxy behavior
|
|
- A key may have one optional proxy attached.
|
|
- If a proxy exists, the first attempt uses the proxy.
|
|
- If the proxy path fails with a transport error, retry the same key directly.
|
|
- Direct fallback does not bypass other business checks.
|
|
- Current runtime policy reads cooldown and manual-review thresholds from environment:
|
|
- `KEY_COOLDOWN_MINUTES`
|
|
- `KEY_FAILURES_BEFORE_MANUAL_REVIEW`
|
|
|
|
## Retry rules
|
|
Retry on the next key only for:
|
|
- network errors
|
|
- connection failures
|
|
- timeouts
|
|
- provider `5xx`
|
|
|
|
Do not retry on the next key for:
|
|
- validation errors
|
|
- unsupported inputs
|
|
- policy rejections
|
|
- other user-caused provider `4xx`
|
|
|
|
## States
|
|
- `active`
|
|
- `cooldown`
|
|
- `out_of_funds`
|
|
- `manual_review`
|
|
- `disabled`
|
|
|
|
## Transitions
|
|
- `active -> cooldown` on retryable failures
|
|
- `cooldown -> active` after successful automatic recheck
|
|
- `cooldown -> manual_review` after more than 10 consecutive retryable failures across recovery cycles
|
|
- `active|cooldown -> out_of_funds` on confirmed insufficient funds
|
|
- `out_of_funds -> active` only by manual admin action
|
|
- `manual_review -> active` only by manual admin action
|
|
- `active -> disabled` by manual admin action
|
|
|
|
## Current runtime note
|
|
- The current worker implementation already applies proxy-first then direct fallback within one provider-key attempt.
|
|
- The current worker implementation writes `GenerationAttempt.usedProxy` and `GenerationAttempt.directFallbackUsed` for auditability.
|
|
- The current worker implementation also runs a background cooldown-recovery sweep and returns keys to `active` after `cooldownUntil` passes.
|
|
|
|
## Balance tracking
|
|
- Primary source of truth is the provider balance API.
|
|
- Balance refresh runs periodically and also after relevant failures.
|
|
- Telegram admin output must show per-key balance snapshots and the count of keys in `out_of_funds`.
|
|
|
|
## Admin expectations
|
|
Web admin and Telegram admin must both be able to:
|
|
- inspect key state
|
|
- inspect last error category and code
|
|
- inspect balance snapshot and refresh time
|
|
- enable or disable a key
|
|
- return a key from `manual_review`
|
|
- return a key from `out_of_funds`
|
|
- add a new key
|