Initial import
This commit is contained in:
sirily
13
docs/AGENTS.md
Normal file
13
docs/AGENTS.md
Normal file
@@ -0,0 +1,13 @@
|
||||
# AGENTS.md
|
||||
|
||||
## Scope
|
||||
Applies within `docs/`.
|
||||
|
||||
## Purpose
|
||||
Documentation in this directory is operational source-of-truth, not marketing copy.
|
||||
|
||||
## Rules
|
||||
- Keep docs aligned with actual architecture decisions.
|
||||
- Prefer short, decision-complete documents over long narrative text.
|
||||
- When architecture changes, update the relevant document in the same change.
|
||||
- Do not store secrets or live credentials here.
|
||||
52
docs/architecture/repository-layout.md
Normal file
52
docs/architecture/repository-layout.md
Normal file
@@ -0,0 +1,52 @@
|
||||
# Repository Layout
|
||||
|
||||
## Tree
|
||||
```text
|
||||
.
|
||||
|- apps/
|
||||
| |- web/
|
||||
| |- worker/
|
||||
| |- bot/
|
||||
| `- cli/
|
||||
|- packages/
|
||||
| |- config/
|
||||
| |- db/
|
||||
| |- domain/
|
||||
| `- providers/
|
||||
|- docs/
|
||||
| |- plan/
|
||||
| |- architecture/
|
||||
| `- ops/
|
||||
|- infra/
|
||||
| |- compose/
|
||||
| `- caddy/
|
||||
`- scripts/
|
||||
```
|
||||
|
||||
## Directory responsibilities
|
||||
### `apps/web`
|
||||
Owns the browser-facing product and HTTP API entrypoints. It should not own core business rules.
|
||||
|
||||
### `apps/worker`
|
||||
Owns asynchronous and scheduled work. It is the execution surface for image-generation jobs, cleanup, and health polling.
|
||||
|
||||
### `apps/bot`
|
||||
Owns Telegram admin interaction only. Business decisions still belong to `packages/domain`.
|
||||
|
||||
### `apps/cli`
|
||||
Owns operator-facing CLI commands such as `nproxy pair`, `nproxy pair list`, and `nproxy pair revoke`.
|
||||
|
||||
### `packages/config`
|
||||
Owns typed environment contracts and config normalization.
|
||||
|
||||
### `packages/db`
|
||||
Owns database schema, migrations, and data-access utilities.
|
||||
|
||||
### `packages/domain`
|
||||
Owns subscription logic, quota logic, key state transitions, and orchestration rules.
|
||||
|
||||
### `packages/providers`
|
||||
Owns provider-specific adapters and low-level HTTP calls. It should not decide business policy.
|
||||
|
||||
### `infra`
|
||||
Owns deployment templates and reverse-proxy configuration for the single-VPS Docker Compose target.
|
||||
34
docs/architecture/system-overview.md
Normal file
34
docs/architecture/system-overview.md
Normal file
@@ -0,0 +1,34 @@
|
||||
# System Overview
|
||||
|
||||
## Runtime components
|
||||
- `apps/web`: public site, user dashboard, admin UI, HTTP API handlers
|
||||
- `apps/worker`: background jobs for generation execution, reconciliation, cleanup, and health checks
|
||||
- `apps/bot`: Telegram admin bot runtime
|
||||
- `apps/cli`: operator commands executed on the server
|
||||
|
||||
## Shared packages
|
||||
- `packages/config`: environment parsing and config contracts
|
||||
- `packages/db`: Prisma schema, migrations, data access helpers
|
||||
- `packages/domain`: business rules and state machines
|
||||
- `packages/providers`: external adapters for model APIs, payment processor, storage, email, and Telegram
|
||||
|
||||
## Core request flow
|
||||
1. User submits a generation request from the chat UI.
|
||||
2. The web app validates auth, subscription, quota, and request shape.
|
||||
3. The app stores a `GenerationRequest` and enqueues work.
|
||||
4. The worker runs provider routing through the key pool.
|
||||
5. The worker persists `GenerationAttempt` rows for each key-level attempt.
|
||||
6. On the first success, the worker stores assets, marks the request succeeded, and consumes quota.
|
||||
7. The web app exposes polling endpoints until the result is ready.
|
||||
|
||||
## Data boundaries
|
||||
- User-visible request lifecycle lives in `GenerationRequest`.
|
||||
- Key-level retries live in `GenerationAttempt`.
|
||||
- Quota accounting lives in `UsageLedgerEntry`.
|
||||
- Provider key health lives in `ProviderKey` plus status-event history.
|
||||
|
||||
## Failure handling
|
||||
- Retryable provider failures are hidden from the user while eligible keys remain.
|
||||
- User-caused provider failures are terminal for that request.
|
||||
- Balance or quota exhaustion removes a key from active rotation.
|
||||
- Provider-key state transitions must be audited.
|
||||
48
docs/ops/deployment.md
Normal file
48
docs/ops/deployment.md
Normal file
@@ -0,0 +1,48 @@
|
||||
# Deployment Plan
|
||||
|
||||
## Chosen target
|
||||
Deploy on one VPS with Docker Compose.
|
||||
|
||||
## Why this target
|
||||
- The system has multiple long-lived components: web, worker, bot, database, and reverse proxy.
|
||||
- Compose gives predictable service boundaries, easier upgrades, and easier recovery than manually managed host processes.
|
||||
- It keeps the path open for later separation of web, worker, and bot without reworking the repository layout.
|
||||
|
||||
## Expected services
|
||||
- `migrate`: one-shot schema bootstrap job run before app services start
|
||||
- `web`: Next.js app serving the site, dashboard, admin UI, and API routes
|
||||
- `worker`: background job processor
|
||||
- `bot`: Telegram admin bot runtime
|
||||
- `postgres`: primary database
|
||||
- `caddy`: TLS termination and reverse proxy
|
||||
- optional `minio`: self-hosted object storage for single-server deployments
|
||||
|
||||
## Deployment notes
|
||||
- Run one Compose project on a single server.
|
||||
- Keep persistent data in named volumes or external storage.
|
||||
- Keep secrets in server-side environment files or a secret manager.
|
||||
- Back up PostgreSQL and object storage separately.
|
||||
- Prefer Telegram long polling in MVP to avoid an extra public webhook surface for the bot.
|
||||
|
||||
## Upgrade strategy
|
||||
- Build new images.
|
||||
- Run the one-shot database schema job.
|
||||
- Restart `web`, `worker`, and `bot` in the same Compose project.
|
||||
- Roll back by redeploying the previous image set if schema changes are backward compatible.
|
||||
|
||||
## Current database bootstrap state
|
||||
- The current Compose template runs a `migrate` service before `web`, `worker`, and `bot`.
|
||||
- The job runs `prisma migrate deploy` from the committed migration history.
|
||||
- The same bootstrap job also ensures the default MVP `SubscriptionPlan` row exists after migrations.
|
||||
- Schema changes must land with a new committed Prisma migration before deployment.
|
||||
|
||||
## Initial operational checklist
|
||||
- provision VPS
|
||||
- install Docker and Compose plugin
|
||||
- provision DNS and TLS
|
||||
- provision PostgreSQL storage
|
||||
- provision S3-compatible storage or enable local MinIO
|
||||
- create `.env`
|
||||
- deploy Compose stack
|
||||
- run database migration job
|
||||
- verify web health, worker job loop, and bot polling
|
||||
67
docs/ops/provider-key-pool.md
Normal file
67
docs/ops/provider-key-pool.md
Normal file
@@ -0,0 +1,67 @@
|
||||
# Provider Key Pool
|
||||
|
||||
## Purpose
|
||||
Route generation traffic through multiple provider API keys while hiding transient failures from end users.
|
||||
|
||||
## Key selection
|
||||
- Only keys in `active` state are eligible for first-pass routing.
|
||||
- Requests start from the next active key by round robin.
|
||||
- A single request must not attempt the same key twice.
|
||||
|
||||
## Optional proxy behavior
|
||||
- A key may have one optional proxy attached.
|
||||
- If a proxy exists, the first attempt uses the proxy.
|
||||
- If the proxy path fails with a transport error, retry the same key directly.
|
||||
- Direct fallback does not bypass other business checks.
|
||||
- Current runtime policy reads cooldown and manual-review thresholds from environment:
|
||||
- `KEY_COOLDOWN_MINUTES`
|
||||
- `KEY_FAILURES_BEFORE_MANUAL_REVIEW`
|
||||
|
||||
## Retry rules
|
||||
Retry on the next key only for:
|
||||
- network errors
|
||||
- connection failures
|
||||
- timeouts
|
||||
- provider `5xx`
|
||||
|
||||
Do not retry on the next key for:
|
||||
- validation errors
|
||||
- unsupported inputs
|
||||
- policy rejections
|
||||
- other user-caused provider `4xx`
|
||||
|
||||
## States
|
||||
- `active`
|
||||
- `cooldown`
|
||||
- `out_of_funds`
|
||||
- `manual_review`
|
||||
- `disabled`
|
||||
|
||||
## Transitions
|
||||
- `active -> cooldown` on retryable failures
|
||||
- `cooldown -> active` after successful automatic recheck
|
||||
- `cooldown -> manual_review` after more than 10 consecutive retryable failures across recovery cycles
|
||||
- `active|cooldown -> out_of_funds` on confirmed insufficient funds
|
||||
- `out_of_funds -> active` only by manual admin action
|
||||
- `manual_review -> active` only by manual admin action
|
||||
- `active -> disabled` by manual admin action
|
||||
|
||||
## Current runtime note
|
||||
- The current worker implementation already applies proxy-first then direct fallback within one provider-key attempt.
|
||||
- The current worker implementation writes `GenerationAttempt.usedProxy` and `GenerationAttempt.directFallbackUsed` for auditability.
|
||||
- The current worker implementation also runs a background cooldown-recovery sweep and returns keys to `active` after `cooldownUntil` passes.
|
||||
|
||||
## Balance tracking
|
||||
- Primary source of truth is the provider balance API.
|
||||
- Balance refresh runs periodically and also after relevant failures.
|
||||
- Telegram admin output must show per-key balance snapshots and the count of keys in `out_of_funds`.
|
||||
|
||||
## Admin expectations
|
||||
Web admin and Telegram admin must both be able to:
|
||||
- inspect key state
|
||||
- inspect last error category and code
|
||||
- inspect balance snapshot and refresh time
|
||||
- enable or disable a key
|
||||
- return a key from `manual_review`
|
||||
- return a key from `out_of_funds`
|
||||
- add a new key
|
||||
48
docs/ops/telegram-pairing.md
Normal file
48
docs/ops/telegram-pairing.md
Normal file
@@ -0,0 +1,48 @@
|
||||
# Telegram Pairing Flow
|
||||
|
||||
## Goal
|
||||
Allow a new Telegram admin to be approved from the server console without editing the database manually.
|
||||
|
||||
## Runtime behavior
|
||||
### Unpaired user
|
||||
1. A user opens the Telegram bot.
|
||||
2. The bot checks whether `telegram_user_id` is present in the allowlist.
|
||||
3. If not present, the bot creates a pending pairing record with:
|
||||
- Telegram user ID
|
||||
- Telegram username and display name snapshot
|
||||
- pairing code hash
|
||||
- expiration timestamp
|
||||
- status `pending`
|
||||
4. The bot replies with a message telling the user to run `nproxy pair <code>` on the server.
|
||||
|
||||
Current runtime note:
|
||||
- The current bot runtime uses Telegram long polling.
|
||||
- On each message from an unpaired user, the bot rotates any previous pending code and issues a fresh pairing code.
|
||||
- Pending pairing creation writes an audit-log entry with actor type `system`.
|
||||
|
||||
### Pair completion
|
||||
1. An operator runs `nproxy pair <code>` on the server.
|
||||
2. The CLI looks up the pending pairing by code.
|
||||
3. The CLI prints the target Telegram identity and asks for confirmation.
|
||||
4. On confirmation, the CLI adds the Telegram user to the allowlist.
|
||||
5. The CLI marks the pending pairing record as `completed`.
|
||||
6. The CLI writes an admin action log entry.
|
||||
|
||||
## Required CLI commands
|
||||
- `nproxy pair <code>`
|
||||
- `nproxy pair list`
|
||||
- `nproxy pair revoke <telegram-user-id>`
|
||||
- `nproxy pair cleanup`
|
||||
|
||||
## Current CLI behavior
|
||||
- `nproxy pair <code>` prints the Telegram identity and requires explicit confirmation unless `--yes` is provided.
|
||||
- `nproxy pair list` prints active allowlist entries and pending pairing records.
|
||||
- `nproxy pair revoke <telegram-user-id>` requires explicit confirmation unless `--yes` is provided.
|
||||
- `nproxy pair cleanup` marks expired pending pairing records as `expired` and writes an audit log entry.
|
||||
|
||||
## Security rules
|
||||
- Pairing codes expire.
|
||||
- Pairing codes are stored hashed, not in plaintext.
|
||||
- Only the server-side CLI can complete a pairing.
|
||||
- Telegram bot access is denied until allowlist membership exists.
|
||||
- Every pairing and revocation action is auditable.
|
||||
103
docs/plan/mvp-system-plan.md
Normal file
103
docs/plan/mvp-system-plan.md
Normal file
@@ -0,0 +1,103 @@
|
||||
# MVP System Plan
|
||||
|
||||
## Summary
|
||||
Build `nproxy`, a B2C web product for image generation through external model APIs. The first model is `nano_banana`. Users register with `email + password`, pay a monthly crypto subscription, receive a monthly request limit, and use a chat-style interface for `text-to-image` and `image-to-image` generation.
|
||||
|
||||
The service hides provider-key failures behind a routed key pool. A user request is attempted against one provider key at a time. Retryable failures move execution to the next eligible key. The user sees an error only after all eligible keys have been exhausted or the request fails for a terminal user-caused reason.
|
||||
|
||||
## Confirmed MVP decisions
|
||||
- One B2C website.
|
||||
- One monthly subscription plan.
|
||||
- Crypto checkout through a payment processor.
|
||||
- Manual renewal in MVP.
|
||||
- Text-to-image and image-to-image.
|
||||
- User-facing synchronous experience implemented with polling over background execution.
|
||||
- Approximate quota buckets only: `100/80/60/40/20/0`.
|
||||
- Storage in S3-compatible object storage.
|
||||
- One VPS deployment with Docker Compose.
|
||||
- Web admin plus Telegram admin bot.
|
||||
- Telegram admin onboarding through pairing on the server console.
|
||||
- Multiple provider API keys with round-robin routing, cooldown, balance tracking, optional per-key proxy, and transparent failover.
|
||||
|
||||
## Core product surfaces
|
||||
### Public web
|
||||
- landing page
|
||||
- register / login / password reset
|
||||
- dashboard with subscription state and approximate quota
|
||||
- chat UI
|
||||
- billing / checkout pages
|
||||
|
||||
### Admin surfaces
|
||||
- web admin for users, subscriptions, payments, generations, provider keys, proxies, and health
|
||||
- Telegram bot for alerts and low-friction admin actions
|
||||
- CLI for server-side operational commands, including Telegram pairing
|
||||
|
||||
## Main backend domains
|
||||
- auth
|
||||
- billing
|
||||
- subscriptions
|
||||
- quota ledger
|
||||
- conversations and generations
|
||||
- provider routing
|
||||
- provider key pool health
|
||||
- asset storage
|
||||
- admin audit
|
||||
- notifications
|
||||
|
||||
## Billing rules
|
||||
- One active plan in MVP.
|
||||
- Each user has an individual billing cycle based on successful activation timestamp.
|
||||
- Limit resets on each successful cycle activation.
|
||||
- One successful generation consumes one request.
|
||||
- Failed generations do not consume quota.
|
||||
|
||||
## Quota display contract
|
||||
Backend tracks exact usage. Normal users see only an approximate bucket:
|
||||
- `81-100%` remaining -> `100%`
|
||||
- `61-80%` remaining -> `80%`
|
||||
- `41-60%` remaining -> `60%`
|
||||
- `21-40%` remaining -> `40%`
|
||||
- `1-20%` remaining -> `20%`
|
||||
- `0%` remaining -> `0%`
|
||||
|
||||
## Generation controls in MVP
|
||||
- mode: `text_to_image` or `image_to_image`
|
||||
- resolution preset
|
||||
- batch size
|
||||
- image strength for `image_to_image`
|
||||
|
||||
## Key pool behavior
|
||||
- Start from the next `active` key by round robin.
|
||||
- Use the key-specific proxy first if configured.
|
||||
- If the proxy path fails with a transport error, retry the same key directly.
|
||||
- Retry on the next key only for retryable failures: network, timeout, provider `5xx`.
|
||||
- Do not retry on the next key for validation, policy, or other user-caused `4xx` errors.
|
||||
- Move a key to `cooldown` on retryable failures.
|
||||
- Default cooldown is `5 minutes`.
|
||||
- After more than `10` consecutive retryable failures across cooldown recoveries, move the key to `manual_review`.
|
||||
- Move a key to `out_of_funds` when the provider balance API or provider response shows insufficient funds.
|
||||
- `out_of_funds` and `manual_review` keys return to service only through a manual admin action.
|
||||
|
||||
## Telegram pairing
|
||||
1. A Telegram user opens the bot.
|
||||
2. If the user is not in the allowlist, the bot generates a short pairing code and stores a pending pairing record.
|
||||
3. The bot tells the user to run `nproxy pair <code>` on the server.
|
||||
4. The server-side CLI confirms the target user and adds the Telegram ID to the allowlist.
|
||||
5. The pairing record is marked complete and the user gains bot access.
|
||||
|
||||
## Deployment target
|
||||
Single VPS with Docker Compose, expected services:
|
||||
- `web`
|
||||
- `worker`
|
||||
- `bot`
|
||||
- `postgres`
|
||||
- `caddy` or `nginx`
|
||||
- optional `minio` when object storage is self-hosted
|
||||
|
||||
## Future-compatible boundaries
|
||||
The codebase should be able to add later:
|
||||
- more image providers
|
||||
- more billing methods
|
||||
- more subscription plans
|
||||
- internal balance wallet
|
||||
- recurring billing if the payment processor supports it natively
|
||||
Reference in New Issue
Block a user