Compare commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
132559cb8e | ||
|
|
aa4e3d0180 | ||
|
|
1ccffc85da | ||
|
|
d38ee68c4e | ||
|
|
aaef47c7a7 | ||
|
|
cbf345b25f | ||
|
|
f548d0671f | ||
|
|
6510a9ca22 | ||
|
|
07c3de4939 | ||
|
|
948dac34fd | ||
|
|
480756d832 | ||
|
|
135fb34e00 | ||
|
|
9efa3bbc8a | ||
|
|
fad2bd3353 | ||
|
|
930e42b0b0 |
@@ -0,0 +1,14 @@
|
||||
{
|
||||
"permissions": {
|
||||
"allow": [
|
||||
"mcp__codegraph__codegraph_explore",
|
||||
"mcp__codegraph__codegraph_search",
|
||||
"mcp__codegraph__codegraph_node",
|
||||
"mcp__codegraph__codegraph_callers",
|
||||
"mcp__codegraph__codegraph_callees",
|
||||
"mcp__codegraph__codegraph_impact",
|
||||
"mcp__codegraph__codegraph_files",
|
||||
"mcp__codegraph__codegraph_status"
|
||||
]
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,16 @@
|
||||
# CodeGraph data files
|
||||
# These are local to each machine and should not be committed
|
||||
|
||||
# Database
|
||||
*.db
|
||||
*.db-wal
|
||||
*.db-shm
|
||||
|
||||
# Cache
|
||||
cache/
|
||||
|
||||
# Logs
|
||||
*.log
|
||||
|
||||
# Hook markers
|
||||
.dirty
|
||||
@@ -0,0 +1,6 @@
|
||||
{
|
||||
"pid": 44608,
|
||||
"version": "0.9.9",
|
||||
"socketPath": "\\\\.\\pipe\\codegraph-97b92efdcc5351da",
|
||||
"startedAt": 1781240018712
|
||||
}
|
||||
@@ -0,0 +1,14 @@
|
||||
{
|
||||
"mcpServers": {
|
||||
"codegraph": {
|
||||
"type": "stdio",
|
||||
"command": "codegraph",
|
||||
"args": [
|
||||
"serve",
|
||||
"--mcp",
|
||||
"--path",
|
||||
"C:\\Users\\shats\\Dev\\EvoBGP"
|
||||
]
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,408 @@
|
||||
---
|
||||
name: Технический аудит EvoBGP
|
||||
overview: "Полный технический аудит EvoBGP для production-сценария (10+ клиентов, нестабильная сеть). Архитектура — hybrid control plane; сильные стороны: stale fallback, CDN/RIPEstat resilience, Ed25519 bundles. Критичные риски: in-process jobs, DoH без retry, misconfiguration demo-seed, отсутствие HA API."
|
||||
todos:
|
||||
- id: quick-ops-checklist
|
||||
content: Применить production-checklist (SEED_DEMO=0, BUNDLE_SEED_HEX, DB/JOB/CONCURRENCY tuning, TLS)
|
||||
status: pending
|
||||
- id: fix-doh-retry
|
||||
content: Добавить DoWithRetry для DoH в internal/pipeline/refresh.go
|
||||
status: pending
|
||||
- id: fix-job-meta-race
|
||||
content: Исправить чтение j.Meta в worker.go через Snapshot() или locked accessor
|
||||
status: pending
|
||||
- id: cdn-preview-resilience
|
||||
content: Перевести CDN preview на upstreamHTTPDo в routes_crud.go
|
||||
status: pending
|
||||
- id: partial-cdn-failure
|
||||
content: "Partial CDN source failure: skip/degrade вместо fail всего модуля"
|
||||
status: pending
|
||||
- id: graceful-shutdown
|
||||
content: Cancel/drain jobs при SIGTERM в cmd/evobgp-api и evobgp-all
|
||||
status: pending
|
||||
- id: ha-job-queue
|
||||
content: "Roadmap: распределённая очередь jobs (PG claim или NATS) для HA API"
|
||||
status: pending
|
||||
isProject: false
|
||||
---
|
||||
|
||||
# Технический аудит EvoBGP
|
||||
|
||||
## Executive summary
|
||||
|
||||
EvoBGP — **hybrid control plane**: один процесс [`evobgp-all`](cmd/evobgp-all/main.go) (monolith) или **reference Compose** с разделёнными воркерами ([`docs/architecture.md`](docs/architecture.md)). Data plane (BIRD + agent) отделён от control plane (API + PostgreSQL + jobs).
|
||||
|
||||
**Сильные стороны для нестабильной сети:**
|
||||
- Stale snapshot fallback по умолчанию (`EVOBGP_STALE_ON_UPSTREAM_ERROR=1`) — [`internal/pipeline/collect_stale.go`](internal/pipeline/collect_stale.go)
|
||||
- CDN/RIPEstat: retry (3×) + per-host circuit breaker — [`internal/httpclient/httpclient.go`](internal/httpclient/httpclient.go), [`circuit.go`](internal/httpclient/circuit.go)
|
||||
- ETag conditional GET, ASN TTL-кэш, parallel collect с cap
|
||||
- Подписанные бандлы Ed25519, verify перед apply
|
||||
|
||||
**Главные риски для 10+ клиентов:**
|
||||
1. `jobs.Registry` — **in-memory, только в процессе API** (ARCH-04)
|
||||
2. DoH — **без retry/breaker** (критично при блокировках провайдеров)
|
||||
3. Один failed CDN source **без stale cache валит весь модуль**
|
||||
4. Production misconfiguration: `Bearer dev`, HTTP API, ephemeral bundle key
|
||||
5. Data race на `Job.Meta` и alias pointers в `store.Memory`
|
||||
|
||||
---
|
||||
|
||||
## 1. Архитектура
|
||||
|
||||
### Стиль
|
||||
|
||||
```mermaid
|
||||
flowchart TB
|
||||
subgraph hybrid [Hybrid deployment]
|
||||
All[evobgp_all monolith]
|
||||
Split[evobgp_api + workers]
|
||||
end
|
||||
subgraph cp [Control plane]
|
||||
API[HTTP API]
|
||||
Jobs[jobs.Registry in-process]
|
||||
PG[(PostgreSQL)]
|
||||
end
|
||||
subgraph dp [Data plane per speaker]
|
||||
Agent[evobgp_agent]
|
||||
BIRD[BIRD2]
|
||||
NodeCLI[evobgp_node]
|
||||
end
|
||||
All --> API
|
||||
Split --> API
|
||||
API --> Jobs
|
||||
API --> PG
|
||||
NodeCLI --> API
|
||||
Agent --> API
|
||||
Agent --> BIRD
|
||||
```
|
||||
|
||||
| Профиль | Стиль | Когда |
|
||||
|---------|-------|-------|
|
||||
| `microvps` / `evobgp-all` | Monolith | 1 VPS, shared Registry |
|
||||
| reference Compose | Microservices-lite | API + scheduler/ingest/render/deploy |
|
||||
| Remote speakers | Edge agents | Panel→Node dispatch |
|
||||
|
||||
### Узкие места (bottlenecks)
|
||||
|
||||
| # | Bottleneck | Где | Impact |
|
||||
|---|------------|-----|--------|
|
||||
| B1 | **In-process job queue** | [`internal/jobs/job.go:175-177`](internal/jobs/job.go) | HA API невозможен без потери/дублирования jobs; scheduler без `EVOBGP_CONTROL_PLANE_URL` создаёт **отдельный Registry** — [`cmd/evobgp-scheduler/main.go:58-60`](cmd/evobgp-scheduler/main.go) |
|
||||
| B2 | **Module refresh = sync upstream fan-out** | [`internal/pipeline/collect_parallel.go`](internal/pipeline/collect_parallel.go) | До `EVOBGP_COLLECT_CONCURRENCY` (8 default, max 32) параллельных HTTP; worst case ~45s × retries на источник |
|
||||
| B3 | **Default job concurrency = 8** | [`internal/jobs/job.go:264-268`](internal/jobs/job.go) | При burst refresh 10+ tenants — очередь растёт, goroutine блокируются на sem |
|
||||
| B4 | **PostgreSQL pool default** | [`internal/db/open.go:28-38`](internal/db/open.go) | pgx default ~4 conns; при `JOB_MAX=16` + HTTP — contention без `EVOBGP_DB_MAX_CONNS=25` |
|
||||
| B5 | **Live endpoints fan-out** | [`internal/httpapi/peers_live.go`](internal/httpapi/peers_live.go) | N goroutines × N speakers, 12s timeout каждый |
|
||||
| B6 | **Broker — заглушка** | [`internal/broker`](internal/broker) | NATS URL логируется, очередь не распределена |
|
||||
|
||||
### Масштабируемость
|
||||
|
||||
- **Вертикальная:** хорошо до ~10–20 tenants при `evobgp-all` + tuning ([`docs/production-checklist.md`](docs/production-checklist.md))
|
||||
- **Горизонтальная API:** **не поддерживается** — два `evobgp-api` = два независимых Registry; `job_audit` в PG — audit only, не очередь исполнения
|
||||
- **Workers (ingest/render/deploy):** координируются через **общую БД**, не через jobs — OK для prefetch/drift
|
||||
|
||||
### Отказоустойчивость
|
||||
|
||||
| Сценарий | Поведение | Оценка |
|
||||
|----------|-----------|--------|
|
||||
| CDN/RIPEstat недоступен | Stale snapshot + circuit breaker | **Хорошо** (если был prior snapshot) |
|
||||
| DoH недоступен | Fail модуля или stale domain snapshot | **Средне** (нет HTTP retry) |
|
||||
| API restart mid-job | Job теряется из Registry; audit может быть inconsistent | **Плохо** |
|
||||
| PG недоступен | API `/v1/ready` → 503 | **OK** |
|
||||
| Agent unreachable | Deploy job succeed, drift в `evobgp-deploy` | **Частичный fail** (by design) |
|
||||
|
||||
**Рекомендация:** для 10+ клиентов — **`evobgp-all` на каждом CP** или один CP + tuning; HA API требует **распределённой очереди** (NATS/Redis + worker pool) — задокументировано как future work.
|
||||
|
||||
---
|
||||
|
||||
## 2. Анализ кода
|
||||
|
||||
### Антипаттерны
|
||||
|
||||
| ID | Проблема | Файл | Критичность |
|
||||
|----|----------|------|-------------|
|
||||
| A1 | **Concurrent map read/write** — worker читает `j.Meta` без lock, handler пишет через `mergeMeta`/`Snapshot` | [`worker.go:108,263,379`](internal/jobs/worker.go), [`job.go:103-111`](internal/jobs/job.go) | **high** |
|
||||
| A2 | **Escape internal pointers** из Memory store | [`store/memory.go:416-475`](internal/store/memory.go) | **high** (tests/dev); **low** (prod PG) |
|
||||
| A3 | **Fire-and-forget goroutine** на каждый auth | [`auth.go:79-81`](internal/httpapi/auth.go) | **medium** |
|
||||
| A4 | **Silent error swallow** в prefetch | [`internal/ingest/run.go`](internal/ingest/run.go), `prefetch.go` | **medium** |
|
||||
| A5 | **Bypass resilience layer** — CDN preview прямой `Do` | [`routes_crud.go:267`](internal/httpapi/routes_crud.go) | **medium** |
|
||||
| A6 | **`EVOBGP_DEV_INSECURE` — dead code** | compose + [`server.go`](internal/httpapi/server.go) | **low** (misleading ops) |
|
||||
| A7 | **Unused Registry** в ingest/render/deploy binaries | [`cmd/evobgp-ingest/main.go`](cmd/evobgp-ingest/main.go) | **low** (resource waste) |
|
||||
|
||||
### Maintainability
|
||||
|
||||
**Плюсы:** чёткое разделение слоёв (ARCH-01..10), `store.Backend`, OpenAPI как контракт, engineering rules, table-driven tests в birdfmt/pipeline.
|
||||
|
||||
**Минусы:**
|
||||
- Дублирование retry-логики (httpclient vs nodedispatch inline loop)
|
||||
- Env-tuning разбросан (`EVOBGP_*` в 15+ местах без central config struct для pipeline)
|
||||
- `Job` comment «персистенция в БД пока не подключена» устарел — hooks есть в [`bootstrap.go:67-92`](internal/httpapi/bootstrap.go)
|
||||
|
||||
### Потенциальные баги и race conditions
|
||||
|
||||
1. **`j.Meta` data race** — `-race` на `TestParallelModuleRefresh_*` + concurrent `GET /v1/jobs/{id}` polling
|
||||
2. **Memory store alias** — `deploy.Run` читает `LastAppliedRevisionID` пока worker пишет
|
||||
3. **peerLiveCache** возвращает slice без копии — [`peers_live.go:82-84`](internal/httpapi/peers_live.go)
|
||||
4. **TOCTOU idempotency** — terminal job удаляется из `byIdempo`, повторный POST создаст новый job (by design, но клиент должен знать)
|
||||
|
||||
### Error handling
|
||||
|
||||
**Хорошо:**
|
||||
- Префиксы ошибок (`httpclient:`, `birdfmt:`)
|
||||
- HTTP 5xx через `writeProblem`, без raw `err.Error()` (ERR-01)
|
||||
- `context.Context` в pipeline workers
|
||||
|
||||
**Пробелы:**
|
||||
- `runRollback` без `workContext` — не отменяется — [`worker.go:500+`](internal/jobs/worker.go)
|
||||
- Prefetch/ingest: ошибки не логируются
|
||||
- `mergeBirdPostApplyMeta` — `context.Background()` 8s, игнорирует job cancel
|
||||
|
||||
---
|
||||
|
||||
## 3. Производительность
|
||||
|
||||
### Блокирующие операции
|
||||
|
||||
| Участок | Блокировка | Риск |
|
||||
|---------|------------|------|
|
||||
| `POST .../cdn-sources/preview` | Sync CDN fetch до 45s в HTTP handler | UI timeout, worker starvation |
|
||||
| `GET /v1/peers/live` | N × agent HTTP, wg.Wait | Slow при многих speakers |
|
||||
| Module refresh job | Sequential: ingest → render revision → optional deploy | Long job chain |
|
||||
| `bird -p` / `birdc configure` | Subprocess в deploy | Disk I/O на ноде |
|
||||
|
||||
### Неэффективные алгоритмы / лишние запросы
|
||||
|
||||
- **Tenant refresh:** `aggregateTenantPrefixRowsAll` — parallel по модулям, но каждый модуль может refetch все CDN/ASN/DoH — [`aggregate.go:28+`](internal/pipeline/aggregate.go). Snapshot skip есть через `module_hash` — проверять hit rate в meta.
|
||||
- **ASN resolve:** `PolitePause()` 150ms между AS — [`asnresolve/ripestat.go`](internal/asnresolve/ripestat.go) — при 50 AS = +7.5s minimum.
|
||||
- **GetModulePrefixSnapshot** вызывается многократно в одном refresh (cdn_snapshot, collect_parallel) — potential duplicate DB reads.
|
||||
- **Auth TouchAPIKeyLastUsed:** UPDATE на каждый request (async) — load на PG при high RPS.
|
||||
|
||||
### Кэширование
|
||||
|
||||
| Кэш | TTL | Gap |
|
||||
|-----|-----|-----|
|
||||
| ASN prefix cache | 1800s (`EVOBGP_ASN_CACHE_TTL_SEC`) | OK |
|
||||
| CDN ETag in DB | Until 304/change | OK |
|
||||
| Module prefix snapshot | Content-hash based skip | OK |
|
||||
| peerLiveCache | In-memory, per-process | Не shared между API replicas; нет defensive copy |
|
||||
| Circuit breaker state | Per-process | Не shared |
|
||||
|
||||
### Конкретные улучшения
|
||||
|
||||
```go
|
||||
// 1. CDN preview — использовать upstreamHTTPDo вместо прямого Do
|
||||
resp, err := pipeline.UpstreamHTTPDo(r.Context(), s.cdnHTTP, req) // extract upstreamHTTPDo
|
||||
|
||||
// 2. Job.Meta — читать под lock или через Snapshot()
|
||||
st := j.Snapshot()
|
||||
mid, _ := st["meta"].(map[string]any)["module_id"].(string)
|
||||
|
||||
// 3. Memory store — возвращать копии (как Postgres)
|
||||
modCopy := *mod
|
||||
return &modCopy, nil
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 4. Сетевое взаимодействие (критично)
|
||||
|
||||
### Текущее состояние
|
||||
|
||||
```mermaid
|
||||
flowchart LR
|
||||
subgraph resilient [Resilient path]
|
||||
CDN[CDN fetch]
|
||||
RIPE[RIPEstat]
|
||||
CDN --> Breaker[Circuit breaker]
|
||||
RIPE --> Breaker
|
||||
Breaker --> Retry[DoWithRetry 3x linear 2s]
|
||||
end
|
||||
subgraph fragile [Fragile path]
|
||||
DoH[DoH resolve]
|
||||
Preview[CDN preview API]
|
||||
AgentHealth[Agent health/bird]
|
||||
DoH --> SingleDo[Single hc.Do]
|
||||
Preview --> SingleDo
|
||||
AgentHealth --> SingleDo
|
||||
end
|
||||
subgraph fallback [App-level fallback]
|
||||
Stale[Stale snapshot]
|
||||
SysDNS[System DNS]
|
||||
DoH --> SysDNS
|
||||
CDN --> Stale
|
||||
RIPE --> Stale
|
||||
end
|
||||
```
|
||||
|
||||
| Upstream | Timeout | Retry | Breaker | Stale fallback |
|
||||
|----------|---------|-------|---------|----------------|
|
||||
| CDN ingest | 45s | 3× linear | per-host | yes |
|
||||
| RIPEstat | 45s | 3× | per-host | yes + cache |
|
||||
| DoH | 10s/profile | **no** | **no** | domain snapshot |
|
||||
| CDN preview | 45s | **no** | **no** | N/A |
|
||||
| Scheduler→API | 45s | 3× | no | N/A |
|
||||
| Node dispatch | 30s | inline 3× | no | N/A |
|
||||
|
||||
### Пробелы для блокировок провайдеров
|
||||
|
||||
1. **DoH без retry** — transient timeout = fail; failover между profiles есть, но каждый profile — single shot
|
||||
2. **429/408 не ретраятся** — только `>= 500`
|
||||
3. **Нет jitter** — thundering herd при mass tenant refresh
|
||||
4. **DNS rebinding TOCTOU** — SSRF check до fetch, HTTP dial без pinned IP — [`cdn_url.go:75-115`](internal/pipeline/cdn_url.go)
|
||||
5. **Circuit breaker без half-open** — после 30s cooldown сразу full traffic — [`circuit.go:29-33`](internal/httpclient/circuit.go)
|
||||
6. **Breaker per-process** — ingest container ≠ API container
|
||||
|
||||
### Рекомендации для нестабильной сети
|
||||
|
||||
| # | Изменение | Effort | Effect |
|
||||
|---|-----------|--------|--------|
|
||||
| N1 | DoH через `DoWithRetry` + optional breaker | Low | **High** для DOMAINS modules |
|
||||
| N2 | Retry 429/503 с `Retry-After` + exponential backoff + jitter | Medium | **High** при rate limits |
|
||||
| N3 | **Partial CDN failure** — continue с stale per-source, не fail whole module | Medium | **High** |
|
||||
| N4 | Multiple DoH profiles + `failover` policy (already exists) — **документировать ops playbook** | Low | **High** (config, not code) |
|
||||
| N5 | Pinned dialer / custom `Transport.DialContext` после SSRF resolve | Medium | **Medium** (SSRF hardening) |
|
||||
| N6 | Proxy support (`HTTP_PROXY` / `EVOBGP_HTTP_PROXY`) для CDN/DoH | Medium | **High** в censored networks |
|
||||
| N7 | Unify CDN preview на `upstreamHTTPDo` | Low | **Medium** |
|
||||
|
||||
---
|
||||
|
||||
## 5. Устойчивость и надёжность
|
||||
|
||||
### Graceful degradation
|
||||
|
||||
**Работает:**
|
||||
- `EVOBGP_STALE_ON_UPSTREAM_ERROR=1` — ASN/CDN/domain stale — [`collect_stale.go`](internal/pipeline/collect_stale.go)
|
||||
- CDN 304 без local cache → forced full GET — [`cdn_snapshot.go:141-159`](internal/pipeline/cdn_snapshot.go)
|
||||
- DoH → system DNS fallback — [`doh_resolve.go:75-93`](internal/pipeline/doh_resolve.go)
|
||||
- Deploy: job succeed even if agent wake fails (drift detection)
|
||||
|
||||
**Не работает / частично:**
|
||||
- Один CDN source fail без cache → **весь module_refresh failed** — [`collect_parallel.go:221-223`](internal/pipeline/collect_parallel.go)
|
||||
- Circuit open → immediate error, stale only if prior data exists
|
||||
- API shutdown: HTTP drain 15s, **jobs не cancel/drain** — [`cmd/evobgp-api/main.go:67-72`](cmd/evobgp-api/main.go)
|
||||
|
||||
### Сценарии отказов
|
||||
|
||||
| Событие | Что произойдёт |
|
||||
|---------|----------------|
|
||||
| **Потеря CP↔PG** | Ready=false; running jobs fail; no new jobs persist audit reliably |
|
||||
| **Потеря CP↔CDN** | Stale prefixes если были; иначе job fail; breaker opens 30s |
|
||||
| **Потеря CP↔agent** | Deploy meta `dispatch_failed`; BIRD на старой ревизии; drift logs |
|
||||
| **RIPEstat rate limit** | 429 → no retry → stale or fail |
|
||||
| **Рост нагрузки** | Job queue; goroutine pile-up; PG pool exhaustion; `/metrics` shows queue depth |
|
||||
| **API restart** | In-flight jobs lost; clients poll 404 or stale terminal state |
|
||||
|
||||
---
|
||||
|
||||
## 6. Безопасность
|
||||
|
||||
| ID | Finding | Severity | Fix |
|
||||
|----|---------|----------|-----|
|
||||
| S1 | `Bearer dev` → operator при demo-seed | **high** (misconfig) | `EVOBGP_SEED_DEMO=0` — [`auth.go:66-92`](internal/httpapi/auth.go) |
|
||||
| S2 | API plain HTTP | **high** (ops) | TLS на edge (Traefik/nginx) |
|
||||
| S3 | Ephemeral bundle key без `EVOBGP_BUNDLE_SEED_HEX` | **high** (ops) | Stable seed + pubkey на нодах |
|
||||
| S4 | Compose defaults: weak PG password, `sslmode=disable` | **high** (ops) | Secrets manager, `sslmode=require` |
|
||||
| S5 | `/metrics` без auth | **medium** | Network policy / mTLS |
|
||||
| S6 | No rate limiting on auth | **medium** | Middleware limiter (e.g. per-IP) |
|
||||
| S7 | CDN SSRF DNS rebinding | **medium** | Pinned dialer after resolve |
|
||||
| S8 | `EVOBGP_CDN_ALLOW_PRIVATE=1` | **medium** | Never in prod |
|
||||
| S9 | `EVOBGP_NODE_DISPATCH_INSECURE_TLS=1` | **medium** | Valid TLS to agent |
|
||||
| S10 | Plaintext `EVOBGP_API_KEYS` in env | **medium** | DB keys via API |
|
||||
| S11 | `editor` can cancel jobs | **low** | Restrict to operator |
|
||||
| S12 | agent_secret `==` compare | **low** | `subtle.ConstantTimeCompare` |
|
||||
|
||||
**SQL injection:** не обнаружено — параметризованные запросы в [`repository/`](internal/repository/).
|
||||
|
||||
**Bundle crypto:** Ed25519 корректно; path traversal blocked в tar extract.
|
||||
|
||||
---
|
||||
|
||||
## 7. Конкретные рекомендации (prioritized backlog)
|
||||
|
||||
### High
|
||||
|
||||
| # | Описание | Как исправить |
|
||||
|---|----------|---------------|
|
||||
| H1 | DoH без retry | Обернуть `hc.Do` в `DoWithRetry(ctx, hc, req, 3)` в [`refresh.go:288,360`](internal/pipeline/refresh.go) |
|
||||
| H2 | Data race `Job.Meta` | Читать через `Snapshot()` или добавить `MetaLocked()` accessor |
|
||||
| H3 | CDN source partial failure | В `collectCDNPrefixRows`: при err без stale — log warning + skip source вместо `return nil, r.err` (config flag `EVOBGP_CDN_PARTIAL_OK=1`) |
|
||||
| H4 | Production checklist enforcement | CI/deploy validation: reject `SEED_DEMO=1`, require `BUNDLE_SEED_HEX` |
|
||||
| H5 | Job queue HA roadmap | Persist queued jobs in PG + worker claim (`SELECT FOR UPDATE SKIP LOCKED`) или NATS — ARCH-04 |
|
||||
|
||||
### Medium
|
||||
|
||||
| # | Описание | Как исправить |
|
||||
|---|----------|---------------|
|
||||
| M1 | CDN preview bypass | [`routes_crud.go:267`](internal/httpapi/routes_crud.go) → `upstreamHTTPDo` |
|
||||
| M2 | Retry 429/503 | Extend `DoWithRetry` status check + parse `Retry-After` |
|
||||
| M3 | Graceful shutdown | On SIGTERM: `Registry.RequestCancelAll()` + wait workers with timeout |
|
||||
| M4 | Auth goroutine storm | Worker pool или sync touch with debounce |
|
||||
| M5 | HTTP proxy support | Custom Transport reading `EVOBGP_HTTP_PROXY` |
|
||||
| M6 | Memory store copies | Defensive copy in Get/List (dev/test safety) |
|
||||
| M7 | Rate limiting | `golang.org/x/time/rate` on auth middleware |
|
||||
|
||||
### Low
|
||||
|
||||
| # | Описание | Как исправить |
|
||||
|---|----------|---------------|
|
||||
| L1 | Jitter in backoff | `wait + rand.Intn(wait/2)` in DoWithRetry |
|
||||
| L2 | Half-open breaker | Single probe request after cooldown |
|
||||
| L3 | Remove dead `EVOBGP_DEV_INSECURE` from compose | Docs + compose cleanup |
|
||||
| L4 | Prefetch error logging | `log.Printf` or structured log in prefetch |
|
||||
| L5 | peerLiveCache defensive copy | `append([]T(nil), views...)` on store |
|
||||
|
||||
---
|
||||
|
||||
## 8. Quick wins (максимальный эффект / минимум усилий)
|
||||
|
||||
1. **Ops (0 code):** [`docs/production-checklist.md`](docs/production-checklist.md) — `SEED_DEMO=0`, `BUNDLE_SEED_HEX`, `DB_MAX_CONNS=25`, `JOB_MAX=16`, `COLLECT_CONCURRENCY=16`, TLS edge, restrict metrics
|
||||
2. **DoH retry** — 5–10 строк в `refresh.go`, reuse existing `DoWithRetry`
|
||||
3. **CDN preview → upstreamHTTPDo** — 1 line change in handler
|
||||
4. **Job.Meta read fix** — replace 4 reads in `worker.go` with `Snapshot()` parsing
|
||||
5. **Log prefetch failures** — visibility без изменения behavior
|
||||
6. **Document DoH failover playbook** — multiple profiles (Cloudflare, Google, Quad9) + `failover` policy for censored regions
|
||||
7. **Run `go test -race ./internal/jobs/...`** in CI — catch Meta race
|
||||
8. **Prefer `evobgp-all`** over split reference for <20 tenants — eliminates Registry split bug
|
||||
|
||||
---
|
||||
|
||||
## Диаграмма: refresh под сетевым stress
|
||||
|
||||
```mermaid
|
||||
sequenceDiagram
|
||||
participant Op as Operator
|
||||
participant API as evobgp_api
|
||||
participant Job as module_refresh
|
||||
participant CDN as CDN_upstream
|
||||
participant PG as PostgreSQL
|
||||
|
||||
Op->>API: POST /modules/id/refresh
|
||||
API->>Job: Enqueue
|
||||
Job->>CDN: GET with ETag
|
||||
alt CDN timeout or 5xx
|
||||
CDN-->>Job: error after 3 retries
|
||||
Job->>PG: load prior snapshot
|
||||
alt stale exists
|
||||
Job->>PG: CreateRenderRevision stale
|
||||
Job-->>API: succeeded degraded
|
||||
else no stale
|
||||
Job-->>API: failed
|
||||
end
|
||||
else CDN 200
|
||||
CDN-->>Job: new prefixes
|
||||
Job->>PG: CreateRenderRevision
|
||||
end
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Итоговая оценка зрелости
|
||||
|
||||
| Область | Оценка | Комментарий |
|
||||
|---------|--------|-------------|
|
||||
| Архитектура | 7/10 | Чистые слои; HA/API scaling — слабое место |
|
||||
| Сеть/resilience | 6/10 | CDN/ASN хорошо; DoH/preview — пробелы |
|
||||
| Concurrency | 6/10 | Registry продуман; Meta race, shutdown |
|
||||
| Performance | 7/10 | Parallel collect, caching; tuning needed at scale |
|
||||
| Security | 6/10 | Crypto OK; ops/config risks dominate |
|
||||
| Maintainability | 8/10 | Docs, rules, OpenAPI, tests |
|
||||
|
||||
**Вердикт:** проект **готов для 10+ клиентов в single-CP deployment** (`evobgp-all` + PostgreSQL + production checklist) при условии ops discipline. Для **multi-CP HA** и **агрессивных сетевых блокировок** — приоритет: DoH retry, partial CDN failure, distributed job queue, HTTP proxy.
|
||||
@@ -0,0 +1,93 @@
|
||||
---
|
||||
description: Context7 — закреплённые ID библиотек и документации стека EvoBGP
|
||||
alwaysApply: true
|
||||
---
|
||||
|
||||
# Context7 — стек EvoBGP
|
||||
|
||||
При вопросах об API, синтаксисе, конфигурации и миграциях библиотек **сначала** `query-docs` с ID из таблицы ниже. Шаг `resolve-library-id` **пропускать**, если библиотека уже перечислена (кроме неоднозначного случая).
|
||||
|
||||
Локальные версии: `go.mod`, `web/package.json`. При расхождении с ID — предпочитать версию из репозитория.
|
||||
|
||||
---
|
||||
|
||||
## Backend (Go)
|
||||
|
||||
| Библиотека | Context7 ID | Версия в проекте | Когда |
|
||||
|------------|-------------|------------------|-------|
|
||||
| Go stdlib | `/golang/go/go1_24_6` | Go 1.24 | `net/http`, `context`, тесты, concurrency |
|
||||
| pgx | `/websites/pkg_go_dev_github_com_jackc_pgx_v5` | v5.7.2 | PostgreSQL, pool, транзакции, типы |
|
||||
| Prometheus Go client | `/prometheus/client_golang` | v1.20.5 | метрики, `/metrics`, middleware |
|
||||
| modernc SQLite | `/websites/pkg_go_dev_modernc_org_sqlite` | v1.34.5 | SQLite-бэкенд, миграции sqlite |
|
||||
| miekg/dns | `/miekg/dns` | v1.1.72 | DNS-запросы, DoH, pipeline |
|
||||
|
||||
---
|
||||
|
||||
## HTTP-контракт и спецификации
|
||||
|
||||
| Библиотека | Context7 ID | Версия в проекте | Когда |
|
||||
|------------|-------------|------------------|-------|
|
||||
| OpenAPI | `/oai/openapi-specification` | 3.x в `docs/openapi.yaml` | схемы, operationId, problem+json |
|
||||
| Redocly CLI | `/redocly/redocly-cli` | CI `@redocly/cli` | lint OpenAPI, `npx @redocly/cli lint` |
|
||||
|
||||
---
|
||||
|
||||
## Web UI (`web/`)
|
||||
|
||||
| Библиотека | Context7 ID | Версия в проекте | Когда |
|
||||
|------------|-------------|------------------|-------|
|
||||
| Svelte | `/websites/svelte_dev` | ^5.54 | runes, компоненты, реактивность |
|
||||
| SvelteKit | `/sveltejs/kit` | ^2.50 | routing, `load`, adapters, SSR |
|
||||
| Vite | `/vitejs/vite/v7.3.1` | ^7.3.1 | dev server, build, plugins |
|
||||
| TypeScript | `/microsoft/typescript/v5.9.3` | ^5.9.3 | типы, strict, tsconfig |
|
||||
| Tailwind CSS | `/tailwindlabs/tailwindcss.com` | ^4.1 | v4, `@tailwindcss/vite`, утилиты |
|
||||
| shadcn-svelte | `/websites/shadcn-svelte` | CLI | примитивы `ui/core`, theming |
|
||||
| Bits UI | `/llmstxt/bits-ui_llms_txt` | ^2.17 | headless-примитивы под shadcn |
|
||||
| sveltekit-superforms | `/ciscoheat/sveltekit-superforms` | ^2.30 | формы, server actions |
|
||||
| Formsnap | `/svecosystem/formsnap` | ^2.0 | доступные поля форм |
|
||||
| Zod | `/websites/zod_dev_v4` | ^4.4 | схемы валидации |
|
||||
| TanStack Table | `/websites/tanstack_table` | table-core ^8.21 | `AppDataTable`, колонки, сортировка |
|
||||
|
||||
UI-правила репозитория: `.cursor/rules/web-shadcn.mdc` (shadcn-svelte docs — первичный источник для компонентов).
|
||||
|
||||
---
|
||||
|
||||
## Data plane / BGP
|
||||
|
||||
| Библиотека | Context7 ID | Версия в проекте | Когда |
|
||||
|------------|-------------|------------------|-------|
|
||||
| BIRD 2 | `/llmstxt/bird_xmsl_dev_llms_txt` | BIRD2 в compose | `birdfmt`, фильтры, протоколы |
|
||||
| BIRD (исходники) | `/cz-nic/bird` | — | низкоуровневый синтаксис daemon |
|
||||
|
||||
Сетевые правила: `.cursor/rules/networking-bird.mdc`.
|
||||
|
||||
---
|
||||
|
||||
## DevOps
|
||||
|
||||
| Библиотека | Context7 ID | Версия в проекте | Когда |
|
||||
|------------|-------------|------------------|-------|
|
||||
| Docker Compose | `/docker/compose` | `deploy/compose/` | сервисы, profiles, volumes |
|
||||
| Docker | `/docker/docs` | — | образы, bake, networking |
|
||||
|
||||
---
|
||||
|
||||
## Приоритет источников
|
||||
|
||||
1. **Контракт HTTP** — `docs/openapi.yaml` (не Context7).
|
||||
2. **Context7** — синтаксис и API библиотек из таблицы.
|
||||
3. **Локальные docs** — `docs/`, `web/README.md`, `AGENTS.md`.
|
||||
4. **Официальный сайт** — BIRD: https://bird.network.cz/?get_doc (если Context7 не покрыл кейс).
|
||||
|
||||
## Примеры запросов
|
||||
|
||||
```
|
||||
/docs /websites/svelte_dev runes $state $derived
|
||||
/docs /golang/go/go1_24_6 net/http ServeMux pattern matching
|
||||
/docs /websites/pkg_go_dev_github_com_jackc_pgx_v5 pool acquire rows
|
||||
/docs /llmstxt/bird_xmsl_dev_llms_txt filter bgp import
|
||||
```
|
||||
|
||||
## Не через Context7
|
||||
|
||||
Рефакторинг `internal/*`, бизнес-логика EvoBGP, code review — код репозитория и `codegraph`. Context7 — только внешние библиотеки и инструменты.
|
||||
@@ -2,6 +2,9 @@
|
||||
"plugins": {
|
||||
"svelte": {
|
||||
"enabled": true
|
||||
},
|
||||
"claude-plugins-official/gopls-lsp": {
|
||||
"enabled": true
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -0,0 +1,31 @@
|
||||
---
|
||||
name: context7-evobgp
|
||||
description: Context7 lookup для стека EvoBGP — использовать закреплённые library ID из .cursor/rules/context7-stack.mdc вместо resolve-library-id.
|
||||
---
|
||||
|
||||
# Context7 — EvoBGP stack
|
||||
|
||||
Перед `query-docs` открой `.cursor/rules/context7-stack.mdc` и выбери ID из таблицы по области задачи.
|
||||
|
||||
## Workflow
|
||||
|
||||
1. Определи область: `internal/` (Go), `web/` (Svelte), `docs/openapi.yaml`, `birdfmt`/`pipeline` (BIRD), `deploy/compose` (Docker).
|
||||
2. Найди строку в таблице `context7-stack.mdc`.
|
||||
3. Вызови `query-docs` с `libraryId` из таблицы и полным вопросом пользователя.
|
||||
4. `resolve-library-id` — только если библиотеки нет в таблице или нужна другая major-версия.
|
||||
|
||||
## Быстрые ID (частые)
|
||||
|
||||
| Задача | libraryId |
|
||||
|--------|-----------|
|
||||
| Svelte 5 runes | `/websites/svelte_dev` |
|
||||
| SvelteKit load/forms | `/sveltejs/kit` |
|
||||
| shadcn-svelte компонент | `/websites/shadcn-svelte` |
|
||||
| pgx pool/query | `/websites/pkg_go_dev_github_com_jackc_pgx_v5` |
|
||||
| Go net/http | `/golang/go/go1_24_6` |
|
||||
| OpenAPI lint | `/redocly/redocly-cli` |
|
||||
| BIRD config | `/llmstxt/bird_xmsl_dev_llms_txt` |
|
||||
| Tailwind v4 | `/tailwindlabs/tailwindcss.com` |
|
||||
| Zod 4 schema | `/websites/zod_dev_v4` |
|
||||
|
||||
Полный список и версии — в `context7-stack.mdc`.
|
||||
@@ -0,0 +1,12 @@
|
||||
{
|
||||
"mcpServers": {
|
||||
"codegraph": {
|
||||
"type": "stdio",
|
||||
"command": "codegraph",
|
||||
"args": [
|
||||
"serve",
|
||||
"--mcp"
|
||||
]
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,12 @@
|
||||
{
|
||||
"mcpServers": {
|
||||
"codegraph": {
|
||||
"type": "stdio",
|
||||
"command": "codegraph",
|
||||
"args": [
|
||||
"serve",
|
||||
"--mcp"
|
||||
]
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -4,7 +4,7 @@
|
||||
|
||||
## С чего начать (минимум чтения)
|
||||
|
||||
0. **Инженерные правила** — при изменении кода следовать [.cursor/rules/engineering.mdc](.cursor/rules/engineering.mdc); для `web/` — [.cursor/rules/web-shadcn.mdc](.cursor/rules/web-shadcn.mdc); для `birdfmt` / `pipeline` / BIRD — [.cursor/rules/networking-bird.mdc](.cursor/rules/networking-bird.mdc).
|
||||
0. **Инженерные правила** — при изменении кода следовать [.cursor/rules/engineering.mdc](.cursor/rules/engineering.mdc); для `web/` — [.cursor/rules/web-shadcn.mdc](.cursor/rules/web-shadcn.mdc); для `birdfmt` / `pipeline` / BIRD — [.cursor/rules/networking-bird.mdc](.cursor/rules/networking-bird.mdc). **Context7 (документация библиотек)** — закреплённые ID стека: [.cursor/rules/context7-stack.mdc](.cursor/rules/context7-stack.mdc); скилл [.cursor/skills/context7-evobgp/SKILL.md](.cursor/skills/context7-evobgp/SKILL.md).
|
||||
1. **[docs/README.md](docs/README.md)** — оглавление и роли читателя.
|
||||
2. **[docs/architecture.md](docs/architecture.md)** — компоненты `cmd/`, карта `internal/`, потоки данных (одного этого файла обычно достаточно для ориентации).
|
||||
3. Задача-специфично: [docs/api.md](docs/api.md), [docs/access.md](docs/access.md), [web/README.md](web/README.md) — только если меняете API, доступ или фронт.
|
||||
|
||||
@@ -12,6 +12,7 @@ import (
|
||||
|
||||
"evobgp/internal/birdfmt"
|
||||
"evobgp/internal/config"
|
||||
"evobgp/internal/dbcli"
|
||||
"evobgp/internal/deploy"
|
||||
"evobgp/internal/httpapi"
|
||||
"evobgp/internal/ingest"
|
||||
@@ -24,6 +25,9 @@ import (
|
||||
|
||||
// microVPS entrypoint: один процесс — HTTP API и фоновые воркеры scheduler, ingest, render, deploy (общий store и jobs.Registry).
|
||||
func main() {
|
||||
if len(os.Args) > 1 && os.Args[1] == "db" {
|
||||
os.Exit(dbcli.Run(os.Args[2:]))
|
||||
}
|
||||
cfg := config.Load()
|
||||
opts := httpapi.Options{
|
||||
APIKeys: os.Getenv("EVOBGP_API_KEYS"),
|
||||
@@ -52,6 +56,7 @@ func main() {
|
||||
go render.Run(ctx, renderDeps)
|
||||
go deploy.Run(ctx, deployDeps)
|
||||
|
||||
srv.StartBackground(ctx)
|
||||
startBirdMetricsPoller(ctx)
|
||||
|
||||
httpSrv := &http.Server{
|
||||
|
||||
@@ -12,6 +12,7 @@ import (
|
||||
|
||||
"evobgp/internal/birdfmt"
|
||||
"evobgp/internal/config"
|
||||
"evobgp/internal/dbcli"
|
||||
"evobgp/internal/httpapi"
|
||||
"evobgp/internal/observability"
|
||||
"evobgp/internal/platform"
|
||||
@@ -19,6 +20,9 @@ import (
|
||||
)
|
||||
|
||||
func main() {
|
||||
if len(os.Args) > 1 && os.Args[1] == "db" {
|
||||
os.Exit(dbcli.Run(os.Args[2:]))
|
||||
}
|
||||
cfg := config.Load()
|
||||
seedDemo := os.Getenv("EVOBGP_SEED_DEMO") != "0"
|
||||
opts := httpapi.Options{
|
||||
@@ -39,6 +43,7 @@ func main() {
|
||||
ctx, stop := signal.NotifyContext(context.Background(), os.Interrupt, syscall.SIGTERM)
|
||||
defer stop()
|
||||
|
||||
srv.StartBackground(ctx)
|
||||
startBirdMetricsPoller(ctx)
|
||||
|
||||
httpSrv := &http.Server{
|
||||
|
||||
@@ -52,6 +52,18 @@ opkey|01ARZ3NDEKTSV4RRFFQ69G5FAV|operator,nodekey|01ARZ3NDEKTSV4RRFFQ69G5FAV|nod
|
||||
|
||||
**Запрещено** в продакшене: не оставляйте demo-seed с известным токеном `dev` на боевых данных. Переменная `EVOBGP_DEV_INSECURE` в текущей версии **не влияет** на аутентификацию (оставлена в compose для совместимости; не включайте в production — см. SEC-02 в инженерных правилах).
|
||||
|
||||
### PostgreSQL monitoring и maintenance (control plane)
|
||||
|
||||
При `EVOBGP_DATABASE_URL` (не memory backend):
|
||||
|
||||
| Операция | Минимальная роль |
|
||||
|----------|------------------|
|
||||
| `GET /v1/monitoring/postgres/*`, `GET /v1/monitoring/correlation` | viewer |
|
||||
| `POST /v1/postgres/vacuum`, `vacuum-analyze`, `analyze`, `reindex`, `cleanup` | **operator** (async job, rate limit 60s на kind) |
|
||||
| `GET /v1/postgres/maintenance/logs` | viewer |
|
||||
|
||||
Метрики **instance-level** (не per-tenant). CLI: `evobgp-api db …` / `evobgp-all db …`.
|
||||
|
||||
### Синхронные «тяжёлые» GET (control plane)
|
||||
|
||||
- `POST /v1/modules/{module_id}/cdn-sources/preview` — загрузка CDN в том же HTTP-запросе (лимит тела ~8 MiB, см. OpenAPI).
|
||||
|
||||
@@ -8,6 +8,28 @@ Runbook для оценки объёма БД и узких мест **пере
|
||||
psql "$EVOBGP_DATABASE_URL"
|
||||
```
|
||||
|
||||
## HTTP API (панель / мониторинг)
|
||||
|
||||
При подключённом PostgreSQL control plane отдаёт instance-level метрики (роль **viewer+**):
|
||||
|
||||
- `GET /v1/monitoring/postgres/overview` — подключения, TPS, cache hit, размер БД
|
||||
- `GET /v1/monitoring/postgres/queries` — top queries (`pg_stat_statements`, если extension включён)
|
||||
- `GET /v1/monitoring/postgres/locks`, `/tables`, `/recommendations`
|
||||
- `GET /v1/monitoring/correlation?window=60` — корреляция refresh jobs и cache hit
|
||||
|
||||
Обслуживание (**operator**, async `202` + `job_id`): `POST /v1/postgres/vacuum`, `vacuum-analyze`, `analyze`, `reindex`, `cleanup`; журнал `GET /v1/postgres/maintenance/logs`.
|
||||
|
||||
CLI на CP: `evobgp-api db report|vacuum|analyze|cleanup` (см. `internal/dbcli`).
|
||||
|
||||
Миграция `000023` создаёт `pg_stat_statements`; для сбора статистики **обязательно** preload и перезапуск Postgres:
|
||||
|
||||
```text
|
||||
# postgresql.conf или command в compose
|
||||
shared_preload_libraries = 'pg_stat_statements'
|
||||
```
|
||||
|
||||
После изменения — restart контейнера/сервиса Postgres. Без этого API `/v1/monitoring/postgres/queries` вернёт пустой список (`statements_available: false`), без 5xx.
|
||||
|
||||
## 1. Размеры таблиц и индексов
|
||||
|
||||
```sql
|
||||
|
||||
@@ -49,6 +49,10 @@ tags:
|
||||
description: Управление API-ключами tenant (operator). Секрет возвращается только при создании и ротации.
|
||||
- name: Auth
|
||||
description: Сессия текущего API-ключа (tenant и роль).
|
||||
- name: Monitoring
|
||||
description: Наблюдаемость PostgreSQL и корреляция (instance-level, viewer+). Maintenance — operator.
|
||||
- name: Maintenance
|
||||
description: Политики обслуживания PostgreSQL (instance-scoped). CRUD и запуск — operator.
|
||||
|
||||
security:
|
||||
- bearerAuth: []
|
||||
@@ -910,6 +914,152 @@ components:
|
||||
type: string
|
||||
additionalProperties: true
|
||||
|
||||
PostgresOverview:
|
||||
type: object
|
||||
description: Instance-level PostgreSQL snapshot (GET /v1/monitoring/postgres/overview).
|
||||
additionalProperties: true
|
||||
|
||||
PostgresQueriesResponse:
|
||||
type: object
|
||||
properties:
|
||||
collected_at:
|
||||
type: string
|
||||
format: date-time
|
||||
source:
|
||||
type: string
|
||||
enum: [live, snapshot]
|
||||
items:
|
||||
type: array
|
||||
items:
|
||||
type: object
|
||||
additionalProperties: true
|
||||
|
||||
PostgresRecommendations:
|
||||
type: object
|
||||
properties:
|
||||
collected_at:
|
||||
type: string
|
||||
format: date-time
|
||||
items:
|
||||
type: array
|
||||
items:
|
||||
type: object
|
||||
properties:
|
||||
severity:
|
||||
type: string
|
||||
code:
|
||||
type: string
|
||||
title:
|
||||
type: string
|
||||
detail:
|
||||
type: string
|
||||
refs:
|
||||
type: array
|
||||
items:
|
||||
type: string
|
||||
|
||||
PostgresMaintenanceBody:
|
||||
type: object
|
||||
properties:
|
||||
table:
|
||||
type: string
|
||||
dry_run:
|
||||
type: boolean
|
||||
default: false
|
||||
policy:
|
||||
type: string
|
||||
description: Deprecated; use maintenance policies API.
|
||||
limit:
|
||||
type: integer
|
||||
|
||||
MaintenancePolicy:
|
||||
type: object
|
||||
required: [name, table_name, schedule, vacuum_strategy]
|
||||
properties:
|
||||
id:
|
||||
$ref: "#/components/schemas/ResourceId"
|
||||
name:
|
||||
type: string
|
||||
table_name:
|
||||
type: string
|
||||
condition:
|
||||
type: string
|
||||
default: "true"
|
||||
retention_period_sec:
|
||||
type: integer
|
||||
minimum: 1
|
||||
max_rows:
|
||||
type: integer
|
||||
minimum: 1
|
||||
maximum: 100000
|
||||
vacuum_strategy:
|
||||
type: string
|
||||
enum: [none, vacuum, analyze, vacuum_analyze, reindex]
|
||||
schedule:
|
||||
type: string
|
||||
description: Cron expression (5-field, UTC).
|
||||
enabled:
|
||||
type: boolean
|
||||
default: true
|
||||
dry_run_enabled:
|
||||
type: boolean
|
||||
default: false
|
||||
last_run_at:
|
||||
type: string
|
||||
format: date-time
|
||||
last_status:
|
||||
type: string
|
||||
last_error:
|
||||
type: string
|
||||
created_at:
|
||||
type: string
|
||||
format: date-time
|
||||
updated_at:
|
||||
type: string
|
||||
format: date-time
|
||||
|
||||
MaintenancePolicyPatch:
|
||||
type: object
|
||||
properties:
|
||||
name:
|
||||
type: string
|
||||
table_name:
|
||||
type: string
|
||||
condition:
|
||||
type: string
|
||||
retention_period_sec:
|
||||
type: integer
|
||||
max_rows:
|
||||
type: integer
|
||||
vacuum_strategy:
|
||||
type: string
|
||||
enum: [none, vacuum, analyze, vacuum_analyze, reindex]
|
||||
schedule:
|
||||
type: string
|
||||
enabled:
|
||||
type: boolean
|
||||
dry_run_enabled:
|
||||
type: boolean
|
||||
|
||||
MaintenanceRunBody:
|
||||
type: object
|
||||
required: [policy_id]
|
||||
properties:
|
||||
policy_id:
|
||||
$ref: "#/components/schemas/ResourceId"
|
||||
|
||||
MaintenancePolicyList:
|
||||
type: object
|
||||
properties:
|
||||
items:
|
||||
type: array
|
||||
items:
|
||||
$ref: "#/components/schemas/MaintenancePolicy"
|
||||
next_cursor:
|
||||
type: string
|
||||
has_more:
|
||||
type: boolean
|
||||
|
||||
BirdLocalStatus:
|
||||
type: object
|
||||
description: Статус локального BIRD на хосте API (GET /v1/bird/status).
|
||||
@@ -3135,6 +3285,482 @@ paths:
|
||||
default:
|
||||
$ref: "#/components/responses/DefaultProblem"
|
||||
|
||||
/v1/monitoring/postgres/overview:
|
||||
get:
|
||||
tags: [Monitoring]
|
||||
summary: PostgreSQL overview (instance-level)
|
||||
operationId: getPostgresOverview
|
||||
parameters:
|
||||
- $ref: "#/components/parameters/TenantId"
|
||||
responses:
|
||||
"200":
|
||||
description: Успешно.
|
||||
content:
|
||||
application/json:
|
||||
schema:
|
||||
$ref: "#/components/schemas/PostgresOverview"
|
||||
"503":
|
||||
description: PostgreSQL backend не подключён.
|
||||
default:
|
||||
$ref: "#/components/responses/DefaultProblem"
|
||||
|
||||
/v1/monitoring/postgres/queries:
|
||||
get:
|
||||
tags: [Monitoring]
|
||||
summary: Top queries (pg_stat_statements or snapshot)
|
||||
operationId: getPostgresQueries
|
||||
parameters:
|
||||
- $ref: "#/components/parameters/TenantId"
|
||||
- $ref: "#/components/parameters/Limit"
|
||||
responses:
|
||||
"200":
|
||||
description: Успешно.
|
||||
content:
|
||||
application/json:
|
||||
schema:
|
||||
$ref: "#/components/schemas/PostgresQueriesResponse"
|
||||
default:
|
||||
$ref: "#/components/responses/DefaultProblem"
|
||||
|
||||
/v1/monitoring/postgres/locks:
|
||||
get:
|
||||
tags: [Monitoring]
|
||||
summary: Active locks
|
||||
operationId: getPostgresLocks
|
||||
parameters:
|
||||
- $ref: "#/components/parameters/TenantId"
|
||||
responses:
|
||||
"200":
|
||||
description: Успешно.
|
||||
content:
|
||||
application/json:
|
||||
schema:
|
||||
type: object
|
||||
properties:
|
||||
items:
|
||||
type: array
|
||||
items:
|
||||
type: object
|
||||
additionalProperties: true
|
||||
default:
|
||||
$ref: "#/components/responses/DefaultProblem"
|
||||
|
||||
/v1/monitoring/postgres/tables:
|
||||
get:
|
||||
tags: [Monitoring]
|
||||
summary: Table sizes and scan stats
|
||||
operationId: getPostgresTables
|
||||
parameters:
|
||||
- $ref: "#/components/parameters/TenantId"
|
||||
- $ref: "#/components/parameters/Limit"
|
||||
responses:
|
||||
"200":
|
||||
description: Успешно.
|
||||
content:
|
||||
application/json:
|
||||
schema:
|
||||
type: object
|
||||
properties:
|
||||
items:
|
||||
type: array
|
||||
items:
|
||||
type: object
|
||||
additionalProperties: true
|
||||
default:
|
||||
$ref: "#/components/responses/DefaultProblem"
|
||||
|
||||
/v1/monitoring/postgres/recommendations:
|
||||
get:
|
||||
tags: [Monitoring]
|
||||
summary: Heuristic optimization recommendations
|
||||
operationId: getPostgresRecommendations
|
||||
parameters:
|
||||
- $ref: "#/components/parameters/TenantId"
|
||||
responses:
|
||||
"200":
|
||||
description: Успешно.
|
||||
content:
|
||||
application/json:
|
||||
schema:
|
||||
$ref: "#/components/schemas/PostgresRecommendations"
|
||||
default:
|
||||
$ref: "#/components/responses/DefaultProblem"
|
||||
|
||||
/v1/monitoring/correlation:
|
||||
get:
|
||||
tags: [Monitoring]
|
||||
summary: Timeline correlation (jobs vs cache hit)
|
||||
operationId: getMonitoringCorrelation
|
||||
parameters:
|
||||
- $ref: "#/components/parameters/TenantId"
|
||||
- name: window
|
||||
in: query
|
||||
schema:
|
||||
type: integer
|
||||
default: 60
|
||||
description: Window in minutes (max 1440).
|
||||
responses:
|
||||
"200":
|
||||
description: Успешно.
|
||||
content:
|
||||
application/json:
|
||||
schema:
|
||||
type: object
|
||||
additionalProperties: true
|
||||
default:
|
||||
$ref: "#/components/responses/DefaultProblem"
|
||||
|
||||
/v1/postgres/vacuum:
|
||||
post:
|
||||
tags: [Monitoring]
|
||||
summary: VACUUM (async job, operator)
|
||||
operationId: postPostgresVacuum
|
||||
parameters:
|
||||
- $ref: "#/components/parameters/TenantId"
|
||||
- $ref: "#/components/parameters/IdempotencyKey"
|
||||
requestBody:
|
||||
content:
|
||||
application/json:
|
||||
schema:
|
||||
$ref: "#/components/schemas/PostgresMaintenanceBody"
|
||||
responses:
|
||||
"202":
|
||||
description: Задача поставлена.
|
||||
content:
|
||||
application/json:
|
||||
schema:
|
||||
$ref: "#/components/schemas/AsyncJobAccepted"
|
||||
"403":
|
||||
$ref: "#/components/responses/Forbidden"
|
||||
default:
|
||||
$ref: "#/components/responses/DefaultProblem"
|
||||
|
||||
/v1/postgres/vacuum-analyze:
|
||||
post:
|
||||
tags: [Monitoring]
|
||||
summary: VACUUM ANALYZE (async job, operator)
|
||||
operationId: postPostgresVacuumAnalyze
|
||||
parameters:
|
||||
- $ref: "#/components/parameters/TenantId"
|
||||
- $ref: "#/components/parameters/IdempotencyKey"
|
||||
requestBody:
|
||||
content:
|
||||
application/json:
|
||||
schema:
|
||||
$ref: "#/components/schemas/PostgresMaintenanceBody"
|
||||
responses:
|
||||
"202":
|
||||
description: Задача поставлена.
|
||||
content:
|
||||
application/json:
|
||||
schema:
|
||||
$ref: "#/components/schemas/AsyncJobAccepted"
|
||||
default:
|
||||
$ref: "#/components/responses/DefaultProblem"
|
||||
|
||||
/v1/postgres/analyze:
|
||||
post:
|
||||
tags: [Monitoring]
|
||||
summary: ANALYZE (async job, operator)
|
||||
operationId: postPostgresAnalyze
|
||||
parameters:
|
||||
- $ref: "#/components/parameters/TenantId"
|
||||
- $ref: "#/components/parameters/IdempotencyKey"
|
||||
requestBody:
|
||||
content:
|
||||
application/json:
|
||||
schema:
|
||||
$ref: "#/components/schemas/PostgresMaintenanceBody"
|
||||
responses:
|
||||
"202":
|
||||
description: Задача поставлена.
|
||||
content:
|
||||
application/json:
|
||||
schema:
|
||||
$ref: "#/components/schemas/AsyncJobAccepted"
|
||||
default:
|
||||
$ref: "#/components/responses/DefaultProblem"
|
||||
|
||||
/v1/postgres/reindex:
|
||||
post:
|
||||
tags: [Monitoring]
|
||||
summary: REINDEX TABLE (async job, operator)
|
||||
operationId: postPostgresReindex
|
||||
parameters:
|
||||
- $ref: "#/components/parameters/TenantId"
|
||||
- $ref: "#/components/parameters/IdempotencyKey"
|
||||
requestBody:
|
||||
content:
|
||||
application/json:
|
||||
schema:
|
||||
$ref: "#/components/schemas/PostgresMaintenanceBody"
|
||||
responses:
|
||||
"202":
|
||||
description: Задача поставлена.
|
||||
content:
|
||||
application/json:
|
||||
schema:
|
||||
$ref: "#/components/schemas/AsyncJobAccepted"
|
||||
default:
|
||||
$ref: "#/components/responses/DefaultProblem"
|
||||
|
||||
/v1/postgres/cleanup:
|
||||
post:
|
||||
tags: [Monitoring]
|
||||
summary: Retention cleanup (async job, operator)
|
||||
operationId: postPostgresCleanup
|
||||
parameters:
|
||||
- $ref: "#/components/parameters/TenantId"
|
||||
- $ref: "#/components/parameters/IdempotencyKey"
|
||||
requestBody:
|
||||
required: true
|
||||
content:
|
||||
application/json:
|
||||
schema:
|
||||
$ref: "#/components/schemas/PostgresMaintenanceBody"
|
||||
responses:
|
||||
"202":
|
||||
description: Задача поставлена.
|
||||
content:
|
||||
application/json:
|
||||
schema:
|
||||
$ref: "#/components/schemas/AsyncJobAccepted"
|
||||
default:
|
||||
$ref: "#/components/responses/DefaultProblem"
|
||||
|
||||
/v1/postgres/maintenance/logs:
|
||||
get:
|
||||
tags: [Monitoring]
|
||||
summary: Maintenance audit log
|
||||
operationId: listPostgresMaintenanceLogs
|
||||
parameters:
|
||||
- $ref: "#/components/parameters/TenantId"
|
||||
- $ref: "#/components/parameters/Cursor"
|
||||
- $ref: "#/components/parameters/Limit"
|
||||
responses:
|
||||
"200":
|
||||
description: Успешно.
|
||||
content:
|
||||
application/json:
|
||||
schema:
|
||||
type: object
|
||||
properties:
|
||||
items:
|
||||
type: array
|
||||
items:
|
||||
type: object
|
||||
additionalProperties: true
|
||||
next_cursor:
|
||||
type: string
|
||||
has_more:
|
||||
type: boolean
|
||||
default:
|
||||
$ref: "#/components/responses/DefaultProblem"
|
||||
|
||||
/v1/maintenance/policies:
|
||||
get:
|
||||
tags: [Maintenance]
|
||||
summary: List maintenance policies
|
||||
operationId: listMaintenancePolicies
|
||||
parameters:
|
||||
- $ref: "#/components/parameters/TenantId"
|
||||
- $ref: "#/components/parameters/Cursor"
|
||||
- $ref: "#/components/parameters/Limit"
|
||||
responses:
|
||||
"200":
|
||||
description: Успешно.
|
||||
content:
|
||||
application/json:
|
||||
schema:
|
||||
$ref: "#/components/schemas/MaintenancePolicyList"
|
||||
default:
|
||||
$ref: "#/components/responses/DefaultProblem"
|
||||
post:
|
||||
tags: [Maintenance]
|
||||
summary: Create maintenance policy
|
||||
operationId: createMaintenancePolicy
|
||||
parameters:
|
||||
- $ref: "#/components/parameters/TenantId"
|
||||
requestBody:
|
||||
required: true
|
||||
content:
|
||||
application/json:
|
||||
schema:
|
||||
$ref: "#/components/schemas/MaintenancePolicy"
|
||||
responses:
|
||||
"201":
|
||||
description: Создано.
|
||||
content:
|
||||
application/json:
|
||||
schema:
|
||||
$ref: "#/components/schemas/MaintenancePolicy"
|
||||
default:
|
||||
$ref: "#/components/responses/DefaultProblem"
|
||||
|
||||
/v1/maintenance/policies/{id}:
|
||||
get:
|
||||
tags: [Maintenance]
|
||||
summary: Get maintenance policy
|
||||
operationId: getMaintenancePolicy
|
||||
parameters:
|
||||
- $ref: "#/components/parameters/TenantId"
|
||||
- name: id
|
||||
in: path
|
||||
required: true
|
||||
schema:
|
||||
$ref: "#/components/schemas/ResourceId"
|
||||
responses:
|
||||
"200":
|
||||
description: Успешно.
|
||||
content:
|
||||
application/json:
|
||||
schema:
|
||||
$ref: "#/components/schemas/MaintenancePolicy"
|
||||
default:
|
||||
$ref: "#/components/responses/DefaultProblem"
|
||||
patch:
|
||||
tags: [Maintenance]
|
||||
summary: Update maintenance policy
|
||||
operationId: patchMaintenancePolicy
|
||||
parameters:
|
||||
- $ref: "#/components/parameters/TenantId"
|
||||
- name: id
|
||||
in: path
|
||||
required: true
|
||||
schema:
|
||||
$ref: "#/components/schemas/ResourceId"
|
||||
requestBody:
|
||||
required: true
|
||||
content:
|
||||
application/json:
|
||||
schema:
|
||||
$ref: "#/components/schemas/MaintenancePolicyPatch"
|
||||
responses:
|
||||
"200":
|
||||
description: Успешно.
|
||||
content:
|
||||
application/json:
|
||||
schema:
|
||||
$ref: "#/components/schemas/MaintenancePolicy"
|
||||
default:
|
||||
$ref: "#/components/responses/DefaultProblem"
|
||||
delete:
|
||||
tags: [Maintenance]
|
||||
summary: Delete maintenance policy
|
||||
operationId: deleteMaintenancePolicy
|
||||
parameters:
|
||||
- $ref: "#/components/parameters/TenantId"
|
||||
- name: id
|
||||
in: path
|
||||
required: true
|
||||
schema:
|
||||
$ref: "#/components/schemas/ResourceId"
|
||||
responses:
|
||||
"204":
|
||||
description: Удалено.
|
||||
default:
|
||||
$ref: "#/components/responses/DefaultProblem"
|
||||
|
||||
/v1/maintenance/policies/{id}/hints:
|
||||
get:
|
||||
tags: [Maintenance]
|
||||
summary: PostgreSQL hints for policy table
|
||||
operationId: getMaintenancePolicyHints
|
||||
parameters:
|
||||
- $ref: "#/components/parameters/TenantId"
|
||||
- name: id
|
||||
in: path
|
||||
required: true
|
||||
schema:
|
||||
$ref: "#/components/schemas/ResourceId"
|
||||
responses:
|
||||
"200":
|
||||
description: Успешно.
|
||||
content:
|
||||
application/json:
|
||||
schema:
|
||||
type: object
|
||||
additionalProperties: true
|
||||
default:
|
||||
$ref: "#/components/responses/DefaultProblem"
|
||||
|
||||
/v1/maintenance/config-audit:
|
||||
get:
|
||||
tags: [Maintenance]
|
||||
summary: Maintenance policy configuration audit log
|
||||
operationId: listMaintenanceConfigAudit
|
||||
parameters:
|
||||
- $ref: "#/components/parameters/TenantId"
|
||||
- $ref: "#/components/parameters/Cursor"
|
||||
- $ref: "#/components/parameters/Limit"
|
||||
responses:
|
||||
"200":
|
||||
description: Успешно.
|
||||
content:
|
||||
application/json:
|
||||
schema:
|
||||
type: object
|
||||
properties:
|
||||
items:
|
||||
type: array
|
||||
items:
|
||||
type: object
|
||||
additionalProperties: true
|
||||
next_cursor:
|
||||
type: string
|
||||
has_more:
|
||||
type: boolean
|
||||
default:
|
||||
$ref: "#/components/responses/DefaultProblem"
|
||||
|
||||
/v1/maintenance/run:
|
||||
post:
|
||||
tags: [Maintenance]
|
||||
summary: Run maintenance policy (async job)
|
||||
operationId: postMaintenanceRun
|
||||
parameters:
|
||||
- $ref: "#/components/parameters/TenantId"
|
||||
- $ref: "#/components/parameters/IdempotencyKey"
|
||||
requestBody:
|
||||
required: true
|
||||
content:
|
||||
application/json:
|
||||
schema:
|
||||
$ref: "#/components/schemas/MaintenanceRunBody"
|
||||
responses:
|
||||
"202":
|
||||
description: Задача поставлена.
|
||||
content:
|
||||
application/json:
|
||||
schema:
|
||||
$ref: "#/components/schemas/AsyncJobAccepted"
|
||||
default:
|
||||
$ref: "#/components/responses/DefaultProblem"
|
||||
|
||||
/v1/maintenance/dry-run:
|
||||
post:
|
||||
tags: [Maintenance]
|
||||
summary: Dry-run maintenance policy (async job)
|
||||
operationId: postMaintenanceDryRun
|
||||
parameters:
|
||||
- $ref: "#/components/parameters/TenantId"
|
||||
- $ref: "#/components/parameters/IdempotencyKey"
|
||||
requestBody:
|
||||
required: true
|
||||
content:
|
||||
application/json:
|
||||
schema:
|
||||
$ref: "#/components/schemas/MaintenanceRunBody"
|
||||
responses:
|
||||
"202":
|
||||
description: Задача поставлена.
|
||||
content:
|
||||
application/json:
|
||||
schema:
|
||||
$ref: "#/components/schemas/AsyncJobAccepted"
|
||||
default:
|
||||
$ref: "#/components/responses/DefaultProblem"
|
||||
|
||||
/v1/settings:
|
||||
get:
|
||||
tags: [Settings]
|
||||
|
||||
@@ -25,6 +25,7 @@ require (
|
||||
github.com/prometheus/common v0.55.0 // indirect
|
||||
github.com/prometheus/procfs v0.15.1 // indirect
|
||||
github.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec // indirect
|
||||
github.com/robfig/cron/v3 v3.0.1 // indirect
|
||||
golang.org/x/crypto v0.46.0 // indirect
|
||||
golang.org/x/mod v0.31.0 // indirect
|
||||
golang.org/x/net v0.48.0 // indirect
|
||||
|
||||
@@ -46,6 +46,8 @@ github.com/prometheus/procfs v0.15.1 h1:YagwOFzUgYfKKHX6Dr+sHT7km/hxC76UB0leargg
|
||||
github.com/prometheus/procfs v0.15.1/go.mod h1:fB45yRUv8NstnjriLhBQLuOUt+WW4BsoGhij/e3PBqk=
|
||||
github.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec h1:W09IVJc94icq4NjY3clb7Lk8O1qJ8BdBEF8z0ibU0rE=
|
||||
github.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec/go.mod h1:qqbHyh8v60DhA7CoWK5oRCqLrMHRGoxYCSS9EjAz6Eo=
|
||||
github.com/robfig/cron/v3 v3.0.1 h1:WdRxkvbJztn8LMz/QEvLN5sBU+xKpSqwwUO1Pjr4qDs=
|
||||
github.com/robfig/cron/v3 v3.0.1/go.mod h1:eQICP3HwyT7UooqI/z+Ov+PtYAWygg1TEWWzGIFLtro=
|
||||
github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
|
||||
github.com/stretchr/testify v1.3.0/go.mod h1:M5WIy9Dh21IEIfnGCwXGc5bZfKNJtfHm1UVUgZn+9EI=
|
||||
github.com/stretchr/testify v1.7.0/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
|
||||
|
||||
@@ -0,0 +1,212 @@
|
||||
// Package dbcli implements control-plane PostgreSQL maintenance CLI (HTTP or local DSN).
|
||||
package dbcli
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"context"
|
||||
"encoding/json"
|
||||
"flag"
|
||||
"fmt"
|
||||
"io"
|
||||
"net/http"
|
||||
"os"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"evobgp/internal/db"
|
||||
"evobgp/internal/httpclient"
|
||||
"evobgp/internal/pgmonitor"
|
||||
)
|
||||
|
||||
// Run executes db subcommands; args exclude program name and "db".
|
||||
func Run(args []string) int {
|
||||
if len(args) == 0 {
|
||||
printUsage()
|
||||
return 2
|
||||
}
|
||||
switch args[0] {
|
||||
case "report":
|
||||
return cmdReport(args[1:])
|
||||
case "vacuum":
|
||||
return cmdMaint(args[1:], "vacuum", "/v1/postgres/vacuum")
|
||||
case "analyze":
|
||||
return cmdMaint(args[1:], "analyze", "/v1/postgres/analyze")
|
||||
case "cleanup":
|
||||
return cmdCleanup(args[1:])
|
||||
default:
|
||||
fmt.Fprintf(os.Stderr, "dbcli: unknown command %q\n", args[0])
|
||||
printUsage()
|
||||
return 2
|
||||
}
|
||||
}
|
||||
|
||||
func printUsage() {
|
||||
fmt.Fprintln(os.Stderr, `usage:
|
||||
evobgp-api db report [--api-url URL] [--token TOKEN] [--format json]
|
||||
evobgp-api db vacuum [--table NAME] [--dry-run] [--api-url URL] [--token TOKEN]
|
||||
evobgp-api db analyze [--table NAME] [--dry-run] [--api-url URL] [--token TOKEN]
|
||||
evobgp-api db cleanup --policy NAME [--dry-run] [--limit N] [--api-url URL] [--token TOKEN]
|
||||
Local break-glass: set EVOBGP_DATABASE_URL (report only uses direct SQL).`)
|
||||
}
|
||||
|
||||
func cmdReport(args []string) int {
|
||||
fs := flag.NewFlagSet("report", flag.ExitOnError)
|
||||
apiURL := fs.String("api-url", "", "control plane base URL")
|
||||
token := fs.String("token", "", "Bearer token (operator)")
|
||||
format := fs.String("format", "json", "output format (json)")
|
||||
_ = fs.Parse(args)
|
||||
|
||||
if dsn := strings.TrimSpace(os.Getenv("EVOBGP_DATABASE_URL")); dsn != "" && *apiURL == "" {
|
||||
ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
|
||||
defer cancel()
|
||||
pool, err := db.OpenPostgresPool(ctx, dsn)
|
||||
if err != nil {
|
||||
fmt.Fprintln(os.Stderr, err)
|
||||
return 1
|
||||
}
|
||||
defer pool.Close()
|
||||
svc := pgmonitor.NewService(pool)
|
||||
ov, err := svc.Overview(ctx)
|
||||
if err != nil {
|
||||
fmt.Fprintln(os.Stderr, err)
|
||||
return 1
|
||||
}
|
||||
return writeJSONStdout(ov, *format)
|
||||
}
|
||||
if *apiURL == "" || *token == "" {
|
||||
fmt.Fprintln(os.Stderr, "report: --api-url and --token required without EVOBGP_DATABASE_URL")
|
||||
return 2
|
||||
}
|
||||
body, err := apiGET(*apiURL, *token, "/v1/monitoring/postgres/overview")
|
||||
if err != nil {
|
||||
fmt.Fprintln(os.Stderr, err)
|
||||
return 1
|
||||
}
|
||||
var pretty any
|
||||
if err := json.Unmarshal(body, &pretty); err != nil {
|
||||
fmt.Fprintln(os.Stderr, err)
|
||||
return 1
|
||||
}
|
||||
return writeJSONStdout(pretty, *format)
|
||||
}
|
||||
|
||||
func cmdMaint(args []string, _ string, path string) int {
|
||||
fs := flag.NewFlagSet("maint", flag.ExitOnError)
|
||||
table := fs.String("table", "", "table name")
|
||||
dryRun := fs.Bool("dry-run", false, "dry run only")
|
||||
apiURL := fs.String("api-url", "", "control plane base URL")
|
||||
token := fs.String("token", "", "Bearer token (operator)")
|
||||
_ = fs.Parse(args)
|
||||
if *apiURL == "" || *token == "" {
|
||||
fmt.Fprintln(os.Stderr, "maintenance: --api-url and --token are required")
|
||||
return 2
|
||||
}
|
||||
payload := map[string]any{"dry_run": *dryRun}
|
||||
if *table != "" {
|
||||
payload["table"] = *table
|
||||
}
|
||||
body, err := apiPOST(*apiURL, *token, path, payload)
|
||||
if err != nil {
|
||||
fmt.Fprintln(os.Stderr, err)
|
||||
return 1
|
||||
}
|
||||
return writeRawJSON(body)
|
||||
}
|
||||
|
||||
func cmdCleanup(args []string) int {
|
||||
fs := flag.NewFlagSet("cleanup", flag.ExitOnError)
|
||||
policyID := fs.String("policy-id", "", "maintenance policy UUID")
|
||||
dryRun := fs.Bool("dry-run", true, "dry run")
|
||||
apiURL := fs.String("api-url", "", "control plane base URL")
|
||||
token := fs.String("token", "", "Bearer token (operator)")
|
||||
_ = fs.Parse(args)
|
||||
if *policyID == "" {
|
||||
fmt.Fprintln(os.Stderr, "cleanup: --policy-id is required")
|
||||
return 2
|
||||
}
|
||||
if *apiURL == "" || *token == "" {
|
||||
fmt.Fprintln(os.Stderr, "cleanup: --api-url and --token are required")
|
||||
return 2
|
||||
}
|
||||
path := "/v1/maintenance/run"
|
||||
if *dryRun {
|
||||
path = "/v1/maintenance/dry-run"
|
||||
}
|
||||
payload := map[string]any{"policy_id": *policyID}
|
||||
body, err := apiPOST(*apiURL, *token, path, payload)
|
||||
if err != nil {
|
||||
fmt.Fprintln(os.Stderr, err)
|
||||
return 1
|
||||
}
|
||||
return writeRawJSON(body)
|
||||
}
|
||||
|
||||
func apiGET(base, token, path string) ([]byte, error) {
|
||||
u := strings.TrimRight(base, "/") + path
|
||||
req, err := http.NewRequest(http.MethodGet, u, nil)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
req.Header.Set("Authorization", "Bearer "+token)
|
||||
ctx, cancel := context.WithTimeout(context.Background(), 60*time.Second)
|
||||
defer cancel()
|
||||
resp, err := httpclient.DoWithRetry(ctx, httpclient.New(60*time.Second), req, 3)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
defer func() { _ = resp.Body.Close() }()
|
||||
b, _ := io.ReadAll(resp.Body)
|
||||
if resp.StatusCode >= 300 {
|
||||
return nil, fmt.Errorf("dbcli: GET %s: %s: %s", path, resp.Status, strings.TrimSpace(string(b)))
|
||||
}
|
||||
return b, nil
|
||||
}
|
||||
|
||||
func apiPOST(base, token, path string, payload map[string]any) ([]byte, error) {
|
||||
b, err := json.Marshal(payload)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
u := strings.TrimRight(base, "/") + path
|
||||
req, err := http.NewRequest(http.MethodPost, u, bytes.NewReader(b))
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
req.Header.Set("Authorization", "Bearer "+token)
|
||||
req.Header.Set("Content-Type", "application/json")
|
||||
ctx, cancel := context.WithTimeout(context.Background(), 60*time.Second)
|
||||
defer cancel()
|
||||
resp, err := httpclient.DoWithRetry(ctx, httpclient.New(60*time.Second), req, 3)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
defer func() { _ = resp.Body.Close() }()
|
||||
out, _ := io.ReadAll(resp.Body)
|
||||
if resp.StatusCode != http.StatusAccepted && resp.StatusCode != http.StatusOK {
|
||||
return nil, fmt.Errorf("dbcli: POST %s: %s: %s", path, resp.Status, strings.TrimSpace(string(out)))
|
||||
}
|
||||
return out, nil
|
||||
}
|
||||
|
||||
func writeJSONStdout(v any, format string) int {
|
||||
if format != "json" {
|
||||
fmt.Fprintln(os.Stderr, "only json format supported")
|
||||
return 2
|
||||
}
|
||||
enc := json.NewEncoder(os.Stdout)
|
||||
enc.SetIndent("", " ")
|
||||
if err := enc.Encode(v); err != nil {
|
||||
fmt.Fprintln(os.Stderr, err)
|
||||
return 1
|
||||
}
|
||||
return 0
|
||||
}
|
||||
|
||||
func writeRawJSON(b []byte) int {
|
||||
var v any
|
||||
if err := json.Unmarshal(b, &v); err != nil {
|
||||
_, _ = os.Stdout.Write(b)
|
||||
return 0
|
||||
}
|
||||
return writeJSONStdout(v, "json")
|
||||
}
|
||||
@@ -51,7 +51,7 @@ func BootstrapWorkers(ctx context.Context, opts Options) (store.Backend, *jobs.R
|
||||
}
|
||||
|
||||
cdnHTTP := NewCDNHTTPClient()
|
||||
wk := &jobs.Worker{Store: backend, HTTPClient: cdnHTTP}
|
||||
wk := &jobs.Worker{Store: backend, PgPool: pool, HTTPClient: cdnHTTP}
|
||||
reg := jobs.NewRegistry(wk.Process)
|
||||
wk.Registry = reg
|
||||
if pool != nil {
|
||||
|
||||
@@ -76,6 +76,9 @@ func (s *Server) registerV1(m *http.ServeMux) {
|
||||
m.HandleFunc("GET /speakers/{speaker_id}/bundle/{revision_id}", s.handleNodeBundle)
|
||||
m.HandleFunc("POST /nodes/enroll", s.handleNodeEnroll)
|
||||
s.registerCRUDRoutes(m)
|
||||
s.registerPostgresMonitoringRoutes(m)
|
||||
s.registerPostgresMaintenanceRoutes(m)
|
||||
s.registerMaintenanceRoutes(m)
|
||||
}
|
||||
|
||||
func (s *Server) handleHealth(w http.ResponseWriter, r *http.Request) {
|
||||
|
||||
@@ -0,0 +1,274 @@
|
||||
package httpapi
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"net/http"
|
||||
"strings"
|
||||
|
||||
"evobgp/internal/jobs"
|
||||
"evobgp/internal/observability"
|
||||
"evobgp/internal/store"
|
||||
)
|
||||
|
||||
func (s *Server) registerMaintenanceRoutes(m *http.ServeMux) {
|
||||
m.HandleFunc("GET /maintenance/policies", s.handleListMaintenancePolicies)
|
||||
m.HandleFunc("POST /maintenance/policies", s.handleCreateMaintenancePolicy)
|
||||
m.HandleFunc("GET /maintenance/policies/{id}", s.handleGetMaintenancePolicy)
|
||||
m.HandleFunc("PATCH /maintenance/policies/{id}", s.handlePatchMaintenancePolicy)
|
||||
m.HandleFunc("DELETE /maintenance/policies/{id}", s.handleDeleteMaintenancePolicy)
|
||||
m.HandleFunc("GET /maintenance/policies/{id}/hints", s.handleMaintenancePolicyHints)
|
||||
m.HandleFunc("GET /maintenance/config-audit", s.handleListMaintenanceConfigAudit)
|
||||
m.HandleFunc("POST /maintenance/run", s.handleMaintenanceRun)
|
||||
m.HandleFunc("POST /maintenance/dry-run", s.handleMaintenanceDryRun)
|
||||
}
|
||||
|
||||
func maintenancePolicyJSON(p *store.MaintenancePolicy) map[string]any {
|
||||
if p == nil {
|
||||
return map[string]any{}
|
||||
}
|
||||
out := map[string]any{
|
||||
"id": p.ID,
|
||||
"name": p.Name,
|
||||
"table_name": p.TableName,
|
||||
"condition": p.Condition,
|
||||
"vacuum_strategy": p.VacuumStrategy,
|
||||
"schedule": p.Schedule,
|
||||
"enabled": p.Enabled,
|
||||
"dry_run_enabled": p.DryRunEnabled,
|
||||
}
|
||||
if p.RetentionPeriodSec != nil {
|
||||
out["retention_period_sec"] = *p.RetentionPeriodSec
|
||||
}
|
||||
if p.MaxRows != nil {
|
||||
out["max_rows"] = *p.MaxRows
|
||||
}
|
||||
if p.LastRunAt != nil {
|
||||
out["last_run_at"] = p.LastRunAt.UTC().Format("2006-01-02T15:04:05Z")
|
||||
}
|
||||
if p.LastStatus != "" {
|
||||
out["last_status"] = p.LastStatus
|
||||
}
|
||||
if p.LastError != "" {
|
||||
out["last_error"] = p.LastError
|
||||
}
|
||||
if !p.CreatedAt.IsZero() {
|
||||
out["created_at"] = p.CreatedAt.UTC().Format("2006-01-02T15:04:05Z")
|
||||
}
|
||||
if !p.UpdatedAt.IsZero() {
|
||||
out["updated_at"] = p.UpdatedAt.UTC().Format("2006-01-02T15:04:05Z")
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
func (s *Server) handleListMaintenancePolicies(w http.ResponseWriter, r *http.Request) {
|
||||
a, ok := authFromContext(r.Context())
|
||||
if !ok || !s.requireAtLeast(w, a, "viewer") || !s.requirePostgres(w) {
|
||||
return
|
||||
}
|
||||
cursor := r.URL.Query().Get("cursor")
|
||||
limit := parseLimitQuery(r, 20, 100)
|
||||
items, next, hasMore, err := s.store.ListMaintenancePolicies(cursor, limit)
|
||||
if err != nil {
|
||||
writeInternalError(w, "maintenance_policies_list", err)
|
||||
return
|
||||
}
|
||||
out := make([]map[string]any, 0, len(items))
|
||||
for _, p := range items {
|
||||
out = append(out, maintenancePolicyJSON(p))
|
||||
}
|
||||
writeJSON(w, http.StatusOK, map[string]any{"items": out, "next_cursor": next, "has_more": hasMore})
|
||||
}
|
||||
|
||||
func (s *Server) handleGetMaintenancePolicy(w http.ResponseWriter, r *http.Request) {
|
||||
a, ok := authFromContext(r.Context())
|
||||
if !ok || !s.requireAtLeast(w, a, "viewer") || !s.requirePostgres(w) {
|
||||
return
|
||||
}
|
||||
p, err := s.store.GetMaintenancePolicy(r.PathValue("id"))
|
||||
if err != nil {
|
||||
writeStoreErr(w, err)
|
||||
return
|
||||
}
|
||||
writeJSON(w, http.StatusOK, maintenancePolicyJSON(p))
|
||||
}
|
||||
|
||||
func (s *Server) handleCreateMaintenancePolicy(w http.ResponseWriter, r *http.Request) {
|
||||
a, ok := authFromContext(r.Context())
|
||||
if !ok || !s.requirePostgres(w) || !s.requireOperatorStrict(w, a) {
|
||||
return
|
||||
}
|
||||
var body store.MaintenancePolicy
|
||||
if err := json.NewDecoder(r.Body).Decode(&body); err != nil {
|
||||
writeProblem(w, http.StatusBadRequest, "Bad Request", invalidInputDetail)
|
||||
return
|
||||
}
|
||||
p, err := s.store.CreateMaintenancePolicy(&body)
|
||||
if err != nil {
|
||||
writeStoreErr(w, err)
|
||||
return
|
||||
}
|
||||
_ = s.store.AppendMaintenancePolicyConfigAudit(actorPrefix(a), p.ID, "create", nil, maintenancePolicyJSON(p))
|
||||
observability.IncMaintenanceConfigChange("create")
|
||||
s.reloadMaintenanceConfig(r)
|
||||
writeJSON(w, http.StatusCreated, maintenancePolicyJSON(p))
|
||||
}
|
||||
|
||||
func (s *Server) handlePatchMaintenancePolicy(w http.ResponseWriter, r *http.Request) {
|
||||
a, ok := authFromContext(r.Context())
|
||||
if !ok || !s.requirePostgres(w) || !s.requireOperatorStrict(w, a) {
|
||||
return
|
||||
}
|
||||
id := r.PathValue("id")
|
||||
before, err := s.store.GetMaintenancePolicy(id)
|
||||
if err != nil {
|
||||
writeStoreErr(w, err)
|
||||
return
|
||||
}
|
||||
var patch store.MaintenancePolicyPatch
|
||||
if err := json.NewDecoder(r.Body).Decode(&patch); err != nil {
|
||||
writeProblem(w, http.StatusBadRequest, "Bad Request", invalidInputDetail)
|
||||
return
|
||||
}
|
||||
updated, err := s.store.UpdateMaintenancePolicy(id, &patch)
|
||||
if err != nil {
|
||||
writeStoreErr(w, err)
|
||||
return
|
||||
}
|
||||
_ = s.store.AppendMaintenancePolicyConfigAudit(actorPrefix(a), id, "update", maintenancePolicyJSON(before), maintenancePolicyJSON(updated))
|
||||
observability.IncMaintenanceConfigChange("update")
|
||||
s.reloadMaintenanceConfig(r)
|
||||
writeJSON(w, http.StatusOK, maintenancePolicyJSON(updated))
|
||||
}
|
||||
|
||||
func (s *Server) handleDeleteMaintenancePolicy(w http.ResponseWriter, r *http.Request) {
|
||||
a, ok := authFromContext(r.Context())
|
||||
if !ok || !s.requirePostgres(w) || !s.requireOperatorStrict(w, a) {
|
||||
return
|
||||
}
|
||||
id := r.PathValue("id")
|
||||
before, err := s.store.GetMaintenancePolicy(id)
|
||||
if err != nil {
|
||||
writeStoreErr(w, err)
|
||||
return
|
||||
}
|
||||
if err := s.store.DeleteMaintenancePolicy(id); err != nil {
|
||||
writeStoreErr(w, err)
|
||||
return
|
||||
}
|
||||
_ = s.store.AppendMaintenancePolicyConfigAudit(actorPrefix(a), id, "delete", maintenancePolicyJSON(before), nil)
|
||||
observability.IncMaintenanceConfigChange("delete")
|
||||
s.reloadMaintenanceConfig(r)
|
||||
w.WriteHeader(http.StatusNoContent)
|
||||
}
|
||||
|
||||
func (s *Server) handleMaintenancePolicyHints(w http.ResponseWriter, r *http.Request) {
|
||||
a, ok := authFromContext(r.Context())
|
||||
if !ok || !s.requireAtLeast(w, a, "viewer") || !s.requirePostgres(w) {
|
||||
return
|
||||
}
|
||||
if s.maintStats == nil {
|
||||
writeProblem(w, http.StatusServiceUnavailable, "Unavailable", "postgresql backend required")
|
||||
return
|
||||
}
|
||||
p, err := s.store.GetMaintenancePolicy(r.PathValue("id"))
|
||||
if err != nil {
|
||||
writeStoreErr(w, err)
|
||||
return
|
||||
}
|
||||
hints, err := s.maintStats.Hints(r.Context(), p.TableName)
|
||||
if err != nil {
|
||||
writeInternalError(w, "maintenance_policy_hints", err)
|
||||
return
|
||||
}
|
||||
writeJSON(w, http.StatusOK, hints)
|
||||
}
|
||||
|
||||
func (s *Server) handleListMaintenanceConfigAudit(w http.ResponseWriter, r *http.Request) {
|
||||
a, ok := authFromContext(r.Context())
|
||||
if !ok || !s.requireAtLeast(w, a, "viewer") || !s.requirePostgres(w) {
|
||||
return
|
||||
}
|
||||
cursor := r.URL.Query().Get("cursor")
|
||||
limit := parseLimitQuery(r, 20, 100)
|
||||
items, next, hasMore, err := s.store.ListMaintenancePolicyConfigAudit(cursor, limit)
|
||||
if err != nil {
|
||||
writeInternalError(w, "maintenance_config_audit", err)
|
||||
return
|
||||
}
|
||||
out := make([]map[string]any, 0, len(items))
|
||||
for _, row := range items {
|
||||
out = append(out, map[string]any{
|
||||
"id": row.ID,
|
||||
"policy_id": row.PolicyID,
|
||||
"actor_prefix": row.ActorPrefix,
|
||||
"action": row.Action,
|
||||
"before": row.Before,
|
||||
"after": row.After,
|
||||
"created_at": row.CreatedAt.UTC().Format("2006-01-02T15:04:05Z"),
|
||||
})
|
||||
}
|
||||
writeJSON(w, http.StatusOK, map[string]any{"items": out, "next_cursor": next, "has_more": hasMore})
|
||||
}
|
||||
|
||||
type maintenanceRunBody struct {
|
||||
PolicyID string `json:"policy_id"`
|
||||
}
|
||||
|
||||
func (s *Server) handleMaintenanceRun(w http.ResponseWriter, r *http.Request) {
|
||||
s.enqueueMaintenancePolicy(w, r, false)
|
||||
}
|
||||
|
||||
func (s *Server) handleMaintenanceDryRun(w http.ResponseWriter, r *http.Request) {
|
||||
s.enqueueMaintenancePolicy(w, r, true)
|
||||
}
|
||||
|
||||
func (s *Server) enqueueMaintenancePolicy(w http.ResponseWriter, r *http.Request, dryRun bool) {
|
||||
a, ok := authFromContext(r.Context())
|
||||
if !ok || !s.requirePostgres(w) || !s.requireOperatorStrict(w, a) {
|
||||
return
|
||||
}
|
||||
var body maintenanceRunBody
|
||||
if err := json.NewDecoder(r.Body).Decode(&body); err != nil {
|
||||
writeProblem(w, http.StatusBadRequest, "Bad Request", invalidInputDetail)
|
||||
return
|
||||
}
|
||||
policyID := strings.TrimSpace(body.PolicyID)
|
||||
if policyID == "" {
|
||||
writeProblem(w, http.StatusBadRequest, "Bad Request", "policy_id is required")
|
||||
return
|
||||
}
|
||||
if _, err := s.store.GetMaintenancePolicy(policyID); err != nil {
|
||||
writeStoreErr(w, err)
|
||||
return
|
||||
}
|
||||
kind := "maintenance_policy_run"
|
||||
if !s.checkPgMaintRateLimit(a.TenantID, kind+":"+policyID) {
|
||||
writeProblem(w, http.StatusTooManyRequests, "Too Many Requests", "wait before repeating this maintenance operation")
|
||||
return
|
||||
}
|
||||
idem := strings.TrimSpace(r.Header.Get("Idempotency-Key"))
|
||||
var idemPtr *string
|
||||
if idem != "" {
|
||||
idemPtr = &idem
|
||||
}
|
||||
title := "Maintenance policy run"
|
||||
if dryRun {
|
||||
title = "Maintenance policy dry-run"
|
||||
}
|
||||
j, _, err := s.jobs.Enqueue(a.TenantID, jobs.KindMaintenancePolicyRun, idemPtr, nil, map[string]any{
|
||||
"policy_id": policyID, "dry_run": dryRun, "actor_prefix": actorPrefix(a), "job_title": title,
|
||||
})
|
||||
if err != nil {
|
||||
writeInternalError(w, "maintenance_policy_enqueue", err)
|
||||
return
|
||||
}
|
||||
w.Header().Set("Location", "/v1/jobs/"+j.ID)
|
||||
snap := j.Snapshot()
|
||||
writeJSON(w, http.StatusAccepted, map[string]any{"job_id": snap["job_id"], "status": snap["status"]})
|
||||
}
|
||||
|
||||
func (s *Server) reloadMaintenanceConfig(r *http.Request) {
|
||||
if s.maintConfig != nil {
|
||||
_ = s.maintConfig.Reload(r.Context())
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,46 @@
|
||||
package httpapi
|
||||
|
||||
import (
|
||||
"net/http"
|
||||
"net/http/httptest"
|
||||
"strings"
|
||||
"testing"
|
||||
)
|
||||
|
||||
func TestMaintenancePoliciesMemoryBackend503(t *testing.T) {
|
||||
srv, err := New(Options{SeedDemo: true, InsecureDev: true})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
defer srv.Close()
|
||||
handler := srv.Handler()
|
||||
|
||||
tests := []struct {
|
||||
method string
|
||||
path string
|
||||
body string
|
||||
}{
|
||||
{http.MethodGet, "/v1/maintenance/policies", ""},
|
||||
{http.MethodPost, "/v1/maintenance/policies", `{"name":"x","table_name":"job_audit","schedule":"0 3 * * *"}`},
|
||||
{http.MethodPost, "/v1/maintenance/run", `{"policy_id":"00000000-0000-0000-0000-000000000001"}`},
|
||||
{http.MethodGet, "/v1/maintenance/config-audit", ""},
|
||||
}
|
||||
|
||||
for _, tc := range tests {
|
||||
t.Run(tc.method+" "+tc.path, func(t *testing.T) {
|
||||
var req *http.Request
|
||||
if tc.body != "" {
|
||||
req = httptest.NewRequest(tc.method, tc.path, strings.NewReader(tc.body))
|
||||
req.Header.Set("Content-Type", "application/json")
|
||||
} else {
|
||||
req = httptest.NewRequest(tc.method, tc.path, nil)
|
||||
}
|
||||
req.Header.Set("Authorization", "Bearer dev")
|
||||
rec := httptest.NewRecorder()
|
||||
handler.ServeHTTP(rec, req)
|
||||
if rec.Code != http.StatusServiceUnavailable {
|
||||
t.Fatalf("status=%d body=%s", rec.Code, rec.Body.String())
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,224 @@
|
||||
package httpapi
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"io"
|
||||
"net/http"
|
||||
"strings"
|
||||
"sync"
|
||||
"time"
|
||||
|
||||
"evobgp/internal/jobs"
|
||||
"evobgp/internal/pgmonitor"
|
||||
)
|
||||
|
||||
var (
|
||||
pgMaintRateMu sync.Mutex
|
||||
pgMaintLastByTK = map[string]time.Time{}
|
||||
)
|
||||
|
||||
func (s *Server) registerPostgresMaintenanceRoutes(m *http.ServeMux) {
|
||||
m.HandleFunc("POST /postgres/vacuum", s.handlePostgresVacuum)
|
||||
m.HandleFunc("POST /postgres/vacuum-analyze", s.handlePostgresVacuumAnalyze)
|
||||
m.HandleFunc("POST /postgres/analyze", s.handlePostgresAnalyze)
|
||||
m.HandleFunc("POST /postgres/reindex", s.handlePostgresReindex)
|
||||
m.HandleFunc("POST /postgres/cleanup", s.handlePostgresCleanup)
|
||||
m.HandleFunc("GET /postgres/maintenance/logs", s.handlePostgresMaintenanceLogs)
|
||||
}
|
||||
|
||||
func (s *Server) requireOperatorStrict(w http.ResponseWriter, a Auth) bool {
|
||||
if strings.ToLower(a.Role) != "operator" {
|
||||
writeProblem(w, http.StatusForbidden, "Forbidden", "operator role required")
|
||||
return false
|
||||
}
|
||||
return true
|
||||
}
|
||||
|
||||
func (s *Server) checkPgMaintRateLimit(tenantID, kind string) bool {
|
||||
key := tenantID + ":" + kind
|
||||
pgMaintRateMu.Lock()
|
||||
defer pgMaintRateMu.Unlock()
|
||||
if t, ok := pgMaintLastByTK[key]; ok && time.Since(t) < 60*time.Second {
|
||||
return false
|
||||
}
|
||||
pgMaintLastByTK[key] = time.Now().UTC()
|
||||
return true
|
||||
}
|
||||
|
||||
type pgMaintBody struct {
|
||||
Table string `json:"table"`
|
||||
DryRun bool `json:"dry_run"`
|
||||
Index string `json:"index"`
|
||||
Policy string `json:"policy"`
|
||||
PolicyID string `json:"policy_id"`
|
||||
Limit int `json:"limit"`
|
||||
}
|
||||
|
||||
func (s *Server) decodePgMaintBody(r *http.Request) (pgMaintBody, bool) {
|
||||
var body pgMaintBody
|
||||
if r.Body == nil || r.ContentLength == 0 {
|
||||
return body, true
|
||||
}
|
||||
if err := json.NewDecoder(io.LimitReader(r.Body, 1<<20)).Decode(&body); err != nil && err != io.EOF {
|
||||
return body, false
|
||||
}
|
||||
return body, true
|
||||
}
|
||||
|
||||
func (s *Server) enqueuePostgresMaint(w http.ResponseWriter, r *http.Request, a Auth, kind string, meta map[string]any) {
|
||||
if !s.requirePostgres(w) || !s.requireOperatorStrict(w, a) {
|
||||
return
|
||||
}
|
||||
if !s.checkPgMaintRateLimit(a.TenantID, kind) {
|
||||
writeProblem(w, http.StatusTooManyRequests, "Too Many Requests", "wait before repeating this maintenance operation")
|
||||
return
|
||||
}
|
||||
idem := strings.TrimSpace(r.Header.Get("Idempotency-Key"))
|
||||
var idemPtr *string
|
||||
if idem != "" {
|
||||
idemPtr = &idem
|
||||
}
|
||||
meta["actor_prefix"] = actorPrefix(a)
|
||||
j, _, err := s.jobs.Enqueue(a.TenantID, kind, idemPtr, nil, meta)
|
||||
if err != nil {
|
||||
writeInternalError(w, "postgres_maint_enqueue", err)
|
||||
return
|
||||
}
|
||||
w.Header().Set("Location", "/v1/jobs/"+j.ID)
|
||||
snap := j.Snapshot()
|
||||
writeJSON(w, http.StatusAccepted, map[string]any{"job_id": snap["job_id"], "status": snap["status"]})
|
||||
}
|
||||
|
||||
func (s *Server) handlePostgresVacuum(w http.ResponseWriter, r *http.Request) {
|
||||
a, ok := authFromContext(r.Context())
|
||||
if !ok {
|
||||
writeProblem(w, http.StatusUnauthorized, "Unauthorized", "missing auth")
|
||||
return
|
||||
}
|
||||
body, ok2 := s.decodePgMaintBody(r)
|
||||
if !ok2 {
|
||||
writeProblem(w, http.StatusBadRequest, "Bad Request", invalidInputDetail)
|
||||
return
|
||||
}
|
||||
s.enqueuePostgresMaint(w, r, a, jobs.KindPostgresVacuum, map[string]any{
|
||||
"table": body.Table, "dry_run": body.DryRun, "job_title": "PostgreSQL VACUUM",
|
||||
})
|
||||
}
|
||||
|
||||
func (s *Server) handlePostgresVacuumAnalyze(w http.ResponseWriter, r *http.Request) {
|
||||
a, ok := authFromContext(r.Context())
|
||||
if !ok {
|
||||
writeProblem(w, http.StatusUnauthorized, "Unauthorized", "missing auth")
|
||||
return
|
||||
}
|
||||
body, ok2 := s.decodePgMaintBody(r)
|
||||
if !ok2 {
|
||||
writeProblem(w, http.StatusBadRequest, "Bad Request", invalidInputDetail)
|
||||
return
|
||||
}
|
||||
s.enqueuePostgresMaint(w, r, a, jobs.KindPostgresVacuumAnalyze, map[string]any{
|
||||
"table": body.Table, "dry_run": body.DryRun, "job_title": "PostgreSQL VACUUM ANALYZE",
|
||||
})
|
||||
}
|
||||
|
||||
func (s *Server) handlePostgresAnalyze(w http.ResponseWriter, r *http.Request) {
|
||||
a, ok := authFromContext(r.Context())
|
||||
if !ok {
|
||||
writeProblem(w, http.StatusUnauthorized, "Unauthorized", "missing auth")
|
||||
return
|
||||
}
|
||||
body, ok2 := s.decodePgMaintBody(r)
|
||||
if !ok2 {
|
||||
writeProblem(w, http.StatusBadRequest, "Bad Request", invalidInputDetail)
|
||||
return
|
||||
}
|
||||
s.enqueuePostgresMaint(w, r, a, jobs.KindPostgresAnalyze, map[string]any{
|
||||
"table": body.Table, "dry_run": body.DryRun, "job_title": "PostgreSQL ANALYZE",
|
||||
})
|
||||
}
|
||||
|
||||
func (s *Server) handlePostgresReindex(w http.ResponseWriter, r *http.Request) {
|
||||
a, ok := authFromContext(r.Context())
|
||||
if !ok {
|
||||
writeProblem(w, http.StatusUnauthorized, "Unauthorized", "missing auth")
|
||||
return
|
||||
}
|
||||
body, ok2 := s.decodePgMaintBody(r)
|
||||
if !ok2 {
|
||||
writeProblem(w, http.StatusBadRequest, "Bad Request", invalidInputDetail)
|
||||
return
|
||||
}
|
||||
table := body.Table
|
||||
if table == "" {
|
||||
table = body.Index
|
||||
}
|
||||
s.enqueuePostgresMaint(w, r, a, jobs.KindPostgresReindex, map[string]any{
|
||||
"table": table, "dry_run": body.DryRun, "job_title": "PostgreSQL REINDEX",
|
||||
})
|
||||
}
|
||||
|
||||
func (s *Server) handlePostgresCleanup(w http.ResponseWriter, r *http.Request) {
|
||||
a, ok := authFromContext(r.Context())
|
||||
if !ok {
|
||||
writeProblem(w, http.StatusUnauthorized, "Unauthorized", "missing auth")
|
||||
return
|
||||
}
|
||||
body, ok2 := s.decodePgMaintBody(r)
|
||||
if !ok2 {
|
||||
writeProblem(w, http.StatusBadRequest, "Bad Request", invalidInputDetail)
|
||||
return
|
||||
}
|
||||
policyID := strings.TrimSpace(body.PolicyID)
|
||||
if policyID == "" {
|
||||
policyID = strings.TrimSpace(body.Policy)
|
||||
}
|
||||
if policyID == "" {
|
||||
writeProblem(w, http.StatusBadRequest, "Bad Request", "policy_id is required")
|
||||
return
|
||||
}
|
||||
if _, err := s.store.GetMaintenancePolicy(policyID); err != nil {
|
||||
writeStoreErr(w, err)
|
||||
return
|
||||
}
|
||||
idem := strings.TrimSpace(r.Header.Get("Idempotency-Key"))
|
||||
var idemPtr *string
|
||||
if idem != "" {
|
||||
idemPtr = &idem
|
||||
}
|
||||
j, _, err := s.jobs.Enqueue(a.TenantID, jobs.KindMaintenancePolicyRun, idemPtr, nil, map[string]any{
|
||||
"policy_id": policyID, "dry_run": body.DryRun, "actor_prefix": actorPrefix(a),
|
||||
"job_title": "PostgreSQL cleanup (deprecated path)",
|
||||
})
|
||||
if err != nil {
|
||||
writeInternalError(w, "postgres_maint_enqueue", err)
|
||||
return
|
||||
}
|
||||
w.Header().Set("Location", "/v1/jobs/"+j.ID)
|
||||
snap := j.Snapshot()
|
||||
writeJSON(w, http.StatusAccepted, map[string]any{"job_id": snap["job_id"], "status": snap["status"]})
|
||||
}
|
||||
|
||||
func (s *Server) handlePostgresMaintenanceLogs(w http.ResponseWriter, r *http.Request) {
|
||||
a, ok := authFromContext(r.Context())
|
||||
if !ok || !s.requireAtLeast(w, a, "viewer") || !s.requirePostgres(w) {
|
||||
return
|
||||
}
|
||||
cursor := r.URL.Query().Get("cursor")
|
||||
limit := parseLimitQuery(r, 20, 100)
|
||||
ctx, cancel := context.WithTimeout(r.Context(), 5*time.Second)
|
||||
defer cancel()
|
||||
items, next, hasMore, err := pgmonitor.ListMaintenanceLogs(ctx, s.pgMonitor.Pool(), cursor, limit)
|
||||
if err != nil {
|
||||
writeInternalError(w, "postgres_maint_logs", err)
|
||||
return
|
||||
}
|
||||
writeJSON(w, http.StatusOK, map[string]any{"items": items, "next_cursor": next, "has_more": hasMore})
|
||||
}
|
||||
|
||||
func actorPrefix(a Auth) string {
|
||||
if len(a.Token) >= 8 {
|
||||
return a.Token[:8]
|
||||
}
|
||||
return a.Role
|
||||
}
|
||||
@@ -0,0 +1,130 @@
|
||||
package httpapi
|
||||
|
||||
import (
|
||||
"context"
|
||||
"net/http"
|
||||
"strconv"
|
||||
"time"
|
||||
)
|
||||
|
||||
func (s *Server) registerPostgresMonitoringRoutes(m *http.ServeMux) {
|
||||
m.HandleFunc("GET /monitoring/postgres/overview", s.handlePostgresOverview)
|
||||
m.HandleFunc("GET /monitoring/postgres/queries", s.handlePostgresQueries)
|
||||
m.HandleFunc("GET /monitoring/postgres/locks", s.handlePostgresLocks)
|
||||
m.HandleFunc("GET /monitoring/postgres/tables", s.handlePostgresTables)
|
||||
m.HandleFunc("GET /monitoring/postgres/recommendations", s.handlePostgresRecommendations)
|
||||
m.HandleFunc("GET /monitoring/correlation", s.handleMonitoringCorrelation)
|
||||
}
|
||||
|
||||
func (s *Server) requirePostgres(w http.ResponseWriter) bool {
|
||||
if s.pgMonitor == nil {
|
||||
writeProblem(w, http.StatusServiceUnavailable, "Unavailable", "postgresql backend required")
|
||||
return false
|
||||
}
|
||||
return true
|
||||
}
|
||||
|
||||
func parseLimitQuery(r *http.Request, def, max int) int {
|
||||
if v := r.URL.Query().Get("limit"); v != "" {
|
||||
if n, err := strconv.Atoi(v); err == nil {
|
||||
return n
|
||||
}
|
||||
}
|
||||
return def
|
||||
}
|
||||
|
||||
func (s *Server) handlePostgresOverview(w http.ResponseWriter, r *http.Request) {
|
||||
a, ok := authFromContext(r.Context())
|
||||
if !ok || !s.requireAtLeast(w, a, "viewer") || !s.requirePostgres(w) {
|
||||
return
|
||||
}
|
||||
ctx, cancel := context.WithTimeout(r.Context(), 3*time.Second)
|
||||
defer cancel()
|
||||
out, err := s.pgMonitor.Overview(ctx)
|
||||
if err != nil {
|
||||
writeInternalError(w, "postgres_overview", err)
|
||||
return
|
||||
}
|
||||
writeJSON(w, http.StatusOK, out)
|
||||
}
|
||||
|
||||
func (s *Server) handlePostgresQueries(w http.ResponseWriter, r *http.Request) {
|
||||
a, ok := authFromContext(r.Context())
|
||||
if !ok || !s.requireAtLeast(w, a, "viewer") || !s.requirePostgres(w) {
|
||||
return
|
||||
}
|
||||
ctx, cancel := context.WithTimeout(r.Context(), 3*time.Second)
|
||||
defer cancel()
|
||||
out, err := s.pgMonitor.TopQueries(ctx, parseLimitQuery(r, 20, 100))
|
||||
if err != nil {
|
||||
writeInternalError(w, "postgres_queries", err)
|
||||
return
|
||||
}
|
||||
writeJSON(w, http.StatusOK, out)
|
||||
}
|
||||
|
||||
func (s *Server) handlePostgresLocks(w http.ResponseWriter, r *http.Request) {
|
||||
a, ok := authFromContext(r.Context())
|
||||
if !ok || !s.requireAtLeast(w, a, "viewer") || !s.requirePostgres(w) {
|
||||
return
|
||||
}
|
||||
ctx, cancel := context.WithTimeout(r.Context(), 3*time.Second)
|
||||
defer cancel()
|
||||
out, err := s.pgMonitor.Locks(ctx)
|
||||
if err != nil {
|
||||
writeInternalError(w, "postgres_locks", err)
|
||||
return
|
||||
}
|
||||
writeJSON(w, http.StatusOK, map[string]any{"items": out})
|
||||
}
|
||||
|
||||
func (s *Server) handlePostgresTables(w http.ResponseWriter, r *http.Request) {
|
||||
a, ok := authFromContext(r.Context())
|
||||
if !ok || !s.requireAtLeast(w, a, "viewer") || !s.requirePostgres(w) {
|
||||
return
|
||||
}
|
||||
ctx, cancel := context.WithTimeout(r.Context(), 3*time.Second)
|
||||
defer cancel()
|
||||
out, err := s.pgMonitor.Tables(ctx, parseLimitQuery(r, 20, 100))
|
||||
if err != nil {
|
||||
writeInternalError(w, "postgres_tables", err)
|
||||
return
|
||||
}
|
||||
writeJSON(w, http.StatusOK, map[string]any{"items": out})
|
||||
}
|
||||
|
||||
func (s *Server) handlePostgresRecommendations(w http.ResponseWriter, r *http.Request) {
|
||||
a, ok := authFromContext(r.Context())
|
||||
if !ok || !s.requireAtLeast(w, a, "viewer") || !s.requirePostgres(w) {
|
||||
return
|
||||
}
|
||||
ctx, cancel := context.WithTimeout(r.Context(), 5*time.Second)
|
||||
defer cancel()
|
||||
out, err := s.pgMonitor.Recommendations(ctx)
|
||||
if err != nil {
|
||||
writeInternalError(w, "postgres_recommendations", err)
|
||||
return
|
||||
}
|
||||
writeJSON(w, http.StatusOK, out)
|
||||
}
|
||||
|
||||
func (s *Server) handleMonitoringCorrelation(w http.ResponseWriter, r *http.Request) {
|
||||
a, ok := authFromContext(r.Context())
|
||||
if !ok || !s.requireAtLeast(w, a, "viewer") || !s.requirePostgres(w) {
|
||||
return
|
||||
}
|
||||
window := 60
|
||||
if v := r.URL.Query().Get("window"); v != "" {
|
||||
if n, err := strconv.Atoi(v); err == nil {
|
||||
window = n
|
||||
}
|
||||
}
|
||||
ctx, cancel := context.WithTimeout(r.Context(), 5*time.Second)
|
||||
defer cancel()
|
||||
out, err := s.pgMonitor.Correlation(ctx, window)
|
||||
if err != nil {
|
||||
writeInternalError(w, "monitoring_correlation", err)
|
||||
return
|
||||
}
|
||||
writeJSON(w, http.StatusOK, out)
|
||||
}
|
||||
@@ -0,0 +1,21 @@
|
||||
package httpapi
|
||||
|
||||
import (
|
||||
"net/http"
|
||||
"net/http/httptest"
|
||||
"testing"
|
||||
)
|
||||
|
||||
func TestPostgresOverviewMemoryBackend503(t *testing.T) {
|
||||
srv, err := New(Options{SeedDemo: true})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
req := httptest.NewRequest(http.MethodGet, "/v1/monitoring/postgres/overview", nil)
|
||||
req.Header.Set("Authorization", "Bearer dev")
|
||||
rec := httptest.NewRecorder()
|
||||
srv.Handler().ServeHTTP(rec, req)
|
||||
if rec.Code != http.StatusServiceUnavailable {
|
||||
t.Fatalf("status=%d body=%s", rec.Code, rec.Body.String())
|
||||
}
|
||||
}
|
||||
@@ -8,8 +8,11 @@ import (
|
||||
"errors"
|
||||
"net/http"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"evobgp/internal/jobs"
|
||||
"evobgp/internal/maintenance"
|
||||
"evobgp/internal/pgmonitor"
|
||||
"evobgp/internal/store"
|
||||
|
||||
"github.com/jackc/pgx/v5/pgxpool"
|
||||
@@ -19,6 +22,9 @@ import (
|
||||
type Server struct {
|
||||
store store.Backend
|
||||
pgPool *pgxpool.Pool
|
||||
pgMonitor *pgmonitor.Service
|
||||
maintConfig *maintenance.ConfigProvider
|
||||
maintStats *maintenance.DBStatsProvider
|
||||
jobs *jobs.Registry
|
||||
bundlePriv ed25519.PrivateKey
|
||||
keyResolver *apiKeyResolver
|
||||
@@ -63,9 +69,21 @@ func New(opts Options) (*Server, error) {
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
var pgMon *pgmonitor.Service
|
||||
var maintCfg *maintenance.ConfigProvider
|
||||
var maintStats *maintenance.DBStatsProvider
|
||||
if pool != nil {
|
||||
pgMon = pgmonitor.NewService(pool)
|
||||
maintCfg = maintenance.NewConfigProvider(backend)
|
||||
_ = maintCfg.Reload(context.Background())
|
||||
maintStats = maintenance.NewDBStatsProvider(pgMon)
|
||||
}
|
||||
s := &Server{
|
||||
store: backend,
|
||||
pgPool: pool,
|
||||
pgMonitor: pgMon,
|
||||
maintConfig: maintCfg,
|
||||
maintStats: maintStats,
|
||||
jobs: reg,
|
||||
bundlePriv: priv,
|
||||
keyResolver: resolver,
|
||||
@@ -89,3 +107,20 @@ func (s *Server) Store() store.Backend { return s.store }
|
||||
|
||||
// Jobs exposes the in-process async job registry (for scheduler / evobgp-all).
|
||||
func (s *Server) Jobs() *jobs.Registry { return s.jobs }
|
||||
|
||||
// StartBackground starts PostgreSQL monitoring and maintenance schedulers until ctx is cancelled.
|
||||
func (s *Server) StartBackground(ctx context.Context) {
|
||||
if s != nil && s.pgPool != nil {
|
||||
pgmonitor.StartScheduler(ctx, s.pgPool)
|
||||
}
|
||||
if s != nil && s.maintConfig != nil && s.jobs != nil {
|
||||
maintenance.StartScheduler(ctx, s.maintConfig, func(policyID string, dryRun bool, idem string) {
|
||||
key := idem
|
||||
_, _, _ = s.jobs.Enqueue("", jobs.KindMaintenancePolicyRun, &key, nil, map[string]any{
|
||||
"policy_id": policyID,
|
||||
"dry_run": dryRun,
|
||||
"trigger": "scheduler",
|
||||
})
|
||||
}, 30*time.Second)
|
||||
}
|
||||
}
|
||||
|
||||
@@ -17,8 +17,6 @@ type Deps struct {
|
||||
Store store.Backend
|
||||
}
|
||||
|
||||
var lastMaintenance time.Time
|
||||
|
||||
// Run blocks until ctx is cancelled.
|
||||
func Run(ctx context.Context, deps *Deps) {
|
||||
cfg := config.Load()
|
||||
@@ -36,10 +34,6 @@ func Run(ctx context.Context, deps *Deps) {
|
||||
log.Printf("evobgp-ingest: stopped")
|
||||
return
|
||||
case <-t.C:
|
||||
if deps.Store != nil && time.Since(lastMaintenance) > time.Hour {
|
||||
deps.Store.RunPeriodicMaintenance(ctx)
|
||||
lastMaintenance = time.Now()
|
||||
}
|
||||
prefetchCtx, cancel := context.WithTimeout(ctx, 50*time.Second)
|
||||
err := pipeline.PrefetchCDNSourceETags(prefetchCtx, deps.Store, hc)
|
||||
cancel()
|
||||
|
||||
@@ -0,0 +1,63 @@
|
||||
package jobs
|
||||
|
||||
import (
|
||||
"errors"
|
||||
"strings"
|
||||
|
||||
"evobgp/internal/maintenance"
|
||||
"evobgp/internal/pgmonitor"
|
||||
"evobgp/internal/store"
|
||||
)
|
||||
|
||||
func (w *Worker) maintenanceExecutor() *maintenance.PolicyExecutor {
|
||||
if w == nil {
|
||||
return nil
|
||||
}
|
||||
return &maintenance.PolicyExecutor{Store: w.Store, Pool: w.PgPool}
|
||||
}
|
||||
|
||||
func (w *Worker) runMaintenancePolicy(j *Job) {
|
||||
if w == nil || w.PgPool == nil {
|
||||
j.Fail("postgresql not configured")
|
||||
return
|
||||
}
|
||||
policyID, _ := j.Meta["policy_id"].(string)
|
||||
policyID = strings.TrimSpace(policyID)
|
||||
if policyID == "" {
|
||||
j.Fail("missing policy_id in job meta")
|
||||
return
|
||||
}
|
||||
dryRun, _ := j.Meta["dry_run"].(bool)
|
||||
actor, _ := j.Meta["actor_prefix"].(string)
|
||||
ctx, cancel := j.workContext()
|
||||
defer cancel()
|
||||
|
||||
pol, err := w.Store.GetMaintenancePolicy(policyID)
|
||||
if err != nil {
|
||||
if errors.Is(err, store.ErrNotFound) {
|
||||
j.Fail("maintenance policy not found")
|
||||
return
|
||||
}
|
||||
j.Fail(err.Error())
|
||||
return
|
||||
}
|
||||
|
||||
auditID, _ := pgmonitor.InsertMaintenanceAuditWithPolicy(ctx, w.PgPool, j.TenantID, actor, "maintenance_policy_run", pol.TableName, policyID, dryRun)
|
||||
exec := w.maintenanceExecutor()
|
||||
detail, err := exec.Execute(ctx, pol, dryRun)
|
||||
var errMsg *string
|
||||
status := StatusSucceeded
|
||||
if err != nil {
|
||||
s := err.Error()
|
||||
errMsg = &s
|
||||
status = StatusFailed
|
||||
_ = w.Store.TouchMaintenancePolicyRun(policyID, status, s)
|
||||
j.Fail(s)
|
||||
} else {
|
||||
j.mergeMeta(map[string]any{"maintenance": detail, "audit_id": auditID, "policy_id": policyID})
|
||||
j.Succeed()
|
||||
}
|
||||
if auditID != "" {
|
||||
_ = pgmonitor.FinishMaintenanceAudit(ctx, w.PgPool, auditID, status, detail, errMsg)
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,140 @@
|
||||
package jobs
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
|
||||
"evobgp/internal/pgmonitor"
|
||||
)
|
||||
|
||||
func (w *Worker) pgService() *pgmonitor.Service {
|
||||
if w == nil || w.PgPool == nil {
|
||||
return nil
|
||||
}
|
||||
return pgmonitor.NewService(w.PgPool)
|
||||
}
|
||||
|
||||
func (w *Worker) runPostgresMetricsRefresh(j *Job) {
|
||||
s := w.pgService()
|
||||
if s == nil {
|
||||
j.Fail("postgresql not configured")
|
||||
return
|
||||
}
|
||||
ctx, cancel := j.workContext()
|
||||
defer cancel()
|
||||
if err := s.RefreshMetricsSnapshot(ctx); err != nil {
|
||||
j.Fail(err.Error())
|
||||
return
|
||||
}
|
||||
j.Succeed()
|
||||
}
|
||||
|
||||
func (w *Worker) runPostgresSlowQueryAgg(j *Job) {
|
||||
s := w.pgService()
|
||||
if s == nil {
|
||||
j.Fail("postgresql not configured")
|
||||
return
|
||||
}
|
||||
ctx, cancel := j.workContext()
|
||||
defer cancel()
|
||||
if err := s.AggregateSlowQueries(ctx, 30); err != nil {
|
||||
j.Fail(err.Error())
|
||||
return
|
||||
}
|
||||
j.Succeed()
|
||||
}
|
||||
|
||||
func (w *Worker) runPostgresTableBloat(j *Job) {
|
||||
s := w.pgService()
|
||||
if s == nil {
|
||||
j.Fail("postgresql not configured")
|
||||
return
|
||||
}
|
||||
ctx, cancel := j.workContext()
|
||||
defer cancel()
|
||||
if err := s.EstimateTableBloat(ctx); err != nil {
|
||||
j.Fail(err.Error())
|
||||
return
|
||||
}
|
||||
j.Succeed()
|
||||
}
|
||||
|
||||
func (w *Worker) runPostgresIndexUsage(j *Job) {
|
||||
s := w.pgService()
|
||||
if s == nil {
|
||||
j.Fail("postgresql not configured")
|
||||
return
|
||||
}
|
||||
ctx, cancel := j.workContext()
|
||||
defer cancel()
|
||||
if err := s.AnalyzeIndexUsage(ctx); err != nil {
|
||||
j.Fail(err.Error())
|
||||
return
|
||||
}
|
||||
j.Succeed()
|
||||
}
|
||||
|
||||
func (w *Worker) runPostgresAutovacuumLag(j *Job) {
|
||||
s := w.pgService()
|
||||
if s == nil {
|
||||
j.Fail("postgresql not configured")
|
||||
return
|
||||
}
|
||||
ctx, cancel := j.workContext()
|
||||
defer cancel()
|
||||
if err := s.DetectAutovacuumLag(ctx); err != nil {
|
||||
j.Fail(err.Error())
|
||||
return
|
||||
}
|
||||
j.Succeed()
|
||||
}
|
||||
|
||||
func (w *Worker) runPostgresMaint(j *Job, kind string) {
|
||||
if w == nil || w.PgPool == nil {
|
||||
j.Fail("postgresql not configured")
|
||||
return
|
||||
}
|
||||
table, _ := j.Meta["table"].(string)
|
||||
dryRun, _ := j.Meta["dry_run"].(bool)
|
||||
actor, _ := j.Meta["actor_prefix"].(string)
|
||||
ctx, cancel := j.workContext()
|
||||
defer cancel()
|
||||
auditID, _ := pgmonitor.InsertMaintenanceAudit(ctx, w.PgPool, j.TenantID, actor, kind, table, dryRun)
|
||||
detail, err := pgmonitor.ExecMaintenance(ctx, w.PgPool, kind, table, dryRun)
|
||||
var errMsg *string
|
||||
status := StatusSucceeded
|
||||
if err != nil {
|
||||
s := err.Error()
|
||||
errMsg = &s
|
||||
status = StatusFailed
|
||||
j.Fail(s)
|
||||
} else {
|
||||
j.mergeMeta(map[string]any{"maintenance": detail, "audit_id": auditID})
|
||||
j.Succeed()
|
||||
}
|
||||
if auditID != "" {
|
||||
_ = pgmonitor.FinishMaintenanceAudit(ctx, w.PgPool, auditID, status, detail, errMsg)
|
||||
}
|
||||
}
|
||||
|
||||
func (w *Worker) runPostgresCleanup(j *Job) {
|
||||
j.Fail("postgres_cleanup deprecated: configure maintenance_policy in UI and use maintenance_policy_run")
|
||||
}
|
||||
|
||||
// EnqueuePostgresAnalyzerJobs enqueues periodic analyzer jobs (global tenant id).
|
||||
func EnqueuePostgresAnalyzerJobs(reg *Registry, tenantID string) {
|
||||
if reg == nil || tenantID == "" {
|
||||
return
|
||||
}
|
||||
kinds := []string{
|
||||
KindPostgresMetricsRefresh,
|
||||
KindPostgresSlowQueryAgg,
|
||||
KindPostgresTableBloat,
|
||||
KindPostgresIndexUsage,
|
||||
KindPostgresAutovacuumLag,
|
||||
}
|
||||
for _, k := range kinds {
|
||||
key := fmt.Sprintf("pgmon-%s-%s", k, tenantID)
|
||||
idem := key
|
||||
_, _, _ = reg.Enqueue(tenantID, k, &idem, nil, map[string]any{"trigger": "scheduler"})
|
||||
}
|
||||
}
|
||||
+42
-6
@@ -18,6 +18,8 @@ import (
|
||||
"evobgp/internal/observability"
|
||||
"evobgp/internal/pipeline"
|
||||
"evobgp/internal/store"
|
||||
|
||||
"github.com/jackc/pgx/v5/pgxpool"
|
||||
)
|
||||
|
||||
// mergeBirdPostApplyMeta attaches a birdc snapshot after deploy/reload (best-effort).
|
||||
@@ -43,17 +45,29 @@ func mergeBirdPostApplyMeta(j *Job) {
|
||||
}
|
||||
|
||||
const (
|
||||
KindModuleRefresh = "module_refresh"
|
||||
KindTenantRefresh = "tenant_refresh"
|
||||
KindPeerReconcile = "peer_reconcile"
|
||||
KindDeployApply = "deploy_apply"
|
||||
KindRevisionRollback = "revision_rollback"
|
||||
KindBirdReload = "bird_reload"
|
||||
KindModuleRefresh = "module_refresh"
|
||||
KindTenantRefresh = "tenant_refresh"
|
||||
KindPeerReconcile = "peer_reconcile"
|
||||
KindDeployApply = "deploy_apply"
|
||||
KindRevisionRollback = "revision_rollback"
|
||||
KindBirdReload = "bird_reload"
|
||||
KindPostgresMetricsRefresh = "postgres_metrics_refresh"
|
||||
KindPostgresSlowQueryAgg = "postgres_slow_query_aggregate"
|
||||
KindPostgresTableBloat = "postgres_table_bloat_estimate"
|
||||
KindPostgresIndexUsage = "postgres_index_usage_analyze"
|
||||
KindPostgresAutovacuumLag = "postgres_autovacuum_lag_detect"
|
||||
KindPostgresVacuum = "postgres_vacuum"
|
||||
KindPostgresVacuumAnalyze = "postgres_vacuum_analyze"
|
||||
KindPostgresAnalyze = "postgres_analyze"
|
||||
KindPostgresReindex = "postgres_reindex"
|
||||
KindPostgresCleanup = "postgres_cleanup"
|
||||
KindMaintenancePolicyRun = "maintenance_policy_run"
|
||||
)
|
||||
|
||||
// Worker executes queued jobs against store.Backend (memory or SQL).
|
||||
type Worker struct {
|
||||
Store store.Backend
|
||||
PgPool *pgxpool.Pool
|
||||
HTTPClient *http.Client // optional; CDN refresh uses this (default 45s timeout).
|
||||
// Registry is set after BootstrapWorkers creates the job queue; used to chain deploy_apply after refresh/rollback.
|
||||
Registry *Registry
|
||||
@@ -155,6 +169,28 @@ func (w *Worker) Process(j *Job) {
|
||||
}
|
||||
mergeBirdPostApplyMeta(j)
|
||||
j.Succeed()
|
||||
case KindPostgresMetricsRefresh:
|
||||
w.runPostgresMetricsRefresh(j)
|
||||
case KindPostgresSlowQueryAgg:
|
||||
w.runPostgresSlowQueryAgg(j)
|
||||
case KindPostgresTableBloat:
|
||||
w.runPostgresTableBloat(j)
|
||||
case KindPostgresIndexUsage:
|
||||
w.runPostgresIndexUsage(j)
|
||||
case KindPostgresAutovacuumLag:
|
||||
w.runPostgresAutovacuumLag(j)
|
||||
case KindPostgresVacuum:
|
||||
w.runPostgresMaint(j, "vacuum")
|
||||
case KindPostgresVacuumAnalyze:
|
||||
w.runPostgresMaint(j, "vacuum_analyze")
|
||||
case KindPostgresAnalyze:
|
||||
w.runPostgresMaint(j, "analyze")
|
||||
case KindPostgresReindex:
|
||||
w.runPostgresMaint(j, "reindex")
|
||||
case KindPostgresCleanup:
|
||||
w.runPostgresCleanup(j)
|
||||
case KindMaintenancePolicyRun:
|
||||
w.runMaintenancePolicy(j)
|
||||
default:
|
||||
j.Fail("unknown job kind")
|
||||
}
|
||||
|
||||
@@ -0,0 +1,67 @@
|
||||
package maintenance
|
||||
|
||||
import (
|
||||
"context"
|
||||
"sync"
|
||||
|
||||
"evobgp/internal/store"
|
||||
)
|
||||
|
||||
// ConfigProvider caches maintenance policies from store.Backend with hot reload.
|
||||
type ConfigProvider struct {
|
||||
store store.Backend
|
||||
mu sync.RWMutex
|
||||
items []*store.MaintenancePolicy
|
||||
}
|
||||
|
||||
// NewConfigProvider constructs a provider; call Reload before use.
|
||||
func NewConfigProvider(st store.Backend) *ConfigProvider {
|
||||
return &ConfigProvider{store: st}
|
||||
}
|
||||
|
||||
// Reload loads all policies from the database into memory.
|
||||
func (c *ConfigProvider) Reload(ctx context.Context) error {
|
||||
if c == nil || c.store == nil {
|
||||
return nil
|
||||
}
|
||||
_ = ctx
|
||||
items, _, _, err := c.store.ListMaintenancePolicies("", 1000)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
cp := make([]*store.MaintenancePolicy, len(items))
|
||||
copy(cp, items)
|
||||
c.mu.Lock()
|
||||
c.items = cp
|
||||
c.mu.Unlock()
|
||||
return nil
|
||||
}
|
||||
|
||||
// Snapshot returns a copy of cached policies.
|
||||
func (c *ConfigProvider) Snapshot() []*store.MaintenancePolicy {
|
||||
if c == nil {
|
||||
return nil
|
||||
}
|
||||
c.mu.RLock()
|
||||
defer c.mu.RUnlock()
|
||||
out := make([]*store.MaintenancePolicy, len(c.items))
|
||||
copy(out, c.items)
|
||||
return out
|
||||
}
|
||||
|
||||
// Get returns one policy by id from cache or store.
|
||||
func (c *ConfigProvider) Get(ctx context.Context, id string) (*store.MaintenancePolicy, error) {
|
||||
if c == nil || c.store == nil {
|
||||
return nil, store.ErrNotFound
|
||||
}
|
||||
c.mu.RLock()
|
||||
for _, p := range c.items {
|
||||
if p.ID == id {
|
||||
cp := *p
|
||||
c.mu.RUnlock()
|
||||
return &cp, nil
|
||||
}
|
||||
}
|
||||
c.mu.RUnlock()
|
||||
return c.store.GetMaintenancePolicy(id)
|
||||
}
|
||||
@@ -0,0 +1,43 @@
|
||||
package maintenance
|
||||
|
||||
import (
|
||||
"context"
|
||||
"testing"
|
||||
|
||||
"evobgp/internal/store"
|
||||
)
|
||||
|
||||
func TestConfigProviderReloadAndSnapshot(t *testing.T) {
|
||||
mem := store.NewMemory()
|
||||
ret := 3600
|
||||
if _, err := mem.CreateMaintenancePolicy(&store.MaintenancePolicy{
|
||||
Name: "p1",
|
||||
TableName: "job_audit",
|
||||
Schedule: "0 3 * * *",
|
||||
RetentionPeriodSec: &ret,
|
||||
Enabled: true,
|
||||
}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
cp := NewConfigProvider(mem)
|
||||
if err := cp.Reload(context.Background()); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
snap := cp.Snapshot()
|
||||
if len(snap) != 1 || snap[0].Name != "p1" {
|
||||
t.Fatalf("snapshot: %+v", snap)
|
||||
}
|
||||
|
||||
newName := "p1-updated"
|
||||
if _, err := mem.UpdateMaintenancePolicy(snap[0].ID, &store.MaintenancePolicyPatch{Name: &newName}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := cp.Reload(context.Background()); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
snap2 := cp.Snapshot()
|
||||
if len(snap2) != 1 || snap2[0].Name != newName {
|
||||
t.Fatalf("after reload: %+v", snap2)
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,62 @@
|
||||
package maintenance
|
||||
|
||||
import (
|
||||
"context"
|
||||
"fmt"
|
||||
|
||||
"evobgp/internal/pgmonitor"
|
||||
)
|
||||
|
||||
// TableHints are PostgreSQL statistics hints for UI recommendations.
|
||||
type TableHints struct {
|
||||
TableName string `json:"table_name"`
|
||||
DeadTuples int64 `json:"n_dead_tup"`
|
||||
BloatRatio float64 `json:"bloat_ratio,omitempty"`
|
||||
LastAutovacuum string `json:"last_autovacuum,omitempty"`
|
||||
RecommendVacuum bool `json:"recommend_vacuum"`
|
||||
Detail string `json:"detail,omitempty"`
|
||||
Refs []string `json:"refs,omitempty"`
|
||||
}
|
||||
|
||||
// DBStatsProvider wraps pgmonitor for maintenance policy hints.
|
||||
type DBStatsProvider struct {
|
||||
pg *pgmonitor.Service
|
||||
}
|
||||
|
||||
// NewDBStatsProvider constructs a stats provider.
|
||||
func NewDBStatsProvider(pg *pgmonitor.Service) *DBStatsProvider {
|
||||
return &DBStatsProvider{pg: pg}
|
||||
}
|
||||
|
||||
// Hints returns table-level vacuum/bloat hints.
|
||||
func (d *DBStatsProvider) Hints(ctx context.Context, tableName string) (TableHints, error) {
|
||||
out := TableHints{TableName: tableName}
|
||||
if d == nil || d.pg == nil {
|
||||
return out, fmt.Errorf("maintenance: postgres monitoring not configured")
|
||||
}
|
||||
if err := ValidateTableName(tableName); err != nil {
|
||||
return out, err
|
||||
}
|
||||
tables, err := d.pg.Tables(ctx, 100)
|
||||
if err != nil {
|
||||
return out, err
|
||||
}
|
||||
for _, t := range tables {
|
||||
if t.Relname != tableName {
|
||||
continue
|
||||
}
|
||||
out.DeadTuples = t.DeadTuples
|
||||
out.BloatRatio = t.BloatRatio
|
||||
if t.LastAutovacuum != nil {
|
||||
out.LastAutovacuum = t.LastAutovacuum.UTC().Format("2006-01-02T15:04:05Z")
|
||||
}
|
||||
if t.BloatRatio > 0.2 && t.DeadTuples > 5000 {
|
||||
out.RecommendVacuum = true
|
||||
out.Detail = "Высокая доля n_dead_tup; рекомендуется VACUUM."
|
||||
out.Refs = []string{t.Relname}
|
||||
}
|
||||
return out, nil
|
||||
}
|
||||
out.Detail = "Таблица не найдена в pg_stat_user_tables (top by size)."
|
||||
return out, nil
|
||||
}
|
||||
@@ -0,0 +1,2 @@
|
||||
// Package maintenance implements PostgreSQL maintenance policies loaded from the database.
|
||||
package maintenance
|
||||
@@ -0,0 +1,195 @@
|
||||
package maintenance
|
||||
|
||||
import (
|
||||
"context"
|
||||
"fmt"
|
||||
"hash/fnv"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"evobgp/internal/observability"
|
||||
"evobgp/internal/pgmonitor"
|
||||
"evobgp/internal/store"
|
||||
|
||||
"github.com/jackc/pgx/v5"
|
||||
"github.com/jackc/pgx/v5/pgxpool"
|
||||
)
|
||||
|
||||
// PolicyExecutor runs maintenance policies against PostgreSQL.
|
||||
type PolicyExecutor struct {
|
||||
Store store.Backend
|
||||
Pool *pgxpool.Pool
|
||||
}
|
||||
|
||||
// Execute runs cleanup and/or vacuum steps for a policy.
|
||||
func (e *PolicyExecutor) Execute(ctx context.Context, policy *store.MaintenancePolicy, dryRun bool) (map[string]any, error) {
|
||||
start := time.Now()
|
||||
if policy == nil {
|
||||
return nil, store.ErrInvalidInput
|
||||
}
|
||||
action := policyAction(policy)
|
||||
record := func(status string, detail map[string]any) {
|
||||
observability.RecordMaintenancePolicyRun(policy.ID, action, status, dryRun, time.Since(start), rowsDeletedFromDetail(detail))
|
||||
}
|
||||
|
||||
if e == nil || e.Pool == nil {
|
||||
record("failed", nil)
|
||||
return nil, fmt.Errorf("maintenance: postgres not configured")
|
||||
}
|
||||
if err := ValidateTableName(policy.TableName); err != nil {
|
||||
record("failed", nil)
|
||||
return nil, err
|
||||
}
|
||||
if err := ValidateCondition(policy.Condition); err != nil {
|
||||
record("failed", nil)
|
||||
return nil, err
|
||||
}
|
||||
if !store.ValidVacuumStrategy(policy.VacuumStrategy) {
|
||||
record("failed", nil)
|
||||
return nil, store.ErrInvalidInput
|
||||
}
|
||||
|
||||
detail := map[string]any{
|
||||
"policy_id": policy.ID,
|
||||
"table": policy.TableName,
|
||||
"dry_run": dryRun,
|
||||
}
|
||||
|
||||
if policy.RetentionPeriodSec != nil || policy.MaxRows != nil {
|
||||
cleanupDetail, err := e.runCleanup(ctx, policy, dryRun)
|
||||
for k, v := range cleanupDetail {
|
||||
detail[k] = v
|
||||
}
|
||||
if err != nil {
|
||||
record("failed", detail)
|
||||
return detail, err
|
||||
}
|
||||
}
|
||||
|
||||
if policy.VacuumStrategy != store.VacuumStrategyNone {
|
||||
kind := vacuumKind(policy.VacuumStrategy)
|
||||
vacDetail, err := pgmonitor.ExecMaintenance(ctx, e.Pool, kind, policy.TableName, dryRun)
|
||||
if vacDetail != nil {
|
||||
detail["vacuum"] = vacDetail
|
||||
}
|
||||
if err != nil {
|
||||
record("failed", detail)
|
||||
return detail, err
|
||||
}
|
||||
}
|
||||
|
||||
if !dryRun && e.Store != nil {
|
||||
_ = e.Store.TouchMaintenancePolicyRun(policy.ID, "succeeded", "")
|
||||
}
|
||||
record("succeeded", detail)
|
||||
return detail, nil
|
||||
}
|
||||
|
||||
func policyAction(p *store.MaintenancePolicy) string {
|
||||
if p == nil {
|
||||
return "run"
|
||||
}
|
||||
if p.RetentionPeriodSec != nil || p.MaxRows != nil {
|
||||
if p.VacuumStrategy != store.VacuumStrategyNone {
|
||||
return "cleanup_vacuum"
|
||||
}
|
||||
return "cleanup"
|
||||
}
|
||||
if p.VacuumStrategy != store.VacuumStrategyNone {
|
||||
return p.VacuumStrategy
|
||||
}
|
||||
return "run"
|
||||
}
|
||||
|
||||
func rowsDeletedFromDetail(detail map[string]any) int64 {
|
||||
if detail == nil {
|
||||
return 0
|
||||
}
|
||||
switch v := detail["deleted"].(type) {
|
||||
case int64:
|
||||
return v
|
||||
case int:
|
||||
return int64(v)
|
||||
case float64:
|
||||
return int64(v)
|
||||
default:
|
||||
return 0
|
||||
}
|
||||
}
|
||||
|
||||
func (e *PolicyExecutor) runCleanup(ctx context.Context, policy *store.MaintenancePolicy, dryRun bool) (map[string]any, error) {
|
||||
detail := map[string]any{"cleanup": true}
|
||||
limit := NormalizeBatchLimit(policy.MaxRows)
|
||||
qualTable := pgx.Identifier{policy.TableName}.Sanitize()
|
||||
cond := store.NormalizeMaintenancePolicyCondition(policy.Condition)
|
||||
|
||||
tx, err := e.Pool.Begin(ctx)
|
||||
if err != nil {
|
||||
return detail, fmt.Errorf("maintenance: begin tx: %w", err)
|
||||
}
|
||||
defer func() { _ = tx.Rollback(ctx) }()
|
||||
|
||||
lockKey := advisoryKey(policy.ID)
|
||||
if _, err := tx.Exec(ctx, `SELECT pg_advisory_xact_lock($1)`, lockKey); err != nil {
|
||||
return detail, fmt.Errorf("maintenance: advisory lock: %w", err)
|
||||
}
|
||||
if _, err := tx.Exec(ctx, fmt.Sprintf(`SET LOCAL statement_timeout = '%ds'`, DefaultStatementTimeoutSec)); err != nil {
|
||||
return detail, fmt.Errorf("maintenance: statement_timeout: %w", err)
|
||||
}
|
||||
|
||||
var args []any
|
||||
where := cond
|
||||
argN := 1
|
||||
if policy.RetentionPeriodSec != nil && *policy.RetentionPeriodSec > 0 {
|
||||
cutoff := time.Now().UTC().Add(-time.Duration(*policy.RetentionPeriodSec) * time.Second)
|
||||
where = fmt.Sprintf("(%s) AND created_at < $%d", cond, argN)
|
||||
args = append(args, cutoff)
|
||||
argN++
|
||||
}
|
||||
|
||||
countSQL := fmt.Sprintf(`SELECT count(*) FROM %s WHERE %s`, qualTable, where)
|
||||
var wouldDelete int64
|
||||
if err := tx.QueryRow(ctx, countSQL, args...).Scan(&wouldDelete); err != nil {
|
||||
return detail, fmt.Errorf("maintenance: count: %w", err)
|
||||
}
|
||||
detail["would_delete"] = wouldDelete
|
||||
if dryRun {
|
||||
return detail, nil
|
||||
}
|
||||
|
||||
deleteSQL := fmt.Sprintf(`
|
||||
DELETE FROM %s WHERE ctid IN (
|
||||
SELECT ctid FROM %s WHERE %s LIMIT $%d
|
||||
)`, qualTable, qualTable, where, argN)
|
||||
args = append(args, limit)
|
||||
tag, err := tx.Exec(ctx, deleteSQL, args...)
|
||||
if err != nil {
|
||||
return detail, fmt.Errorf("maintenance: delete: %w", err)
|
||||
}
|
||||
if err := tx.Commit(ctx); err != nil {
|
||||
return detail, fmt.Errorf("maintenance: commit: %w", err)
|
||||
}
|
||||
detail["deleted"] = tag.RowsAffected()
|
||||
return detail, nil
|
||||
}
|
||||
|
||||
func vacuumKind(strategy string) string {
|
||||
switch strings.TrimSpace(strategy) {
|
||||
case store.VacuumStrategyVacuum:
|
||||
return "vacuum"
|
||||
case store.VacuumStrategyAnalyze:
|
||||
return "analyze"
|
||||
case store.VacuumStrategyVacuumAnalyze:
|
||||
return "vacuum_analyze"
|
||||
case store.VacuumStrategyReindex:
|
||||
return "reindex"
|
||||
default:
|
||||
return "vacuum"
|
||||
}
|
||||
}
|
||||
|
||||
func advisoryKey(policyID string) int64 {
|
||||
h := fnv.New64a()
|
||||
_, _ = h.Write([]byte("maint:" + policyID))
|
||||
return int64(h.Sum64())
|
||||
}
|
||||
@@ -0,0 +1,161 @@
|
||||
package maintenance
|
||||
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"evobgp/internal/store"
|
||||
)
|
||||
|
||||
func validPolicy() *store.MaintenancePolicy {
|
||||
ret := 86400
|
||||
return &store.MaintenancePolicy{
|
||||
ID: "11111111-1111-1111-1111-111111111111",
|
||||
Name: "job audit",
|
||||
TableName: "job_audit",
|
||||
Condition: "true",
|
||||
RetentionPeriodSec: &ret,
|
||||
VacuumStrategy: store.VacuumStrategyNone,
|
||||
Schedule: "0 3 * * *",
|
||||
Enabled: true,
|
||||
}
|
||||
}
|
||||
|
||||
func TestPolicyExecutorExecuteValidation(t *testing.T) {
|
||||
ctx := context.Background()
|
||||
mem := store.NewMemory()
|
||||
base := validPolicy()
|
||||
|
||||
tests := []struct {
|
||||
name string
|
||||
exec *PolicyExecutor
|
||||
policy *store.MaintenancePolicy
|
||||
wantErr error
|
||||
contains string
|
||||
}{
|
||||
{
|
||||
name: "nil policy",
|
||||
exec: &PolicyExecutor{Store: mem},
|
||||
policy: nil,
|
||||
wantErr: store.ErrInvalidInput,
|
||||
},
|
||||
{
|
||||
name: "nil pool",
|
||||
exec: &PolicyExecutor{Store: mem, Pool: nil},
|
||||
policy: base,
|
||||
contains: "postgres not configured",
|
||||
},
|
||||
}
|
||||
|
||||
for _, tc := range tests {
|
||||
t.Run(tc.name, func(t *testing.T) {
|
||||
_, err := tc.exec.Execute(ctx, tc.policy, false)
|
||||
if tc.wantErr != nil {
|
||||
if !errors.Is(err, tc.wantErr) {
|
||||
t.Fatalf("Execute() err=%v want %v", err, tc.wantErr)
|
||||
}
|
||||
return
|
||||
}
|
||||
if err == nil {
|
||||
t.Fatal("Execute() expected error")
|
||||
}
|
||||
if tc.contains != "" && !strings.Contains(err.Error(), tc.contains) {
|
||||
t.Fatalf("Execute() err=%q want substring %q", err, tc.contains)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestPolicyExecutorPreExecuteValidation(t *testing.T) {
|
||||
p := validPolicy()
|
||||
p.TableName = "tenant"
|
||||
if err := ValidateTableName(p.TableName); err == nil {
|
||||
t.Fatal("expected blocked table error")
|
||||
}
|
||||
p = validPolicy()
|
||||
p.Condition = "1=1; DROP TABLE job_audit"
|
||||
if err := ValidateCondition(p.Condition); err == nil {
|
||||
t.Fatal("expected unsafe condition error")
|
||||
}
|
||||
p = validPolicy()
|
||||
p.VacuumStrategy = "invalid"
|
||||
if !store.ValidVacuumStrategy(p.VacuumStrategy) {
|
||||
return
|
||||
}
|
||||
t.Fatal("expected invalid vacuum strategy")
|
||||
}
|
||||
|
||||
func TestPolicyAction(t *testing.T) {
|
||||
ret := 3600
|
||||
tests := []struct {
|
||||
name string
|
||||
p *store.MaintenancePolicy
|
||||
want string
|
||||
}{
|
||||
{"nil", nil, "run"},
|
||||
{"cleanup only", &store.MaintenancePolicy{RetentionPeriodSec: &ret, VacuumStrategy: store.VacuumStrategyNone}, "cleanup"},
|
||||
{"vacuum only", &store.MaintenancePolicy{VacuumStrategy: store.VacuumStrategyVacuum}, "vacuum"},
|
||||
{"cleanup+vacuum", &store.MaintenancePolicy{MaxRows: &ret, VacuumStrategy: store.VacuumStrategyAnalyze}, "cleanup_vacuum"},
|
||||
{"noop run", &store.MaintenancePolicy{VacuumStrategy: store.VacuumStrategyNone}, "run"},
|
||||
}
|
||||
for _, tc := range tests {
|
||||
t.Run(tc.name, func(t *testing.T) {
|
||||
if got := policyAction(tc.p); got != tc.want {
|
||||
t.Fatalf("policyAction()=%q want %q", got, tc.want)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestVacuumKind(t *testing.T) {
|
||||
tests := []struct {
|
||||
strategy string
|
||||
want string
|
||||
}{
|
||||
{store.VacuumStrategyVacuum, "vacuum"},
|
||||
{store.VacuumStrategyAnalyze, "analyze"},
|
||||
{store.VacuumStrategyVacuumAnalyze, "vacuum_analyze"},
|
||||
{store.VacuumStrategyReindex, "reindex"},
|
||||
{"unknown", "vacuum"},
|
||||
}
|
||||
for _, tc := range tests {
|
||||
if got := vacuumKind(tc.strategy); got != tc.want {
|
||||
t.Fatalf("vacuumKind(%q)=%q want %q", tc.strategy, got, tc.want)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestRowsDeletedFromDetail(t *testing.T) {
|
||||
tests := []struct {
|
||||
name string
|
||||
detail map[string]any
|
||||
want int64
|
||||
}{
|
||||
{"nil", nil, 0},
|
||||
{"int64", map[string]any{"deleted": int64(42)}, 42},
|
||||
{"int", map[string]any{"deleted": 7}, 7},
|
||||
{"float64", map[string]any{"deleted": float64(3)}, 3},
|
||||
{"missing", map[string]any{"other": 1}, 0},
|
||||
}
|
||||
for _, tc := range tests {
|
||||
t.Run(tc.name, func(t *testing.T) {
|
||||
if got := rowsDeletedFromDetail(tc.detail); got != tc.want {
|
||||
t.Fatalf("rowsDeletedFromDetail()=%d want %d", got, tc.want)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestAdvisoryKeyStable(t *testing.T) {
|
||||
a := advisoryKey("policy-a")
|
||||
b := advisoryKey("policy-a")
|
||||
c := advisoryKey("policy-b")
|
||||
if a != b {
|
||||
t.Fatal("advisory key not stable for same id")
|
||||
}
|
||||
if a == c {
|
||||
t.Fatal("advisory key collision for different ids")
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,72 @@
|
||||
package maintenance
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"regexp"
|
||||
"strings"
|
||||
)
|
||||
|
||||
const (
|
||||
DefaultBatchRows = 10000
|
||||
MaxBatchRows = 100000
|
||||
DefaultStatementTimeoutSec = 30
|
||||
)
|
||||
|
||||
var (
|
||||
blockedTableNames = map[string]struct{}{
|
||||
"schema_migrations": {},
|
||||
"tenant": {},
|
||||
"maintenance_policy": {},
|
||||
"maintenance_policy_config_audit": {},
|
||||
}
|
||||
|
||||
sqlForbidden = regexp.MustCompile(`(?i)(;|--|/\*|\b(drop|truncate|insert|update|alter|create|grant|revoke|copy)\b)`)
|
||||
)
|
||||
|
||||
// ValidateTableName ensures table is a safe identifier and not blocked.
|
||||
func ValidateTableName(name string) error {
|
||||
name = strings.TrimSpace(name)
|
||||
if name == "" || !isSafeIdent(name) {
|
||||
return fmt.Errorf("maintenance: invalid table name")
|
||||
}
|
||||
if _, blocked := blockedTableNames[strings.ToLower(name)]; blocked {
|
||||
return fmt.Errorf("maintenance: table %q is not allowed", name)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// ValidateCondition ensures the WHERE fragment is safe for parameterized cleanup.
|
||||
func ValidateCondition(condition string) error {
|
||||
c := strings.TrimSpace(condition)
|
||||
if c == "" {
|
||||
return nil
|
||||
}
|
||||
if sqlForbidden.MatchString(c) {
|
||||
return fmt.Errorf("maintenance: unsafe condition")
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// NormalizeBatchLimit clamps delete batch size.
|
||||
func NormalizeBatchLimit(maxRows *int) int {
|
||||
if maxRows == nil || *maxRows <= 0 {
|
||||
return DefaultBatchRows
|
||||
}
|
||||
if *maxRows > MaxBatchRows {
|
||||
return MaxBatchRows
|
||||
}
|
||||
return *maxRows
|
||||
}
|
||||
|
||||
func isSafeIdent(name string) bool {
|
||||
if name == "" {
|
||||
return false
|
||||
}
|
||||
for _, r := range name {
|
||||
if (r >= 'a' && r <= 'z') || (r >= 'A' && r <= 'Z') || (r >= '0' && r <= '9') || r == '_' {
|
||||
continue
|
||||
}
|
||||
return false
|
||||
}
|
||||
return true
|
||||
}
|
||||
@@ -0,0 +1,46 @@
|
||||
package maintenance
|
||||
|
||||
import "testing"
|
||||
|
||||
func TestValidateCondition(t *testing.T) {
|
||||
tests := []struct {
|
||||
cond string
|
||||
ok bool
|
||||
}{
|
||||
{"true", true},
|
||||
{"status IN ('succeeded', 'failed')", true},
|
||||
{"1=1; DROP TABLE tenant", false},
|
||||
{"x -- comment", false},
|
||||
}
|
||||
for _, tc := range tests {
|
||||
err := ValidateCondition(tc.cond)
|
||||
if tc.ok && err != nil {
|
||||
t.Fatalf("cond %q: want ok, got %v", tc.cond, err)
|
||||
}
|
||||
if !tc.ok && err == nil {
|
||||
t.Fatalf("cond %q: want error", tc.cond)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestValidateTableName(t *testing.T) {
|
||||
if err := ValidateTableName("job_audit"); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := ValidateTableName("tenant"); err == nil {
|
||||
t.Fatal("expected blocked table")
|
||||
}
|
||||
if err := ValidateTableName("bad-name"); err == nil {
|
||||
t.Fatal("expected invalid ident")
|
||||
}
|
||||
}
|
||||
|
||||
func TestNormalizeBatchLimit(t *testing.T) {
|
||||
if got := NormalizeBatchLimit(nil); got != DefaultBatchRows {
|
||||
t.Fatalf("default=%d got=%d", DefaultBatchRows, got)
|
||||
}
|
||||
max := 200000
|
||||
if got := NormalizeBatchLimit(&max); got != MaxBatchRows {
|
||||
t.Fatalf("max=%d got=%d", MaxBatchRows, got)
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,85 @@
|
||||
package maintenance
|
||||
|
||||
import (
|
||||
"context"
|
||||
"fmt"
|
||||
"log"
|
||||
"strings"
|
||||
"sync"
|
||||
"time"
|
||||
|
||||
"github.com/robfig/cron/v3"
|
||||
)
|
||||
|
||||
// StartScheduler enqueues maintenance_policy_run jobs when cron schedules match.
|
||||
func StartScheduler(ctx context.Context, provider *ConfigProvider, enqueue func(policyID string, dryRun bool, idempotencyKey string), tick time.Duration) {
|
||||
if provider == nil || enqueue == nil {
|
||||
return
|
||||
}
|
||||
if tick <= 0 {
|
||||
tick = 30 * time.Second
|
||||
}
|
||||
go func() {
|
||||
parser := cron.NewParser(cron.Minute | cron.Hour | cron.Dom | cron.Month | cron.Dow)
|
||||
var mu sync.Mutex
|
||||
schedules := map[string]cron.Schedule{}
|
||||
lastFired := map[string]time.Time{}
|
||||
|
||||
rebuild := func() {
|
||||
mu.Lock()
|
||||
defer mu.Unlock()
|
||||
schedules = map[string]cron.Schedule{}
|
||||
for _, p := range provider.Snapshot() {
|
||||
if p == nil || !p.Enabled || strings.TrimSpace(p.Schedule) == "" {
|
||||
continue
|
||||
}
|
||||
sched, err := parser.Parse(p.Schedule)
|
||||
if err != nil {
|
||||
log.Printf("maintenance: invalid cron for policy %s: %v", p.ID, err)
|
||||
continue
|
||||
}
|
||||
schedules[p.ID] = sched
|
||||
}
|
||||
}
|
||||
|
||||
rebuild()
|
||||
t := time.NewTicker(tick)
|
||||
defer t.Stop()
|
||||
for {
|
||||
select {
|
||||
case <-ctx.Done():
|
||||
return
|
||||
case <-t.C:
|
||||
rebuild()
|
||||
now := time.Now().UTC()
|
||||
mu.Lock()
|
||||
for _, p := range provider.Snapshot() {
|
||||
if p == nil || !p.Enabled {
|
||||
continue
|
||||
}
|
||||
sched, ok := schedules[p.ID]
|
||||
if !ok {
|
||||
continue
|
||||
}
|
||||
prev := lastFired[p.ID]
|
||||
if prev.IsZero() {
|
||||
prev = now.Add(-time.Minute)
|
||||
}
|
||||
next := sched.Next(prev)
|
||||
if next.After(now) {
|
||||
continue
|
||||
}
|
||||
slot := next.Unix() / 60
|
||||
if lf, ok := lastFired[p.ID]; ok && lf.Unix()/60 == slot {
|
||||
continue
|
||||
}
|
||||
lastFired[p.ID] = next
|
||||
idem := fmt.Sprintf("maint-%s-%d", p.ID, slot)
|
||||
enqueue(p.ID, p.DryRunEnabled, idem)
|
||||
}
|
||||
mu.Unlock()
|
||||
}
|
||||
}
|
||||
}()
|
||||
log.Printf("maintenance: policy scheduler started (tick=%s)", tick)
|
||||
}
|
||||
@@ -0,0 +1,72 @@
|
||||
package observability
|
||||
|
||||
import (
|
||||
"strconv"
|
||||
"time"
|
||||
|
||||
"github.com/prometheus/client_golang/prometheus"
|
||||
"github.com/prometheus/client_golang/prometheus/promauto"
|
||||
)
|
||||
|
||||
var (
|
||||
maintenancePolicyRuns = promauto.NewCounterVec(
|
||||
prometheus.CounterOpts{
|
||||
Namespace: namespace,
|
||||
Name: "maintenance_policy_runs_total",
|
||||
Help: "Maintenance policy executions by outcome.",
|
||||
},
|
||||
[]string{"policy_id", "action", "status", "dry_run"},
|
||||
)
|
||||
|
||||
maintenancePolicyDuration = promauto.NewHistogramVec(
|
||||
prometheus.HistogramOpts{
|
||||
Namespace: namespace,
|
||||
Name: "maintenance_policy_duration_seconds",
|
||||
Help: "Maintenance policy execution duration.",
|
||||
Buckets: prometheus.ExponentialBuckets(0.05, 2, 12),
|
||||
},
|
||||
[]string{"policy_id", "action"},
|
||||
)
|
||||
|
||||
maintenanceRowsDeleted = promauto.NewCounterVec(
|
||||
prometheus.CounterOpts{
|
||||
Namespace: namespace,
|
||||
Name: "maintenance_policy_rows_deleted_total",
|
||||
Help: "Rows deleted by maintenance cleanup policies.",
|
||||
},
|
||||
[]string{"policy_id"},
|
||||
)
|
||||
|
||||
maintenanceConfigChanges = promauto.NewCounterVec(
|
||||
prometheus.CounterOpts{
|
||||
Namespace: namespace,
|
||||
Name: "maintenance_config_changes_total",
|
||||
Help: "Maintenance policy configuration changes from UI/API.",
|
||||
},
|
||||
[]string{"action"},
|
||||
)
|
||||
)
|
||||
|
||||
// RecordMaintenancePolicyRun updates run counters and histograms.
|
||||
func RecordMaintenancePolicyRun(policyID, action, status string, dryRun bool, duration time.Duration, rowsDeleted int64) {
|
||||
if policyID == "" {
|
||||
policyID = "unknown"
|
||||
}
|
||||
if action == "" {
|
||||
action = "run"
|
||||
}
|
||||
dry := strconv.FormatBool(dryRun)
|
||||
maintenancePolicyRuns.WithLabelValues(policyID, action, status, dry).Inc()
|
||||
maintenancePolicyDuration.WithLabelValues(policyID, action).Observe(duration.Seconds())
|
||||
if rowsDeleted > 0 && !dryRun {
|
||||
maintenanceRowsDeleted.WithLabelValues(policyID).Add(float64(rowsDeleted))
|
||||
}
|
||||
}
|
||||
|
||||
// IncMaintenanceConfigChange increments config audit metric.
|
||||
func IncMaintenanceConfigChange(action string) {
|
||||
if action == "" {
|
||||
action = "unknown"
|
||||
}
|
||||
maintenanceConfigChanges.WithLabelValues(action).Inc()
|
||||
}
|
||||
@@ -0,0 +1,37 @@
|
||||
package pgmonitor
|
||||
|
||||
import (
|
||||
"sync"
|
||||
"time"
|
||||
)
|
||||
|
||||
type cacheEntry struct {
|
||||
at time.Time
|
||||
data any
|
||||
}
|
||||
|
||||
type ttlCache struct {
|
||||
mu sync.RWMutex
|
||||
ttl time.Duration
|
||||
items map[string]cacheEntry
|
||||
}
|
||||
|
||||
func newTTLCache(ttl time.Duration) *ttlCache {
|
||||
return &ttlCache{ttl: ttl, items: make(map[string]cacheEntry)}
|
||||
}
|
||||
|
||||
func (c *ttlCache) get(key string) (any, bool) {
|
||||
c.mu.RLock()
|
||||
defer c.mu.RUnlock()
|
||||
e, ok := c.items[key]
|
||||
if !ok || time.Since(e.at) > c.ttl {
|
||||
return nil, false
|
||||
}
|
||||
return e.data, true
|
||||
}
|
||||
|
||||
func (c *ttlCache) set(key string, data any) {
|
||||
c.mu.Lock()
|
||||
defer c.mu.Unlock()
|
||||
c.items[key] = cacheEntry{at: time.Now().UTC(), data: data}
|
||||
}
|
||||
@@ -0,0 +1,83 @@
|
||||
package pgmonitor
|
||||
|
||||
import (
|
||||
"context"
|
||||
"fmt"
|
||||
"time"
|
||||
|
||||
"github.com/jackc/pgx/v5/pgxpool"
|
||||
)
|
||||
|
||||
// Correlation builds aligned timeline points from job_audit and overview cache.
|
||||
func (s *Service) Correlation(ctx context.Context, windowMinutes int) (CorrelationResponse, error) {
|
||||
if s == nil || s.pool == nil {
|
||||
return CorrelationResponse{}, fmt.Errorf("pgmonitor: postgres not configured")
|
||||
}
|
||||
if windowMinutes <= 0 {
|
||||
windowMinutes = 60
|
||||
}
|
||||
if windowMinutes > 1440 {
|
||||
windowMinutes = 1440
|
||||
}
|
||||
since := time.Now().UTC().Add(-time.Duration(windowMinutes) * time.Minute)
|
||||
|
||||
rows, err := s.pool.Query(ctx, `
|
||||
SELECT date_trunc('minute', finished_at) AS bucket,
|
||||
percentile_cont(0.99) WITHIN GROUP (ORDER BY
|
||||
EXTRACT(EPOCH FROM (finished_at - started_at)) * 1000)
|
||||
FROM job_audit
|
||||
WHERE finished_at >= $1 AND kind IN ('module_refresh', 'tenant_refresh')
|
||||
AND status = 'succeeded' AND started_at IS NOT NULL
|
||||
GROUP BY 1
|
||||
ORDER BY 1`, since)
|
||||
if err != nil {
|
||||
return CorrelationResponse{}, fmt.Errorf("pgmonitor: correlation jobs: %w", err)
|
||||
}
|
||||
defer rows.Close()
|
||||
|
||||
points := make(map[time.Time]*CorrelationPoint)
|
||||
for rows.Next() {
|
||||
var bucket time.Time
|
||||
var p99 *float64
|
||||
if err := rows.Scan(&bucket, &p99); err != nil {
|
||||
return CorrelationResponse{}, err
|
||||
}
|
||||
bucket = bucket.UTC()
|
||||
pt := points[bucket]
|
||||
if pt == nil {
|
||||
pt = &CorrelationPoint{Timestamp: bucket}
|
||||
points[bucket] = pt
|
||||
}
|
||||
if p99 != nil {
|
||||
pt.PipelineRefreshP99Ms = *p99
|
||||
}
|
||||
}
|
||||
|
||||
ov, err := s.Overview(ctx)
|
||||
if err == nil && ov.Database.CacheHitPct > 0 {
|
||||
now := time.Now().UTC().Truncate(time.Minute)
|
||||
pt := points[now]
|
||||
if pt == nil {
|
||||
pt = &CorrelationPoint{Timestamp: now}
|
||||
points[now] = pt
|
||||
}
|
||||
pt.CacheHitPct = ov.Database.CacheHitPct
|
||||
}
|
||||
|
||||
out := make([]CorrelationPoint, 0, len(points))
|
||||
for _, p := range points {
|
||||
out = append(out, *p)
|
||||
}
|
||||
// simple sort by time
|
||||
for i := 0; i < len(out); i++ {
|
||||
for j := i + 1; j < len(out); j++ {
|
||||
if out[j].Timestamp.Before(out[i].Timestamp) {
|
||||
out[i], out[j] = out[j], out[i]
|
||||
}
|
||||
}
|
||||
}
|
||||
return CorrelationResponse{WindowMinutes: windowMinutes, Points: out}, nil
|
||||
}
|
||||
|
||||
// RecordCorrelationSnapshot is a hook for future Prometheus samples (no-op placeholder).
|
||||
func RecordCorrelationSnapshot(_ *pgxpool.Pool) {}
|
||||
@@ -0,0 +1,96 @@
|
||||
package pgmonitor
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"time"
|
||||
|
||||
"github.com/google/uuid"
|
||||
"github.com/jackc/pgx/v5/pgxpool"
|
||||
)
|
||||
|
||||
// CleanupRequest for deprecated POST /postgres/cleanup (use /v1/maintenance/run).
|
||||
type CleanupRequest struct {
|
||||
PolicyID string `json:"policy_id"`
|
||||
Policy string `json:"policy"`
|
||||
DryRun bool `json:"dry_run"`
|
||||
Limit int `json:"limit"`
|
||||
}
|
||||
|
||||
// InsertMaintenanceAudit records an audit row at job start.
|
||||
func InsertMaintenanceAudit(ctx context.Context, pool *pgxpool.Pool, tenantID, actorPrefix, kind, table string, dryRun bool) (string, error) {
|
||||
return InsertMaintenanceAuditWithPolicy(ctx, pool, tenantID, actorPrefix, kind, table, "", dryRun)
|
||||
}
|
||||
|
||||
// InsertMaintenanceAuditWithPolicy records an audit row linked to maintenance_policy.
|
||||
func InsertMaintenanceAuditWithPolicy(ctx context.Context, pool *pgxpool.Pool, tenantID, actorPrefix, kind, table, policyID string, dryRun bool) (string, error) {
|
||||
id := uuid.New().String()
|
||||
_, err := pool.Exec(ctx, `
|
||||
INSERT INTO postgres_maintenance_audit
|
||||
(id, tenant_id, actor_prefix, kind, target_table, policy_id, dry_run, status, created_at)
|
||||
VALUES ($1, NULLIF($2,''), NULLIF($3,''), $4, NULLIF($5,''), NULLIF($6,''), $7, 'running', now())`,
|
||||
id, tenantID, actorPrefix, kind, table, policyID, dryRun)
|
||||
return id, err
|
||||
}
|
||||
|
||||
// FinishMaintenanceAudit updates terminal state.
|
||||
func FinishMaintenanceAudit(ctx context.Context, pool *pgxpool.Pool, id, status string, detail map[string]any, errMsg *string) error {
|
||||
var detailJSON []byte
|
||||
if detail != nil {
|
||||
detailJSON, _ = json.Marshal(detail)
|
||||
}
|
||||
_, err := pool.Exec(ctx, `
|
||||
UPDATE postgres_maintenance_audit
|
||||
SET status = $2, detail_json = $3::jsonb, error_message = $4,
|
||||
finished_at = now(), started_at = COALESCE(started_at, now())
|
||||
WHERE id = $1`,
|
||||
id, status, string(detailJSON), errMsg)
|
||||
return err
|
||||
}
|
||||
|
||||
// ListMaintenanceLogs returns paginated audit rows.
|
||||
func ListMaintenanceLogs(ctx context.Context, pool *pgxpool.Pool, cursor string, limit int) ([]MaintenanceLogRow, string, bool, error) {
|
||||
limit = clampLimit(limit, 20, 100)
|
||||
args := []any{limit + 1}
|
||||
q := `
|
||||
SELECT id, COALESCE(tenant_id,''), COALESCE(actor_prefix,''), kind,
|
||||
COALESCE(target_table,''), dry_run, status,
|
||||
detail_json, COALESCE(error_message,''), created_at, started_at, finished_at
|
||||
FROM postgres_maintenance_audit`
|
||||
if cursor != "" {
|
||||
q += ` WHERE created_at < (SELECT created_at FROM postgres_maintenance_audit WHERE id = $2)`
|
||||
args = append(args, cursor)
|
||||
}
|
||||
q += ` ORDER BY created_at DESC LIMIT $1`
|
||||
|
||||
rows, err := pool.Query(ctx, q, args...)
|
||||
if err != nil {
|
||||
return nil, "", false, err
|
||||
}
|
||||
defer rows.Close()
|
||||
var out []MaintenanceLogRow
|
||||
for rows.Next() {
|
||||
var r MaintenanceLogRow
|
||||
var detailRaw []byte
|
||||
var started, finished *time.Time
|
||||
if err := rows.Scan(&r.ID, &r.TenantID, &r.ActorPrefix, &r.Kind, &r.TargetTable,
|
||||
&r.DryRun, &r.Status, &detailRaw, &r.Error, &r.CreatedAt, &started, &finished); err != nil {
|
||||
return nil, "", false, err
|
||||
}
|
||||
r.StartedAt = started
|
||||
r.FinishedAt = finished
|
||||
if len(detailRaw) > 0 {
|
||||
_ = json.Unmarshal(detailRaw, &r.Detail)
|
||||
}
|
||||
out = append(out, r)
|
||||
}
|
||||
hasMore := len(out) > limit
|
||||
if hasMore {
|
||||
out = out[:limit]
|
||||
}
|
||||
next := ""
|
||||
if hasMore && len(out) > 0 {
|
||||
next = out[len(out)-1].ID
|
||||
}
|
||||
return out, next, hasMore, rows.Err()
|
||||
}
|
||||
@@ -0,0 +1,350 @@
|
||||
package pgmonitor
|
||||
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
"fmt"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"github.com/jackc/pgx/v5"
|
||||
"github.com/jackc/pgx/v5/pgconn"
|
||||
"github.com/jackc/pgx/v5/pgxpool"
|
||||
)
|
||||
|
||||
func clampLimit(limit, def, max int) int {
|
||||
if limit <= 0 {
|
||||
return def
|
||||
}
|
||||
if limit > max {
|
||||
return max
|
||||
}
|
||||
return limit
|
||||
}
|
||||
|
||||
func (s *Service) fetchOverview(ctx context.Context) (Overview, error) {
|
||||
now := time.Now().UTC()
|
||||
out := Overview{CollectedAt: now}
|
||||
|
||||
var active, idle, total, maxConn int
|
||||
err := s.pool.QueryRow(ctx, `
|
||||
SELECT
|
||||
count(*) FILTER (WHERE state = 'active'),
|
||||
count(*) FILTER (WHERE state = 'idle'),
|
||||
count(*),
|
||||
(SELECT setting::int FROM pg_settings WHERE name = 'max_connections')
|
||||
FROM pg_stat_activity
|
||||
WHERE datname = current_database()`).Scan(&active, &idle, &total, &maxConn)
|
||||
if err != nil {
|
||||
return out, fmt.Errorf("pgmonitor: connections: %w", err)
|
||||
}
|
||||
out.Connections = Connections{Active: active, Idle: idle, Total: total, MaxConnections: maxConn}
|
||||
|
||||
var cachePct *float64
|
||||
err = s.pool.QueryRow(ctx, `
|
||||
SELECT numbackends, xact_commit, xact_rollback, deadlocks, blks_hit, blks_read,
|
||||
CASE WHEN blks_hit + blks_read > 0
|
||||
THEN round(100.0 * blks_hit::numeric / (blks_hit + blks_read), 2) END
|
||||
FROM pg_stat_database WHERE datname = current_database()`).Scan(
|
||||
&out.Database.Backends,
|
||||
&out.Database.XactCommit,
|
||||
&out.Database.XactRollback,
|
||||
&out.Database.Deadlocks,
|
||||
&out.Database.BlksHit,
|
||||
&out.Database.BlksRead,
|
||||
&cachePct,
|
||||
)
|
||||
if err != nil {
|
||||
return out, fmt.Errorf("pgmonitor: database stats: %w", err)
|
||||
}
|
||||
if cachePct != nil {
|
||||
out.Database.CacheHitPct = *cachePct
|
||||
}
|
||||
|
||||
_ = s.pool.QueryRow(ctx, `
|
||||
SELECT checkpoints_timed, checkpoints_req, buffers_checkpoint, buffers_clean,
|
||||
maxwritten_clean, buffers_backend, buffers_alloc
|
||||
FROM pg_stat_bgwriter`).Scan(
|
||||
&out.Bgwriter.CheckpointsTimed,
|
||||
&out.Bgwriter.CheckpointsReq,
|
||||
&out.Bgwriter.BuffersCheckpoint,
|
||||
&out.Bgwriter.BuffersClean,
|
||||
&out.Bgwriter.MaxWrittenClean,
|
||||
&out.Bgwriter.BuffersBackend,
|
||||
&out.Bgwriter.BuffersAlloc,
|
||||
)
|
||||
|
||||
_ = s.pool.QueryRow(ctx, `SELECT pg_database_size(current_database())`).Scan(&out.SizeBytes)
|
||||
|
||||
_ = s.pool.QueryRow(ctx, `
|
||||
SELECT
|
||||
(SELECT setting FROM pg_settings WHERE name = 'shared_buffers'),
|
||||
(SELECT setting FROM pg_settings WHERE name = 'work_mem'),
|
||||
(SELECT setting FROM pg_settings WHERE name = 'effective_cache_size')`).Scan(
|
||||
&out.MemorySettings.SharedBuffers,
|
||||
&out.MemorySettings.WorkMem,
|
||||
&out.MemorySettings.EffectiveCacheSize,
|
||||
)
|
||||
|
||||
rows, err := s.pool.Query(ctx, `
|
||||
SELECT client_addr::text, state, sync_state,
|
||||
EXTRACT(EPOCH FROM COALESCE(write_lag, flush_lag, replay_lag)) * 1000
|
||||
FROM pg_stat_replication`)
|
||||
if err == nil {
|
||||
defer rows.Close()
|
||||
for rows.Next() {
|
||||
var peer ReplicationPeer
|
||||
var lagMs *float64
|
||||
if err := rows.Scan(&peer.ClientAddr, &peer.State, &peer.SyncState, &lagMs); err != nil {
|
||||
continue
|
||||
}
|
||||
if lagMs != nil {
|
||||
v := int64(*lagMs)
|
||||
peer.LagMs = &v
|
||||
}
|
||||
out.Replication = append(out.Replication, peer)
|
||||
}
|
||||
}
|
||||
|
||||
out.StatementsEnabled = s.statementsQueryable(ctx)
|
||||
|
||||
return out, nil
|
||||
}
|
||||
|
||||
func queryLocks(ctx context.Context, pool *pgxpool.Pool) ([]LockRow, error) {
|
||||
rows, err := pool.Query(ctx, `
|
||||
SELECT l.locktype, l.mode, l.granted, a.pid, COALESCE(a.usename, ''),
|
||||
COALESCE(a.state, ''), COALESCE(left(a.query, 300), ''),
|
||||
NOT l.granted AS blocked
|
||||
FROM pg_locks l
|
||||
JOIN pg_stat_activity a ON a.pid = l.pid
|
||||
WHERE a.datname = current_database()
|
||||
AND (NOT l.granted OR l.mode LIKE '%Exclusive%')
|
||||
ORDER BY l.granted ASC, a.query_start NULLS LAST
|
||||
LIMIT 200`)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("pgmonitor: locks: %w", err)
|
||||
}
|
||||
defer rows.Close()
|
||||
var out []LockRow
|
||||
for rows.Next() {
|
||||
var r LockRow
|
||||
if err := rows.Scan(&r.Locktype, &r.Mode, &r.Granted, &r.PID, &r.User, &r.State, &r.Query, &r.Blocked); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
out = append(out, r)
|
||||
}
|
||||
return out, rows.Err()
|
||||
}
|
||||
|
||||
func queryTables(ctx context.Context, pool *pgxpool.Pool, limit int) ([]TableStat, error) {
|
||||
limit = clampLimit(limit, 20, 100)
|
||||
rows, err := pool.Query(ctx, `
|
||||
SELECT t.relname,
|
||||
pg_total_relation_size(t.relid),
|
||||
s.heap_blks_read, s.heap_blks_hit,
|
||||
t.idx_scan, t.seq_scan, t.n_dead_tup, t.last_autovacuum,
|
||||
CASE WHEN t.n_live_tup + t.n_dead_tup > 0
|
||||
THEN round(t.n_dead_tup::numeric / (t.n_live_tup + t.n_dead_tup), 4)
|
||||
ELSE 0 END
|
||||
FROM pg_statio_user_tables s
|
||||
JOIN pg_stat_user_tables t ON t.relid = s.relid
|
||||
WHERE t.schemaname = 'public'
|
||||
ORDER BY pg_total_relation_size(t.relid) DESC
|
||||
LIMIT $1`, limit)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("pgmonitor: tables: %w", err)
|
||||
}
|
||||
defer rows.Close()
|
||||
var out []TableStat
|
||||
for rows.Next() {
|
||||
var r TableStat
|
||||
var last *time.Time
|
||||
if err := rows.Scan(&r.Relname, &r.TotalBytes, &r.HeapBlksRead, &r.HeapBlksHit,
|
||||
&r.IdxScan, &r.SeqScan, &r.DeadTuples, &last, &r.BloatRatio); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
r.LastAutovacuum = last
|
||||
out = append(out, r)
|
||||
}
|
||||
return out, rows.Err()
|
||||
}
|
||||
|
||||
// TopQueries loads from pg_stat_statements when available.
|
||||
func (s *Service) TopQueries(ctx context.Context, limit int) (QueriesResponse, error) {
|
||||
if s == nil || s.pool == nil {
|
||||
return QueriesResponse{}, errors.New("pgmonitor: postgres not configured")
|
||||
}
|
||||
limit = clampLimit(limit, 20, 100)
|
||||
now := time.Now().UTC()
|
||||
|
||||
if snap, ok, err := s.loadSnapshot(ctx, "slow_queries", 15*time.Minute); err == nil && ok {
|
||||
var items []QueryStat
|
||||
if err := decodePayload(snap.Payload, &items); err == nil {
|
||||
return QueriesResponse{
|
||||
CollectedAt: snap.CollectedAt,
|
||||
Source: "snapshot",
|
||||
Items: items,
|
||||
StatementsAvailable: true,
|
||||
}, nil
|
||||
}
|
||||
}
|
||||
|
||||
if !s.statementsQueryable(ctx) {
|
||||
return queriesUnavailable(now), nil
|
||||
}
|
||||
items, err := queryTopStatements(ctx, s.pool, limit)
|
||||
if err != nil {
|
||||
if isPgStatStatementsUnavailable(err) {
|
||||
s.markStatementsUnavailable()
|
||||
return queriesUnavailable(now), nil
|
||||
}
|
||||
return QueriesResponse{}, err
|
||||
}
|
||||
return QueriesResponse{
|
||||
CollectedAt: now,
|
||||
Source: "live",
|
||||
Items: items,
|
||||
StatementsAvailable: true,
|
||||
}, nil
|
||||
}
|
||||
|
||||
func queriesUnavailable(at time.Time) QueriesResponse {
|
||||
return QueriesResponse{
|
||||
CollectedAt: at,
|
||||
Source: "unavailable",
|
||||
Items: nil,
|
||||
StatementsAvailable: false,
|
||||
StatementsHint: statementsUnavailableHint,
|
||||
}
|
||||
}
|
||||
|
||||
const statementsUnavailableHint = "pg_stat_statements requires shared_preload_libraries and PostgreSQL restart (see docs/db-diagnostics.md)"
|
||||
|
||||
// statementsQueryable returns true only when pg_stat_statements can be queried (not merely installed).
|
||||
func (s *Service) statementsQueryable(ctx context.Context) bool {
|
||||
if s == nil || s.pool == nil {
|
||||
return false
|
||||
}
|
||||
if v, ok := s.cache.get("stmt_queryable"); ok {
|
||||
if b, ok := v.(bool); ok {
|
||||
return b
|
||||
}
|
||||
}
|
||||
ok := probePgStatStatements(ctx, s.pool)
|
||||
s.cache.set("stmt_queryable", ok)
|
||||
return ok
|
||||
}
|
||||
|
||||
func (s *Service) markStatementsUnavailable() {
|
||||
s.cache.set("stmt_queryable", false)
|
||||
}
|
||||
|
||||
func probePgStatStatements(ctx context.Context, pool *pgxpool.Pool) bool {
|
||||
var dummy int64
|
||||
err := pool.QueryRow(ctx, `
|
||||
SELECT COALESCE(SUM(calls), 0)::bigint FROM pg_stat_statements LIMIT 1`).Scan(&dummy)
|
||||
if err == nil {
|
||||
return true
|
||||
}
|
||||
return !isPgStatStatementsUnavailable(err)
|
||||
}
|
||||
|
||||
func queryTopStatements(ctx context.Context, pool *pgxpool.Pool, limit int) ([]QueryStat, error) {
|
||||
rows, err := pool.Query(ctx, `
|
||||
SELECT queryid, left(query, 500), calls, total_exec_time, mean_exec_time, rows
|
||||
FROM pg_stat_statements
|
||||
WHERE dbid = (SELECT oid FROM pg_database WHERE datname = current_database())
|
||||
ORDER BY mean_exec_time DESC
|
||||
LIMIT $1`, limit)
|
||||
if err != nil {
|
||||
if isPgStatStatementsUnavailable(err) {
|
||||
return nil, nil
|
||||
}
|
||||
return nil, fmt.Errorf("pgmonitor: pg_stat_statements: %w", err)
|
||||
}
|
||||
defer rows.Close()
|
||||
var out []QueryStat
|
||||
for rows.Next() {
|
||||
var r QueryStat
|
||||
if err := rows.Scan(&r.QueryID, &r.Query, &r.Calls, &r.TotalExecMs, &r.MeanExecMs, &r.Rows); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
out = append(out, r)
|
||||
}
|
||||
return out, rows.Err()
|
||||
}
|
||||
|
||||
// isPgStatStatementsUnavailable reports extension missing or not loaded via shared_preload_libraries.
|
||||
func isPgStatStatementsUnavailable(err error) bool {
|
||||
if err == nil {
|
||||
return false
|
||||
}
|
||||
var pgErr *pgconn.PgError
|
||||
if errors.As(err, &pgErr) {
|
||||
switch pgErr.Code {
|
||||
case "42P01", "42704", "55000":
|
||||
return true
|
||||
}
|
||||
msg := strings.ToLower(pgErr.Message)
|
||||
if strings.Contains(msg, "shared_preload_libraries") || strings.Contains(msg, "pg_stat_statements") {
|
||||
return true
|
||||
}
|
||||
}
|
||||
low := strings.ToLower(err.Error())
|
||||
return strings.Contains(low, "shared_preload_libraries") || strings.Contains(low, "pg_stat_statements")
|
||||
}
|
||||
|
||||
func isSafeIdent(name string) bool {
|
||||
if name == "" {
|
||||
return true
|
||||
}
|
||||
for _, r := range name {
|
||||
if (r >= 'a' && r <= 'z') || (r >= 'A' && r <= 'Z') || (r >= '0' && r <= '9') || r == '_' {
|
||||
continue
|
||||
}
|
||||
return false
|
||||
}
|
||||
return true
|
||||
}
|
||||
|
||||
// ExecMaintenance runs VACUUM/ANALYZE/REINDEX with optional dry-run (returns SQL executed or planned).
|
||||
func ExecMaintenance(ctx context.Context, pool *pgxpool.Pool, kind, table string, dryRun bool) (detail map[string]any, err error) {
|
||||
if pool == nil {
|
||||
return nil, errors.New("pgmonitor: postgres not configured")
|
||||
}
|
||||
table = strings.TrimSpace(table)
|
||||
if table != "" && !isSafeIdent(table) {
|
||||
return nil, errors.New("pgmonitor: invalid table name")
|
||||
}
|
||||
qual := ""
|
||||
if table != "" {
|
||||
qual = " " + pgx.Identifier{table}.Sanitize()
|
||||
}
|
||||
var sql string
|
||||
switch kind {
|
||||
case "vacuum":
|
||||
sql = "VACUUM" + qual
|
||||
case "vacuum_analyze":
|
||||
sql = "VACUUM ANALYZE" + qual
|
||||
case "analyze":
|
||||
sql = "ANALYZE" + qual
|
||||
case "reindex":
|
||||
if table == "" {
|
||||
return nil, errors.New("pgmonitor: reindex requires table")
|
||||
}
|
||||
sql = "REINDEX TABLE" + qual
|
||||
default:
|
||||
return nil, fmt.Errorf("pgmonitor: unknown maintenance kind %q", kind)
|
||||
}
|
||||
detail = map[string]any{"sql": sql, "dry_run": dryRun}
|
||||
if dryRun {
|
||||
return detail, nil
|
||||
}
|
||||
_, err = pool.Exec(ctx, sql)
|
||||
if err != nil {
|
||||
return detail, fmt.Errorf("pgmonitor: %s: %w", kind, err)
|
||||
}
|
||||
detail["executed"] = true
|
||||
return detail, nil
|
||||
}
|
||||
@@ -0,0 +1,104 @@
|
||||
package pgmonitor
|
||||
|
||||
import (
|
||||
"context"
|
||||
"time"
|
||||
)
|
||||
|
||||
// Recommendations builds heuristic items from live stats and snapshots.
|
||||
func (s *Service) Recommendations(ctx context.Context) (RecommendationsResponse, error) {
|
||||
now := time.Now().UTC()
|
||||
var items []RecommendationItem
|
||||
|
||||
ov, err := s.Overview(ctx)
|
||||
if err == nil {
|
||||
if ov.Database.CacheHitPct > 0 && ov.Database.CacheHitPct < 90 {
|
||||
items = append(items, RecommendationItem{
|
||||
Severity: "warn",
|
||||
Code: "low_cache_hit",
|
||||
Title: "Низкий cache hit ratio",
|
||||
Detail: "Buffer cache hit ниже 90%; проверьте shared_buffers и горячие seq scan.",
|
||||
})
|
||||
}
|
||||
if ov.Database.Deadlocks > 0 {
|
||||
items = append(items, RecommendationItem{
|
||||
Severity: "warn",
|
||||
Code: "deadlocks",
|
||||
Title: "Зафиксированы deadlocks",
|
||||
Detail: "Проверьте конкурирующие транзакции и порядок блокировок.",
|
||||
})
|
||||
}
|
||||
if ov.Connections.MaxConnections > 0 &&
|
||||
float64(ov.Connections.Total)/float64(ov.Connections.MaxConnections) > 0.8 {
|
||||
items = append(items, RecommendationItem{
|
||||
Severity: "critical",
|
||||
Code: "connections_high",
|
||||
Title: "Много подключений к PostgreSQL",
|
||||
Detail: "Использование max_connections выше 80%; увеличьте pool tuning или лимит.",
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
tables, err := s.Tables(ctx, 30)
|
||||
if err == nil {
|
||||
for _, t := range tables {
|
||||
if t.SeqScan > 1000 && t.IdxScan < t.SeqScan/10 {
|
||||
items = append(items, RecommendationItem{
|
||||
Severity: "warn",
|
||||
Code: "missing_index",
|
||||
Title: "Высокий seq_scan",
|
||||
Detail: "Таблица часто сканируется последовательно; рассмотрите индекс.",
|
||||
Refs: []string{t.Relname},
|
||||
})
|
||||
}
|
||||
if t.BloatRatio > 0.2 && t.DeadTuples > 5000 {
|
||||
items = append(items, RecommendationItem{
|
||||
Severity: "info",
|
||||
Code: "autovacuum_lag",
|
||||
Title: "Возможный bloat / мёртвые строки",
|
||||
Detail: "Высокая доля n_dead_tup; запланируйте VACUUM.",
|
||||
Refs: []string{t.Relname},
|
||||
})
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
if snap, ok, _ := s.loadSnapshot(ctx, "unused_indexes", 30*time.Minute); ok {
|
||||
type unused struct {
|
||||
Index string `json:"index"`
|
||||
SizeBytes int64 `json:"size_bytes"`
|
||||
}
|
||||
var list []unused
|
||||
if decodePayload(snap.Payload, &list) == nil {
|
||||
for _, u := range list {
|
||||
if u.SizeBytes < 1024*1024 {
|
||||
continue
|
||||
}
|
||||
items = append(items, RecommendationItem{
|
||||
Severity: "info",
|
||||
Code: "unused_index",
|
||||
Title: "Неиспользуемый индекс",
|
||||
Detail: "idx_scan=0; проверьте перед удалением.",
|
||||
Refs: []string{u.Index},
|
||||
})
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
q, err := s.TopQueries(ctx, 5)
|
||||
if err == nil {
|
||||
for _, qs := range q.Items {
|
||||
if qs.MeanExecMs > 500 {
|
||||
items = append(items, RecommendationItem{
|
||||
Severity: "warn",
|
||||
Code: "slow_query",
|
||||
Title: "Медленный запрос",
|
||||
Detail: "Среднее время выполнения выше 500ms.",
|
||||
Refs: []string{qs.Query},
|
||||
})
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
return RecommendationsResponse{CollectedAt: now, Items: items}, nil
|
||||
}
|
||||
@@ -0,0 +1,59 @@
|
||||
package pgmonitor
|
||||
|
||||
import (
|
||||
"context"
|
||||
"log"
|
||||
"time"
|
||||
|
||||
"github.com/jackc/pgx/v5/pgxpool"
|
||||
)
|
||||
|
||||
// StartScheduler runs periodic PostgreSQL analyzer snapshots until ctx is cancelled.
|
||||
func StartScheduler(ctx context.Context, pool *pgxpool.Pool) {
|
||||
if pool == nil {
|
||||
return
|
||||
}
|
||||
go func() {
|
||||
t5 := time.NewTicker(5 * time.Minute)
|
||||
t15 := time.NewTicker(15 * time.Minute)
|
||||
defer t5.Stop()
|
||||
defer t15.Stop()
|
||||
s := NewService(pool)
|
||||
runLight := func() {
|
||||
c, cancel := context.WithTimeout(context.Background(), 2*time.Minute)
|
||||
defer cancel()
|
||||
if err := s.RefreshMetricsSnapshot(c); err != nil {
|
||||
log.Printf("pgmonitor: metrics refresh: %v", err)
|
||||
}
|
||||
if err := s.DetectAutovacuumLag(c); err != nil {
|
||||
log.Printf("pgmonitor: autovacuum lag: %v", err)
|
||||
}
|
||||
}
|
||||
runHeavy := func() {
|
||||
c, cancel := context.WithTimeout(context.Background(), 3*time.Minute)
|
||||
defer cancel()
|
||||
if err := s.AggregateSlowQueries(c, 30); err != nil {
|
||||
log.Printf("pgmonitor: slow queries snapshot: %v", err)
|
||||
}
|
||||
if err := s.EstimateTableBloat(c); err != nil {
|
||||
log.Printf("pgmonitor: bloat: %v", err)
|
||||
}
|
||||
if err := s.AnalyzeIndexUsage(c); err != nil {
|
||||
log.Printf("pgmonitor: index usage: %v", err)
|
||||
}
|
||||
}
|
||||
runLight()
|
||||
runHeavy()
|
||||
for {
|
||||
select {
|
||||
case <-ctx.Done():
|
||||
return
|
||||
case <-t5.C:
|
||||
runLight()
|
||||
case <-t15.C:
|
||||
runHeavy()
|
||||
}
|
||||
}
|
||||
}()
|
||||
log.Printf("pgmonitor: scheduler started (5m light / 15m heavy)")
|
||||
}
|
||||
@@ -0,0 +1,90 @@
|
||||
package pgmonitor
|
||||
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
"fmt"
|
||||
"time"
|
||||
|
||||
"github.com/jackc/pgx/v5/pgxpool"
|
||||
)
|
||||
|
||||
// Service provides PostgreSQL observability and maintenance helpers (control plane instance scope).
|
||||
type Service struct {
|
||||
pool *pgxpool.Pool
|
||||
cache *ttlCache
|
||||
}
|
||||
|
||||
// NewService constructs a metrics service for the API PostgreSQL pool.
|
||||
func NewService(pool *pgxpool.Pool) *Service {
|
||||
if pool == nil {
|
||||
return nil
|
||||
}
|
||||
return &Service{
|
||||
pool: pool,
|
||||
cache: newTTLCache(10 * time.Second),
|
||||
}
|
||||
}
|
||||
|
||||
// Pool exposes the underlying pool for job workers.
|
||||
func (s *Service) Pool() *pgxpool.Pool {
|
||||
if s == nil {
|
||||
return nil
|
||||
}
|
||||
return s.pool
|
||||
}
|
||||
|
||||
// Overview returns cached instance-level stats.
|
||||
func (s *Service) Overview(ctx context.Context) (Overview, error) {
|
||||
if s == nil || s.pool == nil {
|
||||
return Overview{}, errors.New("pgmonitor: postgres not configured")
|
||||
}
|
||||
if v, ok := s.cache.get("overview"); ok {
|
||||
if o, ok := v.(Overview); ok {
|
||||
return o, nil
|
||||
}
|
||||
}
|
||||
o, err := s.fetchOverview(ctx)
|
||||
if err != nil {
|
||||
return Overview{}, err
|
||||
}
|
||||
s.cache.set("overview", o)
|
||||
return o, nil
|
||||
}
|
||||
|
||||
// Locks returns active / blocking locks.
|
||||
func (s *Service) Locks(ctx context.Context) ([]LockRow, error) {
|
||||
if s == nil || s.pool == nil {
|
||||
return nil, errors.New("pgmonitor: postgres not configured")
|
||||
}
|
||||
if v, ok := s.cache.get("locks"); ok {
|
||||
if rows, ok := v.([]LockRow); ok {
|
||||
return rows, nil
|
||||
}
|
||||
}
|
||||
rows, err := queryLocks(ctx, s.pool)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
s.cache.set("locks", rows)
|
||||
return rows, nil
|
||||
}
|
||||
|
||||
// Tables returns top tables by size with I/O stats.
|
||||
func (s *Service) Tables(ctx context.Context, limit int) ([]TableStat, error) {
|
||||
if s == nil || s.pool == nil {
|
||||
return nil, errors.New("pgmonitor: postgres not configured")
|
||||
}
|
||||
key := fmt.Sprintf("tables:%d", limit)
|
||||
if v, ok := s.cache.get(key); ok {
|
||||
if rows, ok := v.([]TableStat); ok {
|
||||
return rows, nil
|
||||
}
|
||||
}
|
||||
rows, err := queryTables(ctx, s.pool, limit)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
s.cache.set(key, rows)
|
||||
return rows, nil
|
||||
}
|
||||
@@ -0,0 +1,48 @@
|
||||
package pgmonitor
|
||||
|
||||
import (
|
||||
"errors"
|
||||
"testing"
|
||||
|
||||
"github.com/jackc/pgx/v5/pgconn"
|
||||
)
|
||||
|
||||
func TestClampLimit(t *testing.T) {
|
||||
if clampLimit(0, 20, 100) != 20 {
|
||||
t.Fatal("default")
|
||||
}
|
||||
if clampLimit(200, 20, 100) != 100 {
|
||||
t.Fatal("max")
|
||||
}
|
||||
if clampLimit(5, 20, 100) != 5 {
|
||||
t.Fatal("value")
|
||||
}
|
||||
}
|
||||
|
||||
func TestIsSafeIdent(t *testing.T) {
|
||||
if !isSafeIdent("revision_materialized_prefix") {
|
||||
t.Fatal("valid")
|
||||
}
|
||||
if isSafeIdent("bad-name") {
|
||||
t.Fatal("invalid")
|
||||
}
|
||||
if !isSafeIdent("") {
|
||||
t.Fatal("empty ok")
|
||||
}
|
||||
}
|
||||
|
||||
func TestNewServiceNilPool(t *testing.T) {
|
||||
if NewService(nil) != nil {
|
||||
t.Fatal("expected nil service")
|
||||
}
|
||||
}
|
||||
|
||||
func TestIsPgStatStatementsUnavailable(t *testing.T) {
|
||||
err := &pgconn.PgError{Code: "55000", Message: "pg_stat_statements must be loaded via shared_preload_libraries"}
|
||||
if !isPgStatStatementsUnavailable(err) {
|
||||
t.Fatal("55000")
|
||||
}
|
||||
if isPgStatStatementsUnavailable(errors.New("other")) {
|
||||
t.Fatal("unrelated")
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,162 @@
|
||||
package pgmonitor
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"time"
|
||||
|
||||
"github.com/jackc/pgx/v5/pgxpool"
|
||||
)
|
||||
|
||||
type snapshotRow struct {
|
||||
ID string
|
||||
CollectedAt time.Time
|
||||
Payload json.RawMessage
|
||||
}
|
||||
|
||||
func (s *Service) loadSnapshot(ctx context.Context, id string, maxAge time.Duration) (snapshotRow, bool, error) {
|
||||
var row snapshotRow
|
||||
err := s.pool.QueryRow(ctx, `
|
||||
SELECT id, collected_at, payload_json
|
||||
FROM postgres_monitor_snapshot
|
||||
WHERE id = $1 AND collected_at >= $2`,
|
||||
id, time.Now().UTC().Add(-maxAge)).Scan(&row.ID, &row.CollectedAt, &row.Payload)
|
||||
if err != nil {
|
||||
return snapshotRow{}, false, nil
|
||||
}
|
||||
return row, true, nil
|
||||
}
|
||||
|
||||
func (s *Service) UpsertSnapshot(ctx context.Context, id string, payload any) error {
|
||||
if s == nil || s.pool == nil {
|
||||
return fmt.Errorf("pgmonitor: postgres not configured")
|
||||
}
|
||||
b, err := json.Marshal(payload)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
_, err = s.pool.Exec(ctx, `
|
||||
INSERT INTO postgres_monitor_snapshot (id, collected_at, payload_json)
|
||||
VALUES ($1, now(), $2::jsonb)
|
||||
ON CONFLICT (id) DO UPDATE SET collected_at = EXCLUDED.collected_at, payload_json = EXCLUDED.payload_json`,
|
||||
id, string(b))
|
||||
return err
|
||||
}
|
||||
|
||||
func decodePayload(raw json.RawMessage, dest any) error {
|
||||
return json.Unmarshal(raw, dest)
|
||||
}
|
||||
|
||||
// RefreshMetricsSnapshot stores overview and tables for heavy reads.
|
||||
func (s *Service) RefreshMetricsSnapshot(ctx context.Context) error {
|
||||
ov, err := s.fetchOverview(ctx)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if err := s.UpsertSnapshot(ctx, "overview", ov); err != nil {
|
||||
return err
|
||||
}
|
||||
tables, err := queryTables(ctx, s.pool, 50)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
return s.UpsertSnapshot(ctx, "tables", tables)
|
||||
}
|
||||
|
||||
// AggregateSlowQueries stores top statements snapshot.
|
||||
func (s *Service) AggregateSlowQueries(ctx context.Context, limit int) error {
|
||||
if !s.statementsQueryable(ctx) {
|
||||
return s.UpsertSnapshot(ctx, "slow_queries", []QueryStat{})
|
||||
}
|
||||
items, err := queryTopStatements(ctx, s.pool, clampLimit(limit, 20, 100))
|
||||
if err != nil {
|
||||
if isPgStatStatementsUnavailable(err) {
|
||||
s.markStatementsUnavailable()
|
||||
return s.UpsertSnapshot(ctx, "slow_queries", []QueryStat{})
|
||||
}
|
||||
return err
|
||||
}
|
||||
return s.UpsertSnapshot(ctx, "slow_queries", items)
|
||||
}
|
||||
|
||||
// EstimateTableBloat refreshes bloat heuristics on tables snapshot.
|
||||
func (s *Service) EstimateTableBloat(ctx context.Context) error {
|
||||
tables, err := queryTables(ctx, s.pool, 100)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
return s.UpsertSnapshot(ctx, "table_bloat", tables)
|
||||
}
|
||||
|
||||
// AnalyzeIndexUsage stores unused indexes.
|
||||
func (s *Service) AnalyzeIndexUsage(ctx context.Context) error {
|
||||
rows, err := s.pool.Query(ctx, `
|
||||
SELECT indexrelname, idx_scan, pg_relation_size(indexrelid)
|
||||
FROM pg_stat_user_indexes
|
||||
WHERE schemaname = 'public' AND idx_scan = 0
|
||||
ORDER BY pg_relation_size(indexrelid) DESC
|
||||
LIMIT 50`)
|
||||
if err != nil {
|
||||
return fmt.Errorf("pgmonitor: index usage: %w", err)
|
||||
}
|
||||
defer rows.Close()
|
||||
type unused struct {
|
||||
Index string `json:"index"`
|
||||
IdxScan int64 `json:"idx_scan"`
|
||||
SizeBytes int64 `json:"size_bytes"`
|
||||
}
|
||||
var items []unused
|
||||
for rows.Next() {
|
||||
var u unused
|
||||
if err := rows.Scan(&u.Index, &u.IdxScan, &u.SizeBytes); err != nil {
|
||||
return err
|
||||
}
|
||||
items = append(items, u)
|
||||
}
|
||||
return s.UpsertSnapshot(ctx, "unused_indexes", items)
|
||||
}
|
||||
|
||||
// DetectAutovacuumLag stores tables with high dead tuple ratio.
|
||||
func (s *Service) DetectAutovacuumLag(ctx context.Context) error {
|
||||
rows, err := s.pool.Query(ctx, `
|
||||
SELECT relname, n_dead_tup, last_autovacuum,
|
||||
CASE WHEN n_live_tup + n_dead_tup > 0
|
||||
THEN round(n_dead_tup::numeric / (n_live_tup + n_dead_tup), 4) ELSE 0 END
|
||||
FROM pg_stat_user_tables
|
||||
WHERE schemaname = 'public' AND n_dead_tup > 1000
|
||||
ORDER BY n_dead_tup DESC
|
||||
LIMIT 30`)
|
||||
if err != nil {
|
||||
return fmt.Errorf("pgmonitor: autovacuum lag: %w", err)
|
||||
}
|
||||
defer rows.Close()
|
||||
type lagRow struct {
|
||||
Relname string `json:"relname"`
|
||||
DeadTuples int64 `json:"n_dead_tup"`
|
||||
LastAutovacuum *time.Time `json:"last_autovacuum,omitempty"`
|
||||
Ratio float64 `json:"ratio"`
|
||||
}
|
||||
var items []lagRow
|
||||
for rows.Next() {
|
||||
var r lagRow
|
||||
if err := rows.Scan(&r.Relname, &r.DeadTuples, &r.LastAutovacuum, &r.Ratio); err != nil {
|
||||
return err
|
||||
}
|
||||
items = append(items, r)
|
||||
}
|
||||
return s.UpsertSnapshot(ctx, "autovacuum_lag", items)
|
||||
}
|
||||
|
||||
// RunPeriodicAnalyzerJobs runs all snapshot analyzers (for scheduler).
|
||||
func RunPeriodicAnalyzerJobs(ctx context.Context, pool *pgxpool.Pool) {
|
||||
s := NewService(pool)
|
||||
if s == nil {
|
||||
return
|
||||
}
|
||||
_ = s.RefreshMetricsSnapshot(ctx)
|
||||
_ = s.AggregateSlowQueries(ctx, 30)
|
||||
_ = s.EstimateTableBloat(ctx)
|
||||
_ = s.AnalyzeIndexUsage(ctx)
|
||||
_ = s.DetectAutovacuumLag(ctx)
|
||||
}
|
||||
@@ -0,0 +1,150 @@
|
||||
package pgmonitor
|
||||
|
||||
import "time"
|
||||
|
||||
// Overview is instance-level PostgreSQL health snapshot.
|
||||
type Overview struct {
|
||||
CollectedAt time.Time `json:"collected_at"`
|
||||
Connections Connections `json:"connections"`
|
||||
Database DatabaseStats `json:"database"`
|
||||
Bgwriter BgwriterStats `json:"bgwriter"`
|
||||
SizeBytes int64 `json:"database_size_bytes"`
|
||||
MemorySettings MemorySettings `json:"memory_settings"`
|
||||
Replication []ReplicationPeer `json:"replication"`
|
||||
StatementsEnabled bool `json:"pg_stat_statements_enabled"`
|
||||
}
|
||||
|
||||
// Connections summarizes pg_stat_activity for current database.
|
||||
type Connections struct {
|
||||
Active int `json:"active"`
|
||||
Idle int `json:"idle"`
|
||||
Total int `json:"total"`
|
||||
MaxConnections int `json:"max_connections"`
|
||||
}
|
||||
|
||||
// DatabaseStats from pg_stat_database.
|
||||
type DatabaseStats struct {
|
||||
Backends int `json:"backends"`
|
||||
XactCommit int64 `json:"xact_commit"`
|
||||
XactRollback int64 `json:"xact_rollback"`
|
||||
Deadlocks int64 `json:"deadlocks"`
|
||||
BlksHit int64 `json:"blks_hit"`
|
||||
BlksRead int64 `json:"blks_read"`
|
||||
CacheHitPct float64 `json:"cache_hit_pct"`
|
||||
}
|
||||
|
||||
// BgwriterStats from pg_stat_bgwriter.
|
||||
type BgwriterStats struct {
|
||||
CheckpointsTimed int64 `json:"checkpoints_timed"`
|
||||
CheckpointsReq int64 `json:"checkpoints_req"`
|
||||
BuffersCheckpoint int64 `json:"buffers_checkpoint"`
|
||||
BuffersClean int64 `json:"buffers_clean"`
|
||||
MaxWrittenClean int64 `json:"maxwritten_clean"`
|
||||
BuffersBackend int64 `json:"buffers_backend"`
|
||||
BuffersAlloc int64 `json:"buffers_alloc"`
|
||||
}
|
||||
|
||||
// MemorySettings is best-effort from pg_settings (not RSS).
|
||||
type MemorySettings struct {
|
||||
SharedBuffers string `json:"shared_buffers"`
|
||||
WorkMem string `json:"work_mem"`
|
||||
EffectiveCacheSize string `json:"effective_cache_size"`
|
||||
}
|
||||
|
||||
// ReplicationPeer from pg_stat_replication.
|
||||
type ReplicationPeer struct {
|
||||
ClientAddr string `json:"client_addr,omitempty"`
|
||||
State string `json:"state"`
|
||||
SyncState string `json:"sync_state,omitempty"`
|
||||
LagMs *int64 `json:"lag_ms,omitempty"`
|
||||
}
|
||||
|
||||
// QueryStat is a row from pg_stat_statements or snapshot.
|
||||
type QueryStat struct {
|
||||
QueryID int64 `json:"queryid,omitempty"`
|
||||
Query string `json:"query"`
|
||||
Calls int64 `json:"calls"`
|
||||
TotalExecMs float64 `json:"total_exec_ms"`
|
||||
MeanExecMs float64 `json:"mean_exec_ms"`
|
||||
Rows int64 `json:"rows"`
|
||||
}
|
||||
|
||||
// QueriesResponse for GET /monitoring/postgres/queries.
|
||||
type QueriesResponse struct {
|
||||
CollectedAt time.Time `json:"collected_at"`
|
||||
Source string `json:"source"` // live | snapshot | unavailable
|
||||
Items []QueryStat `json:"items"`
|
||||
StatementsAvailable bool `json:"statements_available"`
|
||||
StatementsHint string `json:"statements_hint,omitempty"`
|
||||
}
|
||||
|
||||
// LockRow describes a lock / blocking session.
|
||||
type LockRow struct {
|
||||
Locktype string `json:"locktype"`
|
||||
Mode string `json:"mode"`
|
||||
Granted bool `json:"granted"`
|
||||
PID int32 `json:"pid"`
|
||||
User string `json:"usename,omitempty"`
|
||||
State string `json:"state,omitempty"`
|
||||
Query string `json:"query,omitempty"`
|
||||
Blocked bool `json:"blocked"`
|
||||
}
|
||||
|
||||
// TableStat combines size and scan stats for a user table.
|
||||
type TableStat struct {
|
||||
Relname string `json:"relname"`
|
||||
TotalBytes int64 `json:"total_bytes"`
|
||||
HeapBlksRead int64 `json:"heap_blks_read"`
|
||||
HeapBlksHit int64 `json:"heap_blks_hit"`
|
||||
IdxScan int64 `json:"idx_scan"`
|
||||
SeqScan int64 `json:"seq_scan"`
|
||||
DeadTuples int64 `json:"n_dead_tup"`
|
||||
LastAutovacuum *time.Time `json:"last_autovacuum,omitempty"`
|
||||
BloatRatio float64 `json:"bloat_ratio,omitempty"`
|
||||
}
|
||||
|
||||
// RecommendationItem is a heuristic ops hint.
|
||||
type RecommendationItem struct {
|
||||
Severity string `json:"severity"` // info | warn | critical
|
||||
Code string `json:"code"`
|
||||
Title string `json:"title"`
|
||||
Detail string `json:"detail"`
|
||||
Refs []string `json:"refs,omitempty"`
|
||||
}
|
||||
|
||||
// RecommendationsResponse for GET recommendations.
|
||||
type RecommendationsResponse struct {
|
||||
CollectedAt time.Time `json:"collected_at"`
|
||||
Items []RecommendationItem `json:"items"`
|
||||
}
|
||||
|
||||
// CorrelationPoint is one aligned sample for overlay charts.
|
||||
type CorrelationPoint struct {
|
||||
Timestamp time.Time `json:"timestamp"`
|
||||
PipelineRefreshP99Ms float64 `json:"pipeline_refresh_p99_ms,omitempty"`
|
||||
BirdScrapeOK *float64 `json:"bird_scrape_ok,omitempty"`
|
||||
HTTPRequestRate float64 `json:"http_request_rate,omitempty"`
|
||||
CacheHitPct float64 `json:"cache_hit_pct,omitempty"`
|
||||
}
|
||||
|
||||
// CorrelationResponse for GET /monitoring/correlation.
|
||||
type CorrelationResponse struct {
|
||||
WindowMinutes int `json:"window_minutes"`
|
||||
Points []CorrelationPoint `json:"points"`
|
||||
}
|
||||
|
||||
// MaintenanceLogRow is an audit entry.
|
||||
type MaintenanceLogRow struct {
|
||||
ID string `json:"id"`
|
||||
TenantID string `json:"tenant_id,omitempty"`
|
||||
ActorPrefix string `json:"actor_prefix,omitempty"`
|
||||
Kind string `json:"kind"`
|
||||
TargetTable string `json:"target_table,omitempty"`
|
||||
DryRun bool `json:"dry_run"`
|
||||
Status string `json:"status"`
|
||||
Detail map[string]any `json:"detail,omitempty"`
|
||||
Error string `json:"error,omitempty"`
|
||||
CreatedAt time.Time `json:"created_at"`
|
||||
StartedAt *time.Time `json:"started_at,omitempty"`
|
||||
FinishedAt *time.Time `json:"finished_at,omitempty"`
|
||||
}
|
||||
@@ -1,29 +1,8 @@
|
||||
package repository
|
||||
|
||||
import (
|
||||
"context"
|
||||
"time"
|
||||
)
|
||||
import "context"
|
||||
|
||||
const (
|
||||
jobAuditRetentionDays = 90
|
||||
asnCacheRetentionDays = 7
|
||||
)
|
||||
|
||||
// RunPeriodicMaintenance prunes stale job_audit and asn_prefix_cache rows (PostgreSQL).
|
||||
// RunPeriodicMaintenance is a no-op; retention is driven by maintenance_policy rows (UI-configured).
|
||||
func (p *Postgres) RunPeriodicMaintenance(ctx context.Context) {
|
||||
if p == nil || p.pool == nil {
|
||||
return
|
||||
}
|
||||
if ctx == nil {
|
||||
ctx = context.Background()
|
||||
}
|
||||
jobCutoff := time.Now().UTC().Add(-time.Duration(jobAuditRetentionDays) * 24 * time.Hour)
|
||||
_, _ = p.pool.Exec(ctx, `
|
||||
DELETE FROM job_audit
|
||||
WHERE created_at < $1
|
||||
AND status IN ('succeeded', 'failed', 'cancelled')`, jobCutoff)
|
||||
asnCutoff := time.Now().UTC().Add(-time.Duration(asnCacheRetentionDays) * 24 * time.Hour)
|
||||
_, _ = p.pool.Exec(ctx, `
|
||||
DELETE FROM asn_prefix_cache WHERE fetched_at < $1`, asnCutoff)
|
||||
_ = ctx
|
||||
}
|
||||
|
||||
@@ -39,7 +39,7 @@ func (p *Postgres) GetModulePrefixSnapshot(tenantID, moduleID string) (*store.Mo
|
||||
var prefixes []store.PrefixRow
|
||||
if moduleSnapshotRowTableExists(ctx, p.pool) {
|
||||
rows, qerr := p.pool.Query(ctx, `
|
||||
SELECT prefix::text, community_id::text, source
|
||||
SELECT prefix, community_id::text, source
|
||||
FROM module_prefix_snapshot_row
|
||||
WHERE tenant_id = $1::uuid AND module_id = $2::uuid
|
||||
ORDER BY ord`, tenantID, moduleID)
|
||||
@@ -108,7 +108,7 @@ func (p *Postgres) SetModulePrefixSnapshot(tenantID, moduleID, inputHash string,
|
||||
}
|
||||
if _, err := tx.Exec(ctx, `
|
||||
INSERT INTO module_prefix_snapshot_row (tenant_id, module_id, ord, prefix, community_id, source)
|
||||
VALUES ($1::uuid, $2::uuid, $3, $4::cidr, $5::uuid, $6)`,
|
||||
VALUES ($1::uuid, $2::uuid, $3, $4, $5::uuid, $6)`,
|
||||
tenantID, moduleID, i, strings.TrimSpace(pr.Prefix), comm, src); err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
@@ -0,0 +1,304 @@
|
||||
package repository
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"strconv"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"evobgp/internal/store"
|
||||
|
||||
"github.com/google/uuid"
|
||||
"github.com/jackc/pgx/v5"
|
||||
)
|
||||
|
||||
const maintenancePolicySelect = `
|
||||
SELECT id, name, table_name, condition_sql, retention_period_sec, max_rows,
|
||||
vacuum_strategy, schedule_cron, enabled, dry_run_enabled,
|
||||
last_run_at, COALESCE(last_status, ''), COALESCE(last_error, ''),
|
||||
created_at, updated_at
|
||||
FROM maintenance_policy`
|
||||
|
||||
func scanMaintenancePolicy(row pgx.Row) (*store.MaintenancePolicy, error) {
|
||||
var p store.MaintenancePolicy
|
||||
var retention, maxRows *int32
|
||||
var lastRun *time.Time
|
||||
err := row.Scan(
|
||||
&p.ID, &p.Name, &p.TableName, &p.Condition, &retention, &maxRows,
|
||||
&p.VacuumStrategy, &p.Schedule, &p.Enabled, &p.DryRunEnabled,
|
||||
&lastRun, &p.LastStatus, &p.LastError, &p.CreatedAt, &p.UpdatedAt,
|
||||
)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if retention != nil {
|
||||
v := int(*retention)
|
||||
p.RetentionPeriodSec = &v
|
||||
}
|
||||
if maxRows != nil {
|
||||
v := int(*maxRows)
|
||||
p.MaxRows = &v
|
||||
}
|
||||
if lastRun != nil {
|
||||
t := lastRun.UTC()
|
||||
p.LastRunAt = &t
|
||||
}
|
||||
return &p, nil
|
||||
}
|
||||
|
||||
func (p *Postgres) ListMaintenancePolicies(cursor string, limit int) ([]*store.MaintenancePolicy, string, bool, error) {
|
||||
if limit <= 0 {
|
||||
limit = 50
|
||||
}
|
||||
off := 0
|
||||
if cursor != "" {
|
||||
if n, err := strconv.Atoi(cursor); err == nil && n >= 0 {
|
||||
off = n
|
||||
}
|
||||
}
|
||||
ctx := context.Background()
|
||||
rows, err := p.pool.Query(ctx, maintenancePolicySelect+`
|
||||
ORDER BY created_at DESC, id DESC
|
||||
LIMIT $1 OFFSET $2`, limit+1, off)
|
||||
if err != nil {
|
||||
return nil, "", false, err
|
||||
}
|
||||
defer rows.Close()
|
||||
var out []*store.MaintenancePolicy
|
||||
for rows.Next() {
|
||||
pol, err := scanMaintenancePolicy(rows)
|
||||
if err != nil {
|
||||
continue
|
||||
}
|
||||
out = append(out, pol)
|
||||
}
|
||||
more := len(out) > limit
|
||||
if more {
|
||||
out = out[:limit]
|
||||
}
|
||||
next := ""
|
||||
if more {
|
||||
next = strconv.Itoa(off + limit)
|
||||
}
|
||||
return out, next, more, rows.Err()
|
||||
}
|
||||
|
||||
func (p *Postgres) GetMaintenancePolicy(id string) (*store.MaintenancePolicy, error) {
|
||||
ctx := context.Background()
|
||||
row := p.pool.QueryRow(ctx, maintenancePolicySelect+` WHERE id=$1`, id)
|
||||
pol, err := scanMaintenancePolicy(row)
|
||||
if err != nil {
|
||||
if errors.Is(err, pgx.ErrNoRows) {
|
||||
return nil, store.ErrNotFound
|
||||
}
|
||||
return nil, err
|
||||
}
|
||||
return pol, nil
|
||||
}
|
||||
|
||||
func (p *Postgres) CreateMaintenancePolicy(in *store.MaintenancePolicy) (*store.MaintenancePolicy, error) {
|
||||
if in == nil {
|
||||
return nil, store.ErrInvalidInput
|
||||
}
|
||||
vacuum := in.VacuumStrategy
|
||||
if vacuum == "" {
|
||||
vacuum = store.VacuumStrategyNone
|
||||
}
|
||||
if err := store.ValidateMaintenancePolicyInput(in.Name, in.TableName, vacuum, in.Schedule); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
ctx := context.Background()
|
||||
id := uuid.NewString()
|
||||
now := time.Now().UTC()
|
||||
condition := store.NormalizeMaintenancePolicyCondition(in.Condition)
|
||||
var retention, maxRows *int32
|
||||
if in.RetentionPeriodSec != nil {
|
||||
v := int32(*in.RetentionPeriodSec)
|
||||
retention = &v
|
||||
}
|
||||
if in.MaxRows != nil {
|
||||
v := int32(*in.MaxRows)
|
||||
maxRows = &v
|
||||
}
|
||||
_, err := p.pool.Exec(ctx, `
|
||||
INSERT INTO maintenance_policy (
|
||||
id, name, table_name, condition_sql, retention_period_sec, max_rows,
|
||||
vacuum_strategy, schedule_cron, enabled, dry_run_enabled, created_at, updated_at
|
||||
) VALUES ($1,$2,$3,$4,$5,$6,$7,$8,$9,$10,$11,$11)`,
|
||||
id, strings.TrimSpace(in.Name), strings.TrimSpace(in.TableName), condition,
|
||||
retention, maxRows, vacuum, strings.TrimSpace(in.Schedule),
|
||||
in.Enabled, in.DryRunEnabled, now)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return p.GetMaintenancePolicy(id)
|
||||
}
|
||||
|
||||
func (p *Postgres) UpdateMaintenancePolicy(id string, patch *store.MaintenancePolicyPatch) (*store.MaintenancePolicy, error) {
|
||||
if patch == nil {
|
||||
return nil, store.ErrInvalidInput
|
||||
}
|
||||
cur, err := p.GetMaintenancePolicy(id)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if patch.Name != nil {
|
||||
cur.Name = strings.TrimSpace(*patch.Name)
|
||||
}
|
||||
if patch.TableName != nil {
|
||||
cur.TableName = strings.TrimSpace(*patch.TableName)
|
||||
}
|
||||
if patch.Condition != nil {
|
||||
cur.Condition = store.NormalizeMaintenancePolicyCondition(*patch.Condition)
|
||||
}
|
||||
if patch.RetentionPeriodSec != nil {
|
||||
cur.RetentionPeriodSec = patch.RetentionPeriodSec
|
||||
}
|
||||
if patch.MaxRows != nil {
|
||||
cur.MaxRows = patch.MaxRows
|
||||
}
|
||||
if patch.VacuumStrategy != nil {
|
||||
if !store.ValidVacuumStrategy(*patch.VacuumStrategy) {
|
||||
return nil, store.ErrInvalidInput
|
||||
}
|
||||
cur.VacuumStrategy = strings.TrimSpace(*patch.VacuumStrategy)
|
||||
}
|
||||
if patch.Schedule != nil {
|
||||
cur.Schedule = strings.TrimSpace(*patch.Schedule)
|
||||
}
|
||||
if patch.Enabled != nil {
|
||||
cur.Enabled = *patch.Enabled
|
||||
}
|
||||
if patch.DryRunEnabled != nil {
|
||||
cur.DryRunEnabled = *patch.DryRunEnabled
|
||||
}
|
||||
if err := store.ValidateMaintenancePolicyInput(cur.Name, cur.TableName, cur.VacuumStrategy, cur.Schedule); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
var retention, maxRows *int32
|
||||
if cur.RetentionPeriodSec != nil {
|
||||
v := int32(*cur.RetentionPeriodSec)
|
||||
retention = &v
|
||||
}
|
||||
if cur.MaxRows != nil {
|
||||
v := int32(*cur.MaxRows)
|
||||
maxRows = &v
|
||||
}
|
||||
ctx := context.Background()
|
||||
tag, err := p.pool.Exec(ctx, `
|
||||
UPDATE maintenance_policy SET
|
||||
name=$2, table_name=$3, condition_sql=$4, retention_period_sec=$5, max_rows=$6,
|
||||
vacuum_strategy=$7, schedule_cron=$8, enabled=$9, dry_run_enabled=$10, updated_at=now()
|
||||
WHERE id=$1`,
|
||||
id, cur.Name, cur.TableName, cur.Condition, retention, maxRows,
|
||||
cur.VacuumStrategy, cur.Schedule, cur.Enabled, cur.DryRunEnabled)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if tag.RowsAffected() == 0 {
|
||||
return nil, store.ErrNotFound
|
||||
}
|
||||
return p.GetMaintenancePolicy(id)
|
||||
}
|
||||
|
||||
func (p *Postgres) DeleteMaintenancePolicy(id string) error {
|
||||
ctx := context.Background()
|
||||
tag, err := p.pool.Exec(ctx, `DELETE FROM maintenance_policy WHERE id=$1`, id)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if tag.RowsAffected() == 0 {
|
||||
return store.ErrNotFound
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func (p *Postgres) TouchMaintenancePolicyRun(id, status, errMsg string) error {
|
||||
ctx := context.Background()
|
||||
tag, err := p.pool.Exec(ctx, `
|
||||
UPDATE maintenance_policy SET
|
||||
last_run_at=now(), last_status=$2, last_error=NULLIF($3,''), updated_at=now()
|
||||
WHERE id=$1`, id, status, errMsg)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if tag.RowsAffected() == 0 {
|
||||
return store.ErrNotFound
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func (p *Postgres) AppendMaintenancePolicyConfigAudit(actor, policyID, action string, before, after map[string]any) error {
|
||||
ctx := context.Background()
|
||||
id := uuid.NewString()
|
||||
var beforeJSON, afterJSON []byte
|
||||
if before != nil {
|
||||
beforeJSON, _ = json.Marshal(before)
|
||||
}
|
||||
if after != nil {
|
||||
afterJSON, _ = json.Marshal(after)
|
||||
}
|
||||
_, err := p.pool.Exec(ctx, `
|
||||
INSERT INTO maintenance_policy_config_audit
|
||||
(id, policy_id, actor_prefix, action, before_json, after_json, created_at)
|
||||
VALUES ($1, NULLIF($2,''), $3, $4, $5::jsonb, $6::jsonb, now())`,
|
||||
id, policyID, strings.TrimSpace(actor), action,
|
||||
nullJSONBytes(beforeJSON), nullJSONBytes(afterJSON))
|
||||
return err
|
||||
}
|
||||
|
||||
func nullJSONBytes(b []byte) any {
|
||||
if len(b) == 0 {
|
||||
return nil
|
||||
}
|
||||
return string(b)
|
||||
}
|
||||
|
||||
func (p *Postgres) ListMaintenancePolicyConfigAudit(cursor string, limit int) ([]*store.MaintenancePolicyConfigAudit, string, bool, error) {
|
||||
if limit <= 0 {
|
||||
limit = 50
|
||||
}
|
||||
off := 0
|
||||
if cursor != "" {
|
||||
if n, err := strconv.Atoi(cursor); err == nil && n >= 0 {
|
||||
off = n
|
||||
}
|
||||
}
|
||||
ctx := context.Background()
|
||||
rows, err := p.pool.Query(ctx, `
|
||||
SELECT id, COALESCE(policy_id,''), actor_prefix, action,
|
||||
before_json, after_json, created_at
|
||||
FROM maintenance_policy_config_audit
|
||||
ORDER BY created_at DESC, id DESC
|
||||
LIMIT $1 OFFSET $2`, limit+1, off)
|
||||
if err != nil {
|
||||
return nil, "", false, err
|
||||
}
|
||||
defer rows.Close()
|
||||
var out []*store.MaintenancePolicyConfigAudit
|
||||
for rows.Next() {
|
||||
var r store.MaintenancePolicyConfigAudit
|
||||
var beforeRaw, afterRaw []byte
|
||||
if err := rows.Scan(&r.ID, &r.PolicyID, &r.ActorPrefix, &r.Action, &beforeRaw, &afterRaw, &r.CreatedAt); err != nil {
|
||||
continue
|
||||
}
|
||||
if len(beforeRaw) > 0 {
|
||||
_ = json.Unmarshal(beforeRaw, &r.Before)
|
||||
}
|
||||
if len(afterRaw) > 0 {
|
||||
_ = json.Unmarshal(afterRaw, &r.After)
|
||||
}
|
||||
out = append(out, &r)
|
||||
}
|
||||
more := len(out) > limit
|
||||
if more {
|
||||
out = out[:limit]
|
||||
}
|
||||
next := ""
|
||||
if more {
|
||||
next = strconv.Itoa(off + limit)
|
||||
}
|
||||
return out, next, more, rows.Err()
|
||||
}
|
||||
@@ -115,6 +115,16 @@ type Backend interface {
|
||||
|
||||
// RunPeriodicMaintenance prunes stale DB rows (no-op for in-memory).
|
||||
RunPeriodicMaintenance(ctx context.Context)
|
||||
|
||||
// Maintenance policies (instance-scoped PostgreSQL maintenance configuration).
|
||||
ListMaintenancePolicies(cursor string, limit int) ([]*MaintenancePolicy, string, bool, error)
|
||||
GetMaintenancePolicy(id string) (*MaintenancePolicy, error)
|
||||
CreateMaintenancePolicy(in *MaintenancePolicy) (*MaintenancePolicy, error)
|
||||
UpdateMaintenancePolicy(id string, patch *MaintenancePolicyPatch) (*MaintenancePolicy, error)
|
||||
DeleteMaintenancePolicy(id string) error
|
||||
TouchMaintenancePolicyRun(id, status, errMsg string) error
|
||||
AppendMaintenancePolicyConfigAudit(actor, policyID, action string, before, after map[string]any) error
|
||||
ListMaintenancePolicyConfigAudit(cursor string, limit int) ([]*MaintenancePolicyConfigAudit, string, bool, error)
|
||||
}
|
||||
|
||||
// ASNPrefixCacheEntry is a cached RIPEstat response for one ASN.
|
||||
@@ -314,7 +324,7 @@ type SpeakerPatch struct {
|
||||
|
||||
// PrefixRow is one materialized prefix for GET /revisions/.../prefixes.
|
||||
type PrefixRow struct {
|
||||
Prefix string
|
||||
CommunityID *string
|
||||
Source string
|
||||
Prefix string `json:"prefix"`
|
||||
CommunityID *string `json:"community_id,omitempty"`
|
||||
Source string `json:"source,omitempty"`
|
||||
}
|
||||
|
||||
@@ -0,0 +1,89 @@
|
||||
package store
|
||||
|
||||
import (
|
||||
"strings"
|
||||
"time"
|
||||
)
|
||||
|
||||
// Vacuum strategy values for maintenance_policy.vacuum_strategy.
|
||||
const (
|
||||
VacuumStrategyNone = "none"
|
||||
VacuumStrategyVacuum = "vacuum"
|
||||
VacuumStrategyAnalyze = "analyze"
|
||||
VacuumStrategyVacuumAnalyze = "vacuum_analyze"
|
||||
VacuumStrategyReindex = "reindex"
|
||||
)
|
||||
|
||||
// MaintenancePolicy is an instance-scoped PostgreSQL maintenance policy (control plane DB).
|
||||
type MaintenancePolicy struct {
|
||||
ID string `json:"id,omitempty"`
|
||||
Name string `json:"name"`
|
||||
TableName string `json:"table_name"`
|
||||
Condition string `json:"condition"`
|
||||
RetentionPeriodSec *int `json:"retention_period_sec,omitempty"`
|
||||
MaxRows *int `json:"max_rows,omitempty"`
|
||||
VacuumStrategy string `json:"vacuum_strategy"`
|
||||
Schedule string `json:"schedule"`
|
||||
Enabled bool `json:"enabled"`
|
||||
DryRunEnabled bool `json:"dry_run_enabled"`
|
||||
LastRunAt *time.Time `json:"last_run_at,omitempty"`
|
||||
LastStatus string `json:"last_status,omitempty"`
|
||||
LastError string `json:"last_error,omitempty"`
|
||||
CreatedAt time.Time `json:"created_at,omitempty"`
|
||||
UpdatedAt time.Time `json:"updated_at,omitempty"`
|
||||
}
|
||||
|
||||
// MaintenancePolicyPatch is a partial update for maintenance_policy.
|
||||
type MaintenancePolicyPatch struct {
|
||||
Name *string `json:"name,omitempty"`
|
||||
TableName *string `json:"table_name,omitempty"`
|
||||
Condition *string `json:"condition,omitempty"`
|
||||
RetentionPeriodSec *int `json:"retention_period_sec,omitempty"`
|
||||
MaxRows *int `json:"max_rows,omitempty"`
|
||||
VacuumStrategy *string `json:"vacuum_strategy,omitempty"`
|
||||
Schedule *string `json:"schedule,omitempty"`
|
||||
Enabled *bool `json:"enabled,omitempty"`
|
||||
DryRunEnabled *bool `json:"dry_run_enabled,omitempty"`
|
||||
}
|
||||
|
||||
// MaintenancePolicyConfigAudit is a configuration change log entry.
|
||||
type MaintenancePolicyConfigAudit struct {
|
||||
ID string `json:"id"`
|
||||
PolicyID string `json:"policy_id,omitempty"`
|
||||
ActorPrefix string `json:"actor_prefix"`
|
||||
Action string `json:"action"`
|
||||
Before map[string]any `json:"before,omitempty"`
|
||||
After map[string]any `json:"after,omitempty"`
|
||||
CreatedAt time.Time `json:"created_at"`
|
||||
}
|
||||
|
||||
// ValidVacuumStrategy reports whether s is an allowed vacuum_strategy value.
|
||||
func ValidVacuumStrategy(s string) bool {
|
||||
switch strings.TrimSpace(s) {
|
||||
case VacuumStrategyNone, VacuumStrategyVacuum, VacuumStrategyAnalyze,
|
||||
VacuumStrategyVacuumAnalyze, VacuumStrategyReindex:
|
||||
return true
|
||||
default:
|
||||
return false
|
||||
}
|
||||
}
|
||||
|
||||
// NormalizeMaintenancePolicyCondition returns a safe default WHERE fragment.
|
||||
func NormalizeMaintenancePolicyCondition(condition string) string {
|
||||
c := strings.TrimSpace(condition)
|
||||
if c == "" {
|
||||
return "true"
|
||||
}
|
||||
return c
|
||||
}
|
||||
|
||||
// ValidateMaintenancePolicyInput checks required fields for create/update payloads.
|
||||
func ValidateMaintenancePolicyInput(name, tableName, vacuumStrategy, schedule string) error {
|
||||
if strings.TrimSpace(name) == "" || strings.TrimSpace(tableName) == "" || strings.TrimSpace(schedule) == "" {
|
||||
return ErrInvalidInput
|
||||
}
|
||||
if !ValidVacuumStrategy(vacuumStrategy) {
|
||||
return ErrInvalidInput
|
||||
}
|
||||
return nil
|
||||
}
|
||||
+32
-28
@@ -33,17 +33,19 @@ type Memory struct {
|
||||
|
||||
peers map[string]*BGPPeer
|
||||
|
||||
dohProfiles map[string]*DohProfile
|
||||
communities map[string]*Community
|
||||
cdnSources map[string]*CDNSource
|
||||
asEntries map[string]*ASEntry
|
||||
domainEnt map[string]*DomainEntry
|
||||
ipRanges map[string]*IPRangeEntry
|
||||
settings map[string]map[string]any // tenantID -> key -> JSON-compatible value
|
||||
revPrefixes map[string][]PrefixRow
|
||||
moduleSnapshots map[string]*moduleSnapshotRec
|
||||
asnPrefixCache map[int64]*ASNPrefixCacheEntry
|
||||
apiKeys map[string]*apiKeyRec
|
||||
dohProfiles map[string]*DohProfile
|
||||
communities map[string]*Community
|
||||
cdnSources map[string]*CDNSource
|
||||
asEntries map[string]*ASEntry
|
||||
domainEnt map[string]*DomainEntry
|
||||
ipRanges map[string]*IPRangeEntry
|
||||
settings map[string]map[string]any // tenantID -> key -> JSON-compatible value
|
||||
revPrefixes map[string][]PrefixRow
|
||||
moduleSnapshots map[string]*moduleSnapshotRec
|
||||
asnPrefixCache map[int64]*ASNPrefixCacheEntry
|
||||
apiKeys map[string]*apiKeyRec
|
||||
maintenancePolicies map[string]*MaintenancePolicy
|
||||
maintConfigAudit []*MaintenancePolicyConfigAudit
|
||||
|
||||
// DemoIDs valid after SeedDemo()
|
||||
demoTenantID string
|
||||
@@ -123,23 +125,25 @@ type Speaker struct {
|
||||
|
||||
func NewMemory() *Memory {
|
||||
return &Memory{
|
||||
tenants: make(map[string]*Tenant),
|
||||
modules: make(map[string]*Module),
|
||||
revisions: make(map[string]*Revision),
|
||||
speakers: make(map[string]*Speaker),
|
||||
publishedRevision: make(map[string]publishedInfo),
|
||||
peers: make(map[string]*BGPPeer),
|
||||
dohProfiles: make(map[string]*DohProfile),
|
||||
communities: make(map[string]*Community),
|
||||
cdnSources: make(map[string]*CDNSource),
|
||||
asEntries: make(map[string]*ASEntry),
|
||||
domainEnt: make(map[string]*DomainEntry),
|
||||
ipRanges: make(map[string]*IPRangeEntry),
|
||||
settings: make(map[string]map[string]any),
|
||||
revPrefixes: make(map[string][]PrefixRow),
|
||||
moduleSnapshots: make(map[string]*moduleSnapshotRec),
|
||||
asnPrefixCache: make(map[int64]*ASNPrefixCacheEntry),
|
||||
apiKeys: make(map[string]*apiKeyRec),
|
||||
tenants: make(map[string]*Tenant),
|
||||
modules: make(map[string]*Module),
|
||||
revisions: make(map[string]*Revision),
|
||||
speakers: make(map[string]*Speaker),
|
||||
publishedRevision: make(map[string]publishedInfo),
|
||||
peers: make(map[string]*BGPPeer),
|
||||
dohProfiles: make(map[string]*DohProfile),
|
||||
communities: make(map[string]*Community),
|
||||
cdnSources: make(map[string]*CDNSource),
|
||||
asEntries: make(map[string]*ASEntry),
|
||||
domainEnt: make(map[string]*DomainEntry),
|
||||
ipRanges: make(map[string]*IPRangeEntry),
|
||||
settings: make(map[string]map[string]any),
|
||||
revPrefixes: make(map[string][]PrefixRow),
|
||||
moduleSnapshots: make(map[string]*moduleSnapshotRec),
|
||||
asnPrefixCache: make(map[int64]*ASNPrefixCacheEntry),
|
||||
apiKeys: make(map[string]*apiKeyRec),
|
||||
maintenancePolicies: make(map[string]*MaintenancePolicy),
|
||||
maintConfigAudit: nil,
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
@@ -0,0 +1,244 @@
|
||||
package store
|
||||
|
||||
import (
|
||||
"sort"
|
||||
"strconv"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"github.com/google/uuid"
|
||||
)
|
||||
|
||||
func (m *Memory) ListMaintenancePolicies(cursor string, limit int) ([]*MaintenancePolicy, string, bool, error) {
|
||||
if limit <= 0 {
|
||||
limit = 50
|
||||
}
|
||||
m.mu.RLock()
|
||||
defer m.mu.RUnlock()
|
||||
all := make([]*MaintenancePolicy, 0, len(m.maintenancePolicies))
|
||||
for _, p := range m.maintenancePolicies {
|
||||
all = append(all, p)
|
||||
}
|
||||
sort.Slice(all, func(i, j int) bool {
|
||||
if all[i].CreatedAt.Equal(all[j].CreatedAt) {
|
||||
return all[i].ID > all[j].ID
|
||||
}
|
||||
return all[i].CreatedAt.After(all[j].CreatedAt)
|
||||
})
|
||||
off := parseMaintCursor(cursor)
|
||||
end := off + limit
|
||||
next := ""
|
||||
hasMore := false
|
||||
if end > len(all) {
|
||||
end = len(all)
|
||||
} else if end < len(all) {
|
||||
hasMore = true
|
||||
next = formatMaintCursor(end)
|
||||
}
|
||||
if off >= len(all) {
|
||||
return nil, "", false, nil
|
||||
}
|
||||
out := make([]*MaintenancePolicy, end-off)
|
||||
copy(out, all[off:end])
|
||||
return out, next, hasMore, nil
|
||||
}
|
||||
|
||||
func (m *Memory) GetMaintenancePolicy(id string) (*MaintenancePolicy, error) {
|
||||
m.mu.RLock()
|
||||
defer m.mu.RUnlock()
|
||||
p, ok := m.maintenancePolicies[id]
|
||||
if !ok {
|
||||
return nil, ErrNotFound
|
||||
}
|
||||
return cloneMaintenancePolicy(p), nil
|
||||
}
|
||||
|
||||
func (m *Memory) CreateMaintenancePolicy(in *MaintenancePolicy) (*MaintenancePolicy, error) {
|
||||
if in == nil {
|
||||
return nil, ErrInvalidInput
|
||||
}
|
||||
vacuum := in.VacuumStrategy
|
||||
if vacuum == "" {
|
||||
vacuum = VacuumStrategyNone
|
||||
}
|
||||
if err := ValidateMaintenancePolicyInput(in.Name, in.TableName, vacuum, in.Schedule); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
m.mu.Lock()
|
||||
defer m.mu.Unlock()
|
||||
now := time.Now().UTC()
|
||||
id := uuid.NewString()
|
||||
p := &MaintenancePolicy{
|
||||
ID: id,
|
||||
Name: strings.TrimSpace(in.Name),
|
||||
TableName: strings.TrimSpace(in.TableName),
|
||||
Condition: NormalizeMaintenancePolicyCondition(in.Condition),
|
||||
RetentionPeriodSec: in.RetentionPeriodSec,
|
||||
MaxRows: in.MaxRows,
|
||||
VacuumStrategy: vacuum,
|
||||
Schedule: strings.TrimSpace(in.Schedule),
|
||||
Enabled: in.Enabled,
|
||||
DryRunEnabled: in.DryRunEnabled,
|
||||
CreatedAt: now,
|
||||
UpdatedAt: now,
|
||||
}
|
||||
m.maintenancePolicies[id] = p
|
||||
return cloneMaintenancePolicy(p), nil
|
||||
}
|
||||
|
||||
func (m *Memory) UpdateMaintenancePolicy(id string, patch *MaintenancePolicyPatch) (*MaintenancePolicy, error) {
|
||||
if patch == nil {
|
||||
return nil, ErrInvalidInput
|
||||
}
|
||||
m.mu.Lock()
|
||||
defer m.mu.Unlock()
|
||||
p, ok := m.maintenancePolicies[id]
|
||||
if !ok {
|
||||
return nil, ErrNotFound
|
||||
}
|
||||
if patch.Name != nil {
|
||||
p.Name = strings.TrimSpace(*patch.Name)
|
||||
}
|
||||
if patch.TableName != nil {
|
||||
p.TableName = strings.TrimSpace(*patch.TableName)
|
||||
}
|
||||
if patch.Condition != nil {
|
||||
p.Condition = NormalizeMaintenancePolicyCondition(*patch.Condition)
|
||||
}
|
||||
if patch.RetentionPeriodSec != nil {
|
||||
p.RetentionPeriodSec = patch.RetentionPeriodSec
|
||||
}
|
||||
if patch.MaxRows != nil {
|
||||
p.MaxRows = patch.MaxRows
|
||||
}
|
||||
if patch.VacuumStrategy != nil {
|
||||
if !ValidVacuumStrategy(*patch.VacuumStrategy) {
|
||||
return nil, ErrInvalidInput
|
||||
}
|
||||
p.VacuumStrategy = strings.TrimSpace(*patch.VacuumStrategy)
|
||||
}
|
||||
if patch.Schedule != nil {
|
||||
p.Schedule = strings.TrimSpace(*patch.Schedule)
|
||||
}
|
||||
if patch.Enabled != nil {
|
||||
p.Enabled = *patch.Enabled
|
||||
}
|
||||
if patch.DryRunEnabled != nil {
|
||||
p.DryRunEnabled = *patch.DryRunEnabled
|
||||
}
|
||||
if err := ValidateMaintenancePolicyInput(p.Name, p.TableName, p.VacuumStrategy, p.Schedule); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
p.UpdatedAt = time.Now().UTC()
|
||||
return cloneMaintenancePolicy(p), nil
|
||||
}
|
||||
|
||||
func (m *Memory) DeleteMaintenancePolicy(id string) error {
|
||||
m.mu.Lock()
|
||||
defer m.mu.Unlock()
|
||||
if _, ok := m.maintenancePolicies[id]; !ok {
|
||||
return ErrNotFound
|
||||
}
|
||||
delete(m.maintenancePolicies, id)
|
||||
return nil
|
||||
}
|
||||
|
||||
func (m *Memory) TouchMaintenancePolicyRun(id, status, errMsg string) error {
|
||||
m.mu.Lock()
|
||||
defer m.mu.Unlock()
|
||||
p, ok := m.maintenancePolicies[id]
|
||||
if !ok {
|
||||
return ErrNotFound
|
||||
}
|
||||
now := time.Now().UTC()
|
||||
p.LastRunAt = &now
|
||||
p.LastStatus = status
|
||||
p.LastError = errMsg
|
||||
p.UpdatedAt = now
|
||||
return nil
|
||||
}
|
||||
|
||||
func (m *Memory) AppendMaintenancePolicyConfigAudit(actor, policyID, action string, before, after map[string]any) error {
|
||||
m.mu.Lock()
|
||||
defer m.mu.Unlock()
|
||||
row := &MaintenancePolicyConfigAudit{
|
||||
ID: uuid.NewString(),
|
||||
PolicyID: policyID,
|
||||
ActorPrefix: strings.TrimSpace(actor),
|
||||
Action: action,
|
||||
Before: before,
|
||||
After: after,
|
||||
CreatedAt: time.Now().UTC(),
|
||||
}
|
||||
m.maintConfigAudit = append(m.maintConfigAudit, row)
|
||||
return nil
|
||||
}
|
||||
|
||||
func (m *Memory) ListMaintenancePolicyConfigAudit(cursor string, limit int) ([]*MaintenancePolicyConfigAudit, string, bool, error) {
|
||||
if limit <= 0 {
|
||||
limit = 50
|
||||
}
|
||||
m.mu.RLock()
|
||||
defer m.mu.RUnlock()
|
||||
all := append([]*MaintenancePolicyConfigAudit(nil), m.maintConfigAudit...)
|
||||
sort.Slice(all, func(i, j int) bool {
|
||||
if all[i].CreatedAt.Equal(all[j].CreatedAt) {
|
||||
return all[i].ID > all[j].ID
|
||||
}
|
||||
return all[i].CreatedAt.After(all[j].CreatedAt)
|
||||
})
|
||||
off := parseMaintCursor(cursor)
|
||||
end := off + limit
|
||||
next := ""
|
||||
hasMore := false
|
||||
if end > len(all) {
|
||||
end = len(all)
|
||||
} else if end < len(all) {
|
||||
hasMore = true
|
||||
next = formatMaintCursor(end)
|
||||
}
|
||||
if off >= len(all) {
|
||||
return nil, "", false, nil
|
||||
}
|
||||
out := make([]*MaintenancePolicyConfigAudit, end-off)
|
||||
copy(out, all[off:end])
|
||||
return out, next, hasMore, nil
|
||||
}
|
||||
|
||||
func cloneMaintenancePolicy(p *MaintenancePolicy) *MaintenancePolicy {
|
||||
if p == nil {
|
||||
return nil
|
||||
}
|
||||
cp := *p
|
||||
if p.RetentionPeriodSec != nil {
|
||||
v := *p.RetentionPeriodSec
|
||||
cp.RetentionPeriodSec = &v
|
||||
}
|
||||
if p.MaxRows != nil {
|
||||
v := *p.MaxRows
|
||||
cp.MaxRows = &v
|
||||
}
|
||||
if p.LastRunAt != nil {
|
||||
t := *p.LastRunAt
|
||||
cp.LastRunAt = &t
|
||||
}
|
||||
return &cp
|
||||
}
|
||||
|
||||
func parseMaintCursor(cursor string) int {
|
||||
if cursor == "" {
|
||||
return 0
|
||||
}
|
||||
var off int
|
||||
for _, r := range cursor {
|
||||
if r < '0' || r > '9' {
|
||||
return 0
|
||||
}
|
||||
off = off*10 + int(r-'0')
|
||||
}
|
||||
return off
|
||||
}
|
||||
|
||||
func formatMaintCursor(off int) string {
|
||||
return strconv.Itoa(off)
|
||||
}
|
||||
@@ -0,0 +1,96 @@
|
||||
package store
|
||||
|
||||
import "testing"
|
||||
|
||||
func TestMemoryMaintenancePolicyCRUD(t *testing.T) {
|
||||
m := NewMemory()
|
||||
ret := int(86400)
|
||||
max := 5000
|
||||
created, err := m.CreateMaintenancePolicy(&MaintenancePolicy{
|
||||
Name: "audit cleanup",
|
||||
TableName: "job_audit",
|
||||
Condition: "status = 'succeeded'",
|
||||
RetentionPeriodSec: &ret,
|
||||
MaxRows: &max,
|
||||
VacuumStrategy: VacuumStrategyVacuumAnalyze,
|
||||
Schedule: "0 4 * * *",
|
||||
Enabled: true,
|
||||
DryRunEnabled: true,
|
||||
})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if created.ID == "" {
|
||||
t.Fatal("missing id")
|
||||
}
|
||||
|
||||
items, _, hasMore, err := m.ListMaintenancePolicies("", 10)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if len(items) != 1 || hasMore {
|
||||
t.Fatalf("list: len=%d hasMore=%v", len(items), hasMore)
|
||||
}
|
||||
|
||||
got, err := m.GetMaintenancePolicy(created.ID)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if got.Name != "audit cleanup" || got.VacuumStrategy != VacuumStrategyVacuumAnalyze {
|
||||
t.Fatalf("get: %+v", got)
|
||||
}
|
||||
|
||||
newName := "renamed"
|
||||
disabled := false
|
||||
updated, err := m.UpdateMaintenancePolicy(created.ID, &MaintenancePolicyPatch{
|
||||
Name: &newName,
|
||||
Enabled: &disabled,
|
||||
})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if updated.Name != newName || updated.Enabled {
|
||||
t.Fatalf("update: %+v", updated)
|
||||
}
|
||||
|
||||
if err := m.TouchMaintenancePolicyRun(created.ID, "succeeded", ""); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
afterTouch, err := m.GetMaintenancePolicy(created.ID)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if afterTouch.LastStatus != "succeeded" || afterTouch.LastRunAt == nil {
|
||||
t.Fatalf("touch: %+v", afterTouch)
|
||||
}
|
||||
|
||||
if err := m.AppendMaintenancePolicyConfigAudit("op:test", created.ID, "update", map[string]any{"name": "old"}, map[string]any{"name": newName}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
audit, next, hasMore, err := m.ListMaintenancePolicyConfigAudit("", 10)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if len(audit) != 1 || audit[0].Action != "update" || next != "" || hasMore {
|
||||
t.Fatalf("audit: %+v next=%q hasMore=%v", audit, next, hasMore)
|
||||
}
|
||||
|
||||
if err := m.DeleteMaintenancePolicy(created.ID); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if _, err := m.GetMaintenancePolicy(created.ID); err != ErrNotFound {
|
||||
t.Fatalf("after delete: %v", err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestCreateMaintenancePolicyInvalid(t *testing.T) {
|
||||
m := NewMemory()
|
||||
_, err := m.CreateMaintenancePolicy(&MaintenancePolicy{Name: "", TableName: "job_audit", Schedule: "0 3 * * *"})
|
||||
if err != ErrInvalidInput {
|
||||
t.Fatalf("want ErrInvalidInput got %v", err)
|
||||
}
|
||||
_, err = m.CreateMaintenancePolicy(&MaintenancePolicy{Name: "x", TableName: "job_audit", Schedule: "0 3 * * *", VacuumStrategy: "bad"})
|
||||
if err != ErrInvalidInput {
|
||||
t.Fatalf("want ErrInvalidInput got %v", err)
|
||||
}
|
||||
}
|
||||
@@ -2,7 +2,7 @@ CREATE TABLE module_prefix_snapshot_row (
|
||||
tenant_id UUID NOT NULL,
|
||||
module_id UUID NOT NULL,
|
||||
ord INTEGER NOT NULL,
|
||||
prefix CIDR NOT NULL,
|
||||
prefix TEXT NOT NULL,
|
||||
community_id UUID,
|
||||
source TEXT NOT NULL DEFAULT '',
|
||||
PRIMARY KEY (tenant_id, module_id, ord),
|
||||
@@ -10,16 +10,37 @@ CREATE TABLE module_prefix_snapshot_row (
|
||||
REFERENCES module_prefix_snapshot (tenant_id, module_id) ON DELETE CASCADE
|
||||
);
|
||||
|
||||
-- prefixes_json from Go json.Marshal(PrefixRow) used "Prefix"/"CommunityID"/"Source" before json tags.
|
||||
INSERT INTO module_prefix_snapshot_row (tenant_id, module_id, ord, prefix, community_id, source)
|
||||
SELECT mps.tenant_id,
|
||||
mps.module_id,
|
||||
(t.ordinality - 1)::int,
|
||||
(t.elem->>'prefix')::cidr,
|
||||
NULLIF(t.elem->>'community_id', '')::uuid,
|
||||
COALESCE(NULLIF(t.elem->>'source', ''), '')
|
||||
FROM module_prefix_snapshot mps
|
||||
CROSS JOIN LATERAL jsonb_array_elements(mps.prefixes_json) WITH ORDINALITY AS t(elem, ordinality)
|
||||
WHERE jsonb_typeof(mps.prefixes_json) = 'array'
|
||||
AND jsonb_array_length(mps.prefixes_json) > 0;
|
||||
SELECT tenant_id,
|
||||
module_id,
|
||||
(row_number() OVER (PARTITION BY tenant_id, module_id ORDER BY ordinality) - 1)::int,
|
||||
prefix,
|
||||
NULLIF(community_id, '')::uuid,
|
||||
COALESCE(source, '')
|
||||
FROM (
|
||||
SELECT mps.tenant_id,
|
||||
mps.module_id,
|
||||
t.ordinality,
|
||||
COALESCE(
|
||||
NULLIF(trim(t.elem->>'prefix'), ''),
|
||||
NULLIF(trim(t.elem->>'Prefix'), '')
|
||||
) AS prefix,
|
||||
COALESCE(
|
||||
NULLIF(trim(t.elem->>'community_id'), ''),
|
||||
NULLIF(trim(t.elem->>'CommunityID'), '')
|
||||
) AS community_id,
|
||||
COALESCE(
|
||||
NULLIF(trim(t.elem->>'source'), ''),
|
||||
NULLIF(trim(t.elem->>'Source'), ''),
|
||||
''
|
||||
) AS source
|
||||
FROM module_prefix_snapshot mps
|
||||
CROSS JOIN LATERAL jsonb_array_elements(mps.prefixes_json) WITH ORDINALITY AS t(elem, ordinality)
|
||||
WHERE jsonb_typeof(mps.prefixes_json) = 'array'
|
||||
AND jsonb_array_length(mps.prefixes_json) > 0
|
||||
) parsed
|
||||
WHERE parsed.prefix IS NOT NULL
|
||||
AND trim(parsed.prefix) <> '';
|
||||
|
||||
ALTER TABLE module_prefix_snapshot DROP COLUMN prefixes_json;
|
||||
|
||||
@@ -0,0 +1 @@
|
||||
DROP EXTENSION IF EXISTS pg_stat_statements;
|
||||
@@ -0,0 +1,2 @@
|
||||
-- pg_stat_statements requires shared_preload_libraries on the server; extension may fail on dev without restart.
|
||||
CREATE EXTENSION IF NOT EXISTS pg_stat_statements;
|
||||
@@ -0,0 +1,2 @@
|
||||
DROP TABLE IF EXISTS postgres_maintenance_audit;
|
||||
DROP TABLE IF EXISTS postgres_monitor_snapshot;
|
||||
@@ -0,0 +1,29 @@
|
||||
CREATE TABLE IF NOT EXISTS postgres_monitor_snapshot (
|
||||
id TEXT PRIMARY KEY,
|
||||
collected_at TIMESTAMPTZ NOT NULL,
|
||||
payload_json JSONB NOT NULL
|
||||
);
|
||||
|
||||
CREATE INDEX IF NOT EXISTS idx_postgres_monitor_snapshot_collected
|
||||
ON postgres_monitor_snapshot (collected_at DESC);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS postgres_maintenance_audit (
|
||||
id TEXT PRIMARY KEY,
|
||||
tenant_id TEXT,
|
||||
actor_prefix TEXT,
|
||||
kind TEXT NOT NULL,
|
||||
target_table TEXT,
|
||||
dry_run BOOLEAN NOT NULL DEFAULT false,
|
||||
status TEXT NOT NULL,
|
||||
detail_json JSONB,
|
||||
error_message TEXT,
|
||||
created_at TIMESTAMPTZ NOT NULL DEFAULT now(),
|
||||
started_at TIMESTAMPTZ,
|
||||
finished_at TIMESTAMPTZ,
|
||||
CONSTRAINT postgres_maintenance_audit_status_chk CHECK (
|
||||
status IN ('queued', 'running', 'succeeded', 'failed', 'cancelled')
|
||||
)
|
||||
);
|
||||
|
||||
CREATE INDEX IF NOT EXISTS idx_postgres_maintenance_audit_created
|
||||
ON postgres_maintenance_audit (created_at DESC);
|
||||
@@ -0,0 +1,7 @@
|
||||
DROP INDEX IF EXISTS idx_postgres_maintenance_audit_policy;
|
||||
|
||||
ALTER TABLE postgres_maintenance_audit DROP COLUMN IF EXISTS policy_id;
|
||||
|
||||
DROP TABLE IF EXISTS maintenance_policy_config_audit;
|
||||
|
||||
DROP TABLE IF EXISTS maintenance_policy;
|
||||
@@ -0,0 +1,47 @@
|
||||
CREATE TABLE IF NOT EXISTS maintenance_policy (
|
||||
id TEXT PRIMARY KEY,
|
||||
name TEXT NOT NULL,
|
||||
table_name TEXT NOT NULL,
|
||||
condition_sql TEXT NOT NULL DEFAULT 'true',
|
||||
retention_period_sec INTEGER,
|
||||
max_rows INTEGER,
|
||||
vacuum_strategy TEXT NOT NULL DEFAULT 'none',
|
||||
schedule_cron TEXT NOT NULL,
|
||||
enabled BOOLEAN NOT NULL DEFAULT true,
|
||||
dry_run_enabled BOOLEAN NOT NULL DEFAULT false,
|
||||
last_run_at TIMESTAMPTZ,
|
||||
last_status TEXT,
|
||||
last_error TEXT,
|
||||
created_at TIMESTAMPTZ NOT NULL DEFAULT now(),
|
||||
updated_at TIMESTAMPTZ NOT NULL DEFAULT now(),
|
||||
CONSTRAINT maintenance_policy_name_chk CHECK (length(trim(name)) > 0),
|
||||
CONSTRAINT maintenance_policy_table_name_chk CHECK (length(trim(table_name)) > 0),
|
||||
CONSTRAINT maintenance_policy_vacuum_strategy_chk CHECK (
|
||||
vacuum_strategy IN ('none', 'vacuum', 'analyze', 'vacuum_analyze', 'reindex')
|
||||
)
|
||||
);
|
||||
|
||||
CREATE INDEX IF NOT EXISTS idx_maintenance_policy_enabled
|
||||
ON maintenance_policy (enabled);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS maintenance_policy_config_audit (
|
||||
id TEXT PRIMARY KEY,
|
||||
policy_id TEXT,
|
||||
actor_prefix TEXT NOT NULL,
|
||||
action TEXT NOT NULL,
|
||||
before_json JSONB,
|
||||
after_json JSONB,
|
||||
created_at TIMESTAMPTZ NOT NULL DEFAULT now(),
|
||||
CONSTRAINT maintenance_policy_config_audit_action_chk CHECK (
|
||||
action IN ('create', 'update', 'delete')
|
||||
)
|
||||
);
|
||||
|
||||
CREATE INDEX IF NOT EXISTS idx_maintenance_policy_config_audit_created
|
||||
ON maintenance_policy_config_audit (created_at DESC);
|
||||
|
||||
ALTER TABLE postgres_maintenance_audit
|
||||
ADD COLUMN IF NOT EXISTS policy_id TEXT REFERENCES maintenance_policy (id) ON DELETE SET NULL;
|
||||
|
||||
CREATE INDEX IF NOT EXISTS idx_postgres_maintenance_audit_policy
|
||||
ON postgres_maintenance_audit (policy_id);
|
||||
@@ -0,0 +1 @@
|
||||
-- no-op
|
||||
@@ -0,0 +1 @@
|
||||
-- no-op: pg_stat_statements is PostgreSQL-only
|
||||
@@ -0,0 +1,2 @@
|
||||
DROP TABLE IF EXISTS postgres_maintenance_audit;
|
||||
DROP TABLE IF EXISTS postgres_monitor_snapshot;
|
||||
@@ -0,0 +1,20 @@
|
||||
CREATE TABLE IF NOT EXISTS postgres_monitor_snapshot (
|
||||
id TEXT PRIMARY KEY,
|
||||
collected_at TEXT NOT NULL,
|
||||
payload_json TEXT NOT NULL
|
||||
);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS postgres_maintenance_audit (
|
||||
id TEXT PRIMARY KEY,
|
||||
tenant_id TEXT,
|
||||
actor_prefix TEXT,
|
||||
kind TEXT NOT NULL,
|
||||
target_table TEXT,
|
||||
dry_run INTEGER NOT NULL DEFAULT 0,
|
||||
status TEXT NOT NULL,
|
||||
detail_json TEXT,
|
||||
error_message TEXT,
|
||||
created_at TEXT NOT NULL,
|
||||
started_at TEXT,
|
||||
finished_at TEXT
|
||||
);
|
||||
@@ -0,0 +1,5 @@
|
||||
ALTER TABLE postgres_maintenance_audit DROP COLUMN policy_id;
|
||||
|
||||
DROP TABLE IF EXISTS maintenance_policy_config_audit;
|
||||
|
||||
DROP TABLE IF EXISTS maintenance_policy;
|
||||
@@ -0,0 +1,29 @@
|
||||
CREATE TABLE IF NOT EXISTS maintenance_policy (
|
||||
id TEXT PRIMARY KEY,
|
||||
name TEXT NOT NULL,
|
||||
table_name TEXT NOT NULL,
|
||||
condition_sql TEXT NOT NULL DEFAULT 'true',
|
||||
retention_period_sec INTEGER,
|
||||
max_rows INTEGER,
|
||||
vacuum_strategy TEXT NOT NULL DEFAULT 'none',
|
||||
schedule_cron TEXT NOT NULL,
|
||||
enabled INTEGER NOT NULL DEFAULT 1,
|
||||
dry_run_enabled INTEGER NOT NULL DEFAULT 0,
|
||||
last_run_at TEXT,
|
||||
last_status TEXT,
|
||||
last_error TEXT,
|
||||
created_at TEXT NOT NULL,
|
||||
updated_at TEXT NOT NULL
|
||||
);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS maintenance_policy_config_audit (
|
||||
id TEXT PRIMARY KEY,
|
||||
policy_id TEXT,
|
||||
actor_prefix TEXT NOT NULL,
|
||||
action TEXT NOT NULL,
|
||||
before_json TEXT,
|
||||
after_json TEXT,
|
||||
created_at TEXT NOT NULL
|
||||
);
|
||||
|
||||
ALTER TABLE postgres_maintenance_audit ADD COLUMN policy_id TEXT;
|
||||
@@ -0,0 +1,14 @@
|
||||
{
|
||||
"$schema": "https://opencode.ai/config.json",
|
||||
"mcp": {
|
||||
"codegraph": {
|
||||
"type": "local",
|
||||
"command": [
|
||||
"codegraph",
|
||||
"serve",
|
||||
"--mcp"
|
||||
],
|
||||
"enabled": true
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,511 @@
|
||||
<script lang="ts">
|
||||
import { onMount } from 'svelte';
|
||||
import type { AuthSession } from '$lib/api/types.js';
|
||||
import type { PostgresTableRow } from '$lib/monitoring/postgres.js';
|
||||
import {
|
||||
createMaintenancePolicy,
|
||||
deleteMaintenancePolicy,
|
||||
fetchPolicyHints,
|
||||
listMaintenancePolicies,
|
||||
runMaintenancePolicy,
|
||||
updateMaintenancePolicy,
|
||||
type MaintenancePolicy,
|
||||
type MaintenancePolicyHints
|
||||
} from '$lib/maintenance/policy-api.js';
|
||||
import {
|
||||
emptyMaintenancePolicyForm,
|
||||
formToPayload,
|
||||
vacuumStrategies,
|
||||
type MaintenancePolicyForm
|
||||
} from '$lib/maintenance/policy.schema.js';
|
||||
import {
|
||||
filterAvailablePresets,
|
||||
isPresetAlreadyApplied,
|
||||
maintenancePolicyPresets,
|
||||
presetForm,
|
||||
type MaintenancePolicyPreset
|
||||
} from '$lib/maintenance/policy-presets.js';
|
||||
import { Button } from '$lib/ui/core/button/index.js';
|
||||
import {
|
||||
Card,
|
||||
CardContent,
|
||||
CardHeader,
|
||||
CardTitle,
|
||||
CardDescription
|
||||
} from '$lib/ui/core/card/index.js';
|
||||
import { Badge } from '$lib/ui/core/badge/index.js';
|
||||
import { Alert, AlertDescription, AlertTitle } from '$lib/ui/core/alert/index.js';
|
||||
import {
|
||||
Dialog,
|
||||
DialogContent,
|
||||
DialogHeader,
|
||||
DialogTitle,
|
||||
DialogFooter
|
||||
} from '$lib/ui/core/dialog/index.js';
|
||||
import { Switch } from '$lib/ui/core/switch/index.js';
|
||||
import { Label } from '$lib/ui/core/label/index.js';
|
||||
import { Checkbox } from '$lib/ui/core/checkbox/index.js';
|
||||
import FormField from '$lib/ui/patterns/form/form-field.svelte';
|
||||
import AppInput from '$lib/ui/patterns/form/app-input.svelte';
|
||||
import AppDataTable from '$lib/ui/patterns/data-table/app-data-table.svelte';
|
||||
import type { DataTableColumn } from '$lib/ui/patterns/data-table/types.js';
|
||||
import { confirm } from '$lib/ui/patterns/confirm/confirm-state.svelte.js';
|
||||
import { notify, notifyApiError } from '$lib/ui/app/toast.js';
|
||||
import Plus from '@lucide/svelte/icons/plus';
|
||||
import Pencil from '@lucide/svelte/icons/pencil';
|
||||
import Trash2 from '@lucide/svelte/icons/trash-2';
|
||||
import Play from '@lucide/svelte/icons/play';
|
||||
import FlaskConical from '@lucide/svelte/icons/flask-conical';
|
||||
import Info from '@lucide/svelte/icons/info';
|
||||
import Layers from '@lucide/svelte/icons/layers';
|
||||
|
||||
type Props = {
|
||||
session: AuthSession | null;
|
||||
tables: PostgresTableRow[];
|
||||
onJobQueued?: () => void | Promise<void>;
|
||||
};
|
||||
|
||||
let { session, tables = [], onJobQueued }: Props = $props();
|
||||
|
||||
let policies = $state<MaintenancePolicy[]>([]);
|
||||
let loading = $state(true);
|
||||
let dialogOpen = $state(false);
|
||||
let editTarget = $state<MaintenancePolicy | null>(null);
|
||||
let form = $state<MaintenancePolicyForm>(emptyMaintenancePolicyForm());
|
||||
let saving = $state(false);
|
||||
let hints = $state<MaintenancePolicyHints | null>(null);
|
||||
let hintsLoading = $state(false);
|
||||
let selectedPresetIds = $state<string[]>([]);
|
||||
let applyingPresets = $state(false);
|
||||
let activePresetId = $state<string | null>(null);
|
||||
|
||||
const creatablePresets = $derived(filterAvailablePresets(policies, selectedPresetIds));
|
||||
|
||||
function togglePresetSelection(id: string, checked: boolean) {
|
||||
if (checked) {
|
||||
if (!selectedPresetIds.includes(id)) {
|
||||
selectedPresetIds = [...selectedPresetIds, id];
|
||||
}
|
||||
} else {
|
||||
selectedPresetIds = selectedPresetIds.filter((x) => x !== id);
|
||||
}
|
||||
}
|
||||
|
||||
function applyPresetToForm(preset: MaintenancePolicyPreset) {
|
||||
form = presetForm(preset);
|
||||
activePresetId = preset.id;
|
||||
}
|
||||
|
||||
async function createSelectedPresets() {
|
||||
const toCreate = creatablePresets;
|
||||
if (toCreate.length === 0) {
|
||||
notify.error('Выберите пресеты, которые ещё не созданы');
|
||||
return;
|
||||
}
|
||||
applyingPresets = true;
|
||||
let created = 0;
|
||||
try {
|
||||
for (const preset of toCreate) {
|
||||
await createMaintenancePolicy(formToPayload(preset.form));
|
||||
created++;
|
||||
}
|
||||
selectedPresetIds = selectedPresetIds.filter((id) => !toCreate.some((p) => p.id === id));
|
||||
notify.success(`Создано политик: ${created}`);
|
||||
await loadPolicies();
|
||||
} catch (e) {
|
||||
notifyApiError(e, created > 0 ? `Создано ${created} из ${toCreate.length}` : undefined);
|
||||
if (created > 0) await loadPolicies();
|
||||
} finally {
|
||||
applyingPresets = false;
|
||||
}
|
||||
}
|
||||
|
||||
const isOperator = $derived(session?.role === 'operator');
|
||||
|
||||
const tableOptions = $derived.by(() => {
|
||||
const names = new Set(tables.map((t) => t.relname));
|
||||
if (form.table_name.trim()) names.add(form.table_name.trim());
|
||||
return [...names].sort();
|
||||
});
|
||||
|
||||
const columns: DataTableColumn<MaintenancePolicy>[] = [
|
||||
{ id: 'name', label: 'Название', sortable: true, sortValue: (p) => p.name },
|
||||
{ id: 'table_name', label: 'Таблица', sortable: true, sortValue: (p) => p.table_name },
|
||||
{ id: 'schedule', label: 'Cron (UTC)' },
|
||||
{ id: 'status', label: 'Статус' },
|
||||
{ id: 'actions', label: '', class: 'w-40' }
|
||||
];
|
||||
|
||||
async function loadPolicies() {
|
||||
loading = true;
|
||||
try {
|
||||
policies = await listMaintenancePolicies();
|
||||
} catch (e) {
|
||||
notifyApiError(e, 'Не удалось загрузить политики');
|
||||
} finally {
|
||||
loading = false;
|
||||
}
|
||||
}
|
||||
|
||||
function openCreate() {
|
||||
editTarget = null;
|
||||
form = emptyMaintenancePolicyForm();
|
||||
hints = null;
|
||||
activePresetId = null;
|
||||
dialogOpen = true;
|
||||
}
|
||||
|
||||
function openEdit(p: MaintenancePolicy) {
|
||||
editTarget = p;
|
||||
form = {
|
||||
name: p.name,
|
||||
table_name: p.table_name,
|
||||
condition: p.condition || 'true',
|
||||
retention_period_sec: p.retention_period_sec ? String(p.retention_period_sec) : '',
|
||||
max_rows: p.max_rows ? String(p.max_rows) : '',
|
||||
vacuum_strategy: (vacuumStrategies.includes(
|
||||
p.vacuum_strategy as (typeof vacuumStrategies)[number]
|
||||
)
|
||||
? p.vacuum_strategy
|
||||
: 'none') as MaintenancePolicyForm['vacuum_strategy'],
|
||||
schedule: p.schedule,
|
||||
enabled: p.enabled,
|
||||
dry_run_enabled: p.dry_run_enabled
|
||||
};
|
||||
hints = null;
|
||||
dialogOpen = true;
|
||||
void loadHints(p.id);
|
||||
}
|
||||
|
||||
async function loadHints(id: string) {
|
||||
hintsLoading = true;
|
||||
try {
|
||||
hints = await fetchPolicyHints(id);
|
||||
} catch {
|
||||
hints = null;
|
||||
} finally {
|
||||
hintsLoading = false;
|
||||
}
|
||||
}
|
||||
|
||||
function requestDelete(p: MaintenancePolicy) {
|
||||
void confirm({
|
||||
title: `Удалить политику «${p.name}»?`,
|
||||
description: 'Расписание и очистка по этой политике прекратятся.',
|
||||
confirmLabel: 'Удалить',
|
||||
destructive: true,
|
||||
onConfirm: async () => {
|
||||
await deleteMaintenancePolicy(p.id);
|
||||
notify.success('Политика удалена');
|
||||
await loadPolicies();
|
||||
}
|
||||
});
|
||||
}
|
||||
|
||||
async function save() {
|
||||
if (!form.name.trim() || !form.table_name.trim() || !form.schedule.trim()) {
|
||||
notify.error('Заполните обязательные поля');
|
||||
return;
|
||||
}
|
||||
saving = true;
|
||||
try {
|
||||
const payload = formToPayload(form);
|
||||
if (editTarget) {
|
||||
await updateMaintenancePolicy(editTarget.id, payload);
|
||||
notify.success('Политика обновлена');
|
||||
} else {
|
||||
await createMaintenancePolicy(payload);
|
||||
notify.success('Политика создана');
|
||||
}
|
||||
dialogOpen = false;
|
||||
await loadPolicies();
|
||||
} catch (e) {
|
||||
notifyApiError(e);
|
||||
} finally {
|
||||
saving = false;
|
||||
}
|
||||
}
|
||||
|
||||
function queueRun(p: MaintenancePolicy, dryRun: boolean) {
|
||||
void confirm({
|
||||
title: dryRun ? `Dry-run: ${p.name}` : `Запуск: ${p.name}`,
|
||||
description: dryRun
|
||||
? 'Изменения в БД не применяются — только оценка.'
|
||||
: 'Задача будет поставлена в очередь jobs.',
|
||||
confirmLabel: dryRun ? 'Dry-run' : 'Запустить',
|
||||
destructive: !dryRun,
|
||||
onConfirm: async () => {
|
||||
const res = await runMaintenancePolicy(p.id, dryRun);
|
||||
notify.success(`Задача ${res.job_id}`);
|
||||
await onJobQueued?.();
|
||||
}
|
||||
});
|
||||
}
|
||||
|
||||
function statusBadge(p: MaintenancePolicy) {
|
||||
if (!p.enabled) return 'выкл';
|
||||
if (p.dry_run_enabled) return 'dry-run sched';
|
||||
return p.last_status || '—';
|
||||
}
|
||||
|
||||
onMount(() => {
|
||||
void loadPolicies();
|
||||
});
|
||||
</script>
|
||||
|
||||
{#if !isOperator}
|
||||
<Alert>
|
||||
<AlertTitle>Только operator</AlertTitle>
|
||||
<AlertDescription>Политики обслуживания БД настраиваются с ролью operator.</AlertDescription>
|
||||
</Alert>
|
||||
{/if}
|
||||
|
||||
<Card>
|
||||
<CardHeader
|
||||
class="flex flex-col gap-3 border-b py-3 sm:flex-row sm:items-center sm:justify-between"
|
||||
>
|
||||
<div>
|
||||
<CardTitle>Политики обслуживания</CardTitle>
|
||||
<CardDescription>
|
||||
Единственный источник конфигурации retention, vacuum и расписания (UTC cron).
|
||||
</CardDescription>
|
||||
</div>
|
||||
{#if isOperator}
|
||||
<Button size="sm" onclick={openCreate}><Plus class="size-4" /> Новая политика</Button>
|
||||
{/if}
|
||||
</CardHeader>
|
||||
<CardContent class="flex flex-col gap-4 pt-4">
|
||||
{#if isOperator}
|
||||
<div class="rounded-lg border border-border/80 bg-muted/20 p-4">
|
||||
<div class="mb-3 flex flex-wrap items-start justify-between gap-2">
|
||||
<div>
|
||||
<p class="flex items-center gap-2 text-sm font-medium">
|
||||
<Layers class="size-4 text-muted-foreground" />
|
||||
Пресеты стратегий
|
||||
</p>
|
||||
<p class="mt-1 text-xs text-muted-foreground">
|
||||
Выберите шаблоны и создайте политики одним действием или примените шаблон в форме.
|
||||
</p>
|
||||
</div>
|
||||
<Button
|
||||
size="sm"
|
||||
disabled={creatablePresets.length === 0 || applyingPresets}
|
||||
onclick={createSelectedPresets}
|
||||
>
|
||||
{applyingPresets ? 'Создание…' : `Создать выбранные (${creatablePresets.length})`}
|
||||
</Button>
|
||||
</div>
|
||||
<div class="grid gap-2 sm:grid-cols-2">
|
||||
{#each maintenancePolicyPresets as preset (preset.id)}
|
||||
{@const applied = isPresetAlreadyApplied(preset, policies)}
|
||||
{@const checked = selectedPresetIds.includes(preset.id)}
|
||||
<label
|
||||
class="flex cursor-pointer gap-3 rounded-md border border-border/60 bg-background p-3 transition-colors hover:bg-muted/30 has-[:disabled]:cursor-not-allowed has-[:disabled]:opacity-60"
|
||||
>
|
||||
<Checkbox
|
||||
id="preset-{preset.id}"
|
||||
{checked}
|
||||
disabled={applied}
|
||||
onCheckedChange={(v) => togglePresetSelection(preset.id, v === true)}
|
||||
/>
|
||||
<span class="min-w-0 flex-1 space-y-1">
|
||||
<span class="flex flex-wrap items-center gap-2">
|
||||
<span class="text-sm font-medium">{preset.label}</span>
|
||||
{#if applied}
|
||||
<Badge variant="outline" class="text-xs">уже есть</Badge>
|
||||
{/if}
|
||||
</span>
|
||||
<span class="block text-xs text-muted-foreground">{preset.description}</span>
|
||||
<span class="block font-mono text-[11px] text-muted-foreground">
|
||||
{preset.form.table_name} · cron {preset.form.schedule}
|
||||
</span>
|
||||
</span>
|
||||
<Button
|
||||
type="button"
|
||||
variant="ghost"
|
||||
size="sm"
|
||||
class="shrink-0 self-start"
|
||||
disabled={!isOperator}
|
||||
onclick={(e) => {
|
||||
e.preventDefault();
|
||||
e.stopPropagation();
|
||||
editTarget = null;
|
||||
applyPresetToForm(preset);
|
||||
dialogOpen = true;
|
||||
}}
|
||||
>
|
||||
В форму
|
||||
</Button>
|
||||
</label>
|
||||
{/each}
|
||||
</div>
|
||||
</div>
|
||||
{/if}
|
||||
|
||||
<AppDataTable
|
||||
{columns}
|
||||
rows={policies}
|
||||
rowKey={(p) => p.id}
|
||||
{loading}
|
||||
emptyTitle="Политики не созданы"
|
||||
emptyDescription="Добавьте первую политику через UI — это единственный способ настройки."
|
||||
>
|
||||
{#snippet cell({ row, column })}
|
||||
{#if column.id === 'status'}
|
||||
<Badge variant={row.enabled ? 'secondary' : 'outline'}>{statusBadge(row)}</Badge>
|
||||
{#if row.last_run_at}
|
||||
<p class="mt-1 text-xs text-muted-foreground">{row.last_run_at}</p>
|
||||
{/if}
|
||||
{:else if column.id === 'actions' && isOperator}
|
||||
<div class="flex flex-wrap gap-1">
|
||||
<Button
|
||||
variant="ghost"
|
||||
size="icon-sm"
|
||||
onclick={() => openEdit(row)}
|
||||
aria-label="Изменить"
|
||||
>
|
||||
<Pencil class="size-4" />
|
||||
</Button>
|
||||
<Button
|
||||
variant="ghost"
|
||||
size="icon-sm"
|
||||
onclick={() => queueRun(row, true)}
|
||||
aria-label="Dry-run"
|
||||
>
|
||||
<FlaskConical class="size-4" />
|
||||
</Button>
|
||||
<Button
|
||||
variant="ghost"
|
||||
size="icon-sm"
|
||||
onclick={() => queueRun(row, false)}
|
||||
aria-label="Run"
|
||||
>
|
||||
<Play class="size-4" />
|
||||
</Button>
|
||||
<Button
|
||||
variant="ghost"
|
||||
size="icon-sm"
|
||||
onclick={() => requestDelete(row)}
|
||||
aria-label="Удалить"
|
||||
>
|
||||
<Trash2 class="size-4" />
|
||||
</Button>
|
||||
</div>
|
||||
{:else if column.id === 'name'}
|
||||
{row.name}
|
||||
{:else if column.id === 'table_name'}
|
||||
{row.table_name}
|
||||
{:else if column.id === 'schedule'}
|
||||
<span class="font-mono text-xs">{row.schedule}</span>
|
||||
{:else if column.id !== 'actions'}
|
||||
—
|
||||
{/if}
|
||||
{/snippet}
|
||||
</AppDataTable>
|
||||
</CardContent>
|
||||
</Card>
|
||||
|
||||
<Dialog bind:open={dialogOpen}>
|
||||
<DialogContent class="max-h-[90vh] overflow-y-auto sm:max-w-lg">
|
||||
<DialogHeader>
|
||||
<DialogTitle>{editTarget ? 'Изменить политику' : 'Новая политика'}</DialogTitle>
|
||||
</DialogHeader>
|
||||
|
||||
{#if hints?.recommend_vacuum}
|
||||
<Alert class="border-warning/30 bg-warning/5">
|
||||
<Info class="text-warning" />
|
||||
<AlertTitle>Рекомендация</AlertTitle>
|
||||
<AlertDescription>{hints.detail ?? 'Рекомендуется VACUUM.'}</AlertDescription>
|
||||
</Alert>
|
||||
{:else if hintsLoading}
|
||||
<p class="text-sm text-muted-foreground">Загрузка подсказок pg_stat…</p>
|
||||
{/if}
|
||||
|
||||
{#if !editTarget}
|
||||
<div class="space-y-2">
|
||||
<p class="text-sm font-medium">Шаблон (опционально)</p>
|
||||
<div class="flex flex-wrap gap-2">
|
||||
{#each maintenancePolicyPresets as preset (preset.id)}
|
||||
<Button
|
||||
type="button"
|
||||
variant={activePresetId === preset.id ? 'secondary' : 'outline'}
|
||||
size="sm"
|
||||
class="h-auto max-w-full py-1.5 text-left whitespace-normal"
|
||||
disabled={!isOperator}
|
||||
onclick={() => applyPresetToForm(preset)}
|
||||
>
|
||||
{preset.label}
|
||||
</Button>
|
||||
{/each}
|
||||
</div>
|
||||
<p class="text-xs text-muted-foreground">
|
||||
Поля формы заполняются из шаблона; перед сохранением можно изменить любое значение.
|
||||
</p>
|
||||
</div>
|
||||
{/if}
|
||||
|
||||
<div class="grid gap-4 py-2">
|
||||
<FormField label="Название" id="mp-name" required>
|
||||
<AppInput bind:value={form.name} disabled={!isOperator} />
|
||||
</FormField>
|
||||
<FormField label="Таблица" id="mp-table" required>
|
||||
<select
|
||||
class="flex h-9 w-full rounded-md border border-input bg-background px-3 text-sm"
|
||||
bind:value={form.table_name}
|
||||
disabled={!isOperator}
|
||||
>
|
||||
<option value="">— выберите —</option>
|
||||
{#each tableOptions as name (name)}
|
||||
<option value={name}>{name}</option>
|
||||
{/each}
|
||||
</select>
|
||||
</FormField>
|
||||
<FormField label="Condition (SQL WHERE)" id="mp-condition" required>
|
||||
<textarea
|
||||
class="min-h-20 w-full rounded-md border border-input bg-background px-3 py-2 font-mono text-xs"
|
||||
bind:value={form.condition}
|
||||
disabled={!isOperator}
|
||||
></textarea>
|
||||
</FormField>
|
||||
<div class="grid gap-4 sm:grid-cols-2">
|
||||
<FormField label="Retention (сек)" id="mp-retention">
|
||||
<AppInput bind:value={form.retention_period_sec} type="number" disabled={!isOperator} />
|
||||
</FormField>
|
||||
<FormField label="Max rows (batch)" id="mp-max-rows">
|
||||
<AppInput bind:value={form.max_rows} type="number" disabled={!isOperator} />
|
||||
</FormField>
|
||||
</div>
|
||||
<FormField label="Vacuum strategy" id="mp-vacuum">
|
||||
<select
|
||||
class="flex h-9 w-full rounded-md border border-input bg-background px-3 text-sm"
|
||||
bind:value={form.vacuum_strategy}
|
||||
disabled={!isOperator}
|
||||
>
|
||||
{#each vacuumStrategies as s (s)}
|
||||
<option value={s}>{s}</option>
|
||||
{/each}
|
||||
</select>
|
||||
</FormField>
|
||||
<FormField label="Schedule (cron, UTC)" id="mp-schedule" required>
|
||||
<AppInput bind:value={form.schedule} class="font-mono" disabled={!isOperator} />
|
||||
</FormField>
|
||||
<div class="flex flex-wrap gap-6">
|
||||
<div class="flex items-center gap-2">
|
||||
<Switch id="mp-enabled" bind:checked={form.enabled} disabled={!isOperator} />
|
||||
<Label for="mp-enabled">Включена</Label>
|
||||
</div>
|
||||
<div class="flex items-center gap-2">
|
||||
<Switch id="mp-dry" bind:checked={form.dry_run_enabled} disabled={!isOperator} />
|
||||
<Label for="mp-dry">Scheduler только dry-run</Label>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<DialogFooter>
|
||||
<Button variant="outline" onclick={() => (dialogOpen = false)}>Отмена</Button>
|
||||
{#if isOperator}
|
||||
<Button onclick={save} disabled={saving}>{saving ? 'Сохранение…' : 'Сохранить'}</Button>
|
||||
{/if}
|
||||
</DialogFooter>
|
||||
</DialogContent>
|
||||
</Dialog>
|
||||
@@ -0,0 +1,436 @@
|
||||
<script lang="ts">
|
||||
import { onMount } from 'svelte';
|
||||
import { apiJSON } from '$lib/api/client.js';
|
||||
import type { AuthSession } from '$lib/api/types.js';
|
||||
import { notifyApiError } from '$lib/ui/app/toast.js';
|
||||
import MaintenancePoliciesTab from '$lib/components/monitoring/MaintenancePoliciesTab.svelte';
|
||||
import {
|
||||
POSTGRES_POLL_MS,
|
||||
POSTGRES_SLOW_POLL_MS,
|
||||
formatBytes,
|
||||
connUsagePct,
|
||||
type PostgresOverview,
|
||||
type PostgresQueriesResponse,
|
||||
type PostgresLockRow,
|
||||
type PostgresTableRow,
|
||||
type PostgresRecommendationsResponse,
|
||||
type PostgresMaintLog,
|
||||
type CorrelationResponse
|
||||
} from '$lib/monitoring/postgres.js';
|
||||
import { Tabs, TabsContent, TabsList, TabsTrigger } from '$lib/ui/core/tabs/index.js';
|
||||
import {
|
||||
Card,
|
||||
CardContent,
|
||||
CardHeader,
|
||||
CardTitle,
|
||||
CardDescription
|
||||
} from '$lib/ui/core/card/index.js';
|
||||
import { Button } from '$lib/ui/core/button/index.js';
|
||||
import { Badge } from '$lib/ui/core/badge/index.js';
|
||||
import { Alert, AlertDescription, AlertTitle } from '$lib/ui/core/alert/index.js';
|
||||
import {
|
||||
Table,
|
||||
TableBody,
|
||||
TableCell,
|
||||
TableHead,
|
||||
TableHeader,
|
||||
TableRow
|
||||
} from '$lib/ui/core/table/index.js';
|
||||
import { Switch } from '$lib/ui/core/switch/index.js';
|
||||
import { Label } from '$lib/ui/core/label/index.js';
|
||||
import Database from '@lucide/svelte/icons/database';
|
||||
import RefreshCw from '@lucide/svelte/icons/refresh-cw';
|
||||
|
||||
let pgTab = $state('overview');
|
||||
let autoRefresh = $state(true);
|
||||
let session = $state<AuthSession | null>(null);
|
||||
let unavailable = $state(false);
|
||||
|
||||
let overview = $state<PostgresOverview | null>(null);
|
||||
let queries = $state<PostgresQueriesResponse | null>(null);
|
||||
let locks = $state<PostgresLockRow[]>([]);
|
||||
let tables = $state<PostgresTableRow[]>([]);
|
||||
let recommendations = $state<PostgresRecommendationsResponse | null>(null);
|
||||
let maintLogs = $state<PostgresMaintLog[]>([]);
|
||||
let correlation = $state<CorrelationResponse | null>(null);
|
||||
let loading = $state(true);
|
||||
|
||||
async function loadCore() {
|
||||
try {
|
||||
overview = await apiJSON<PostgresOverview>('/v1/monitoring/postgres/overview');
|
||||
locks = (await apiJSON<{ items: PostgresLockRow[] }>('/v1/monitoring/postgres/locks')).items;
|
||||
unavailable = false;
|
||||
} catch (e) {
|
||||
unavailable = true;
|
||||
overview = null;
|
||||
throw e;
|
||||
}
|
||||
}
|
||||
|
||||
async function loadSlow() {
|
||||
queries = await apiJSON<PostgresQueriesResponse>('/v1/monitoring/postgres/queries?limit=20');
|
||||
tables = (
|
||||
await apiJSON<{ items: PostgresTableRow[] }>('/v1/monitoring/postgres/tables?limit=30')
|
||||
).items;
|
||||
recommendations = await apiJSON<PostgresRecommendationsResponse>(
|
||||
'/v1/monitoring/postgres/recommendations'
|
||||
);
|
||||
correlation = await apiJSON<CorrelationResponse>('/v1/monitoring/correlation?window=60');
|
||||
maintLogs = (
|
||||
await apiJSON<{ items: PostgresMaintLog[] }>('/v1/postgres/maintenance/logs?limit=20')
|
||||
).items;
|
||||
}
|
||||
|
||||
async function loadAll() {
|
||||
loading = true;
|
||||
try {
|
||||
await loadCore();
|
||||
await loadSlow();
|
||||
} catch (e) {
|
||||
notifyApiError(e, 'PostgreSQL monitoring');
|
||||
} finally {
|
||||
loading = false;
|
||||
}
|
||||
}
|
||||
|
||||
onMount(() => {
|
||||
void (async () => {
|
||||
try {
|
||||
session = await apiJSON<AuthSession>('/v1/auth/session');
|
||||
} catch {
|
||||
session = null;
|
||||
}
|
||||
await loadAll();
|
||||
})();
|
||||
});
|
||||
|
||||
$effect(() => {
|
||||
if (!autoRefresh || unavailable) return;
|
||||
const fast = setInterval(() => {
|
||||
void loadCore().catch(() => {});
|
||||
}, POSTGRES_POLL_MS);
|
||||
const slow = setInterval(() => {
|
||||
void loadSlow().catch(() => {});
|
||||
}, POSTGRES_SLOW_POLL_MS);
|
||||
return () => {
|
||||
clearInterval(fast);
|
||||
clearInterval(slow);
|
||||
};
|
||||
});
|
||||
</script>
|
||||
|
||||
<div class="flex flex-wrap items-center justify-between gap-3">
|
||||
<div class="flex items-center gap-2 text-sm text-muted-foreground">
|
||||
<Database class="size-4" />
|
||||
<span>Instance-level PostgreSQL (control plane)</span>
|
||||
</div>
|
||||
<div class="flex items-center gap-4">
|
||||
<div class="flex items-center gap-2">
|
||||
<Switch id="pg-auto" bind:checked={autoRefresh} />
|
||||
<Label for="pg-auto">Автообновление</Label>
|
||||
</div>
|
||||
<Button variant="outline" size="sm" onclick={() => loadAll()} disabled={loading}>
|
||||
<RefreshCw class="mr-1 size-4 {loading ? 'animate-spin' : ''}" />
|
||||
Обновить
|
||||
</Button>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
{#if unavailable}
|
||||
<Alert variant="destructive" class="mt-4">
|
||||
<AlertTitle>PostgreSQL недоступен</AlertTitle>
|
||||
<AlertDescription>
|
||||
Мониторинг требует <code class="text-xs">EVOBGP_DATABASE_URL</code> (не memory backend).
|
||||
</AlertDescription>
|
||||
</Alert>
|
||||
{:else}
|
||||
<Tabs bind:value={pgTab} class="mt-4">
|
||||
<TabsList>
|
||||
<TabsTrigger value="overview">Обзор</TabsTrigger>
|
||||
<TabsTrigger value="queries">Запросы</TabsTrigger>
|
||||
<TabsTrigger value="locks">Блокировки</TabsTrigger>
|
||||
<TabsTrigger value="tables">Таблицы</TabsTrigger>
|
||||
<TabsTrigger value="maintenance">Обслуживание</TabsTrigger>
|
||||
<TabsTrigger value="correlation">Корреляция</TabsTrigger>
|
||||
</TabsList>
|
||||
|
||||
<TabsContent value="overview" class="mt-4 space-y-4">
|
||||
{#if overview}
|
||||
<div class="grid gap-4 md:grid-cols-2 lg:grid-cols-4">
|
||||
<Card>
|
||||
<CardHeader class="pb-2">
|
||||
<CardTitle class="text-sm font-medium">Подключения</CardTitle>
|
||||
</CardHeader>
|
||||
<CardContent>
|
||||
<p class="text-2xl font-semibold tabular-nums">
|
||||
{overview.connections.active} / {overview.connections.max_connections}
|
||||
</p>
|
||||
<div class="mt-2 h-2 overflow-hidden rounded-full bg-muted">
|
||||
<div
|
||||
class="h-full bg-chart-1 transition-all"
|
||||
style="width: {connUsagePct(overview)}%"
|
||||
></div>
|
||||
</div>
|
||||
<p class="mt-1 text-xs text-muted-foreground">
|
||||
idle {overview.connections.idle}, total {overview.connections.total}
|
||||
</p>
|
||||
</CardContent>
|
||||
</Card>
|
||||
<Card>
|
||||
<CardHeader class="pb-2">
|
||||
<CardTitle class="text-sm font-medium">Cache hit</CardTitle>
|
||||
</CardHeader>
|
||||
<CardContent>
|
||||
<p class="text-2xl font-semibold tabular-nums">
|
||||
{overview.database.cache_hit_pct ?? '—'}%
|
||||
</p>
|
||||
<div class="mt-2 h-2 overflow-hidden rounded-full bg-muted">
|
||||
<div
|
||||
class="h-full bg-chart-2 transition-all"
|
||||
style="width: {overview.database.cache_hit_pct ?? 0}%"
|
||||
></div>
|
||||
</div>
|
||||
</CardContent>
|
||||
</Card>
|
||||
<Card>
|
||||
<CardHeader class="pb-2">
|
||||
<CardTitle class="text-sm font-medium">TPS (commits)</CardTitle>
|
||||
</CardHeader>
|
||||
<CardContent>
|
||||
<p class="text-2xl font-semibold tabular-nums">
|
||||
{overview.database.xact_commit.toLocaleString()}
|
||||
</p>
|
||||
<p class="text-xs text-muted-foreground">
|
||||
rollback {overview.database.xact_rollback.toLocaleString()}, deadlocks {overview
|
||||
.database.deadlocks}
|
||||
</p>
|
||||
</CardContent>
|
||||
</Card>
|
||||
<Card>
|
||||
<CardHeader class="pb-2">
|
||||
<CardTitle class="text-sm font-medium">Размер БД</CardTitle>
|
||||
</CardHeader>
|
||||
<CardContent>
|
||||
<p class="text-2xl font-semibold">{formatBytes(overview.database_size_bytes)}</p>
|
||||
<p class="text-xs text-muted-foreground">
|
||||
shared_buffers {overview.memory_settings.shared_buffers}
|
||||
</p>
|
||||
</CardContent>
|
||||
</Card>
|
||||
</div>
|
||||
{#if overview.replication?.length}
|
||||
<Card>
|
||||
<CardHeader>
|
||||
<CardTitle>Репликация</CardTitle>
|
||||
</CardHeader>
|
||||
<CardContent>
|
||||
<Table>
|
||||
<TableHeader>
|
||||
<TableRow>
|
||||
<TableHead>Адрес</TableHead>
|
||||
<TableHead>Состояние</TableHead>
|
||||
<TableHead>Lag ms</TableHead>
|
||||
</TableRow>
|
||||
</TableHeader>
|
||||
<TableBody>
|
||||
{#each overview.replication as r (r.client_addr ?? r.state)}
|
||||
<TableRow>
|
||||
<TableCell>{r.client_addr ?? '—'}</TableCell>
|
||||
<TableCell>{r.state}</TableCell>
|
||||
<TableCell>{r.lag_ms ?? '—'}</TableCell>
|
||||
</TableRow>
|
||||
{/each}
|
||||
</TableBody>
|
||||
</Table>
|
||||
</CardContent>
|
||||
</Card>
|
||||
{/if}
|
||||
{/if}
|
||||
</TabsContent>
|
||||
|
||||
<TabsContent value="queries" class="mt-4">
|
||||
<Card>
|
||||
<CardHeader>
|
||||
<CardTitle>Медленные запросы</CardTitle>
|
||||
<CardDescription>
|
||||
Источник: {queries?.source ?? '—'}
|
||||
{#if queries?.statements_available === false || (overview && !overview.pg_stat_statements_enabled)}
|
||||
· pg_stat_statements недоступен
|
||||
{/if}
|
||||
</CardDescription>
|
||||
</CardHeader>
|
||||
<CardContent>
|
||||
{#if queries?.statements_hint}
|
||||
<Alert class="mb-4">
|
||||
<AlertTitle>Нет статистики запросов</AlertTitle>
|
||||
<AlertDescription>{queries.statements_hint}</AlertDescription>
|
||||
</Alert>
|
||||
{/if}
|
||||
<Table>
|
||||
<TableHeader>
|
||||
<TableRow>
|
||||
<TableHead>mean ms</TableHead>
|
||||
<TableHead>calls</TableHead>
|
||||
<TableHead>query</TableHead>
|
||||
</TableRow>
|
||||
</TableHeader>
|
||||
<TableBody>
|
||||
{#each queries?.items ?? [] as q (q.queryid ?? q.query)}
|
||||
<TableRow>
|
||||
<TableCell class="tabular-nums">{q.mean_exec_ms.toFixed(1)}</TableCell>
|
||||
<TableCell>{q.calls}</TableCell>
|
||||
<TableCell class="max-w-md truncate font-mono text-xs">{q.query}</TableCell>
|
||||
</TableRow>
|
||||
{:else}
|
||||
<TableRow>
|
||||
<TableCell colspan={3} class="text-muted-foreground">Нет данных</TableCell>
|
||||
</TableRow>
|
||||
{/each}
|
||||
</TableBody>
|
||||
</Table>
|
||||
</CardContent>
|
||||
</Card>
|
||||
</TabsContent>
|
||||
|
||||
<TabsContent value="locks" class="mt-4">
|
||||
<Card>
|
||||
<CardHeader>
|
||||
<CardTitle>Блокировки</CardTitle>
|
||||
</CardHeader>
|
||||
<CardContent>
|
||||
<Table>
|
||||
<TableHeader>
|
||||
<TableRow>
|
||||
<TableHead>pid</TableHead>
|
||||
<TableHead>mode</TableHead>
|
||||
<TableHead>granted</TableHead>
|
||||
<TableHead>query</TableHead>
|
||||
</TableRow>
|
||||
</TableHeader>
|
||||
<TableBody>
|
||||
{#each locks as l (l.pid)}
|
||||
<TableRow>
|
||||
<TableCell>{l.pid}</TableCell>
|
||||
<TableCell>
|
||||
<Badge variant={l.blocked ? 'destructive' : 'secondary'}>{l.mode}</Badge>
|
||||
</TableCell>
|
||||
<TableCell>{l.granted ? 'да' : 'нет'}</TableCell>
|
||||
<TableCell class="max-w-lg truncate font-mono text-xs">{l.query ?? '—'}</TableCell
|
||||
>
|
||||
</TableRow>
|
||||
{:else}
|
||||
<TableRow>
|
||||
<TableCell colspan={4} class="text-muted-foreground"
|
||||
>Нет активных блокировок</TableCell
|
||||
>
|
||||
</TableRow>
|
||||
{/each}
|
||||
</TableBody>
|
||||
</Table>
|
||||
</CardContent>
|
||||
</Card>
|
||||
</TabsContent>
|
||||
|
||||
<TabsContent value="tables" class="mt-4 space-y-4">
|
||||
<Card>
|
||||
<CardHeader>
|
||||
<CardTitle>Таблицы и хранилище</CardTitle>
|
||||
</CardHeader>
|
||||
<CardContent>
|
||||
<Table>
|
||||
<TableHeader>
|
||||
<TableRow>
|
||||
<TableHead>table</TableHead>
|
||||
<TableHead>size</TableHead>
|
||||
<TableHead>seq_scan</TableHead>
|
||||
<TableHead>idx_scan</TableHead>
|
||||
<TableHead>bloat</TableHead>
|
||||
</TableRow>
|
||||
</TableHeader>
|
||||
<TableBody>
|
||||
{#each tables as t (t.relname)}
|
||||
<TableRow>
|
||||
<TableCell class="font-mono text-xs">{t.relname}</TableCell>
|
||||
<TableCell>{formatBytes(t.total_bytes)}</TableCell>
|
||||
<TableCell>{t.seq_scan}</TableCell>
|
||||
<TableCell>{t.idx_scan}</TableCell>
|
||||
<TableCell>{(t.bloat_ratio ?? 0).toFixed(2)}</TableCell>
|
||||
</TableRow>
|
||||
{/each}
|
||||
</TableBody>
|
||||
</Table>
|
||||
</CardContent>
|
||||
</Card>
|
||||
{#if recommendations?.items?.length}
|
||||
<Card>
|
||||
<CardHeader>
|
||||
<CardTitle>Рекомендации</CardTitle>
|
||||
</CardHeader>
|
||||
<CardContent class="space-y-2">
|
||||
{#each recommendations.items as item (item.code + item.title)}
|
||||
<Alert>
|
||||
<AlertTitle>{item.title}</AlertTitle>
|
||||
<AlertDescription>{item.detail}</AlertDescription>
|
||||
</Alert>
|
||||
{/each}
|
||||
</CardContent>
|
||||
</Card>
|
||||
{/if}
|
||||
</TabsContent>
|
||||
|
||||
<TabsContent value="maintenance" class="mt-4 space-y-4">
|
||||
<MaintenancePoliciesTab {session} {tables} onJobQueued={loadSlow} />
|
||||
<Card>
|
||||
<CardHeader>
|
||||
<CardTitle>Журнал обслуживания</CardTitle>
|
||||
</CardHeader>
|
||||
<CardContent>
|
||||
<Table>
|
||||
<TableHeader>
|
||||
<TableRow>
|
||||
<TableHead>время</TableHead>
|
||||
<TableHead>kind</TableHead>
|
||||
<TableHead>status</TableHead>
|
||||
<TableHead>dry_run</TableHead>
|
||||
</TableRow>
|
||||
</TableHeader>
|
||||
<TableBody>
|
||||
{#each maintLogs as log (log.id)}
|
||||
<TableRow>
|
||||
<TableCell class="text-xs">{log.created_at}</TableCell>
|
||||
<TableCell>{log.kind}</TableCell>
|
||||
<TableCell>{log.status}</TableCell>
|
||||
<TableCell>{log.dry_run ? 'да' : 'нет'}</TableCell>
|
||||
</TableRow>
|
||||
{:else}
|
||||
<TableRow>
|
||||
<TableCell colspan={4} class="text-muted-foreground">Пусто</TableCell>
|
||||
</TableRow>
|
||||
{/each}
|
||||
</TableBody>
|
||||
</Table>
|
||||
</CardContent>
|
||||
</Card>
|
||||
</TabsContent>
|
||||
|
||||
<TabsContent value="correlation" class="mt-4">
|
||||
<Card>
|
||||
<CardHeader>
|
||||
<CardTitle>Корреляция (1ч)</CardTitle>
|
||||
<CardDescription>Pipeline refresh p99 vs cache hit по минутам</CardDescription>
|
||||
</CardHeader>
|
||||
<CardContent class="space-y-3">
|
||||
{#each correlation?.points ?? [] as p (p.timestamp)}
|
||||
<div class="grid gap-2 rounded-md border p-2 text-xs md:grid-cols-3">
|
||||
<span>{p.timestamp}</span>
|
||||
<span>p99 refresh: {p.pipeline_refresh_p99_ms?.toFixed(0) ?? '—'} ms</span>
|
||||
<span>cache hit: {p.cache_hit_pct?.toFixed(1) ?? '—'}%</span>
|
||||
</div>
|
||||
{:else}
|
||||
<p class="text-muted-foreground">Нет точек за окно</p>
|
||||
{/each}
|
||||
</CardContent>
|
||||
</Card>
|
||||
</TabsContent>
|
||||
</Tabs>
|
||||
{/if}
|
||||
@@ -0,0 +1,68 @@
|
||||
import { apiJSON, apiMutate } from '$lib/api/client.js';
|
||||
|
||||
export type MaintenancePolicy = {
|
||||
id: string;
|
||||
name: string;
|
||||
table_name: string;
|
||||
condition: string;
|
||||
retention_period_sec?: number;
|
||||
max_rows?: number;
|
||||
vacuum_strategy: string;
|
||||
schedule: string;
|
||||
enabled: boolean;
|
||||
dry_run_enabled: boolean;
|
||||
last_run_at?: string;
|
||||
last_status?: string;
|
||||
last_error?: string;
|
||||
created_at?: string;
|
||||
updated_at?: string;
|
||||
};
|
||||
|
||||
export type MaintenancePolicyHints = {
|
||||
table_name: string;
|
||||
n_dead_tup: number;
|
||||
bloat_ratio?: number;
|
||||
last_autovacuum?: string;
|
||||
recommend_vacuum: boolean;
|
||||
detail?: string;
|
||||
};
|
||||
|
||||
export type MaintenancePoliciesResponse = {
|
||||
items: MaintenancePolicy[];
|
||||
next_cursor?: string;
|
||||
has_more?: boolean;
|
||||
};
|
||||
|
||||
export async function listMaintenancePolicies(limit = 100): Promise<MaintenancePolicy[]> {
|
||||
const r = await apiJSON<MaintenancePoliciesResponse>(`/v1/maintenance/policies?limit=${limit}`);
|
||||
return r.items ?? [];
|
||||
}
|
||||
|
||||
export async function createMaintenancePolicy(
|
||||
body: Record<string, unknown>
|
||||
): Promise<MaintenancePolicy> {
|
||||
return apiMutate<MaintenancePolicy>('/v1/maintenance/policies', 'POST', body);
|
||||
}
|
||||
|
||||
export async function updateMaintenancePolicy(
|
||||
id: string,
|
||||
body: Record<string, unknown>
|
||||
): Promise<MaintenancePolicy> {
|
||||
return apiMutate<MaintenancePolicy>(`/v1/maintenance/policies/${id}`, 'PATCH', body);
|
||||
}
|
||||
|
||||
export async function deleteMaintenancePolicy(id: string): Promise<void> {
|
||||
await apiMutate(`/v1/maintenance/policies/${id}`, 'DELETE', undefined, { idempotent: false });
|
||||
}
|
||||
|
||||
export async function runMaintenancePolicy(
|
||||
id: string,
|
||||
dryRun: boolean
|
||||
): Promise<{ job_id: string }> {
|
||||
const path = dryRun ? '/v1/maintenance/dry-run' : '/v1/maintenance/run';
|
||||
return apiMutate<{ job_id: string; status: string }>(path, 'POST', { policy_id: id });
|
||||
}
|
||||
|
||||
export async function fetchPolicyHints(id: string): Promise<MaintenancePolicyHints> {
|
||||
return apiJSON<MaintenancePolicyHints>(`/v1/maintenance/policies/${id}/hints`);
|
||||
}
|
||||
@@ -0,0 +1,130 @@
|
||||
import type { MaintenancePolicy } from '$lib/maintenance/policy-api.js';
|
||||
import type { MaintenancePolicyForm } from '$lib/maintenance/policy.schema.js';
|
||||
|
||||
const DAY_SEC = 86_400;
|
||||
|
||||
/** Рекомендуемый шаблон политики (только UI; в БД не seed'ится). */
|
||||
export type MaintenancePolicyPreset = {
|
||||
id: string;
|
||||
label: string;
|
||||
description: string;
|
||||
/** Подсказка: таблица должна быть видна в pg_stat (не блокирует создание). */
|
||||
tableHint?: string;
|
||||
form: MaintenancePolicyForm;
|
||||
};
|
||||
|
||||
/** Базовые пресеты EvoBGP — оператор выбирает, какие создать. */
|
||||
export const maintenancePolicyPresets: MaintenancePolicyPreset[] = [
|
||||
{
|
||||
id: 'job_audit_retention',
|
||||
label: 'Job audit — retention 90d',
|
||||
description:
|
||||
'Удаляет завершённые записи job_audit старше 90 дней; VACUUM ANALYZE после очистки. Расписание 03:00 UTC.',
|
||||
tableHint: 'job_audit',
|
||||
form: {
|
||||
name: 'Job audit retention (90d)',
|
||||
table_name: 'job_audit',
|
||||
condition: "status IN ('succeeded', 'failed', 'cancelled')",
|
||||
retention_period_sec: String(90 * DAY_SEC),
|
||||
max_rows: '10000',
|
||||
vacuum_strategy: 'vacuum_analyze',
|
||||
schedule: '0 3 * * *',
|
||||
enabled: true,
|
||||
dry_run_enabled: true
|
||||
}
|
||||
},
|
||||
{
|
||||
id: 'postgres_maintenance_audit_retention',
|
||||
label: 'Maintenance audit — 30d',
|
||||
description: 'Очищает postgres_maintenance_audit старше 30 дней без vacuum.',
|
||||
tableHint: 'postgres_maintenance_audit',
|
||||
form: {
|
||||
name: 'Postgres maintenance audit (30d)',
|
||||
table_name: 'postgres_maintenance_audit',
|
||||
condition: 'true',
|
||||
retention_period_sec: String(30 * DAY_SEC),
|
||||
max_rows: '5000',
|
||||
vacuum_strategy: 'none',
|
||||
schedule: '0 4 * * *',
|
||||
enabled: true,
|
||||
dry_run_enabled: true
|
||||
}
|
||||
},
|
||||
{
|
||||
id: 'job_audit_vacuum_weekly',
|
||||
label: 'Job audit — VACUUM weekly',
|
||||
description: 'Только VACUUM ANALYZE job_audit по воскресеньям, без удаления строк.',
|
||||
tableHint: 'job_audit',
|
||||
form: {
|
||||
name: 'Job audit vacuum (weekly)',
|
||||
table_name: 'job_audit',
|
||||
condition: 'true',
|
||||
retention_period_sec: '',
|
||||
max_rows: '',
|
||||
vacuum_strategy: 'vacuum_analyze',
|
||||
schedule: '0 2 * * 0',
|
||||
enabled: true,
|
||||
dry_run_enabled: false
|
||||
}
|
||||
},
|
||||
{
|
||||
id: 'postgres_monitor_snapshot',
|
||||
label: 'PG monitor snapshots — 14d',
|
||||
description:
|
||||
'Удаляет снимки postgres_monitor_snapshot старше 14 дней (фильтр по collected_at в condition).',
|
||||
tableHint: 'postgres_monitor_snapshot',
|
||||
form: {
|
||||
name: 'Postgres monitor snapshots (14d)',
|
||||
table_name: 'postgres_monitor_snapshot',
|
||||
condition: "collected_at < NOW() - INTERVAL '14 days'",
|
||||
retention_period_sec: '',
|
||||
max_rows: '10000',
|
||||
vacuum_strategy: 'none',
|
||||
schedule: '0 5 * * *',
|
||||
enabled: true,
|
||||
dry_run_enabled: true
|
||||
}
|
||||
},
|
||||
{
|
||||
id: 'config_revision_retention',
|
||||
label: 'Config revisions — 180d',
|
||||
description:
|
||||
'Долгое хранение старых config_revision (180d). Перед включением проверьте revision_retention в настройках.',
|
||||
tableHint: 'config_revision',
|
||||
form: {
|
||||
name: 'Config revision retention (180d)',
|
||||
table_name: 'config_revision',
|
||||
condition: 'true',
|
||||
retention_period_sec: String(180 * DAY_SEC),
|
||||
max_rows: '5000',
|
||||
vacuum_strategy: 'vacuum',
|
||||
schedule: '0 6 * * 0',
|
||||
enabled: false,
|
||||
dry_run_enabled: true
|
||||
}
|
||||
}
|
||||
];
|
||||
|
||||
export function presetForm(preset: MaintenancePolicyPreset): MaintenancePolicyForm {
|
||||
return structuredClone(preset.form);
|
||||
}
|
||||
|
||||
/** Политика с тем же именем и таблицей считается уже созданной из пресета. */
|
||||
export function isPresetAlreadyApplied(
|
||||
preset: MaintenancePolicyPreset,
|
||||
policies: MaintenancePolicy[]
|
||||
): boolean {
|
||||
return policies.some(
|
||||
(p) => p.name === preset.form.name.trim() && p.table_name === preset.form.table_name.trim()
|
||||
);
|
||||
}
|
||||
|
||||
export function filterAvailablePresets(
|
||||
policies: MaintenancePolicy[],
|
||||
selected: Iterable<string>
|
||||
): MaintenancePolicyPreset[] {
|
||||
const ids = new Set(selected);
|
||||
return maintenancePolicyPresets.filter(
|
||||
(p) => ids.has(p.id) && !isPresetAlreadyApplied(p, policies)
|
||||
);
|
||||
}
|
||||
@@ -0,0 +1,58 @@
|
||||
import { z } from 'zod';
|
||||
|
||||
export const vacuumStrategies = ['none', 'vacuum', 'analyze', 'vacuum_analyze', 'reindex'] as const;
|
||||
|
||||
export type VacuumStrategy = (typeof vacuumStrategies)[number];
|
||||
|
||||
export const maintenancePolicySchema = z.object({
|
||||
name: z.string().trim().min(1, 'Укажите название'),
|
||||
table_name: z.string().trim().min(1, 'Укажите таблицу'),
|
||||
condition: z
|
||||
.string()
|
||||
.trim()
|
||||
.min(1, 'Укажите условие')
|
||||
.refine((v) => !/[;]|--|\/\*/.test(v), 'Недопустимые символы в condition'),
|
||||
retention_period_sec: z.string().optional(),
|
||||
max_rows: z.string().optional(),
|
||||
vacuum_strategy: z.enum(vacuumStrategies),
|
||||
schedule: z.string().trim().min(1, 'Укажите cron (UTC)'),
|
||||
enabled: z.boolean(),
|
||||
dry_run_enabled: z.boolean()
|
||||
});
|
||||
|
||||
export type MaintenancePolicyForm = z.infer<typeof maintenancePolicySchema>;
|
||||
|
||||
export function emptyMaintenancePolicyForm(): MaintenancePolicyForm {
|
||||
return {
|
||||
name: '',
|
||||
table_name: '',
|
||||
condition: 'true',
|
||||
retention_period_sec: '',
|
||||
max_rows: '10000',
|
||||
vacuum_strategy: 'none',
|
||||
schedule: '0 3 * * *',
|
||||
enabled: true,
|
||||
dry_run_enabled: true
|
||||
};
|
||||
}
|
||||
|
||||
export function parseOptionalInt(raw: string | undefined): number | undefined {
|
||||
const v = String(raw ?? '').trim();
|
||||
if (!v) return undefined;
|
||||
const n = Number(v);
|
||||
return Number.isFinite(n) && n > 0 ? Math.floor(n) : undefined;
|
||||
}
|
||||
|
||||
export function formToPayload(form: MaintenancePolicyForm) {
|
||||
return {
|
||||
name: form.name.trim(),
|
||||
table_name: form.table_name.trim(),
|
||||
condition: form.condition.trim() || 'true',
|
||||
retention_period_sec: parseOptionalInt(form.retention_period_sec),
|
||||
max_rows: parseOptionalInt(form.max_rows),
|
||||
vacuum_strategy: form.vacuum_strategy,
|
||||
schedule: form.schedule.trim(),
|
||||
enabled: form.enabled,
|
||||
dry_run_enabled: form.dry_run_enabled
|
||||
};
|
||||
}
|
||||
@@ -0,0 +1,118 @@
|
||||
/** Types and helpers for PostgreSQL monitoring API. */
|
||||
|
||||
export const POSTGRES_POLL_MS = 20_000;
|
||||
export const POSTGRES_SLOW_POLL_MS = 60_000;
|
||||
|
||||
export type PostgresOverview = {
|
||||
collected_at: string;
|
||||
connections: {
|
||||
active: number;
|
||||
idle: number;
|
||||
total: number;
|
||||
max_connections: number;
|
||||
};
|
||||
database: {
|
||||
backends: number;
|
||||
xact_commit: number;
|
||||
xact_rollback: number;
|
||||
deadlocks: number;
|
||||
blks_hit: number;
|
||||
blks_read: number;
|
||||
cache_hit_pct: number;
|
||||
};
|
||||
database_size_bytes: number;
|
||||
memory_settings: {
|
||||
shared_buffers: string;
|
||||
work_mem: string;
|
||||
effective_cache_size: string;
|
||||
};
|
||||
replication: Array<{
|
||||
client_addr?: string;
|
||||
state: string;
|
||||
sync_state?: string;
|
||||
lag_ms?: number;
|
||||
}>;
|
||||
pg_stat_statements_enabled: boolean;
|
||||
};
|
||||
|
||||
export type PostgresQueryRow = {
|
||||
queryid?: number;
|
||||
query: string;
|
||||
calls: number;
|
||||
total_exec_ms: number;
|
||||
mean_exec_ms: number;
|
||||
rows: number;
|
||||
};
|
||||
|
||||
export type PostgresQueriesResponse = {
|
||||
collected_at: string;
|
||||
source: string;
|
||||
items: PostgresQueryRow[];
|
||||
statements_available?: boolean;
|
||||
statements_hint?: string;
|
||||
};
|
||||
|
||||
export type PostgresLockRow = {
|
||||
locktype: string;
|
||||
mode: string;
|
||||
granted: boolean;
|
||||
pid: number;
|
||||
usename?: string;
|
||||
state?: string;
|
||||
query?: string;
|
||||
blocked: boolean;
|
||||
};
|
||||
|
||||
export type PostgresTableRow = {
|
||||
relname: string;
|
||||
total_bytes: number;
|
||||
idx_scan: number;
|
||||
seq_scan: number;
|
||||
n_dead_tup: number;
|
||||
bloat_ratio?: number;
|
||||
last_autovacuum?: string;
|
||||
};
|
||||
|
||||
export type PostgresRecommendation = {
|
||||
severity: string;
|
||||
code: string;
|
||||
title: string;
|
||||
detail: string;
|
||||
refs?: string[];
|
||||
};
|
||||
|
||||
export type PostgresRecommendationsResponse = {
|
||||
collected_at: string;
|
||||
items: PostgresRecommendation[];
|
||||
};
|
||||
|
||||
export type PostgresMaintLog = {
|
||||
id: string;
|
||||
kind: string;
|
||||
target_table?: string;
|
||||
dry_run: boolean;
|
||||
status: string;
|
||||
error?: string;
|
||||
created_at: string;
|
||||
};
|
||||
|
||||
export type CorrelationResponse = {
|
||||
window_minutes: number;
|
||||
points: Array<{
|
||||
timestamp: string;
|
||||
pipeline_refresh_p99_ms?: number;
|
||||
cache_hit_pct?: number;
|
||||
}>;
|
||||
};
|
||||
|
||||
export function formatBytes(n: number): string {
|
||||
if (n >= 1 << 30) return `${(n / (1 << 30)).toFixed(1)} GiB`;
|
||||
if (n >= 1 << 20) return `${(n / (1 << 20)).toFixed(1)} MiB`;
|
||||
if (n >= 1 << 10) return `${(n / (1 << 10)).toFixed(1)} KiB`;
|
||||
return `${n} B`;
|
||||
}
|
||||
|
||||
export function connUsagePct(ov: PostgresOverview | null): number {
|
||||
if (!ov?.connections.max_connections) return 0;
|
||||
return Math.min(100, (ov.connections.total / ov.connections.max_connections) * 100);
|
||||
}
|
||||
@@ -23,6 +23,8 @@ export function jobKindTitle(job: JobRow, moduleNameById?: ReadonlyMap<string, s
|
||||
return 'Откат ревизии';
|
||||
case 'bird_reload':
|
||||
return 'Перезагрузка BIRD';
|
||||
case 'maintenance_policy_run':
|
||||
return 'Обслуживание PostgreSQL (политика)';
|
||||
default:
|
||||
return job.kind;
|
||||
}
|
||||
|
||||
@@ -1,7 +1,18 @@
|
||||
import { z } from 'zod';
|
||||
|
||||
export const revisionSettingsSchema = z.object({
|
||||
revision_retention_minutes: z.string().refine(
|
||||
/** HTML type=number binds number; API/store may return number — normalize to string for validation. */
|
||||
function retentionMinutesInput(val: unknown): string {
|
||||
if (val === undefined || val === null) return '';
|
||||
if (typeof val === 'number') {
|
||||
if (!Number.isFinite(val)) return '';
|
||||
return String(Math.trunc(val));
|
||||
}
|
||||
return String(val);
|
||||
}
|
||||
|
||||
const revisionRetentionMinutes = z.preprocess(
|
||||
retentionMinutesInput,
|
||||
z.string().refine(
|
||||
(v) => {
|
||||
const s = v.trim();
|
||||
if (s === '') return true;
|
||||
@@ -10,6 +21,10 @@ export const revisionSettingsSchema = z.object({
|
||||
},
|
||||
{ message: 'TTL ревизий должен быть целым числом от 15 до 43200 минут' }
|
||||
)
|
||||
);
|
||||
|
||||
export const revisionSettingsSchema = z.object({
|
||||
revision_retention_minutes: revisionRetentionMinutes
|
||||
});
|
||||
|
||||
export type RevisionSettingsForm = z.infer<typeof revisionSettingsSchema>;
|
||||
|
||||
@@ -49,6 +49,8 @@
|
||||
import KpiMetricsGrid from '$lib/ui/patterns/kpi/kpi-metrics-grid.svelte';
|
||||
import ScrollPreBlock from '$lib/components/app/scroll-pre-block.svelte';
|
||||
import PageHeader from '$lib/ui/app/page-header/page-header.svelte';
|
||||
import MonitoringPostgresTab from '$lib/components/monitoring/MonitoringPostgresTab.svelte';
|
||||
import { Tabs, TabsContent, TabsList, TabsTrigger } from '$lib/ui/core/tabs/index.js';
|
||||
import { cn } from '$lib/utils.js';
|
||||
import RefreshCw from '@lucide/svelte/icons/refresh-cw';
|
||||
import Gauge from '@lucide/svelte/icons/gauge';
|
||||
@@ -80,6 +82,7 @@
|
||||
let lastUpdated = $state<Date | null>(null);
|
||||
let initialLoading = $state(true);
|
||||
let refreshing = $state(false);
|
||||
let mainTab = $state('system');
|
||||
|
||||
const statAccents = [
|
||||
{
|
||||
@@ -322,337 +325,362 @@
|
||||
{/snippet}
|
||||
</PageHeader>
|
||||
|
||||
{#if !initialLoading}
|
||||
{#if overallStatus === 'ok'}
|
||||
<Alert class="border-success/30 bg-success/5">
|
||||
<CheckCircle class="text-success" />
|
||||
<AlertTitle>Система в норме</AlertTitle>
|
||||
<AlertDescription>{overallHint}</AlertDescription>
|
||||
</Alert>
|
||||
{:else if overallStatus === 'warn'}
|
||||
<Alert class="border-warning/30 bg-warning/5">
|
||||
<AlertTriangle class="text-warning" />
|
||||
<AlertTitle>Требуется внимание</AlertTitle>
|
||||
<AlertDescription>{overallHint}</AlertDescription>
|
||||
</Alert>
|
||||
{:else if overallStatus === 'error'}
|
||||
<Alert variant="destructive">
|
||||
<XCircle />
|
||||
<AlertTitle>Обнаружена проблема</AlertTitle>
|
||||
<AlertDescription>{overallHint}</AlertDescription>
|
||||
</Alert>
|
||||
{/if}
|
||||
{/if}
|
||||
<Tabs bind:value={mainTab}>
|
||||
<TabsList>
|
||||
<TabsTrigger value="system">Система</TabsTrigger>
|
||||
<TabsTrigger value="postgres">PostgreSQL</TabsTrigger>
|
||||
</TabsList>
|
||||
|
||||
<KpiMetricsGrid
|
||||
cards={kpiCards}
|
||||
loading={initialLoading}
|
||||
skeletonCount={4}
|
||||
class="sm:grid-cols-2 xl:grid-cols-4"
|
||||
/>
|
||||
<TabsContent value="system" class="mt-4 flex flex-col gap-6">
|
||||
{#if !initialLoading}
|
||||
{#if overallStatus === 'ok'}
|
||||
<Alert class="border-success/30 bg-success/5">
|
||||
<CheckCircle class="text-success" />
|
||||
<AlertTitle>Система в норме</AlertTitle>
|
||||
<AlertDescription>{overallHint}</AlertDescription>
|
||||
</Alert>
|
||||
{:else if overallStatus === 'warn'}
|
||||
<Alert class="border-warning/30 bg-warning/5">
|
||||
<AlertTriangle class="text-warning" />
|
||||
<AlertTitle>Требуется внимание</AlertTitle>
|
||||
<AlertDescription>{overallHint}</AlertDescription>
|
||||
</Alert>
|
||||
{:else if overallStatus === 'error'}
|
||||
<Alert variant="destructive">
|
||||
<XCircle />
|
||||
<AlertTitle>Обнаружена проблема</AlertTitle>
|
||||
<AlertDescription>{overallHint}</AlertDescription>
|
||||
</Alert>
|
||||
{/if}
|
||||
{/if}
|
||||
|
||||
<div class="grid gap-4 lg:grid-cols-2">
|
||||
{#if initialLoading}
|
||||
<CardSkeleton />
|
||||
<CardSkeleton />
|
||||
{:else}
|
||||
<Card>
|
||||
<CardHeader>
|
||||
<CardTitle class="text-base">Доступность и готовность</CardTitle>
|
||||
<CardDescription>GET /v1/health · GET /v1/ready</CardDescription>
|
||||
</CardHeader>
|
||||
<CardContent class="space-y-4">
|
||||
{#if health?.error || readyError}
|
||||
<Alert variant="destructive">
|
||||
<XCircle />
|
||||
<AlertTitle>Ошибка проверки</AlertTitle>
|
||||
<AlertDescription>
|
||||
{#if health?.error}{health.error}{/if}
|
||||
{#if health?.error && readyError}<br />{/if}
|
||||
{#if readyError}{readyError}{/if}
|
||||
</AlertDescription>
|
||||
</Alert>
|
||||
{/if}
|
||||
<KpiMetricsGrid
|
||||
cards={kpiCards}
|
||||
loading={initialLoading}
|
||||
skeletonCount={4}
|
||||
class="sm:grid-cols-2 xl:grid-cols-4"
|
||||
/>
|
||||
|
||||
<Table>
|
||||
<TableHeader>
|
||||
<TableRow>
|
||||
<TableHead class="w-[55%]">Проверка</TableHead>
|
||||
<TableHead>Статус</TableHead>
|
||||
</TableRow>
|
||||
</TableHeader>
|
||||
<TableBody>
|
||||
{@const liveBadge = livenessBadge(health)}
|
||||
<TableRow>
|
||||
<TableCell>
|
||||
<div class="flex items-center gap-2">
|
||||
<HeartPulse class="size-4 shrink-0 text-muted-foreground" aria-hidden="true" />
|
||||
<div>
|
||||
<p class="text-sm font-medium">Liveness</p>
|
||||
<p class="text-xs text-muted-foreground">/v1/health</p>
|
||||
</div>
|
||||
</div>
|
||||
</TableCell>
|
||||
<TableCell>
|
||||
<Badge variant={liveBadge.variant} class={liveBadge.class}
|
||||
>{liveBadge.label}</Badge
|
||||
>
|
||||
</TableCell>
|
||||
</TableRow>
|
||||
{@const readyBadge = readinessBadge(ready)}
|
||||
<TableRow>
|
||||
<TableCell>
|
||||
<div class="flex items-center gap-2">
|
||||
<ShieldCheck class="size-4 shrink-0 text-muted-foreground" aria-hidden="true" />
|
||||
<div>
|
||||
<p class="text-sm font-medium">Readiness</p>
|
||||
<p class="text-xs text-muted-foreground">/v1/ready</p>
|
||||
</div>
|
||||
</div>
|
||||
</TableCell>
|
||||
<TableCell>
|
||||
<Badge variant={readyBadge.variant} class={readyBadge.class}
|
||||
>{readyBadge.label}</Badge
|
||||
>
|
||||
</TableCell>
|
||||
</TableRow>
|
||||
{#if ready?.checks && Object.keys(ready.checks).length > 0}
|
||||
<TableRow>
|
||||
<TableCell colspan={2} class="bg-muted/30 py-2">
|
||||
<p class="text-xs font-medium text-muted-foreground">Зависимости</p>
|
||||
</TableCell>
|
||||
</TableRow>
|
||||
{#each Object.entries(ready.checks) as [key, value] (key)}
|
||||
{@const badge = checkStatusBadge(value)}
|
||||
{@const CheckIcon = checkIconByKey[key] ?? ListTodo}
|
||||
<div class="grid gap-4 lg:grid-cols-2">
|
||||
{#if initialLoading}
|
||||
<CardSkeleton />
|
||||
<CardSkeleton />
|
||||
{:else}
|
||||
<Card>
|
||||
<CardHeader>
|
||||
<CardTitle class="text-base">Доступность и готовность</CardTitle>
|
||||
<CardDescription>GET /v1/health · GET /v1/ready</CardDescription>
|
||||
</CardHeader>
|
||||
<CardContent class="space-y-4">
|
||||
{#if health?.error || readyError}
|
||||
<Alert variant="destructive">
|
||||
<XCircle />
|
||||
<AlertTitle>Ошибка проверки</AlertTitle>
|
||||
<AlertDescription>
|
||||
{#if health?.error}{health.error}{/if}
|
||||
{#if health?.error && readyError}<br />{/if}
|
||||
{#if readyError}{readyError}{/if}
|
||||
</AlertDescription>
|
||||
</Alert>
|
||||
{/if}
|
||||
|
||||
<Table>
|
||||
<TableHeader>
|
||||
<TableRow>
|
||||
<TableHead class="w-[55%]">Проверка</TableHead>
|
||||
<TableHead>Статус</TableHead>
|
||||
</TableRow>
|
||||
</TableHeader>
|
||||
<TableBody>
|
||||
{@const liveBadge = livenessBadge(health)}
|
||||
<TableRow>
|
||||
<TableCell>
|
||||
<div class="flex items-center gap-2">
|
||||
<CheckIcon
|
||||
<HeartPulse
|
||||
class="size-4 shrink-0 text-muted-foreground"
|
||||
aria-hidden="true"
|
||||
/>
|
||||
<div>
|
||||
<p class="text-sm font-medium">{checkDisplayName(key)}</p>
|
||||
<p class="text-xs text-muted-foreground">{key}</p>
|
||||
<p class="text-sm font-medium">Liveness</p>
|
||||
<p class="text-xs text-muted-foreground">/v1/health</p>
|
||||
</div>
|
||||
</div>
|
||||
</TableCell>
|
||||
<TableCell>
|
||||
<div class="space-y-1">
|
||||
<Badge variant={badge.variant} class={badge.class}>{badge.label}</Badge>
|
||||
{#if badge.hint}
|
||||
<p class="text-xs text-muted-foreground">{badge.hint}</p>
|
||||
{/if}
|
||||
</div>
|
||||
<Badge variant={liveBadge.variant} class={liveBadge.class}
|
||||
>{liveBadge.label}</Badge
|
||||
>
|
||||
</TableCell>
|
||||
</TableRow>
|
||||
{/each}
|
||||
{/if}
|
||||
</TableBody>
|
||||
</Table>
|
||||
|
||||
<p class="text-xs text-muted-foreground">
|
||||
HTTP 503 на readiness означает недоступность одной из зависимостей в checks.
|
||||
</p>
|
||||
</CardContent>
|
||||
</Card>
|
||||
|
||||
<Card>
|
||||
<CardHeader>
|
||||
<div class="flex items-start justify-between gap-2">
|
||||
<div>
|
||||
<CardTitle class="flex items-center gap-2 text-base">
|
||||
<Bird class="size-4" />
|
||||
BGP на API-хосте
|
||||
</CardTitle>
|
||||
<CardDescription>GET /v1/bird/status</CardDescription>
|
||||
</div>
|
||||
<Button variant="outline" size="sm" href={resolve('/network?tab=overview')}
|
||||
>Пиры и спикеры</Button
|
||||
>
|
||||
</div>
|
||||
</CardHeader>
|
||||
<CardContent class="space-y-4">
|
||||
{#if birdError}
|
||||
<Alert variant="destructive">
|
||||
<XCircle />
|
||||
<AlertTitle>Ошибка birdc</AlertTitle>
|
||||
<AlertDescription>{birdError}</AlertDescription>
|
||||
</Alert>
|
||||
{:else if bird && !bird.birdc_configured}
|
||||
<p class="text-sm text-muted-foreground">
|
||||
{bird.message ?? 'birdc не настроен на API-хосте (EVOBGP_BIRDC_SOCKET).'}
|
||||
</p>
|
||||
{:else if bird}
|
||||
<div class="space-y-2">
|
||||
<div class="flex items-center justify-between text-sm">
|
||||
<span class="text-muted-foreground">Established / total</span>
|
||||
<span class="font-medium tabular-nums">
|
||||
{bird.bgp_established} / {bird.bgp_sessions_total}
|
||||
{#if bgpRatio !== null}
|
||||
<span class="text-muted-foreground">({bgpRatio}%)</span>
|
||||
{/if}
|
||||
</span>
|
||||
</div>
|
||||
{#if bgpRatio !== null}
|
||||
<div class="h-2 overflow-hidden rounded-full bg-muted">
|
||||
<div
|
||||
class={cn(
|
||||
'h-full rounded-full transition-all',
|
||||
bgpRatio >= 100
|
||||
? 'bg-success'
|
||||
: bgpRatio >= 50
|
||||
? 'bg-warning'
|
||||
: 'bg-destructive'
|
||||
)}
|
||||
style="width: {bgpRatio}%"
|
||||
></div>
|
||||
</div>
|
||||
{/if}
|
||||
{#if bird.error}
|
||||
<p class="text-xs text-destructive">{bird.error}</p>
|
||||
{/if}
|
||||
</div>
|
||||
{#if bird.protocols_excerpt}
|
||||
<Separator />
|
||||
<div class="space-y-2">
|
||||
<p class="text-sm font-medium">Вывод birdc (protocols)</p>
|
||||
<ScrollPreBlock variant="preserve" text={bird.protocols_excerpt} class="max-h-48" />
|
||||
</div>
|
||||
{/if}
|
||||
{/if}
|
||||
</CardContent>
|
||||
</Card>
|
||||
{/if}
|
||||
</div>
|
||||
|
||||
<div class="grid gap-4 lg:grid-cols-2">
|
||||
{#if initialLoading}
|
||||
<CardSkeleton />
|
||||
<CardSkeleton />
|
||||
{:else}
|
||||
<Card>
|
||||
<CardHeader>
|
||||
<div class="flex items-start justify-between gap-2">
|
||||
<div>
|
||||
<CardTitle class="flex items-center gap-2 text-base">
|
||||
<Activity class="size-4" />
|
||||
Задачи
|
||||
</CardTitle>
|
||||
<CardDescription>Последние 100 задач · GET /v1/jobs</CardDescription>
|
||||
</div>
|
||||
<Button variant="outline" size="sm" href={resolve('/operations')}>Все операции</Button>
|
||||
</div>
|
||||
</CardHeader>
|
||||
<CardContent class="space-y-4">
|
||||
{#if jobsError}
|
||||
<Alert variant="destructive">
|
||||
<XCircle />
|
||||
<AlertTitle>Не удалось загрузить задачи</AlertTitle>
|
||||
<AlertDescription>{jobsError}</AlertDescription>
|
||||
</Alert>
|
||||
{:else if jobs}
|
||||
<div class="flex flex-wrap gap-4 text-sm">
|
||||
<div>
|
||||
<p class="text-muted-foreground">Активных</p>
|
||||
<p class="text-2xl font-bold tabular-nums">{jobs.running}</p>
|
||||
</div>
|
||||
<div>
|
||||
<p class="text-muted-foreground">С ошибками</p>
|
||||
<p
|
||||
class={cn(
|
||||
'text-2xl font-bold tabular-nums',
|
||||
jobs.failed > 0 ? 'text-warning' : 'text-success'
|
||||
)}
|
||||
>
|
||||
{jobs.failed}
|
||||
</p>
|
||||
</div>
|
||||
<div>
|
||||
<p class="text-muted-foreground">В выборке</p>
|
||||
<p class="text-2xl font-bold tabular-nums">{jobs.total}</p>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<Separator />
|
||||
|
||||
{#if failedJobs.length > 0}
|
||||
<div class="space-y-3">
|
||||
<p class="text-sm font-medium">Последние ошибки</p>
|
||||
<ul class="space-y-2">
|
||||
{#each failedJobs as job (job.job_id)}
|
||||
<li class="rounded-lg border px-3 py-2 text-sm">
|
||||
<div class="flex items-start justify-between gap-2">
|
||||
<p class="font-medium">{jobKindTitle(job)}</p>
|
||||
<Badge variant="destructive">{jobStatusRu(job.status)}</Badge>
|
||||
{@const readyBadge = readinessBadge(ready)}
|
||||
<TableRow>
|
||||
<TableCell>
|
||||
<div class="flex items-center gap-2">
|
||||
<ShieldCheck
|
||||
class="size-4 shrink-0 text-muted-foreground"
|
||||
aria-hidden="true"
|
||||
/>
|
||||
<div>
|
||||
<p class="text-sm font-medium">Readiness</p>
|
||||
<p class="text-xs text-muted-foreground">/v1/ready</p>
|
||||
</div>
|
||||
</div>
|
||||
{#if job.error}
|
||||
<p class="mt-1 text-xs text-muted-foreground">
|
||||
{truncateError(job.error)}
|
||||
</p>
|
||||
{/if}
|
||||
</li>
|
||||
{/each}
|
||||
</ul>
|
||||
</div>
|
||||
{:else}
|
||||
<p class="text-sm text-muted-foreground">
|
||||
Критичных сбоев в последних 100 задачах нет.
|
||||
</p>
|
||||
{/if}
|
||||
{/if}
|
||||
</CardContent>
|
||||
</Card>
|
||||
</TableCell>
|
||||
<TableCell>
|
||||
<Badge variant={readyBadge.variant} class={readyBadge.class}
|
||||
>{readyBadge.label}</Badge
|
||||
>
|
||||
</TableCell>
|
||||
</TableRow>
|
||||
{#if ready?.checks && Object.keys(ready.checks).length > 0}
|
||||
<TableRow>
|
||||
<TableCell colspan={2} class="bg-muted/30 py-2">
|
||||
<p class="text-xs font-medium text-muted-foreground">Зависимости</p>
|
||||
</TableCell>
|
||||
</TableRow>
|
||||
{#each Object.entries(ready.checks) as [key, value] (key)}
|
||||
{@const badge = checkStatusBadge(value)}
|
||||
{@const CheckIcon = checkIconByKey[key] ?? ListTodo}
|
||||
<TableRow>
|
||||
<TableCell>
|
||||
<div class="flex items-center gap-2">
|
||||
<CheckIcon
|
||||
class="size-4 shrink-0 text-muted-foreground"
|
||||
aria-hidden="true"
|
||||
/>
|
||||
<div>
|
||||
<p class="text-sm font-medium">{checkDisplayName(key)}</p>
|
||||
<p class="text-xs text-muted-foreground">{key}</p>
|
||||
</div>
|
||||
</div>
|
||||
</TableCell>
|
||||
<TableCell>
|
||||
<div class="space-y-1">
|
||||
<Badge variant={badge.variant} class={badge.class}>{badge.label}</Badge>
|
||||
{#if badge.hint}
|
||||
<p class="text-xs text-muted-foreground">{badge.hint}</p>
|
||||
{/if}
|
||||
</div>
|
||||
</TableCell>
|
||||
</TableRow>
|
||||
{/each}
|
||||
{/if}
|
||||
</TableBody>
|
||||
</Table>
|
||||
|
||||
<Card>
|
||||
<CardHeader>
|
||||
<CardTitle class="flex items-center gap-2 text-base">
|
||||
<AlertTriangle class="size-4 text-muted-foreground" />
|
||||
Что проверять при деградации
|
||||
</CardTitle>
|
||||
<CardDescription>Короткая шпаргалка для triage</CardDescription>
|
||||
</CardHeader>
|
||||
<CardContent class="space-y-3">
|
||||
<Alert>
|
||||
<HeartPulse class="size-4" />
|
||||
<AlertTitle>API недоступен</AlertTitle>
|
||||
<AlertDescription>
|
||||
Если <code class="text-xs">/v1/health</code> возвращает ошибку — проверьте процесс API и
|
||||
его логи.
|
||||
</AlertDescription>
|
||||
</Alert>
|
||||
<Alert>
|
||||
<Database class="size-4" />
|
||||
<AlertTitle>Readiness не «Готов»</AlertTitle>
|
||||
<AlertDescription>
|
||||
Сначала <code class="text-xs">postgres</code>, затем
|
||||
<code class="text-xs">store</code> и <code class="text-xs">jobs</code> в checks.
|
||||
</AlertDescription>
|
||||
</Alert>
|
||||
<Alert>
|
||||
<Bird class="size-4" />
|
||||
<AlertTitle>Низкий ratio BGP</AlertTitle>
|
||||
<AlertDescription>
|
||||
Проверьте <code class="text-xs">/v1/bird/status</code>, затем состояние пиров в
|
||||
<Button variant="link" class="h-auto p-0" href={resolve('/network?tab=overview')}
|
||||
>Сети</Button
|
||||
>.
|
||||
</AlertDescription>
|
||||
</Alert>
|
||||
<Alert>
|
||||
<ListTodo class="size-4" />
|
||||
<AlertTitle>Ошибки задач</AlertTitle>
|
||||
<AlertDescription>
|
||||
Откройте
|
||||
<Button variant="link" class="h-auto p-0" href={resolve('/operations')}
|
||||
>Операции</Button
|
||||
>
|
||||
и проверьте последние неуспешные jobs.
|
||||
</AlertDescription>
|
||||
</Alert>
|
||||
</CardContent>
|
||||
</Card>
|
||||
{/if}
|
||||
</div>
|
||||
<p class="text-xs text-muted-foreground">
|
||||
HTTP 503 на readiness означает недоступность одной из зависимостей в checks.
|
||||
</p>
|
||||
</CardContent>
|
||||
</Card>
|
||||
|
||||
<Card>
|
||||
<CardHeader>
|
||||
<div class="flex items-start justify-between gap-2">
|
||||
<div>
|
||||
<CardTitle class="flex items-center gap-2 text-base">
|
||||
<Bird class="size-4" />
|
||||
BGP на API-хосте
|
||||
</CardTitle>
|
||||
<CardDescription>GET /v1/bird/status</CardDescription>
|
||||
</div>
|
||||
<Button variant="outline" size="sm" href={resolve('/network?tab=overview')}
|
||||
>Пиры и спикеры</Button
|
||||
>
|
||||
</div>
|
||||
</CardHeader>
|
||||
<CardContent class="space-y-4">
|
||||
{#if birdError}
|
||||
<Alert variant="destructive">
|
||||
<XCircle />
|
||||
<AlertTitle>Ошибка birdc</AlertTitle>
|
||||
<AlertDescription>{birdError}</AlertDescription>
|
||||
</Alert>
|
||||
{:else if bird && !bird.birdc_configured}
|
||||
<p class="text-sm text-muted-foreground">
|
||||
{bird.message ?? 'birdc не настроен на API-хосте (EVOBGP_BIRDC_SOCKET).'}
|
||||
</p>
|
||||
{:else if bird}
|
||||
<div class="space-y-2">
|
||||
<div class="flex items-center justify-between text-sm">
|
||||
<span class="text-muted-foreground">Established / total</span>
|
||||
<span class="font-medium tabular-nums">
|
||||
{bird.bgp_established} / {bird.bgp_sessions_total}
|
||||
{#if bgpRatio !== null}
|
||||
<span class="text-muted-foreground">({bgpRatio}%)</span>
|
||||
{/if}
|
||||
</span>
|
||||
</div>
|
||||
{#if bgpRatio !== null}
|
||||
<div class="h-2 overflow-hidden rounded-full bg-muted">
|
||||
<div
|
||||
class={cn(
|
||||
'h-full rounded-full transition-all',
|
||||
bgpRatio >= 100
|
||||
? 'bg-success'
|
||||
: bgpRatio >= 50
|
||||
? 'bg-warning'
|
||||
: 'bg-destructive'
|
||||
)}
|
||||
style="width: {bgpRatio}%"
|
||||
></div>
|
||||
</div>
|
||||
{/if}
|
||||
{#if bird.error}
|
||||
<p class="text-xs text-destructive">{bird.error}</p>
|
||||
{/if}
|
||||
</div>
|
||||
{#if bird.protocols_excerpt}
|
||||
<Separator />
|
||||
<div class="space-y-2">
|
||||
<p class="text-sm font-medium">Вывод birdc (protocols)</p>
|
||||
<ScrollPreBlock
|
||||
variant="preserve"
|
||||
text={bird.protocols_excerpt}
|
||||
class="max-h-48"
|
||||
/>
|
||||
</div>
|
||||
{/if}
|
||||
{/if}
|
||||
</CardContent>
|
||||
</Card>
|
||||
{/if}
|
||||
</div>
|
||||
|
||||
<div class="grid gap-4 lg:grid-cols-2">
|
||||
{#if initialLoading}
|
||||
<CardSkeleton />
|
||||
<CardSkeleton />
|
||||
{:else}
|
||||
<Card>
|
||||
<CardHeader>
|
||||
<div class="flex items-start justify-between gap-2">
|
||||
<div>
|
||||
<CardTitle class="flex items-center gap-2 text-base">
|
||||
<Activity class="size-4" />
|
||||
Задачи
|
||||
</CardTitle>
|
||||
<CardDescription>Последние 100 задач · GET /v1/jobs</CardDescription>
|
||||
</div>
|
||||
<Button variant="outline" size="sm" href={resolve('/operations')}
|
||||
>Все операции</Button
|
||||
>
|
||||
</div>
|
||||
</CardHeader>
|
||||
<CardContent class="space-y-4">
|
||||
{#if jobsError}
|
||||
<Alert variant="destructive">
|
||||
<XCircle />
|
||||
<AlertTitle>Не удалось загрузить задачи</AlertTitle>
|
||||
<AlertDescription>{jobsError}</AlertDescription>
|
||||
</Alert>
|
||||
{:else if jobs}
|
||||
<div class="flex flex-wrap gap-4 text-sm">
|
||||
<div>
|
||||
<p class="text-muted-foreground">Активных</p>
|
||||
<p class="text-2xl font-bold tabular-nums">{jobs.running}</p>
|
||||
</div>
|
||||
<div>
|
||||
<p class="text-muted-foreground">С ошибками</p>
|
||||
<p
|
||||
class={cn(
|
||||
'text-2xl font-bold tabular-nums',
|
||||
jobs.failed > 0 ? 'text-warning' : 'text-success'
|
||||
)}
|
||||
>
|
||||
{jobs.failed}
|
||||
</p>
|
||||
</div>
|
||||
<div>
|
||||
<p class="text-muted-foreground">В выборке</p>
|
||||
<p class="text-2xl font-bold tabular-nums">{jobs.total}</p>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<Separator />
|
||||
|
||||
{#if failedJobs.length > 0}
|
||||
<div class="space-y-3">
|
||||
<p class="text-sm font-medium">Последние ошибки</p>
|
||||
<ul class="space-y-2">
|
||||
{#each failedJobs as job (job.job_id)}
|
||||
<li class="rounded-lg border px-3 py-2 text-sm">
|
||||
<div class="flex items-start justify-between gap-2">
|
||||
<p class="font-medium">{jobKindTitle(job)}</p>
|
||||
<Badge variant="destructive">{jobStatusRu(job.status)}</Badge>
|
||||
</div>
|
||||
{#if job.error}
|
||||
<p class="mt-1 text-xs text-muted-foreground">
|
||||
{truncateError(job.error)}
|
||||
</p>
|
||||
{/if}
|
||||
</li>
|
||||
{/each}
|
||||
</ul>
|
||||
</div>
|
||||
{:else}
|
||||
<p class="text-sm text-muted-foreground">
|
||||
Критичных сбоев в последних 100 задачах нет.
|
||||
</p>
|
||||
{/if}
|
||||
{/if}
|
||||
</CardContent>
|
||||
</Card>
|
||||
|
||||
<Card>
|
||||
<CardHeader>
|
||||
<CardTitle class="flex items-center gap-2 text-base">
|
||||
<AlertTriangle class="size-4 text-muted-foreground" />
|
||||
Что проверять при деградации
|
||||
</CardTitle>
|
||||
<CardDescription>Короткая шпаргалка для triage</CardDescription>
|
||||
</CardHeader>
|
||||
<CardContent class="space-y-3">
|
||||
<Alert>
|
||||
<HeartPulse class="size-4" />
|
||||
<AlertTitle>API недоступен</AlertTitle>
|
||||
<AlertDescription>
|
||||
Если <code class="text-xs">/v1/health</code> возвращает ошибку — проверьте процесс API
|
||||
и его логи.
|
||||
</AlertDescription>
|
||||
</Alert>
|
||||
<Alert>
|
||||
<Database class="size-4" />
|
||||
<AlertTitle>Readiness не «Готов»</AlertTitle>
|
||||
<AlertDescription>
|
||||
Сначала <code class="text-xs">postgres</code>, затем
|
||||
<code class="text-xs">store</code> и <code class="text-xs">jobs</code> в checks.
|
||||
</AlertDescription>
|
||||
</Alert>
|
||||
<Alert>
|
||||
<Bird class="size-4" />
|
||||
<AlertTitle>Низкий ratio BGP</AlertTitle>
|
||||
<AlertDescription>
|
||||
Проверьте <code class="text-xs">/v1/bird/status</code>, затем состояние пиров в
|
||||
<Button variant="link" class="h-auto p-0" href={resolve('/network?tab=overview')}
|
||||
>Сети</Button
|
||||
>.
|
||||
</AlertDescription>
|
||||
</Alert>
|
||||
<Alert>
|
||||
<ListTodo class="size-4" />
|
||||
<AlertTitle>Ошибки задач</AlertTitle>
|
||||
<AlertDescription>
|
||||
Откройте
|
||||
<Button variant="link" class="h-auto p-0" href={resolve('/operations')}
|
||||
>Операции</Button
|
||||
>
|
||||
и проверьте последние неуспешные jobs.
|
||||
</AlertDescription>
|
||||
</Alert>
|
||||
</CardContent>
|
||||
</Card>
|
||||
{/if}
|
||||
</div>
|
||||
</TabsContent>
|
||||
|
||||
<TabsContent value="postgres" class="mt-4">
|
||||
<MonitoringPostgresTab />
|
||||
</TabsContent>
|
||||
</Tabs>
|
||||
</div>
|
||||
|
||||
Reference in New Issue
Block a user