Enhance API and UI for incident management and live updates
- Added a new endpoint `/api/agg/incidents` to provide a normalized snapshot of incidents for fleet triage, including severity and recommended actions. - Implemented live event streaming via `/api/live/events` for real-time updates on fleet status and incidents, enhancing observability. - Updated the Web UI to include dedicated sections for incidents and live updates, improving user navigation and access to critical information. - Enhanced API documentation to reflect new endpoints and their functionalities, ensuring clarity for developers and users.
This commit is contained in:
@@ -1,6 +1,6 @@
|
|||||||
# telemt-api
|
# telemt-api
|
||||||
|
|
||||||
HTTP‑шлюз на Go для [Telemt Control API](docs/API.md): один порт, **белый список IP (CIDR)**, маршруты вида `/api/{alias}/…` → `{base_url}/v1/…`, агрегация нескольких инстансов — [`/api/agg/…`](docs/AGGREGATE.md), метрики Prometheus на `/metrics`. **Web UI** (SvelteKit) встроен в тот же процесс/образ: статика на `/`, API на `/api/…` и `/health`.
|
HTTP‑шлюз на Go для [Telemt Control API](docs/API.md): один порт, **белый список IP (CIDR)**, маршруты вида `/api/{alias}/…` → `{base_url}/v1/…`, агрегация нескольких инстансов — [`/api/agg/…`](docs/AGGREGATE.md), live SSE поток — `/api/live/events`, метрики Prometheus на `/metrics`. **Web UI** (SvelteKit) встроен в тот же процесс/образ: статика на `/`, API на `/api/…` и `/health`.
|
||||||
|
|
||||||
## Быстрый старт (Linux)
|
## Быстрый старт (Linux)
|
||||||
|
|
||||||
@@ -81,6 +81,8 @@ cors_allowed_origins:
|
|||||||
| **[docs/API.md](docs/API.md)** | Контракт Telemt Control API (`/v1/…`) |
|
| **[docs/API.md](docs/API.md)** | Контракт Telemt Control API (`/v1/…`) |
|
||||||
| **[docs/AGGREGATE.md](docs/AGGREGATE.md)** | Агрегирующие эндпоинты шлюза (`/api/agg/…`), CORS, кэш |
|
| **[docs/AGGREGATE.md](docs/AGGREGATE.md)** | Агрегирующие эндпоинты шлюза (`/api/agg/…`), CORS, кэш |
|
||||||
| **[docs/AGGREGATE_OPENAPI.yaml](docs/AGGREGATE_OPENAPI.yaml)** | OpenAPI 3 черновик для `/api/agg/*` (генерация типов для UI) |
|
| **[docs/AGGREGATE_OPENAPI.yaml](docs/AGGREGATE_OPENAPI.yaml)** | OpenAPI 3 черновик для `/api/agg/*` (генерация типов для UI) |
|
||||||
|
| **[docs/OPERATIONS_BASELINE.md](docs/OPERATIONS_BASELINE.md)** | Baseline UX/SLO для панели быстрого реагирования (MTTD/MTTR и критерии успеха) |
|
||||||
|
| **[docs/INCIDENT_ROLLOUT.md](docs/INCIDENT_ROLLOUT.md)** | Пошаговый rollout incidents/live функций и настройка alert policy |
|
||||||
| **[web/README.md](web/README.md)** | Web UI (SvelteKit): разработка с Vite, `PUBLIC_TELEMT_GATEWAY_URL`, встраивание в образ шлюза |
|
| **[web/README.md](web/README.md)** | Web UI (SvelteKit): разработка с Vite, `PUBLIC_TELEMT_GATEWAY_URL`, встраивание в образ шлюза |
|
||||||
| **[docs/GEOIP.md](docs/GEOIP.md)** | GeoLite2 City (страна/город) и опционально ASN (номер AS, организация) для IP в `unique-ips` |
|
| **[docs/GEOIP.md](docs/GEOIP.md)** | GeoLite2 City (страна/город) и опционально ASN (номер AS, организация) для IP в `unique-ips` |
|
||||||
|
|
||||||
@@ -90,6 +92,14 @@ cors_allowed_origins:
|
|||||||
go mod tidy && go test ./...
|
go mod tidy && go test ./...
|
||||||
```
|
```
|
||||||
|
|
||||||
|
## Observability (gateway)
|
||||||
|
|
||||||
|
Prometheus метрики доступны на `/metrics`, включая:
|
||||||
|
|
||||||
|
- `telemt_gateway_http_in_flight`
|
||||||
|
- `telemt_gateway_http_requests_total{code,method,alias,endpoint}`
|
||||||
|
- `telemt_gateway_http_request_duration_seconds{method,alias,endpoint}`
|
||||||
|
|
||||||
## CI/CD
|
## CI/CD
|
||||||
|
|
||||||
В репозитории: [.gitea/workflows/docker.yaml](.gitea/workflows/docker.yaml) — тесты Go, сборка и публикация образа в Container Registry Gitea (см. раздел «Обновление и CI/CD» в [docs/GATEWAY_RUN.md](docs/GATEWAY_RUN.md)).
|
В репозитории: [.gitea/workflows/docker.yaml](.gitea/workflows/docker.yaml) — тесты Go, сборка и публикация образа в Container Registry Gitea (см. раздел «Обновление и CI/CD» в [docs/GATEWAY_RUN.md](docs/GATEWAY_RUN.md)).
|
||||||
|
|||||||
@@ -33,6 +33,7 @@
|
|||||||
| GET | `/api/agg/users` | Объединённый список пользователей с `by_server`, суммарным `total_megabytes` и **смерженными лимитами** (см. ниже). |
|
| GET | `/api/agg/users` | Объединённый список пользователей с `by_server`, суммарным `total_megabytes` и **смерженными лимитами** (см. ниже). |
|
||||||
| GET | `/api/agg/user/{username}` | Один пользователь в том же формате, что элементы `/api/agg/users` (без списка всех). Имя в пути: `[A-Za-z0-9_.-]+`. Ответ **`404`**, если пользователь не найден ни на одном успешном upstream. |
|
| GET | `/api/agg/user/{username}` | Один пользователь в том же формате, что элементы `/api/agg/users` (без списка всех). Имя в пути: `[A-Za-z0-9_.-]+`. Ответ **`404`**, если пользователь не найден ни на одном успешном upstream. |
|
||||||
| GET | `/api/agg/fleet-status` | По каждому алиасу: параллельно health + system/info; в `data.servers[]` — статусы подзапросов и тела `health` / `system_info` при успехе. См. [AGGREGATE_OPENAPI.yaml](AGGREGATE_OPENAPI.yaml). |
|
| GET | `/api/agg/fleet-status` | По каждому алиасу: параллельно health + system/info; в `data.servers[]` — статусы подзапросов и тела `health` / `system_info` при успехе. См. [AGGREGATE_OPENAPI.yaml](AGGREGATE_OPENAPI.yaml). |
|
||||||
|
| GET | `/api/agg/incidents` | Нормализованный snapshot инцидентов для triage-панели: `critical/warning/info`, `affected_aliases`, рекомендуемые `actions` (runbook/deep links), счётчики по severity. |
|
||||||
|
|
||||||
Все методы — **GET**; действует тот же whitelist, что и для остального API шлюза.
|
Все методы — **GET**; действует тот же whitelist, что и для остального API шлюза.
|
||||||
|
|
||||||
@@ -57,6 +58,18 @@
|
|||||||
| `include_links` | `users`, `user/…` | `true` — добавить сгенерированные `tg://proxy` ссылки (берётся первая успешная запись по пользователю). |
|
| `include_links` | `users`, `user/…` | `true` — добавить сгенерированные `tg://proxy` ссылки (берётся первая успешная запись по пользователю). |
|
||||||
| `min_total_megabytes` | `users` | Порог суммарного трафика пользователя в MiB (строго больше 0). |
|
| `min_total_megabytes` | `users` | Порог суммарного трафика пользователя в MiB (строго больше 0). |
|
||||||
| `min_total_octets` | `users` | Устаревший вариант порога в октетах (если задан `min_total_megabytes`, он приоритетнее). |
|
| `min_total_octets` | `users` | Устаревший вариант порога в октетах (если задан `min_total_megabytes`, он приоритетнее). |
|
||||||
|
| `aliases` | `incidents` | Список алиасов через запятую; позволяет строить incidents snapshot по выбранной группе нод. |
|
||||||
|
|
||||||
|
## Live stream (SSE)
|
||||||
|
|
||||||
|
Для оперативного режима NOC доступен поток событий:
|
||||||
|
|
||||||
|
- **`GET /api/live/events`** (`text/event-stream`)
|
||||||
|
- query: `aliases` (опционально, как в `/api/agg/*`)
|
||||||
|
- событие: `event: snapshot`
|
||||||
|
- payload: JSON со статусом флота (`healthy/degraded/critical`), `partial`, и массивом `incidents`
|
||||||
|
|
||||||
|
Поток рассчитан на UI-клиент с авто-reconnect (на фронте используется экспоненциальный backoff).
|
||||||
|
|
||||||
## Конфигурация (опционально)
|
## Конфигурация (опционально)
|
||||||
|
|
||||||
@@ -99,4 +112,8 @@ curl -sS "http://127.0.0.1:8080/api/agg/unique-ips"
|
|||||||
curl -sS "http://127.0.0.1:8080/api/agg/users?include_links=false&min_total_megabytes=1"
|
curl -sS "http://127.0.0.1:8080/api/agg/users?include_links=false&min_total_megabytes=1"
|
||||||
curl -sS "http://127.0.0.1:8080/api/agg/fleet-status"
|
curl -sS "http://127.0.0.1:8080/api/agg/fleet-status"
|
||||||
curl -sS "http://127.0.0.1:8080/api/agg/user/myuser?aliases=gt1"
|
curl -sS "http://127.0.0.1:8080/api/agg/user/myuser?aliases=gt1"
|
||||||
|
curl -sS "http://127.0.0.1:8080/api/agg/incidents?aliases=gt1,gt2"
|
||||||
|
|
||||||
|
# SSE поток snapshot-событий (пример с curl)
|
||||||
|
curl -N "http://127.0.0.1:8080/api/live/events?aliases=gt1,gt2"
|
||||||
```
|
```
|
||||||
|
|||||||
@@ -106,6 +106,18 @@ paths:
|
|||||||
application/json:
|
application/json:
|
||||||
schema: { $ref: '#/components/schemas/AggEnvelopeFleetStatus' }
|
schema: { $ref: '#/components/schemas/AggEnvelopeFleetStatus' }
|
||||||
|
|
||||||
|
/api/agg/incidents:
|
||||||
|
get:
|
||||||
|
summary: Нормализованный snapshot инцидентов по флоту
|
||||||
|
parameters:
|
||||||
|
- $ref: '#/components/parameters/aliases'
|
||||||
|
responses:
|
||||||
|
'200':
|
||||||
|
description: OK
|
||||||
|
content:
|
||||||
|
application/json:
|
||||||
|
schema: { $ref: '#/components/schemas/AggEnvelopeIncidents' }
|
||||||
|
|
||||||
components:
|
components:
|
||||||
parameters:
|
parameters:
|
||||||
aliases:
|
aliases:
|
||||||
@@ -173,6 +185,13 @@ components:
|
|||||||
properties:
|
properties:
|
||||||
data: { $ref: '#/components/schemas/FleetStatusData' }
|
data: { $ref: '#/components/schemas/FleetStatusData' }
|
||||||
|
|
||||||
|
AggEnvelopeIncidents:
|
||||||
|
allOf:
|
||||||
|
- $ref: '#/components/schemas/AggSuccessBase'
|
||||||
|
- type: object
|
||||||
|
properties:
|
||||||
|
data: { $ref: '#/components/schemas/IncidentsData' }
|
||||||
|
|
||||||
TrafficRow:
|
TrafficRow:
|
||||||
type: object
|
type: object
|
||||||
properties:
|
properties:
|
||||||
@@ -297,3 +316,43 @@ components:
|
|||||||
servers_total: { type: integer }
|
servers_total: { type: integer }
|
||||||
servers_all_ok: { type: integer }
|
servers_all_ok: { type: integer }
|
||||||
servers_failed: { type: integer }
|
servers_failed: { type: integer }
|
||||||
|
|
||||||
|
IncidentAction:
|
||||||
|
type: object
|
||||||
|
properties:
|
||||||
|
label: { type: string }
|
||||||
|
href: { type: string }
|
||||||
|
|
||||||
|
IncidentItem:
|
||||||
|
type: object
|
||||||
|
properties:
|
||||||
|
id: { type: string }
|
||||||
|
kind: { type: string }
|
||||||
|
severity:
|
||||||
|
type: string
|
||||||
|
enum: [info, warning, critical]
|
||||||
|
status:
|
||||||
|
type: string
|
||||||
|
enum: [firing]
|
||||||
|
title: { type: string }
|
||||||
|
summary: { type: string }
|
||||||
|
affected_aliases:
|
||||||
|
type: array
|
||||||
|
items: { type: string }
|
||||||
|
metric_name: { type: string }
|
||||||
|
metric_value: { type: number, format: float }
|
||||||
|
metric_threshold: { type: number, format: float }
|
||||||
|
actions:
|
||||||
|
type: array
|
||||||
|
items: { $ref: '#/components/schemas/IncidentAction' }
|
||||||
|
|
||||||
|
IncidentsData:
|
||||||
|
type: object
|
||||||
|
properties:
|
||||||
|
items:
|
||||||
|
type: array
|
||||||
|
items: { $ref: '#/components/schemas/IncidentItem' }
|
||||||
|
total: { type: integer }
|
||||||
|
critical_total: { type: integer }
|
||||||
|
warning_total: { type: integer }
|
||||||
|
info_total: { type: integer }
|
||||||
|
|||||||
@@ -0,0 +1,73 @@
|
|||||||
|
# Incident Rollout Playbook
|
||||||
|
|
||||||
|
## Scope
|
||||||
|
|
||||||
|
This playbook describes staged rollout for:
|
||||||
|
|
||||||
|
- `/api/agg/incidents`
|
||||||
|
- `/api/live/events` (SSE snapshots)
|
||||||
|
- UI pages `/incidents` and `/live`
|
||||||
|
- Live polling controls and stale indicators on key pages
|
||||||
|
|
||||||
|
## Stage 0 - Baseline Capture (2-3 days)
|
||||||
|
|
||||||
|
- Record current MTTD and MTTR from on-call logs.
|
||||||
|
- Record manual refresh usage on main pages.
|
||||||
|
- Save top recurring failure patterns (degraded nodes, read-only modes, bad connections spikes).
|
||||||
|
|
||||||
|
Outputs:
|
||||||
|
|
||||||
|
- baseline MTTD / MTTR
|
||||||
|
- top 5 incident categories by frequency
|
||||||
|
|
||||||
|
## Stage 1 - Shadow Mode (3-5 days)
|
||||||
|
|
||||||
|
- Enable incidents and live pages for operators.
|
||||||
|
- Do not change paging/escalation yet.
|
||||||
|
- Compare incident feed against existing monitoring and mark false positives.
|
||||||
|
|
||||||
|
Targets:
|
||||||
|
|
||||||
|
- false positive ratio < 20%
|
||||||
|
- no increase in upstream load beyond acceptable budget
|
||||||
|
|
||||||
|
## Stage 2 - Assisted Triage (1 week)
|
||||||
|
|
||||||
|
- Use `/incidents` as primary triage board.
|
||||||
|
- Require owner + ack for active critical incidents.
|
||||||
|
- Use runbook links from incident items.
|
||||||
|
|
||||||
|
Targets:
|
||||||
|
|
||||||
|
- ack coverage for critical incidents >= 90%
|
||||||
|
- owner coverage for critical incidents >= 90%
|
||||||
|
|
||||||
|
## Stage 3 - Policy Tuning (ongoing)
|
||||||
|
|
||||||
|
- Adjust thresholds:
|
||||||
|
- `bad_connections_warn` (default 1000)
|
||||||
|
- `bad_connections_high` (default 10000)
|
||||||
|
- Review alert fatigue weekly.
|
||||||
|
- Promote stable thresholds into documented policy.
|
||||||
|
|
||||||
|
## KPI Tracking
|
||||||
|
|
||||||
|
- MTTD (minutes): incident first observed -> first ack
|
||||||
|
- MTTR (minutes): incident first observed -> resolved
|
||||||
|
- Stale time share: percentage of time live views are stale
|
||||||
|
- Manual refresh share: manual refresh / total data update actions
|
||||||
|
|
||||||
|
## Fast Rollback
|
||||||
|
|
||||||
|
If noise or load is excessive:
|
||||||
|
|
||||||
|
1. disable auto-refresh by setting `refresh=0` in shared ops links
|
||||||
|
2. switch operators back to dashboard summary only
|
||||||
|
3. keep `/api/agg/incidents` for diagnostics while disabling SSE consumers
|
||||||
|
|
||||||
|
## Weekly Review Template
|
||||||
|
|
||||||
|
- KPI delta (MTTD, MTTR) vs baseline
|
||||||
|
- top noisy rules
|
||||||
|
- incidents with missing owner/ack
|
||||||
|
- policy changes applied this week
|
||||||
@@ -0,0 +1,49 @@
|
|||||||
|
# Operations Baseline (Telemt Panel)
|
||||||
|
|
||||||
|
## Purpose
|
||||||
|
|
||||||
|
This document defines the baseline operating model and target SLO metrics for the Telemt response panel.
|
||||||
|
It is used to measure impact of live updates, incidents workflow, and observability improvements.
|
||||||
|
|
||||||
|
## Current Baseline (Before Rollout)
|
||||||
|
|
||||||
|
- Dashboard refresh mode: mostly manual (`Refresh` buttons on key pages).
|
||||||
|
- Unified incident queue: not present.
|
||||||
|
- Alert ownership and acknowledgement flow: not present.
|
||||||
|
- MTTR tracking: not formalized in product UI.
|
||||||
|
- MTTD tracking: not formalized in product UI.
|
||||||
|
- Cross-page filtering consistency: partial, per-page controls only.
|
||||||
|
|
||||||
|
## Baseline Risks
|
||||||
|
|
||||||
|
- Slow detection when operators do not refresh at the right time.
|
||||||
|
- High context switching cost between pages during incidents.
|
||||||
|
- No single place to triage partial/degraded node states.
|
||||||
|
- Weak evidence trail for post-incident analysis.
|
||||||
|
|
||||||
|
## SLO Targets (Phase Goals)
|
||||||
|
|
||||||
|
- MTTD reduction: at least 30% versus baseline.
|
||||||
|
- MTTR reduction: at least 25% versus baseline.
|
||||||
|
- Manual refresh share during incident windows: under 10%.
|
||||||
|
- Incident cards with `owner + ack + runbook`: over 90%.
|
||||||
|
|
||||||
|
## Measurement Inputs
|
||||||
|
|
||||||
|
- Gateway API:
|
||||||
|
- `/api/agg/fleet-status`
|
||||||
|
- `/api/agg/summary`
|
||||||
|
- `/api/agg/incidents` (planned/implemented in this rollout)
|
||||||
|
- Gateway metrics endpoint:
|
||||||
|
- `/metrics`
|
||||||
|
- Frontend telemetry (local panel interaction metrics):
|
||||||
|
- refresh actions
|
||||||
|
- ack/resolve actions
|
||||||
|
- stale/live state durations
|
||||||
|
|
||||||
|
## Rollout Validation Checklist
|
||||||
|
|
||||||
|
- [ ] Baseline values captured before enabling auto-refresh.
|
||||||
|
- [ ] Incident flow tested with simulated degraded upstream.
|
||||||
|
- [ ] Alert noise review completed after first week.
|
||||||
|
- [ ] MTTD/MTTR comparison published for phase review.
|
||||||
@@ -35,6 +35,11 @@ type Handler struct {
|
|||||||
cache map[string]cacheEntry
|
cache map[string]cacheEntry
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// ResolveAliasesForLive resolves aliases for external endpoints (SSE/live).
|
||||||
|
func (h *Handler) ResolveAliasesForLive(r *http.Request) ([]string, error) {
|
||||||
|
return h.resolveAliases(r)
|
||||||
|
}
|
||||||
|
|
||||||
// NewHandler builds an aggregate handler; client must use a non-nil Transport (e.g. gateway shared transport).
|
// NewHandler builds an aggregate handler; client must use a non-nil Transport (e.g. gateway shared transport).
|
||||||
// Geo may be nil (no GeoLite2 lookups). cacheTTL 0 disables response caching.
|
// Geo may be nil (no GeoLite2 lookups). cacheTTL 0 disables response caching.
|
||||||
func NewHandler(p *config.Parsed, client *http.Client, geo *geoip.Service, cacheTTL time.Duration) *Handler {
|
func NewHandler(p *config.Parsed, client *http.Client, geo *geoip.Service, cacheTTL time.Duration) *Handler {
|
||||||
@@ -108,6 +113,8 @@ func (h *Handler) dispatch(w http.ResponseWriter, r *http.Request, sub string) {
|
|||||||
h.handleUsers(w, r)
|
h.handleUsers(w, r)
|
||||||
case sub == "fleet-status":
|
case sub == "fleet-status":
|
||||||
h.handleFleetStatus(w, r)
|
h.handleFleetStatus(w, r)
|
||||||
|
case sub == "incidents":
|
||||||
|
h.handleIncidents(w, r)
|
||||||
case strings.HasPrefix(sub, "user/"):
|
case strings.HasPrefix(sub, "user/"):
|
||||||
username := strings.TrimPrefix(sub, "user/")
|
username := strings.TrimPrefix(sub, "user/")
|
||||||
if username == "" {
|
if username == "" {
|
||||||
@@ -304,6 +311,18 @@ func (h *Handler) handleUserOne(w http.ResponseWriter, r *http.Request, username
|
|||||||
writeAggOK(w, partial, row)
|
writeAggOK(w, partial, row)
|
||||||
}
|
}
|
||||||
|
|
||||||
|
func (h *Handler) handleIncidents(w http.ResponseWriter, r *http.Request) {
|
||||||
|
aliases, err := h.resolveAliases(r)
|
||||||
|
if err != nil {
|
||||||
|
writeBadRequest(w, err)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
ctx, cancel := context.WithTimeout(r.Context(), 60*time.Second)
|
||||||
|
defer cancel()
|
||||||
|
data, partial := BuildIncidents(ctx, h, aliases)
|
||||||
|
writeAggOK(w, partial, data)
|
||||||
|
}
|
||||||
|
|
||||||
func writeBadRequest(w http.ResponseWriter, err error) {
|
func writeBadRequest(w http.ResponseWriter, err error) {
|
||||||
w.Header().Set("Content-Type", "application/json; charset=utf-8")
|
w.Header().Set("Content-Type", "application/json; charset=utf-8")
|
||||||
w.WriteHeader(http.StatusBadRequest)
|
w.WriteHeader(http.StatusBadRequest)
|
||||||
|
|||||||
@@ -0,0 +1,154 @@
|
|||||||
|
package aggregate
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"fmt"
|
||||||
|
"sort"
|
||||||
|
"strings"
|
||||||
|
"time"
|
||||||
|
)
|
||||||
|
|
||||||
|
type statsSummaryLite struct {
|
||||||
|
ConnectionsBadTotal uint64 `json:"connections_bad_total"`
|
||||||
|
ConnectionsTotal uint64 `json:"connections_total"`
|
||||||
|
}
|
||||||
|
|
||||||
|
func BuildIncidents(ctx context.Context, h *Handler, aliases []string) (IncidentsData, bool) {
|
||||||
|
out := IncidentsData{Items: make([]IncidentItem, 0)}
|
||||||
|
partial := false
|
||||||
|
|
||||||
|
fleet := FetchFleetStatus(ctx, h.Client, h.Parsed, aliases)
|
||||||
|
if fleet.ServersFailed > 0 {
|
||||||
|
partial = true
|
||||||
|
}
|
||||||
|
|
||||||
|
for _, s := range fleet.Servers {
|
||||||
|
if !s.OK {
|
||||||
|
severity := SeverityWarning
|
||||||
|
if !s.HealthOK && !s.SystemInfoOK {
|
||||||
|
severity = SeverityCritical
|
||||||
|
}
|
||||||
|
out.Items = append(out.Items, IncidentItem{
|
||||||
|
ID: fmt.Sprintf("node_degraded:%s", s.Alias),
|
||||||
|
Kind: "node_degraded",
|
||||||
|
Severity: severity,
|
||||||
|
Status: IncidentStatusFiring,
|
||||||
|
Title: fmt.Sprintf("Нода %s degraded", s.Alias),
|
||||||
|
Summary: "health/system_info не прошли полностью",
|
||||||
|
AffectedAliases: []string{s.Alias},
|
||||||
|
Actions: []IncidentAction{
|
||||||
|
{Label: "Открыть ноду", Href: "/servers/" + s.Alias},
|
||||||
|
{Label: "Runtime", Href: "/servers/" + s.Alias + "/runtime"},
|
||||||
|
},
|
||||||
|
})
|
||||||
|
}
|
||||||
|
if s.Health != nil && s.Health.ReadOnly {
|
||||||
|
out.Items = append(out.Items, IncidentItem{
|
||||||
|
ID: fmt.Sprintf("node_read_only:%s", s.Alias),
|
||||||
|
Kind: "node_read_only",
|
||||||
|
Severity: SeverityWarning,
|
||||||
|
Status: IncidentStatusFiring,
|
||||||
|
Title: fmt.Sprintf("Нода %s в read_only", s.Alias),
|
||||||
|
Summary: "API ноды не принимает mutating операции",
|
||||||
|
AffectedAliases: []string{s.Alias},
|
||||||
|
Actions: []IncidentAction{
|
||||||
|
{Label: "Users", Href: "/servers/" + s.Alias + "/users"},
|
||||||
|
{Label: "Security", Href: "/servers/" + s.Alias + "/security"},
|
||||||
|
},
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
for _, alias := range aliases {
|
||||||
|
data, meta := FetchTelemtGET[statsSummaryLite](ctx, h.Client, h.Parsed, alias, "stats/summary")
|
||||||
|
if !meta.OK {
|
||||||
|
partial = true
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
if data.ConnectionsBadTotal >= 10000 {
|
||||||
|
out.Items = append(out.Items, IncidentItem{
|
||||||
|
ID: fmt.Sprintf("bad_connections_high:%s", alias),
|
||||||
|
Kind: "bad_connections_high",
|
||||||
|
Severity: SeverityCritical,
|
||||||
|
Status: IncidentStatusFiring,
|
||||||
|
Title: fmt.Sprintf("Высокий bad connections на %s", alias),
|
||||||
|
Summary: "Резкий рост ошибок клиентских соединений",
|
||||||
|
AffectedAliases: []string{alias},
|
||||||
|
MetricName: "connections_bad_total",
|
||||||
|
MetricValue: float64(data.ConnectionsBadTotal),
|
||||||
|
MetricThreshold: 10000,
|
||||||
|
Actions: []IncidentAction{
|
||||||
|
{Label: "Node dashboard", Href: "/servers/" + alias},
|
||||||
|
},
|
||||||
|
})
|
||||||
|
} else if data.ConnectionsBadTotal >= 1000 {
|
||||||
|
out.Items = append(out.Items, IncidentItem{
|
||||||
|
ID: fmt.Sprintf("bad_connections_warn:%s", alias),
|
||||||
|
Kind: "bad_connections_warn",
|
||||||
|
Severity: SeverityWarning,
|
||||||
|
Status: IncidentStatusFiring,
|
||||||
|
Title: fmt.Sprintf("Рост bad connections на %s", alias),
|
||||||
|
Summary: "Наблюдается рост ошибок клиентских соединений",
|
||||||
|
AffectedAliases: []string{alias},
|
||||||
|
MetricName: "connections_bad_total",
|
||||||
|
MetricValue: float64(data.ConnectionsBadTotal),
|
||||||
|
MetricThreshold: 1000,
|
||||||
|
Actions: []IncidentAction{
|
||||||
|
{Label: "Node dashboard", Href: "/servers/" + alias},
|
||||||
|
},
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
sort.Slice(out.Items, func(i, j int) bool {
|
||||||
|
if severityWeight(out.Items[i].Severity) != severityWeight(out.Items[j].Severity) {
|
||||||
|
return severityWeight(out.Items[i].Severity) > severityWeight(out.Items[j].Severity)
|
||||||
|
}
|
||||||
|
return strings.Compare(out.Items[i].ID, out.Items[j].ID) < 0
|
||||||
|
})
|
||||||
|
|
||||||
|
out.Total = len(out.Items)
|
||||||
|
for _, it := range out.Items {
|
||||||
|
switch it.Severity {
|
||||||
|
case SeverityCritical:
|
||||||
|
out.CriticalTotal++
|
||||||
|
case SeverityWarning:
|
||||||
|
out.WarningTotal++
|
||||||
|
default:
|
||||||
|
out.InfoTotal++
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return out, partial
|
||||||
|
}
|
||||||
|
|
||||||
|
func severityWeight(s IncidentSeverity) int {
|
||||||
|
switch s {
|
||||||
|
case SeverityCritical:
|
||||||
|
return 3
|
||||||
|
case SeverityWarning:
|
||||||
|
return 2
|
||||||
|
case SeverityInfo:
|
||||||
|
return 1
|
||||||
|
default:
|
||||||
|
return 0
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func BuildLiveEnvelope(ctx context.Context, h *Handler, aliases []string) map[string]any {
|
||||||
|
inc, partial := BuildIncidents(ctx, h, aliases)
|
||||||
|
status := "healthy"
|
||||||
|
if inc.CriticalTotal > 0 {
|
||||||
|
status = "critical"
|
||||||
|
} else if inc.WarningTotal > 0 {
|
||||||
|
status = "degraded"
|
||||||
|
}
|
||||||
|
return map[string]any{
|
||||||
|
"type": "live_snapshot",
|
||||||
|
"timestamp": time.Now().UTC().Format(time.RFC3339Nano),
|
||||||
|
"status": status,
|
||||||
|
"partial": partial,
|
||||||
|
"incidents": inc.Items,
|
||||||
|
"counts": map[string]int{"total": inc.Total, "critical": inc.CriticalTotal, "warning": inc.WarningTotal, "info": inc.InfoTotal},
|
||||||
|
"aliases_used": aliases,
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -149,6 +149,52 @@ type SummaryData struct {
|
|||||||
TopUsersByUniqueIPs []TopUserByUniqueIPs `json:"top_users_by_unique_ips"`
|
TopUsersByUniqueIPs []TopUserByUniqueIPs `json:"top_users_by_unique_ips"`
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// IncidentSeverity is normalized severity used by aggregate incidents.
|
||||||
|
type IncidentSeverity string
|
||||||
|
|
||||||
|
const (
|
||||||
|
SeverityInfo IncidentSeverity = "info"
|
||||||
|
SeverityWarning IncidentSeverity = "warning"
|
||||||
|
SeverityCritical IncidentSeverity = "critical"
|
||||||
|
)
|
||||||
|
|
||||||
|
// IncidentStatus is current calculated incident state.
|
||||||
|
type IncidentStatus string
|
||||||
|
|
||||||
|
const (
|
||||||
|
IncidentStatusFiring IncidentStatus = "firing"
|
||||||
|
)
|
||||||
|
|
||||||
|
// IncidentAction points to a UI route with details/runbook context.
|
||||||
|
type IncidentAction struct {
|
||||||
|
Label string `json:"label"`
|
||||||
|
Href string `json:"href"`
|
||||||
|
}
|
||||||
|
|
||||||
|
// IncidentItem is one normalized fleet incident.
|
||||||
|
type IncidentItem struct {
|
||||||
|
ID string `json:"id"`
|
||||||
|
Kind string `json:"kind"`
|
||||||
|
Severity IncidentSeverity `json:"severity"`
|
||||||
|
Status IncidentStatus `json:"status"`
|
||||||
|
Title string `json:"title"`
|
||||||
|
Summary string `json:"summary"`
|
||||||
|
AffectedAliases []string `json:"affected_aliases,omitempty"`
|
||||||
|
MetricName string `json:"metric_name,omitempty"`
|
||||||
|
MetricValue float64 `json:"metric_value,omitempty"`
|
||||||
|
MetricThreshold float64 `json:"metric_threshold,omitempty"`
|
||||||
|
Actions []IncidentAction `json:"actions,omitempty"`
|
||||||
|
}
|
||||||
|
|
||||||
|
// IncidentsData is aggregate response payload for /api/agg/incidents.
|
||||||
|
type IncidentsData struct {
|
||||||
|
Items []IncidentItem `json:"items"`
|
||||||
|
Total int `json:"total"`
|
||||||
|
CriticalTotal int `json:"critical_total"`
|
||||||
|
WarningTotal int `json:"warning_total"`
|
||||||
|
InfoTotal int `json:"info_total"`
|
||||||
|
}
|
||||||
|
|
||||||
// TopUser by summed traffic across servers for one username (мегабайты).
|
// TopUser by summed traffic across servers for one username (мегабайты).
|
||||||
type TopUser struct {
|
type TopUser struct {
|
||||||
Username string `json:"username"`
|
Username string `json:"username"`
|
||||||
|
|||||||
@@ -3,6 +3,7 @@ package server
|
|||||||
import (
|
import (
|
||||||
"context"
|
"context"
|
||||||
"encoding/json"
|
"encoding/json"
|
||||||
|
"fmt"
|
||||||
"log/slog"
|
"log/slog"
|
||||||
"net"
|
"net"
|
||||||
"net/http"
|
"net/http"
|
||||||
@@ -181,8 +182,9 @@ func (g *Gateway) withMetrics(next http.Handler) http.Handler {
|
|||||||
httpInFlight.Inc()
|
httpInFlight.Inc()
|
||||||
start := time.Now()
|
start := time.Now()
|
||||||
alias := routeAlias(r.URL.Path)
|
alias := routeAlias(r.URL.Path)
|
||||||
|
endpoint := routeEndpoint(r.URL.Path)
|
||||||
lw := &statusWriter{ResponseWriter: w, status: http.StatusOK}
|
lw := &statusWriter{ResponseWriter: w, status: http.StatusOK}
|
||||||
defer observeRequest(r.Method, alias, lw.status, start)
|
defer observeRequest(r.Method, alias, endpoint, lw.status, start)
|
||||||
next.ServeHTTP(lw, r)
|
next.ServeHTTP(lw, r)
|
||||||
})
|
})
|
||||||
}
|
}
|
||||||
@@ -206,6 +208,33 @@ func routeAlias(path string) string {
|
|||||||
return rest[:i]
|
return rest[:i]
|
||||||
}
|
}
|
||||||
|
|
||||||
|
func routeEndpoint(path string) string {
|
||||||
|
if path == "" || path == "/" {
|
||||||
|
return "ui_root"
|
||||||
|
}
|
||||||
|
if path == "/health" || path == "/metrics" {
|
||||||
|
return strings.TrimPrefix(path, "/")
|
||||||
|
}
|
||||||
|
if strings.HasPrefix(path, "/api/agg/") {
|
||||||
|
return strings.TrimPrefix(path, "/api/agg/")
|
||||||
|
}
|
||||||
|
if path == "/api/agg" {
|
||||||
|
return "agg"
|
||||||
|
}
|
||||||
|
if strings.HasPrefix(path, "/api/live/events") {
|
||||||
|
return "live_events"
|
||||||
|
}
|
||||||
|
if strings.HasPrefix(path, "/api/") {
|
||||||
|
rest := strings.TrimPrefix(path, "/api/")
|
||||||
|
i := strings.IndexByte(rest, '/')
|
||||||
|
if i < 0 {
|
||||||
|
return "proxy_root"
|
||||||
|
}
|
||||||
|
return "proxy_" + rest[i+1:]
|
||||||
|
}
|
||||||
|
return "ui"
|
||||||
|
}
|
||||||
|
|
||||||
func (g *Gateway) serve(w http.ResponseWriter, r *http.Request) {
|
func (g *Gateway) serve(w http.ResponseWriter, r *http.Request) {
|
||||||
switch r.URL.Path {
|
switch r.URL.Path {
|
||||||
case "/health":
|
case "/health":
|
||||||
@@ -233,6 +262,10 @@ func (g *Gateway) serve(w http.ResponseWriter, r *http.Request) {
|
|||||||
g.agg.ServeHTTP(w, r)
|
g.agg.ServeHTTP(w, r)
|
||||||
return
|
return
|
||||||
}
|
}
|
||||||
|
if r.URL.Path == "/api/live/events" {
|
||||||
|
g.serveLiveEvents(w, r)
|
||||||
|
return
|
||||||
|
}
|
||||||
if !strings.HasPrefix(r.URL.Path, prefix) {
|
if !strings.HasPrefix(r.URL.Path, prefix) {
|
||||||
g.webUI.ServeHTTP(w, r)
|
g.webUI.ServeHTTP(w, r)
|
||||||
return
|
return
|
||||||
@@ -265,6 +298,54 @@ func (g *Gateway) serve(w http.ResponseWriter, r *http.Request) {
|
|||||||
rp.ServeHTTP(w, r)
|
rp.ServeHTTP(w, r)
|
||||||
}
|
}
|
||||||
|
|
||||||
|
func (g *Gateway) serveLiveEvents(w http.ResponseWriter, r *http.Request) {
|
||||||
|
if r.Method != http.MethodGet {
|
||||||
|
http.Error(w, "method not allowed", http.StatusMethodNotAllowed)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
flusher, ok := w.(http.Flusher)
|
||||||
|
if !ok {
|
||||||
|
http.Error(w, "stream unsupported", http.StatusInternalServerError)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
aliases, err := g.agg.ResolveAliasesForLive(r)
|
||||||
|
if err != nil {
|
||||||
|
w.Header().Set("Content-Type", "application/json; charset=utf-8")
|
||||||
|
w.WriteHeader(http.StatusBadRequest)
|
||||||
|
_ = json.NewEncoder(w).Encode(map[string]any{
|
||||||
|
"ok": false,
|
||||||
|
"error": map[string]string{"code": "bad_request", "message": err.Error()},
|
||||||
|
})
|
||||||
|
return
|
||||||
|
}
|
||||||
|
w.Header().Set("Content-Type", "text/event-stream")
|
||||||
|
w.Header().Set("Cache-Control", "no-cache")
|
||||||
|
w.Header().Set("Connection", "keep-alive")
|
||||||
|
w.WriteHeader(http.StatusOK)
|
||||||
|
|
||||||
|
ticker := time.NewTicker(5 * time.Second)
|
||||||
|
defer ticker.Stop()
|
||||||
|
|
||||||
|
push := func() {
|
||||||
|
ctx, cancel := context.WithTimeout(r.Context(), 10*time.Second)
|
||||||
|
defer cancel()
|
||||||
|
payload := aggregate.BuildLiveEnvelope(ctx, g.agg, aliases)
|
||||||
|
b, _ := json.Marshal(payload)
|
||||||
|
_, _ = fmt.Fprintf(w, "event: snapshot\n")
|
||||||
|
_, _ = fmt.Fprintf(w, "data: %s\n\n", string(b))
|
||||||
|
flusher.Flush()
|
||||||
|
}
|
||||||
|
push()
|
||||||
|
for {
|
||||||
|
select {
|
||||||
|
case <-r.Context().Done():
|
||||||
|
return
|
||||||
|
case <-ticker.C:
|
||||||
|
push()
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
type statusWriter struct {
|
type statusWriter struct {
|
||||||
http.ResponseWriter
|
http.ResponseWriter
|
||||||
status int
|
status int
|
||||||
|
|||||||
@@ -15,17 +15,17 @@ var (
|
|||||||
})
|
})
|
||||||
httpRequests = promauto.NewCounterVec(prometheus.CounterOpts{
|
httpRequests = promauto.NewCounterVec(prometheus.CounterOpts{
|
||||||
Name: "telemt_gateway_http_requests_total",
|
Name: "telemt_gateway_http_requests_total",
|
||||||
Help: "HTTP requests by status, method, alias.",
|
Help: "HTTP requests by status, method, alias, endpoint.",
|
||||||
}, []string{"code", "method", "alias"})
|
}, []string{"code", "method", "alias", "endpoint"})
|
||||||
httpDuration = promauto.NewHistogramVec(prometheus.HistogramOpts{
|
httpDuration = promauto.NewHistogramVec(prometheus.HistogramOpts{
|
||||||
Name: "telemt_gateway_http_request_duration_seconds",
|
Name: "telemt_gateway_http_request_duration_seconds",
|
||||||
Help: "Request duration in seconds.",
|
Help: "Request duration in seconds.",
|
||||||
Buckets: prometheus.DefBuckets,
|
Buckets: prometheus.DefBuckets,
|
||||||
}, []string{"method", "alias"})
|
}, []string{"method", "alias", "endpoint"})
|
||||||
)
|
)
|
||||||
|
|
||||||
func observeRequest(method, alias string, status int, started time.Time) {
|
func observeRequest(method, alias, endpoint string, status int, started time.Time) {
|
||||||
httpInFlight.Dec()
|
httpInFlight.Dec()
|
||||||
httpRequests.WithLabelValues(strconv.Itoa(status), method, alias).Inc()
|
httpRequests.WithLabelValues(strconv.Itoa(status), method, alias, endpoint).Inc()
|
||||||
httpDuration.WithLabelValues(method, alias).Observe(time.Since(started).Seconds())
|
httpDuration.WithLabelValues(method, alias, endpoint).Observe(time.Since(started).Seconds())
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -2,6 +2,14 @@
|
|||||||
|
|
||||||
SvelteKit + shadcn-svelte. В **production** статика собирается и **встраивается в образ шлюза** ([Dockerfile](../Dockerfile) в корне репозитория): панель и API на **одном порту** (например `http://127.0.0.1:8080/` — UI, `/api/…` — шлюз).
|
SvelteKit + shadcn-svelte. В **production** статика собирается и **встраивается в образ шлюза** ([Dockerfile](../Dockerfile) в корне репозитория): панель и API на **одном порту** (например `http://127.0.0.1:8080/` — UI, `/api/…` — шлюз).
|
||||||
|
|
||||||
|
## Основные разделы панели
|
||||||
|
|
||||||
|
- `/` — обзор флота (KPI, активные IP, сводка по нодам)
|
||||||
|
- `/users`, `/users/[username]` — пользователи и детали
|
||||||
|
- `/ips` — unique IP + GeoIP карта
|
||||||
|
- `/incidents` — triage-интерфейс инцидентов (`ack/resolved/owner/note` в localStorage)
|
||||||
|
- `/live` — live snapshot (SSE, авто-reconnect)
|
||||||
|
|
||||||
## Переменная `PUBLIC_TELEMT_GATEWAY_URL`
|
## Переменная `PUBLIC_TELEMT_GATEWAY_URL`
|
||||||
|
|
||||||
| Значение | Когда |
|
| Значение | Когда |
|
||||||
@@ -42,6 +50,17 @@ npm run gen:api
|
|||||||
|
|
||||||
Источник: [../docs/AGGREGATE_OPENAPI.yaml](../docs/AGGREGATE_OPENAPI.yaml).
|
Источник: [../docs/AGGREGATE_OPENAPI.yaml](../docs/AGGREGATE_OPENAPI.yaml).
|
||||||
|
|
||||||
|
## URL-driven controls (операторский режим)
|
||||||
|
|
||||||
|
На ключевых страницах используются query-параметры:
|
||||||
|
|
||||||
|
- `aliases=node-a,node-b` — фильтр по нодам
|
||||||
|
- `refresh=0|10..300` — auto-refresh (0 выключает polling)
|
||||||
|
- `include_links=0|1` — для `/users`
|
||||||
|
- `geo=0|1` — для `/ips`
|
||||||
|
|
||||||
|
Live-страница `/live` работает через SSE endpoint шлюза: `/api/live/events`.
|
||||||
|
|
||||||
## Ограничения
|
## Ограничения
|
||||||
|
|
||||||
- Секреты upstream к Telemt задаются на шлюзе (`authorization_env`), не в браузере.
|
- Секреты upstream к Telemt задаются на шлюзе (`authorization_env`), не в браузере.
|
||||||
|
|||||||
@@ -20,6 +20,37 @@ export type AggEnvelope<T> = {
|
|||||||
data: T;
|
data: T;
|
||||||
};
|
};
|
||||||
|
|
||||||
|
export type IncidentSeverity = 'info' | 'warning' | 'critical';
|
||||||
|
|
||||||
|
export type IncidentStatus = 'firing';
|
||||||
|
|
||||||
|
export type IncidentAction = {
|
||||||
|
label: string;
|
||||||
|
href: string;
|
||||||
|
};
|
||||||
|
|
||||||
|
export type IncidentItem = {
|
||||||
|
id: string;
|
||||||
|
kind: string;
|
||||||
|
severity: IncidentSeverity;
|
||||||
|
status: IncidentStatus;
|
||||||
|
title: string;
|
||||||
|
summary: string;
|
||||||
|
affected_aliases?: string[];
|
||||||
|
metric_name?: string;
|
||||||
|
metric_value?: number;
|
||||||
|
metric_threshold?: number;
|
||||||
|
actions?: IncidentAction[];
|
||||||
|
};
|
||||||
|
|
||||||
|
export type IncidentsData = {
|
||||||
|
items: IncidentItem[];
|
||||||
|
total: number;
|
||||||
|
critical_total: number;
|
||||||
|
warning_total: number;
|
||||||
|
info_total: number;
|
||||||
|
};
|
||||||
|
|
||||||
export class ApiError extends Error {
|
export class ApiError extends Error {
|
||||||
constructor(
|
constructor(
|
||||||
message: string,
|
message: string,
|
||||||
@@ -120,6 +151,23 @@ export async function fetchAggUser(
|
|||||||
return body as AggEnvelope<components['schemas']['UsersRow']>;
|
return body as AggEnvelope<components['schemas']['UsersRow']>;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
export async function fetchAggIncidents(params?: { aliases?: string }): Promise<AggEnvelope<IncidentsData>> {
|
||||||
|
const q = new URLSearchParams();
|
||||||
|
if (params?.aliases) q.set('aliases', params.aliases);
|
||||||
|
const url = `${gatewayBase()}/api/agg/incidents${q.toString() ? `?${q}` : ''}`;
|
||||||
|
const res = await fetch(url);
|
||||||
|
const body = (await parseJson(res)) as Record<string, unknown> | null;
|
||||||
|
if (!res.ok) throw new ApiError(`incidents HTTP ${res.status}`, res.status, body);
|
||||||
|
if (!body || body.ok !== true) throw new ApiError('incidents: ok !== true', res.status, body);
|
||||||
|
return body as AggEnvelope<IncidentsData>;
|
||||||
|
}
|
||||||
|
|
||||||
|
export function liveEventsUrl(params?: { aliases?: string }): string {
|
||||||
|
const q = new URLSearchParams();
|
||||||
|
if (params?.aliases) q.set('aliases', params.aliases);
|
||||||
|
return `${gatewayBase()}/api/live/events${q.toString() ? `?${q}` : ''}`;
|
||||||
|
}
|
||||||
|
|
||||||
/** Путь к upstream без префикса /v1 — шлюз сам добавляет path_prefix. */
|
/** Путь к upstream без префикса /v1 — шлюз сам добавляет path_prefix. */
|
||||||
function apiUrl(alias: string, path: string): string {
|
function apiUrl(alias: string, path: string): string {
|
||||||
const p = path.replace(/^\/+/, '');
|
const p = path.replace(/^\/+/, '');
|
||||||
|
|||||||
@@ -9,6 +9,8 @@
|
|||||||
import NetworkIcon from '@lucide/svelte/icons/network';
|
import NetworkIcon from '@lucide/svelte/icons/network';
|
||||||
import SettingsIcon from '@lucide/svelte/icons/settings';
|
import SettingsIcon from '@lucide/svelte/icons/settings';
|
||||||
import RefreshCwIcon from '@lucide/svelte/icons/refresh-cw';
|
import RefreshCwIcon from '@lucide/svelte/icons/refresh-cw';
|
||||||
|
import SirenIcon from '@lucide/svelte/icons/siren';
|
||||||
|
import RadioIcon from '@lucide/svelte/icons/radio';
|
||||||
import * as Sidebar from '$lib/components/ui/sidebar/index.js';
|
import * as Sidebar from '$lib/components/ui/sidebar/index.js';
|
||||||
|
|
||||||
let {
|
let {
|
||||||
@@ -76,6 +78,26 @@
|
|||||||
{/snippet}
|
{/snippet}
|
||||||
</Sidebar.MenuButton>
|
</Sidebar.MenuButton>
|
||||||
</Sidebar.MenuItem>
|
</Sidebar.MenuItem>
|
||||||
|
<Sidebar.MenuItem>
|
||||||
|
<Sidebar.MenuButton isActive={active('/incidents')} tooltipContent="Incidents">
|
||||||
|
{#snippet child({ props })}
|
||||||
|
<a href="/incidents" {...props}>
|
||||||
|
<SirenIcon />
|
||||||
|
<span>Инциденты</span>
|
||||||
|
</a>
|
||||||
|
{/snippet}
|
||||||
|
</Sidebar.MenuButton>
|
||||||
|
</Sidebar.MenuItem>
|
||||||
|
<Sidebar.MenuItem>
|
||||||
|
<Sidebar.MenuButton isActive={active('/live')} tooltipContent="Live">
|
||||||
|
{#snippet child({ props })}
|
||||||
|
<a href="/live" {...props}>
|
||||||
|
<RadioIcon />
|
||||||
|
<span>Live</span>
|
||||||
|
</a>
|
||||||
|
{/snippet}
|
||||||
|
</Sidebar.MenuButton>
|
||||||
|
</Sidebar.MenuItem>
|
||||||
</Sidebar.Menu>
|
</Sidebar.Menu>
|
||||||
</Sidebar.Group>
|
</Sidebar.Group>
|
||||||
{#if serverAliases.length > 0}
|
{#if serverAliases.length > 0}
|
||||||
|
|||||||
+112
-8
@@ -1,5 +1,7 @@
|
|||||||
<script lang="ts">
|
<script lang="ts">
|
||||||
import { onMount } from 'svelte';
|
import { onMount } from 'svelte';
|
||||||
|
import { page } from '$app/state';
|
||||||
|
import { goto } from '$app/navigation';
|
||||||
import {
|
import {
|
||||||
ApiError,
|
ApiError,
|
||||||
fetchAggFleetStatus,
|
fetchAggFleetStatus,
|
||||||
@@ -73,6 +75,14 @@
|
|||||||
let uniqueIps = $state<components['schemas']['UniqueIPsRow'][] | null>(null);
|
let uniqueIps = $state<components['schemas']['UniqueIPsRow'][] | null>(null);
|
||||||
let generatedAt = $state<string | null>(null);
|
let generatedAt = $state<string | null>(null);
|
||||||
let partial = $state(false);
|
let partial = $state(false);
|
||||||
|
let aliases = $state('');
|
||||||
|
let aliasesInput = $state('');
|
||||||
|
let refreshSeconds = $state(30);
|
||||||
|
let refreshInput = $state('30');
|
||||||
|
let lastSuccessAtMs = $state<number | null>(null);
|
||||||
|
let nowMs = $state(Date.now());
|
||||||
|
let pollTimer: ReturnType<typeof setInterval> | null = null;
|
||||||
|
let staleTimer: ReturnType<typeof setInterval> | null = null;
|
||||||
let nodeStats = $state<
|
let nodeStats = $state<
|
||||||
Record<
|
Record<
|
||||||
string,
|
string,
|
||||||
@@ -85,24 +95,25 @@
|
|||||||
err = null;
|
err = null;
|
||||||
try {
|
try {
|
||||||
const [s, f, u] = await Promise.all([
|
const [s, f, u] = await Promise.all([
|
||||||
fetchAggSummary({ top_n: 15 }),
|
fetchAggSummary({ top_n: 15, aliases: aliases || undefined }),
|
||||||
fetchAggFleetStatus(),
|
fetchAggFleetStatus({ aliases: aliases || undefined }),
|
||||||
fetchAggUniqueIps({ geo: true })
|
fetchAggUniqueIps({ aliases: aliases || undefined, geo: true })
|
||||||
]);
|
]);
|
||||||
generatedAt = s.generated_at;
|
generatedAt = s.generated_at;
|
||||||
partial = !!(s.partial || f.partial || u.partial);
|
partial = !!(s.partial || f.partial || u.partial);
|
||||||
summary = s.data;
|
summary = s.data;
|
||||||
fleet = f.data ?? null;
|
fleet = f.data ?? null;
|
||||||
uniqueIps = u.data ?? null;
|
uniqueIps = u.data ?? null;
|
||||||
|
lastSuccessAtMs = Date.now();
|
||||||
|
|
||||||
// Как бейдж «OK» в таблице: только health + system/info (без x.ok — в JSON поле ok опционально).
|
// Как бейдж «OK» в таблице: только health + system/info (без x.ok — в JSON поле ok опционально).
|
||||||
const aliases = (fleet?.servers ?? [])
|
const okAliases = (fleet?.servers ?? [])
|
||||||
.filter((x) => x.alias && x.health_ok && x.system_info_ok)
|
.filter((x) => x.alias && x.health_ok && x.system_info_ok)
|
||||||
.map((x) => x.alias as string);
|
.map((x) => x.alias as string);
|
||||||
const next: typeof nodeStats = {};
|
const next: typeof nodeStats = {};
|
||||||
const concurrency = 4;
|
const concurrency = 4;
|
||||||
for (let i = 0; i < aliases.length; i += concurrency) {
|
for (let i = 0; i < okAliases.length; i += concurrency) {
|
||||||
const batch = aliases.slice(i, i + concurrency);
|
const batch = okAliases.slice(i, i + concurrency);
|
||||||
await Promise.all(
|
await Promise.all(
|
||||||
batch.map(async (alias) => {
|
batch.map(async (alias) => {
|
||||||
try {
|
try {
|
||||||
@@ -130,7 +141,70 @@
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
onMount(load);
|
function parseRefresh(raw: string | null): number {
|
||||||
|
if (raw == null || raw.trim() === '') return 30;
|
||||||
|
const n = Number(raw);
|
||||||
|
if (!Number.isFinite(n)) return 30;
|
||||||
|
if (n === 0) return 0;
|
||||||
|
return Math.max(10, Math.min(300, Math.floor(n)));
|
||||||
|
}
|
||||||
|
|
||||||
|
function restartPolling() {
|
||||||
|
if (pollTimer) {
|
||||||
|
clearInterval(pollTimer);
|
||||||
|
pollTimer = null;
|
||||||
|
}
|
||||||
|
if (refreshSeconds > 0) {
|
||||||
|
pollTimer = setInterval(() => {
|
||||||
|
void load();
|
||||||
|
}, refreshSeconds * 1000);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
async function applyControls() {
|
||||||
|
const q = new URLSearchParams(page.url.searchParams);
|
||||||
|
const normalizedAliases = aliasesInput.trim();
|
||||||
|
const parsedRefresh = parseRefresh(refreshInput);
|
||||||
|
if (normalizedAliases) q.set('aliases', normalizedAliases);
|
||||||
|
else q.delete('aliases');
|
||||||
|
if (parsedRefresh === 30) q.delete('refresh');
|
||||||
|
else q.set('refresh', String(parsedRefresh));
|
||||||
|
const qs = q.toString();
|
||||||
|
await goto(`${page.url.pathname}${qs ? `?${qs}` : ''}`, {
|
||||||
|
replaceState: true,
|
||||||
|
noScroll: true,
|
||||||
|
keepFocus: true
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
|
let queryKey = $derived(page.url.searchParams.toString());
|
||||||
|
$effect(() => {
|
||||||
|
queryKey;
|
||||||
|
if (typeof window === 'undefined') return;
|
||||||
|
aliases = page.url.searchParams.get('aliases')?.trim() ?? '';
|
||||||
|
refreshSeconds = parseRefresh(page.url.searchParams.get('refresh'));
|
||||||
|
aliasesInput = aliases;
|
||||||
|
refreshInput = String(refreshSeconds);
|
||||||
|
restartPolling();
|
||||||
|
void load();
|
||||||
|
});
|
||||||
|
|
||||||
|
let staleSeconds = $derived.by(() =>
|
||||||
|
lastSuccessAtMs == null ? null : Math.max(0, Math.floor((nowMs - lastSuccessAtMs) / 1000))
|
||||||
|
);
|
||||||
|
let isStale = $derived.by(() =>
|
||||||
|
staleSeconds == null ? true : staleSeconds > (refreshSeconds > 0 ? refreshSeconds * 2 : 60)
|
||||||
|
);
|
||||||
|
|
||||||
|
onMount(() => {
|
||||||
|
staleTimer = setInterval(() => {
|
||||||
|
nowMs = Date.now();
|
||||||
|
}, 1000);
|
||||||
|
return () => {
|
||||||
|
if (pollTimer) clearInterval(pollTimer);
|
||||||
|
if (staleTimer) clearInterval(staleTimer);
|
||||||
|
};
|
||||||
|
});
|
||||||
|
|
||||||
/** Число из ответа Telemt (поле может отсутствовать или прийти строкой). */
|
/** Число из ответа Telemt (поле может отсутствовать или прийти строкой). */
|
||||||
function numStat(v: unknown, fallback = 0): number {
|
function numStat(v: unknown, fallback = 0): number {
|
||||||
@@ -284,11 +358,41 @@
|
|||||||
</p>
|
</p>
|
||||||
{/if}
|
{/if}
|
||||||
</div>
|
</div>
|
||||||
<Button variant="outline" size="sm" onclick={load} disabled={loading}>
|
<div class="flex items-center gap-2">
|
||||||
|
<Badge variant={isStale ? 'secondary' : 'default'}>
|
||||||
|
{isStale ? 'stale' : 'live'}{#if staleSeconds != null} · {staleSeconds}s{/if}
|
||||||
|
</Badge>
|
||||||
|
<Button variant="outline" size="sm" onclick={() => void load()} disabled={loading}>
|
||||||
<RefreshCwIcon class="mr-1 size-4 {loading ? 'animate-spin' : ''}" />
|
<RefreshCwIcon class="mr-1 size-4 {loading ? 'animate-spin' : ''}" />
|
||||||
Обновить
|
Обновить
|
||||||
</Button>
|
</Button>
|
||||||
</div>
|
</div>
|
||||||
|
</div>
|
||||||
|
|
||||||
|
<div class="mb-4 flex flex-wrap items-end gap-2">
|
||||||
|
<label class="text-xs text-muted-foreground">
|
||||||
|
Aliases
|
||||||
|
<input
|
||||||
|
type="text"
|
||||||
|
class="mt-1 h-9 w-56 rounded-md border bg-background px-2 text-sm"
|
||||||
|
bind:value={aliasesInput}
|
||||||
|
placeholder="node-a,node-b"
|
||||||
|
/>
|
||||||
|
</label>
|
||||||
|
<label class="text-xs text-muted-foreground">
|
||||||
|
Refresh (sec)
|
||||||
|
<input
|
||||||
|
type="number"
|
||||||
|
min="0"
|
||||||
|
max="300"
|
||||||
|
class="mt-1 h-9 w-28 rounded-md border bg-background px-2 text-sm"
|
||||||
|
bind:value={refreshInput}
|
||||||
|
/>
|
||||||
|
</label>
|
||||||
|
<Button variant="outline" size="sm" onclick={() => void applyControls()} disabled={loading}>
|
||||||
|
Apply
|
||||||
|
</Button>
|
||||||
|
</div>
|
||||||
|
|
||||||
{#if partial}
|
{#if partial}
|
||||||
<Alert variant="destructive" class="mb-4">
|
<Alert variant="destructive" class="mb-4">
|
||||||
|
|||||||
@@ -0,0 +1,351 @@
|
|||||||
|
<script lang="ts">
|
||||||
|
import { onMount } from 'svelte';
|
||||||
|
import { page } from '$app/state';
|
||||||
|
import { goto } from '$app/navigation';
|
||||||
|
import {
|
||||||
|
ApiError,
|
||||||
|
fetchAggIncidents,
|
||||||
|
type IncidentItem,
|
||||||
|
type IncidentsData
|
||||||
|
} from '$lib/api/client.js';
|
||||||
|
import * as Card from '$lib/components/ui/card/index.js';
|
||||||
|
import * as Table from '$lib/components/ui/table/index.js';
|
||||||
|
import { Alert, AlertDescription, AlertTitle } from '$lib/components/ui/alert/index.js';
|
||||||
|
import { Badge } from '$lib/components/ui/badge/index.js';
|
||||||
|
import { Button } from '$lib/components/ui/button/index.js';
|
||||||
|
import RefreshCwIcon from '@lucide/svelte/icons/refresh-cw';
|
||||||
|
|
||||||
|
type TriageState = {
|
||||||
|
ack?: boolean;
|
||||||
|
resolved?: boolean;
|
||||||
|
owner?: string;
|
||||||
|
note?: string;
|
||||||
|
};
|
||||||
|
|
||||||
|
const TRIAGE_KEY = 'telemt.incidents.triage.v1';
|
||||||
|
|
||||||
|
let loading = $state(true);
|
||||||
|
let err = $state<string | null>(null);
|
||||||
|
let partial = $state(false);
|
||||||
|
let generatedAt = $state<string | null>(null);
|
||||||
|
let data = $state<IncidentsData | null>(null);
|
||||||
|
let aliases = $state('');
|
||||||
|
let aliasesInput = $state('');
|
||||||
|
let refreshSeconds = $state(30);
|
||||||
|
let refreshInput = $state('30');
|
||||||
|
let lastSuccessAtMs = $state<number | null>(null);
|
||||||
|
let nowMs = $state(Date.now());
|
||||||
|
let pollTimer: ReturnType<typeof setInterval> | null = null;
|
||||||
|
let staleTimer: ReturnType<typeof setInterval> | null = null;
|
||||||
|
let triageById = $state<Record<string, TriageState>>({});
|
||||||
|
|
||||||
|
function parseRefresh(raw: string | null): number {
|
||||||
|
if (raw == null || raw.trim() === '') return 30;
|
||||||
|
const n = Number(raw);
|
||||||
|
if (!Number.isFinite(n)) return 30;
|
||||||
|
if (n === 0) return 0;
|
||||||
|
return Math.max(10, Math.min(300, Math.floor(n)));
|
||||||
|
}
|
||||||
|
|
||||||
|
function triageFor(id: string): TriageState {
|
||||||
|
return triageById[id] ?? {};
|
||||||
|
}
|
||||||
|
|
||||||
|
function saveTriage() {
|
||||||
|
if (typeof localStorage === 'undefined') return;
|
||||||
|
localStorage.setItem(TRIAGE_KEY, JSON.stringify(triageById));
|
||||||
|
}
|
||||||
|
|
||||||
|
function loadTriage() {
|
||||||
|
if (typeof localStorage === 'undefined') return;
|
||||||
|
try {
|
||||||
|
const raw = localStorage.getItem(TRIAGE_KEY);
|
||||||
|
if (!raw) return;
|
||||||
|
const parsed = JSON.parse(raw) as Record<string, TriageState>;
|
||||||
|
if (parsed && typeof parsed === 'object') triageById = parsed;
|
||||||
|
} catch {
|
||||||
|
triageById = {};
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
function patchTriage(id: string, patch: Partial<TriageState>) {
|
||||||
|
triageById = {
|
||||||
|
...triageById,
|
||||||
|
[id]: {
|
||||||
|
...triageFor(id),
|
||||||
|
...patch
|
||||||
|
}
|
||||||
|
};
|
||||||
|
saveTriage();
|
||||||
|
}
|
||||||
|
|
||||||
|
function toggleAck(id: string) {
|
||||||
|
const t = triageFor(id);
|
||||||
|
patchTriage(id, { ack: !t.ack });
|
||||||
|
}
|
||||||
|
|
||||||
|
function toggleResolved(id: string) {
|
||||||
|
const t = triageFor(id);
|
||||||
|
patchTriage(id, { resolved: !t.resolved });
|
||||||
|
}
|
||||||
|
|
||||||
|
function severityVariant(sev: IncidentItem['severity']): 'default' | 'secondary' | 'destructive' | 'outline' {
|
||||||
|
if (sev === 'critical') return 'destructive';
|
||||||
|
if (sev === 'warning') return 'secondary';
|
||||||
|
return 'outline';
|
||||||
|
}
|
||||||
|
|
||||||
|
function restartPolling() {
|
||||||
|
if (pollTimer) {
|
||||||
|
clearInterval(pollTimer);
|
||||||
|
pollTimer = null;
|
||||||
|
}
|
||||||
|
if (refreshSeconds > 0) {
|
||||||
|
pollTimer = setInterval(() => {
|
||||||
|
void load();
|
||||||
|
}, refreshSeconds * 1000);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
async function load() {
|
||||||
|
loading = true;
|
||||||
|
err = null;
|
||||||
|
try {
|
||||||
|
const env = await fetchAggIncidents({ aliases: aliases || undefined });
|
||||||
|
data = env.data ?? null;
|
||||||
|
partial = !!env.partial;
|
||||||
|
generatedAt = env.generated_at;
|
||||||
|
lastSuccessAtMs = Date.now();
|
||||||
|
} catch (e) {
|
||||||
|
err = e instanceof ApiError ? e.message : String(e);
|
||||||
|
} finally {
|
||||||
|
loading = false;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
async function applyControls() {
|
||||||
|
const q = new URLSearchParams(page.url.searchParams);
|
||||||
|
const normalizedAliases = aliasesInput.trim();
|
||||||
|
const parsedRefresh = parseRefresh(refreshInput);
|
||||||
|
if (normalizedAliases) q.set('aliases', normalizedAliases);
|
||||||
|
else q.delete('aliases');
|
||||||
|
if (parsedRefresh === 30) q.delete('refresh');
|
||||||
|
else q.set('refresh', String(parsedRefresh));
|
||||||
|
const qs = q.toString();
|
||||||
|
await goto(`${page.url.pathname}${qs ? `?${qs}` : ''}`, {
|
||||||
|
replaceState: true,
|
||||||
|
noScroll: true,
|
||||||
|
keepFocus: true
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
|
let queryKey = $derived(page.url.searchParams.toString());
|
||||||
|
$effect(() => {
|
||||||
|
queryKey;
|
||||||
|
if (typeof window === 'undefined') return;
|
||||||
|
aliases = page.url.searchParams.get('aliases')?.trim() ?? '';
|
||||||
|
refreshSeconds = parseRefresh(page.url.searchParams.get('refresh'));
|
||||||
|
aliasesInput = aliases;
|
||||||
|
refreshInput = String(refreshSeconds);
|
||||||
|
restartPolling();
|
||||||
|
void load();
|
||||||
|
});
|
||||||
|
|
||||||
|
let staleSeconds = $derived.by(() =>
|
||||||
|
lastSuccessAtMs == null ? null : Math.max(0, Math.floor((nowMs - lastSuccessAtMs) / 1000))
|
||||||
|
);
|
||||||
|
let isStale = $derived.by(() =>
|
||||||
|
staleSeconds == null ? true : staleSeconds > (refreshSeconds > 0 ? refreshSeconds * 2 : 60)
|
||||||
|
);
|
||||||
|
|
||||||
|
onMount(() => {
|
||||||
|
loadTriage();
|
||||||
|
staleTimer = setInterval(() => {
|
||||||
|
nowMs = Date.now();
|
||||||
|
}, 1000);
|
||||||
|
return () => {
|
||||||
|
if (pollTimer) clearInterval(pollTimer);
|
||||||
|
if (staleTimer) clearInterval(staleTimer);
|
||||||
|
};
|
||||||
|
});
|
||||||
|
</script>
|
||||||
|
|
||||||
|
<div class="mb-6 flex flex-wrap items-center justify-between gap-4">
|
||||||
|
<div>
|
||||||
|
<h1 class="text-2xl font-semibold tracking-tight">Инциденты</h1>
|
||||||
|
<p class="text-sm text-muted-foreground">Сводка и triage по всему флоту.</p>
|
||||||
|
{#if generatedAt}
|
||||||
|
<p class="mt-1 text-xs text-muted-foreground">Снимок: {new Date(generatedAt).toLocaleString()}</p>
|
||||||
|
{/if}
|
||||||
|
</div>
|
||||||
|
<div class="flex items-center gap-2">
|
||||||
|
<Badge variant={isStale ? 'secondary' : 'default'}>
|
||||||
|
{isStale ? 'stale' : 'live'}{#if staleSeconds != null} · {staleSeconds}s{/if}
|
||||||
|
</Badge>
|
||||||
|
<Button variant="outline" size="sm" onclick={() => void load()} disabled={loading}>
|
||||||
|
<RefreshCwIcon class="mr-1 size-4 {loading ? 'animate-spin' : ''}" />
|
||||||
|
Обновить
|
||||||
|
</Button>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
|
||||||
|
<div class="mb-4 flex flex-wrap items-end gap-2">
|
||||||
|
<label class="text-xs text-muted-foreground">
|
||||||
|
Aliases
|
||||||
|
<input
|
||||||
|
type="text"
|
||||||
|
class="mt-1 h-9 w-56 rounded-md border bg-background px-2 text-sm"
|
||||||
|
bind:value={aliasesInput}
|
||||||
|
placeholder="node-a,node-b"
|
||||||
|
/>
|
||||||
|
</label>
|
||||||
|
<label class="text-xs text-muted-foreground">
|
||||||
|
Refresh (sec)
|
||||||
|
<input
|
||||||
|
type="number"
|
||||||
|
min="0"
|
||||||
|
max="300"
|
||||||
|
class="mt-1 h-9 w-28 rounded-md border bg-background px-2 text-sm"
|
||||||
|
bind:value={refreshInput}
|
||||||
|
/>
|
||||||
|
</label>
|
||||||
|
<Button variant="outline" size="sm" onclick={() => void applyControls()} disabled={loading}>
|
||||||
|
Apply
|
||||||
|
</Button>
|
||||||
|
</div>
|
||||||
|
|
||||||
|
{#if partial}
|
||||||
|
<Alert class="mb-4">
|
||||||
|
<AlertTitle>Частичные данные</AlertTitle>
|
||||||
|
<AlertDescription>Не все upstream ответили успешно.</AlertDescription>
|
||||||
|
</Alert>
|
||||||
|
{/if}
|
||||||
|
|
||||||
|
{#if err}
|
||||||
|
<Alert variant="destructive">
|
||||||
|
<AlertTitle>Ошибка</AlertTitle>
|
||||||
|
<AlertDescription>{err}</AlertDescription>
|
||||||
|
</Alert>
|
||||||
|
{:else}
|
||||||
|
<div class="mb-4 grid gap-4 sm:grid-cols-3">
|
||||||
|
<Card.Root>
|
||||||
|
<Card.Header class="pb-2">
|
||||||
|
<Card.Description>Critical</Card.Description>
|
||||||
|
<Card.Title class="text-2xl text-red-500 tabular-nums">{data?.critical_total ?? 0}</Card.Title>
|
||||||
|
</Card.Header>
|
||||||
|
</Card.Root>
|
||||||
|
<Card.Root>
|
||||||
|
<Card.Header class="pb-2">
|
||||||
|
<Card.Description>Warning</Card.Description>
|
||||||
|
<Card.Title class="text-2xl text-amber-500 tabular-nums">{data?.warning_total ?? 0}</Card.Title>
|
||||||
|
</Card.Header>
|
||||||
|
</Card.Root>
|
||||||
|
<Card.Root>
|
||||||
|
<Card.Header class="pb-2">
|
||||||
|
<Card.Description>Info</Card.Description>
|
||||||
|
<Card.Title class="text-2xl tabular-nums">{data?.info_total ?? 0}</Card.Title>
|
||||||
|
</Card.Header>
|
||||||
|
</Card.Root>
|
||||||
|
</div>
|
||||||
|
|
||||||
|
<Card.Root>
|
||||||
|
<Card.Content class="p-0">
|
||||||
|
<Table.Root>
|
||||||
|
<Table.Header>
|
||||||
|
<Table.Row>
|
||||||
|
<Table.Head>Severity</Table.Head>
|
||||||
|
<Table.Head>Инцидент</Table.Head>
|
||||||
|
<Table.Head>Серверы</Table.Head>
|
||||||
|
<Table.Head>Runbook</Table.Head>
|
||||||
|
<Table.Head>Triage</Table.Head>
|
||||||
|
</Table.Row>
|
||||||
|
</Table.Header>
|
||||||
|
<Table.Body>
|
||||||
|
{#if (data?.items.length ?? 0) === 0}
|
||||||
|
<Table.Row>
|
||||||
|
<Table.Cell colspan={5} class="p-4 text-center text-muted-foreground">
|
||||||
|
Инцидентов нет
|
||||||
|
</Table.Cell>
|
||||||
|
</Table.Row>
|
||||||
|
{:else}
|
||||||
|
{#each data?.items ?? [] as item (item.id)}
|
||||||
|
<Table.Row>
|
||||||
|
<Table.Cell>
|
||||||
|
<Badge variant={severityVariant(item.severity)}>{item.severity}</Badge>
|
||||||
|
</Table.Cell>
|
||||||
|
<Table.Cell class="max-w-[360px]">
|
||||||
|
<div class="font-medium">{item.title}</div>
|
||||||
|
<div class="text-xs text-muted-foreground">{item.summary}</div>
|
||||||
|
{#if item.metric_name}
|
||||||
|
<div class="mt-1 text-xs text-muted-foreground">
|
||||||
|
{item.metric_name}: {item.metric_value} / {item.metric_threshold}
|
||||||
|
</div>
|
||||||
|
{/if}
|
||||||
|
</Table.Cell>
|
||||||
|
<Table.Cell>
|
||||||
|
<div class="flex flex-wrap gap-1">
|
||||||
|
{#each item.affected_aliases ?? [] as alias (alias)}
|
||||||
|
<Badge variant="outline">{alias}</Badge>
|
||||||
|
{/each}
|
||||||
|
</div>
|
||||||
|
</Table.Cell>
|
||||||
|
<Table.Cell class="max-w-[240px]">
|
||||||
|
<div class="flex flex-wrap gap-1">
|
||||||
|
{#each item.actions ?? [] as action (action.label + action.href)}
|
||||||
|
<a href={action.href} class="text-primary hover:underline text-xs">
|
||||||
|
{action.label}
|
||||||
|
</a>
|
||||||
|
{/each}
|
||||||
|
</div>
|
||||||
|
</Table.Cell>
|
||||||
|
<Table.Cell class="min-w-[280px]">
|
||||||
|
{@const triage = triageFor(item.id)}
|
||||||
|
<div class="mb-2 flex flex-wrap gap-1">
|
||||||
|
<Button
|
||||||
|
size="xs"
|
||||||
|
variant={triage.ack ? 'default' : 'outline'}
|
||||||
|
onclick={() => toggleAck(item.id)}
|
||||||
|
>
|
||||||
|
{triage.ack ? 'unack' : 'ack'}
|
||||||
|
</Button>
|
||||||
|
<Button
|
||||||
|
size="xs"
|
||||||
|
variant={triage.resolved ? 'default' : 'outline'}
|
||||||
|
onclick={() => toggleResolved(item.id)}
|
||||||
|
>
|
||||||
|
{triage.resolved ? 'unresolve' : 'resolve'}
|
||||||
|
</Button>
|
||||||
|
</div>
|
||||||
|
<div class="mb-1">
|
||||||
|
<input
|
||||||
|
type="text"
|
||||||
|
class="h-8 w-full rounded-md border bg-background px-2 text-xs"
|
||||||
|
placeholder="owner"
|
||||||
|
value={triage.owner ?? ''}
|
||||||
|
oninput={(e) =>
|
||||||
|
patchTriage(item.id, {
|
||||||
|
owner: (e.currentTarget as HTMLInputElement).value
|
||||||
|
})}
|
||||||
|
/>
|
||||||
|
</div>
|
||||||
|
<div>
|
||||||
|
<input
|
||||||
|
type="text"
|
||||||
|
class="h-8 w-full rounded-md border bg-background px-2 text-xs"
|
||||||
|
placeholder="note"
|
||||||
|
value={triage.note ?? ''}
|
||||||
|
oninput={(e) =>
|
||||||
|
patchTriage(item.id, {
|
||||||
|
note: (e.currentTarget as HTMLInputElement).value
|
||||||
|
})}
|
||||||
|
/>
|
||||||
|
</div>
|
||||||
|
</Table.Cell>
|
||||||
|
</Table.Row>
|
||||||
|
{/each}
|
||||||
|
{/if}
|
||||||
|
</Table.Body>
|
||||||
|
</Table.Root>
|
||||||
|
</Card.Content>
|
||||||
|
</Card.Root>
|
||||||
|
{/if}
|
||||||
@@ -1,5 +1,7 @@
|
|||||||
<script lang="ts">
|
<script lang="ts">
|
||||||
import { onMount, tick } from 'svelte';
|
import { onMount, tick } from 'svelte';
|
||||||
|
import { page } from '$app/state';
|
||||||
|
import { goto } from '$app/navigation';
|
||||||
import { fetchAggUniqueIps, ApiError } from '$lib/api/client.js';
|
import { fetchAggUniqueIps, ApiError } from '$lib/api/client.js';
|
||||||
import type { components } from '$lib/api/aggregate.gen.js';
|
import type { components } from '$lib/api/aggregate.gen.js';
|
||||||
import type * as Leaflet from 'leaflet';
|
import type * as Leaflet from 'leaflet';
|
||||||
@@ -17,6 +19,14 @@
|
|||||||
let rows = $state<components['schemas']['UniqueIPsRow'][]>([]);
|
let rows = $state<components['schemas']['UniqueIPsRow'][]>([]);
|
||||||
let partial = $state(false);
|
let partial = $state(false);
|
||||||
let geo = $state(true);
|
let geo = $state(true);
|
||||||
|
let aliases = $state('');
|
||||||
|
let aliasesInput = $state('');
|
||||||
|
let refreshSeconds = $state(30);
|
||||||
|
let refreshInput = $state('30');
|
||||||
|
let lastSuccessAtMs = $state<number | null>(null);
|
||||||
|
let nowMs = $state(Date.now());
|
||||||
|
let pollTimer: ReturnType<typeof setInterval> | null = null;
|
||||||
|
let staleTimer: ReturnType<typeof setInterval> | null = null;
|
||||||
|
|
||||||
let mapEl = $state<HTMLDivElement | null>(null);
|
let mapEl = $state<HTMLDivElement | null>(null);
|
||||||
let leaflet: typeof import('leaflet') | null = null;
|
let leaflet: typeof import('leaflet') | null = null;
|
||||||
@@ -273,9 +283,10 @@
|
|||||||
// Ждём, пока bind:this завершится и контейнеру карты будет доступен размер.
|
// Ждём, пока bind:this завершится и контейнеру карты будет доступен размер.
|
||||||
await tick();
|
await tick();
|
||||||
initMap();
|
initMap();
|
||||||
const env = await fetchAggUniqueIps({ geo });
|
const env = await fetchAggUniqueIps({ aliases: aliases || undefined, geo });
|
||||||
partial = !!env.partial;
|
partial = !!env.partial;
|
||||||
rows = env.data ?? [];
|
rows = env.data ?? [];
|
||||||
|
lastSuccessAtMs = Date.now();
|
||||||
} catch (e) {
|
} catch (e) {
|
||||||
err = e instanceof ApiError ? e.message : String(e);
|
err = e instanceof ApiError ? e.message : String(e);
|
||||||
} finally {
|
} finally {
|
||||||
@@ -288,10 +299,76 @@
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
onMount(() => {
|
function parseRefresh(raw: string | null): number {
|
||||||
// Инициализация Leaflet и загрузка данных происходит в `load()`.
|
if (raw == null || raw.trim() === '') return 30;
|
||||||
// Это избегает проблем с SSR (если он включён).
|
const n = Number(raw);
|
||||||
|
if (!Number.isFinite(n)) return 30;
|
||||||
|
if (n === 0) return 0;
|
||||||
|
return Math.max(10, Math.min(300, Math.floor(n)));
|
||||||
|
}
|
||||||
|
|
||||||
|
function parse01(raw: string | null, fallback: boolean): boolean {
|
||||||
|
if (raw == null || raw === '') return fallback;
|
||||||
|
return raw === '1';
|
||||||
|
}
|
||||||
|
|
||||||
|
function restartPolling() {
|
||||||
|
if (pollTimer) {
|
||||||
|
clearInterval(pollTimer);
|
||||||
|
pollTimer = null;
|
||||||
|
}
|
||||||
|
if (refreshSeconds > 0) {
|
||||||
|
pollTimer = setInterval(() => {
|
||||||
void load();
|
void load();
|
||||||
|
}, refreshSeconds * 1000);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
async function applyControls() {
|
||||||
|
const q = new URLSearchParams(page.url.searchParams);
|
||||||
|
const normalizedAliases = aliasesInput.trim();
|
||||||
|
const parsedRefresh = parseRefresh(refreshInput);
|
||||||
|
if (normalizedAliases) q.set('aliases', normalizedAliases);
|
||||||
|
else q.delete('aliases');
|
||||||
|
if (parsedRefresh === 30) q.delete('refresh');
|
||||||
|
else q.set('refresh', String(parsedRefresh));
|
||||||
|
q.set('geo', geo ? '1' : '0');
|
||||||
|
const qs = q.toString();
|
||||||
|
await goto(`${page.url.pathname}${qs ? `?${qs}` : ''}`, {
|
||||||
|
replaceState: true,
|
||||||
|
noScroll: true,
|
||||||
|
keepFocus: true
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
|
let queryKey = $derived(page.url.searchParams.toString());
|
||||||
|
$effect(() => {
|
||||||
|
queryKey;
|
||||||
|
if (typeof window === 'undefined') return;
|
||||||
|
aliases = page.url.searchParams.get('aliases')?.trim() ?? '';
|
||||||
|
refreshSeconds = parseRefresh(page.url.searchParams.get('refresh'));
|
||||||
|
geo = parse01(page.url.searchParams.get('geo'), true);
|
||||||
|
aliasesInput = aliases;
|
||||||
|
refreshInput = String(refreshSeconds);
|
||||||
|
restartPolling();
|
||||||
|
void load();
|
||||||
|
});
|
||||||
|
|
||||||
|
let staleSeconds = $derived.by(() =>
|
||||||
|
lastSuccessAtMs == null ? null : Math.max(0, Math.floor((nowMs - lastSuccessAtMs) / 1000))
|
||||||
|
);
|
||||||
|
let isStale = $derived.by(() =>
|
||||||
|
staleSeconds == null ? true : staleSeconds > (refreshSeconds > 0 ? refreshSeconds * 2 : 60)
|
||||||
|
);
|
||||||
|
|
||||||
|
onMount(() => {
|
||||||
|
staleTimer = setInterval(() => {
|
||||||
|
nowMs = Date.now();
|
||||||
|
}, 1000);
|
||||||
|
return () => {
|
||||||
|
if (pollTimer) clearInterval(pollTimer);
|
||||||
|
if (staleTimer) clearInterval(staleTimer);
|
||||||
|
};
|
||||||
});
|
});
|
||||||
</script>
|
</script>
|
||||||
|
|
||||||
@@ -301,17 +378,50 @@
|
|||||||
<p class="text-sm text-muted-foreground">Снимок active/recent с шлюза; GeoIP из конфига шлюза.</p>
|
<p class="text-sm text-muted-foreground">Снимок active/recent с шлюза; GeoIP из конфига шлюза.</p>
|
||||||
</div>
|
</div>
|
||||||
<div class="flex items-center gap-2">
|
<div class="flex items-center gap-2">
|
||||||
|
<Badge variant={isStale ? 'secondary' : 'default'}>
|
||||||
|
{isStale ? 'stale' : 'live'}{#if staleSeconds != null} · {staleSeconds}s{/if}
|
||||||
|
</Badge>
|
||||||
<label class="flex cursor-pointer items-center gap-2 text-sm">
|
<label class="flex cursor-pointer items-center gap-2 text-sm">
|
||||||
<input type="checkbox" bind:checked={geo} onchange={load} class="rounded border" />
|
<input
|
||||||
|
type="checkbox"
|
||||||
|
bind:checked={geo}
|
||||||
|
onchange={() => void applyControls()}
|
||||||
|
class="rounded border"
|
||||||
|
/>
|
||||||
GeoIP
|
GeoIP
|
||||||
</label>
|
</label>
|
||||||
<Button variant="outline" size="sm" onclick={load} disabled={loading}>
|
<Button variant="outline" size="sm" onclick={() => void load()} disabled={loading}>
|
||||||
<RefreshCwIcon class="mr-1 size-4 {loading ? 'animate-spin' : ''}" />
|
<RefreshCwIcon class="mr-1 size-4 {loading ? 'animate-spin' : ''}" />
|
||||||
Обновить
|
Обновить
|
||||||
</Button>
|
</Button>
|
||||||
</div>
|
</div>
|
||||||
</div>
|
</div>
|
||||||
|
|
||||||
|
<div class="mb-4 flex flex-wrap items-end gap-2">
|
||||||
|
<label class="text-xs text-muted-foreground">
|
||||||
|
Aliases
|
||||||
|
<input
|
||||||
|
type="text"
|
||||||
|
class="mt-1 h-9 w-56 rounded-md border bg-background px-2 text-sm"
|
||||||
|
bind:value={aliasesInput}
|
||||||
|
placeholder="node-a,node-b"
|
||||||
|
/>
|
||||||
|
</label>
|
||||||
|
<label class="text-xs text-muted-foreground">
|
||||||
|
Refresh (sec)
|
||||||
|
<input
|
||||||
|
type="number"
|
||||||
|
min="0"
|
||||||
|
max="300"
|
||||||
|
class="mt-1 h-9 w-28 rounded-md border bg-background px-2 text-sm"
|
||||||
|
bind:value={refreshInput}
|
||||||
|
/>
|
||||||
|
</label>
|
||||||
|
<Button variant="outline" size="sm" onclick={() => void applyControls()} disabled={loading}>
|
||||||
|
Apply
|
||||||
|
</Button>
|
||||||
|
</div>
|
||||||
|
|
||||||
{#if partial}
|
{#if partial}
|
||||||
<Alert class="mb-4">
|
<Alert class="mb-4">
|
||||||
<AlertTitle>Частичные данные</AlertTitle>
|
<AlertTitle>Частичные данные</AlertTitle>
|
||||||
|
|||||||
@@ -0,0 +1,348 @@
|
|||||||
|
<script lang="ts">
|
||||||
|
import { onMount } from 'svelte';
|
||||||
|
import { page } from '$app/state';
|
||||||
|
import { goto } from '$app/navigation';
|
||||||
|
import {
|
||||||
|
liveEventsUrl,
|
||||||
|
type IncidentItem,
|
||||||
|
type IncidentSeverity,
|
||||||
|
type IncidentStatus
|
||||||
|
} from '$lib/api/client.js';
|
||||||
|
import * as Card from '$lib/components/ui/card/index.js';
|
||||||
|
import { Alert, AlertDescription, AlertTitle } from '$lib/components/ui/alert/index.js';
|
||||||
|
import { Badge } from '$lib/components/ui/badge/index.js';
|
||||||
|
import { Button } from '$lib/components/ui/button/index.js';
|
||||||
|
import * as Table from '$lib/components/ui/table/index.js';
|
||||||
|
import RefreshCwIcon from '@lucide/svelte/icons/refresh-cw';
|
||||||
|
|
||||||
|
type LiveStatus = 'healthy' | 'degraded' | 'critical';
|
||||||
|
|
||||||
|
type LiveSnapshot = {
|
||||||
|
type?: string;
|
||||||
|
timestamp?: string;
|
||||||
|
status?: LiveStatus;
|
||||||
|
partial?: boolean;
|
||||||
|
incidents?: IncidentItem[];
|
||||||
|
counts?: {
|
||||||
|
total?: number;
|
||||||
|
critical?: number;
|
||||||
|
warning?: number;
|
||||||
|
info?: number;
|
||||||
|
};
|
||||||
|
aliases_used?: string[];
|
||||||
|
};
|
||||||
|
|
||||||
|
let aliases = $state('');
|
||||||
|
let aliasesInput = $state('');
|
||||||
|
let err = $state<string | null>(null);
|
||||||
|
let connected = $state(false);
|
||||||
|
let reconnectInSec = $state<number | null>(null);
|
||||||
|
let reconnectAttempt = $state(0);
|
||||||
|
let snapshot = $state<LiveSnapshot | null>(null);
|
||||||
|
let lastEventAtMs = $state<number | null>(null);
|
||||||
|
let nowMs = $state(Date.now());
|
||||||
|
let reconnectTimer: ReturnType<typeof setTimeout> | null = null;
|
||||||
|
let staleTimer: ReturnType<typeof setInterval> | null = null;
|
||||||
|
let es: EventSource | null = null;
|
||||||
|
|
||||||
|
function severityVariant(sev: IncidentSeverity): 'default' | 'secondary' | 'destructive' | 'outline' {
|
||||||
|
if (sev === 'critical') return 'destructive';
|
||||||
|
if (sev === 'warning') return 'secondary';
|
||||||
|
return 'outline';
|
||||||
|
}
|
||||||
|
|
||||||
|
function statusVariant(st: LiveStatus | undefined): 'default' | 'secondary' | 'destructive' {
|
||||||
|
if (st === 'critical') return 'destructive';
|
||||||
|
if (st === 'degraded') return 'secondary';
|
||||||
|
return 'default';
|
||||||
|
}
|
||||||
|
|
||||||
|
function normIncident(v: unknown): IncidentItem | null {
|
||||||
|
if (!v || typeof v !== 'object') return null;
|
||||||
|
const x = v as Record<string, unknown>;
|
||||||
|
const id = typeof x.id === 'string' ? x.id : '';
|
||||||
|
if (!id) return null;
|
||||||
|
const severity = (x.severity ?? 'info') as IncidentSeverity;
|
||||||
|
const status = (x.status ?? 'firing') as IncidentStatus;
|
||||||
|
return {
|
||||||
|
id,
|
||||||
|
kind: typeof x.kind === 'string' ? x.kind : '',
|
||||||
|
severity,
|
||||||
|
status,
|
||||||
|
title: typeof x.title === 'string' ? x.title : id,
|
||||||
|
summary: typeof x.summary === 'string' ? x.summary : '',
|
||||||
|
affected_aliases: Array.isArray(x.affected_aliases)
|
||||||
|
? x.affected_aliases.filter((a): a is string => typeof a === 'string')
|
||||||
|
: [],
|
||||||
|
metric_name: typeof x.metric_name === 'string' ? x.metric_name : undefined,
|
||||||
|
metric_value: typeof x.metric_value === 'number' ? x.metric_value : undefined,
|
||||||
|
metric_threshold: typeof x.metric_threshold === 'number' ? x.metric_threshold : undefined,
|
||||||
|
actions: Array.isArray(x.actions)
|
||||||
|
? x.actions
|
||||||
|
.filter((a) => a && typeof a === 'object')
|
||||||
|
.map((a) => a as Record<string, unknown>)
|
||||||
|
.filter((a) => typeof a.label === 'string' && typeof a.href === 'string')
|
||||||
|
.map((a) => ({ label: String(a.label), href: String(a.href) }))
|
||||||
|
: []
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
function parseSnapshot(raw: unknown): LiveSnapshot | null {
|
||||||
|
if (!raw || typeof raw !== 'object') return null;
|
||||||
|
const x = raw as Record<string, unknown>;
|
||||||
|
const incidents = Array.isArray(x.incidents) ? x.incidents.map(normIncident).filter(Boolean) : [];
|
||||||
|
const countsRaw = x.counts && typeof x.counts === 'object' ? (x.counts as Record<string, unknown>) : {};
|
||||||
|
return {
|
||||||
|
type: typeof x.type === 'string' ? x.type : undefined,
|
||||||
|
timestamp: typeof x.timestamp === 'string' ? x.timestamp : undefined,
|
||||||
|
status:
|
||||||
|
x.status === 'healthy' || x.status === 'degraded' || x.status === 'critical'
|
||||||
|
? x.status
|
||||||
|
: 'healthy',
|
||||||
|
partial: !!x.partial,
|
||||||
|
incidents: incidents as IncidentItem[],
|
||||||
|
counts: {
|
||||||
|
total: typeof countsRaw.total === 'number' ? countsRaw.total : incidents.length,
|
||||||
|
critical: typeof countsRaw.critical === 'number' ? countsRaw.critical : undefined,
|
||||||
|
warning: typeof countsRaw.warning === 'number' ? countsRaw.warning : undefined,
|
||||||
|
info: typeof countsRaw.info === 'number' ? countsRaw.info : undefined
|
||||||
|
},
|
||||||
|
aliases_used: Array.isArray(x.aliases_used)
|
||||||
|
? x.aliases_used.filter((a): a is string => typeof a === 'string')
|
||||||
|
: []
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
function clearReconnectTimer() {
|
||||||
|
if (!reconnectTimer) return;
|
||||||
|
clearTimeout(reconnectTimer);
|
||||||
|
reconnectTimer = null;
|
||||||
|
}
|
||||||
|
|
||||||
|
function closeStream() {
|
||||||
|
if (es) {
|
||||||
|
es.close();
|
||||||
|
es = null;
|
||||||
|
}
|
||||||
|
connected = false;
|
||||||
|
}
|
||||||
|
|
||||||
|
function scheduleReconnect() {
|
||||||
|
clearReconnectTimer();
|
||||||
|
const delayMs = Math.min(10_000, 1000 * 2 ** Math.min(8, reconnectAttempt));
|
||||||
|
reconnectAttempt += 1;
|
||||||
|
reconnectInSec = Math.ceil(delayMs / 1000);
|
||||||
|
const target = Date.now() + delayMs;
|
||||||
|
reconnectTimer = setTimeout(() => {
|
||||||
|
reconnectInSec = null;
|
||||||
|
connectStream();
|
||||||
|
}, delayMs);
|
||||||
|
const countdown = setInterval(() => {
|
||||||
|
const left = Math.max(0, Math.ceil((target - Date.now()) / 1000));
|
||||||
|
reconnectInSec = left;
|
||||||
|
if (left <= 0) clearInterval(countdown);
|
||||||
|
}, 250);
|
||||||
|
}
|
||||||
|
|
||||||
|
function connectStream() {
|
||||||
|
closeStream();
|
||||||
|
clearReconnectTimer();
|
||||||
|
err = null;
|
||||||
|
const url = liveEventsUrl({ aliases: aliases || undefined });
|
||||||
|
const next = new EventSource(url);
|
||||||
|
next.onopen = () => {
|
||||||
|
connected = true;
|
||||||
|
reconnectAttempt = 0;
|
||||||
|
reconnectInSec = null;
|
||||||
|
};
|
||||||
|
next.onmessage = (ev) => {
|
||||||
|
try {
|
||||||
|
const parsed = parseSnapshot(JSON.parse(ev.data));
|
||||||
|
if (!parsed) return;
|
||||||
|
snapshot = parsed;
|
||||||
|
lastEventAtMs = Date.now();
|
||||||
|
err = null;
|
||||||
|
connected = true;
|
||||||
|
} catch (e) {
|
||||||
|
err = `Невалидный snapshot: ${String(e)}`;
|
||||||
|
}
|
||||||
|
};
|
||||||
|
next.addEventListener('snapshot', (ev) => {
|
||||||
|
try {
|
||||||
|
const msg = ev as MessageEvent<string>;
|
||||||
|
const parsed = parseSnapshot(JSON.parse(msg.data));
|
||||||
|
if (!parsed) return;
|
||||||
|
snapshot = parsed;
|
||||||
|
lastEventAtMs = Date.now();
|
||||||
|
err = null;
|
||||||
|
connected = true;
|
||||||
|
} catch (e) {
|
||||||
|
err = `Невалидный snapshot: ${String(e)}`;
|
||||||
|
}
|
||||||
|
});
|
||||||
|
next.onerror = () => {
|
||||||
|
connected = false;
|
||||||
|
err = 'Поток SSE разорван, переподключение...';
|
||||||
|
next.close();
|
||||||
|
if (es === next) es = null;
|
||||||
|
scheduleReconnect();
|
||||||
|
};
|
||||||
|
es = next;
|
||||||
|
}
|
||||||
|
|
||||||
|
async function applyControls() {
|
||||||
|
const q = new URLSearchParams(page.url.searchParams);
|
||||||
|
const normalizedAliases = aliasesInput.trim();
|
||||||
|
if (normalizedAliases) q.set('aliases', normalizedAliases);
|
||||||
|
else q.delete('aliases');
|
||||||
|
const qs = q.toString();
|
||||||
|
await goto(`${page.url.pathname}${qs ? `?${qs}` : ''}`, {
|
||||||
|
replaceState: true,
|
||||||
|
noScroll: true,
|
||||||
|
keepFocus: true
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
|
let queryKey = $derived(page.url.searchParams.toString());
|
||||||
|
$effect(() => {
|
||||||
|
queryKey;
|
||||||
|
if (typeof window === 'undefined') return;
|
||||||
|
aliases = page.url.searchParams.get('aliases')?.trim() ?? '';
|
||||||
|
aliasesInput = aliases;
|
||||||
|
connectStream();
|
||||||
|
});
|
||||||
|
|
||||||
|
let staleSeconds = $derived.by(() =>
|
||||||
|
lastEventAtMs == null ? null : Math.max(0, Math.floor((nowMs - lastEventAtMs) / 1000))
|
||||||
|
);
|
||||||
|
let isStale = $derived.by(() => staleSeconds == null || staleSeconds > 12);
|
||||||
|
|
||||||
|
onMount(() => {
|
||||||
|
staleTimer = setInterval(() => {
|
||||||
|
nowMs = Date.now();
|
||||||
|
}, 1000);
|
||||||
|
return () => {
|
||||||
|
closeStream();
|
||||||
|
clearReconnectTimer();
|
||||||
|
if (staleTimer) clearInterval(staleTimer);
|
||||||
|
};
|
||||||
|
});
|
||||||
|
</script>
|
||||||
|
|
||||||
|
<div class="mb-6 flex flex-wrap items-center justify-between gap-4">
|
||||||
|
<div>
|
||||||
|
<h1 class="text-2xl font-semibold tracking-tight">Live</h1>
|
||||||
|
<p class="text-sm text-muted-foreground">SSE поток инцидентов и статуса флота.</p>
|
||||||
|
</div>
|
||||||
|
<div class="flex items-center gap-2">
|
||||||
|
<Badge variant={connected ? 'default' : 'secondary'}>
|
||||||
|
{connected ? 'connected' : 'disconnected'}
|
||||||
|
</Badge>
|
||||||
|
<Badge variant={isStale ? 'secondary' : 'default'}>
|
||||||
|
{isStale ? 'stale' : 'live'}{#if staleSeconds != null} · {staleSeconds}s{/if}
|
||||||
|
</Badge>
|
||||||
|
<Button variant="outline" size="sm" onclick={() => connectStream()}>
|
||||||
|
<RefreshCwIcon class="mr-1 size-4" />
|
||||||
|
Reconnect
|
||||||
|
</Button>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
|
||||||
|
<div class="mb-4 flex flex-wrap items-end gap-2">
|
||||||
|
<label class="text-xs text-muted-foreground">
|
||||||
|
Aliases
|
||||||
|
<input
|
||||||
|
type="text"
|
||||||
|
class="mt-1 h-9 w-56 rounded-md border bg-background px-2 text-sm"
|
||||||
|
bind:value={aliasesInput}
|
||||||
|
placeholder="node-a,node-b"
|
||||||
|
/>
|
||||||
|
</label>
|
||||||
|
<Button variant="outline" size="sm" onclick={() => void applyControls()}>
|
||||||
|
Apply
|
||||||
|
</Button>
|
||||||
|
</div>
|
||||||
|
|
||||||
|
{#if err}
|
||||||
|
<Alert class="mb-4">
|
||||||
|
<AlertTitle>Live stream</AlertTitle>
|
||||||
|
<AlertDescription>
|
||||||
|
{err}
|
||||||
|
{#if reconnectInSec != null}
|
||||||
|
Переподключение через {reconnectInSec}s.
|
||||||
|
{/if}
|
||||||
|
</AlertDescription>
|
||||||
|
</Alert>
|
||||||
|
{/if}
|
||||||
|
|
||||||
|
<div class="mb-4 grid gap-4 sm:grid-cols-3">
|
||||||
|
<Card.Root>
|
||||||
|
<Card.Header class="pb-2">
|
||||||
|
<Card.Description>Общий статус</Card.Description>
|
||||||
|
<Card.Title>
|
||||||
|
<Badge variant={statusVariant(snapshot?.status)}>
|
||||||
|
{snapshot?.status ?? 'healthy'}
|
||||||
|
</Badge>
|
||||||
|
</Card.Title>
|
||||||
|
</Card.Header>
|
||||||
|
</Card.Root>
|
||||||
|
<Card.Root>
|
||||||
|
<Card.Header class="pb-2">
|
||||||
|
<Card.Description>Partial</Card.Description>
|
||||||
|
<Card.Title class="text-2xl tabular-nums">{snapshot?.partial ? 'yes' : 'no'}</Card.Title>
|
||||||
|
</Card.Header>
|
||||||
|
</Card.Root>
|
||||||
|
<Card.Root>
|
||||||
|
<Card.Header class="pb-2">
|
||||||
|
<Card.Description>Timestamp</Card.Description>
|
||||||
|
<Card.Title class="text-sm">
|
||||||
|
{snapshot?.timestamp ? new Date(snapshot.timestamp).toLocaleString() : '—'}
|
||||||
|
</Card.Title>
|
||||||
|
</Card.Header>
|
||||||
|
</Card.Root>
|
||||||
|
</div>
|
||||||
|
|
||||||
|
<Card.Root>
|
||||||
|
<Card.Header>
|
||||||
|
<Card.Title>Последние incidents из snapshot</Card.Title>
|
||||||
|
<Card.Description>Всего: {snapshot?.counts?.total ?? snapshot?.incidents?.length ?? 0}</Card.Description>
|
||||||
|
</Card.Header>
|
||||||
|
<Card.Content class="p-0">
|
||||||
|
<Table.Root>
|
||||||
|
<Table.Header>
|
||||||
|
<Table.Row>
|
||||||
|
<Table.Head>Severity</Table.Head>
|
||||||
|
<Table.Head>Title</Table.Head>
|
||||||
|
<Table.Head>Summary</Table.Head>
|
||||||
|
<Table.Head>Aliases</Table.Head>
|
||||||
|
</Table.Row>
|
||||||
|
</Table.Header>
|
||||||
|
<Table.Body>
|
||||||
|
{#if (snapshot?.incidents?.length ?? 0) === 0}
|
||||||
|
<Table.Row>
|
||||||
|
<Table.Cell colspan={4} class="p-4 text-center text-muted-foreground">
|
||||||
|
Инцидентов нет
|
||||||
|
</Table.Cell>
|
||||||
|
</Table.Row>
|
||||||
|
{:else}
|
||||||
|
{#each snapshot?.incidents ?? [] as item (item.id)}
|
||||||
|
<Table.Row>
|
||||||
|
<Table.Cell>
|
||||||
|
<Badge variant={severityVariant(item.severity)}>{item.severity}</Badge>
|
||||||
|
</Table.Cell>
|
||||||
|
<Table.Cell class="font-medium">{item.title}</Table.Cell>
|
||||||
|
<Table.Cell class="max-w-[380px] text-sm text-muted-foreground">{item.summary}</Table.Cell>
|
||||||
|
<Table.Cell>
|
||||||
|
<div class="flex flex-wrap gap-1">
|
||||||
|
{#each item.affected_aliases ?? [] as alias (alias)}
|
||||||
|
<Badge variant="outline">{alias}</Badge>
|
||||||
|
{/each}
|
||||||
|
</div>
|
||||||
|
</Table.Cell>
|
||||||
|
</Table.Row>
|
||||||
|
{/each}
|
||||||
|
{/if}
|
||||||
|
</Table.Body>
|
||||||
|
</Table.Root>
|
||||||
|
</Card.Content>
|
||||||
|
</Card.Root>
|
||||||
@@ -1,9 +1,11 @@
|
|||||||
<script lang="ts">
|
<script lang="ts">
|
||||||
import { onMount } from 'svelte';
|
import { onMount } from 'svelte';
|
||||||
|
import { page } from '$app/state';
|
||||||
import { goto } from '$app/navigation';
|
import { goto } from '$app/navigation';
|
||||||
import { fetchAggUsers, ApiError } from '$lib/api/client.js';
|
import { fetchAggUsers, ApiError } from '$lib/api/client.js';
|
||||||
import type { components } from '$lib/api/aggregate.gen.js';
|
import type { components } from '$lib/api/aggregate.gen.js';
|
||||||
import { formatMiB } from '$lib/format.js';
|
import { formatMiB } from '$lib/format.js';
|
||||||
|
import { Badge } from '$lib/components/ui/badge/index.js';
|
||||||
import { Button } from '$lib/components/ui/button/index.js';
|
import { Button } from '$lib/components/ui/button/index.js';
|
||||||
import * as Table from '$lib/components/ui/table/index.js';
|
import * as Table from '$lib/components/ui/table/index.js';
|
||||||
import * as Card from '$lib/components/ui/card/index.js';
|
import * as Card from '$lib/components/ui/card/index.js';
|
||||||
@@ -16,14 +18,39 @@
|
|||||||
let rows = $state<components['schemas']['UsersRow'][]>([]);
|
let rows = $state<components['schemas']['UsersRow'][]>([]);
|
||||||
let partial = $state(false);
|
let partial = $state(false);
|
||||||
let includeLinks = $state(false);
|
let includeLinks = $state(false);
|
||||||
|
let aliases = $state('');
|
||||||
|
let aliasesInput = $state('');
|
||||||
|
let refreshSeconds = $state(30);
|
||||||
|
let refreshInput = $state('30');
|
||||||
|
let lastSuccessAtMs = $state<number | null>(null);
|
||||||
|
let nowMs = $state(Date.now());
|
||||||
|
let pollTimer: ReturnType<typeof setInterval> | null = null;
|
||||||
|
let staleTimer: ReturnType<typeof setInterval> | null = null;
|
||||||
|
|
||||||
|
function parseRefresh(raw: string | null): number {
|
||||||
|
if (raw == null || raw.trim() === '') return 30;
|
||||||
|
const n = Number(raw);
|
||||||
|
if (!Number.isFinite(n)) return 30;
|
||||||
|
if (n === 0) return 0;
|
||||||
|
return Math.max(10, Math.min(300, Math.floor(n)));
|
||||||
|
}
|
||||||
|
|
||||||
|
function parse01(raw: string | null, fallback: boolean): boolean {
|
||||||
|
if (raw == null || raw === '') return fallback;
|
||||||
|
return raw === '1';
|
||||||
|
}
|
||||||
|
|
||||||
async function load() {
|
async function load() {
|
||||||
loading = true;
|
loading = true;
|
||||||
err = null;
|
err = null;
|
||||||
try {
|
try {
|
||||||
const usersEnv = await fetchAggUsers({ include_links: includeLinks });
|
const usersEnv = await fetchAggUsers({
|
||||||
|
aliases: aliases || undefined,
|
||||||
|
include_links: includeLinks
|
||||||
|
});
|
||||||
partial = !!usersEnv.partial;
|
partial = !!usersEnv.partial;
|
||||||
rows = usersEnv.data ?? [];
|
rows = usersEnv.data ?? [];
|
||||||
|
lastSuccessAtMs = Date.now();
|
||||||
} catch (e) {
|
} catch (e) {
|
||||||
err = e instanceof ApiError ? e.message : String(e);
|
err = e instanceof ApiError ? e.message : String(e);
|
||||||
} finally {
|
} finally {
|
||||||
@@ -31,7 +58,64 @@
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
onMount(load);
|
function restartPolling() {
|
||||||
|
if (pollTimer) {
|
||||||
|
clearInterval(pollTimer);
|
||||||
|
pollTimer = null;
|
||||||
|
}
|
||||||
|
if (refreshSeconds > 0) {
|
||||||
|
pollTimer = setInterval(() => {
|
||||||
|
void load();
|
||||||
|
}, refreshSeconds * 1000);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
async function applyControls() {
|
||||||
|
const q = new URLSearchParams(page.url.searchParams);
|
||||||
|
const normalizedAliases = aliasesInput.trim();
|
||||||
|
const parsedRefresh = parseRefresh(refreshInput);
|
||||||
|
if (normalizedAliases) q.set('aliases', normalizedAliases);
|
||||||
|
else q.delete('aliases');
|
||||||
|
if (parsedRefresh === 30) q.delete('refresh');
|
||||||
|
else q.set('refresh', String(parsedRefresh));
|
||||||
|
q.set('include_links', includeLinks ? '1' : '0');
|
||||||
|
const qs = q.toString();
|
||||||
|
await goto(`${page.url.pathname}${qs ? `?${qs}` : ''}`, {
|
||||||
|
replaceState: true,
|
||||||
|
noScroll: true,
|
||||||
|
keepFocus: true
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
|
let queryKey = $derived(page.url.searchParams.toString());
|
||||||
|
$effect(() => {
|
||||||
|
queryKey;
|
||||||
|
if (typeof window === 'undefined') return;
|
||||||
|
aliases = page.url.searchParams.get('aliases')?.trim() ?? '';
|
||||||
|
refreshSeconds = parseRefresh(page.url.searchParams.get('refresh'));
|
||||||
|
includeLinks = parse01(page.url.searchParams.get('include_links'), false);
|
||||||
|
aliasesInput = aliases;
|
||||||
|
refreshInput = String(refreshSeconds);
|
||||||
|
restartPolling();
|
||||||
|
void load();
|
||||||
|
});
|
||||||
|
|
||||||
|
let staleSeconds = $derived.by(() =>
|
||||||
|
lastSuccessAtMs == null ? null : Math.max(0, Math.floor((nowMs - lastSuccessAtMs) / 1000))
|
||||||
|
);
|
||||||
|
let isStale = $derived.by(() =>
|
||||||
|
staleSeconds == null ? true : staleSeconds > (refreshSeconds > 0 ? refreshSeconds * 2 : 60)
|
||||||
|
);
|
||||||
|
|
||||||
|
onMount(() => {
|
||||||
|
staleTimer = setInterval(() => {
|
||||||
|
nowMs = Date.now();
|
||||||
|
}, 1000);
|
||||||
|
return () => {
|
||||||
|
if (pollTimer) clearInterval(pollTimer);
|
||||||
|
if (staleTimer) clearInterval(staleTimer);
|
||||||
|
};
|
||||||
|
});
|
||||||
|
|
||||||
function copy(text: string) {
|
function copy(text: string) {
|
||||||
const value = String(text ?? '');
|
const value = String(text ?? '');
|
||||||
@@ -76,17 +160,50 @@
|
|||||||
</p>
|
</p>
|
||||||
</div>
|
</div>
|
||||||
<div class="flex items-center gap-2">
|
<div class="flex items-center gap-2">
|
||||||
|
<Badge variant={isStale ? 'secondary' : 'default'}>
|
||||||
|
{isStale ? 'stale' : 'live'}{#if staleSeconds != null} · {staleSeconds}s{/if}
|
||||||
|
</Badge>
|
||||||
<label class="flex cursor-pointer items-center gap-2 text-sm">
|
<label class="flex cursor-pointer items-center gap-2 text-sm">
|
||||||
<input type="checkbox" bind:checked={includeLinks} onchange={load} class="rounded border" />
|
<input
|
||||||
|
type="checkbox"
|
||||||
|
bind:checked={includeLinks}
|
||||||
|
onchange={() => void applyControls()}
|
||||||
|
class="rounded border"
|
||||||
|
/>
|
||||||
Ссылки tg://proxy
|
Ссылки tg://proxy
|
||||||
</label>
|
</label>
|
||||||
<Button variant="outline" size="sm" onclick={load} disabled={loading}>
|
<Button variant="outline" size="sm" onclick={() => void load()} disabled={loading}>
|
||||||
<RefreshCwIcon class="mr-1 size-4 {loading ? 'animate-spin' : ''}" />
|
<RefreshCwIcon class="mr-1 size-4 {loading ? 'animate-spin' : ''}" />
|
||||||
Обновить
|
Обновить
|
||||||
</Button>
|
</Button>
|
||||||
</div>
|
</div>
|
||||||
</div>
|
</div>
|
||||||
|
|
||||||
|
<div class="mb-4 flex flex-wrap items-end gap-2">
|
||||||
|
<label class="text-xs text-muted-foreground">
|
||||||
|
Aliases
|
||||||
|
<input
|
||||||
|
type="text"
|
||||||
|
class="mt-1 h-9 w-56 rounded-md border bg-background px-2 text-sm"
|
||||||
|
bind:value={aliasesInput}
|
||||||
|
placeholder="node-a,node-b"
|
||||||
|
/>
|
||||||
|
</label>
|
||||||
|
<label class="text-xs text-muted-foreground">
|
||||||
|
Refresh (sec)
|
||||||
|
<input
|
||||||
|
type="number"
|
||||||
|
min="0"
|
||||||
|
max="300"
|
||||||
|
class="mt-1 h-9 w-28 rounded-md border bg-background px-2 text-sm"
|
||||||
|
bind:value={refreshInput}
|
||||||
|
/>
|
||||||
|
</label>
|
||||||
|
<Button variant="outline" size="sm" onclick={() => void applyControls()} disabled={loading}>
|
||||||
|
Apply
|
||||||
|
</Button>
|
||||||
|
</div>
|
||||||
|
|
||||||
{#if partial}
|
{#if partial}
|
||||||
<Alert class="mb-4">
|
<Alert class="mb-4">
|
||||||
<AlertTitle>Частичные данные</AlertTitle>
|
<AlertTitle>Частичные данные</AlertTitle>
|
||||||
|
|||||||
Reference in New Issue
Block a user