Compare commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
c162a41bc0 | ||
|
|
97e43b2335 | ||
|
|
db21e1217c | ||
|
|
5bb9066be8 | ||
|
|
b1fd259f10 | ||
|
|
3687bb8fa2 | ||
|
|
a80caf5676 | ||
|
|
d39e3454aa | ||
|
|
a8f2055c77 | ||
|
|
cdeb97d841 | ||
|
|
36c5305db7 | ||
|
|
5f2b4e2d40 | ||
|
|
cff26813b9 | ||
|
|
39c8ec4a02 | ||
|
|
c6c859a495 | ||
|
|
bf5cfff12c | ||
|
|
30a8ec4420 |
@@ -53,30 +53,39 @@ jobs:
|
||||
shell: bash
|
||||
run: |
|
||||
set -euo pipefail
|
||||
docker run -d --name mm-pg \
|
||||
NAME="mm-pg-${{ gitea.run_id }}-${{ gitea.run_attempt }}"
|
||||
docker rm -f "$NAME" mm-pg 2>/dev/null || true
|
||||
HOST_PORT=$(python3 -c 'import socket; s=socket.socket(); s.bind(("127.0.0.1", 0)); print(s.getsockname()[1]); s.close()')
|
||||
docker run -d --name "$NAME" --rm \
|
||||
-e POSTGRES_USER=mmapp \
|
||||
-e POSTGRES_PASSWORD=mmapp \
|
||||
-e POSTGRES_DB=mmapp \
|
||||
-p 5432:5432 \
|
||||
-p "127.0.0.1:${HOST_PORT}:5432" \
|
||||
postgres:18-alpine
|
||||
echo "PG_CONTAINER=$NAME" >> "${GITHUB_ENV}"
|
||||
echo "DATABASE_URL=postgres://mmapp:[email protected]:${HOST_PORT}/mmapp" >> "${GITHUB_ENV}"
|
||||
for i in $(seq 1 40); do
|
||||
if docker exec mm-pg pg_isready -U mmapp -d mmapp; then
|
||||
if docker exec "$NAME" pg_isready -U mmapp -d mmapp; then
|
||||
exit 0
|
||||
fi
|
||||
sleep 1
|
||||
done
|
||||
echo "PostgreSQL не поднялся"
|
||||
docker logs "$NAME" || true
|
||||
exit 1
|
||||
|
||||
- name: Install and test backend
|
||||
shell: bash
|
||||
env:
|
||||
DATABASE_URL: postgres://mmapp:[email protected]:5432/mmapp
|
||||
run: |
|
||||
set -euo pipefail
|
||||
npm ci
|
||||
npm run test --prefix backend
|
||||
|
||||
- name: Stop PostgreSQL
|
||||
if: always()
|
||||
shell: bash
|
||||
run: docker rm -f "${PG_CONTAINER:-}" mm-pg 2>/dev/null || true
|
||||
|
||||
backend-image:
|
||||
needs:
|
||||
- prepare-release
|
||||
|
||||
@@ -0,0 +1,63 @@
|
||||
# Локальные GeoLite2-базы (Country + ASN) для потоков по странам и ASN
|
||||
|
||||
## Контекст
|
||||
|
||||
Сейчас страна и ASN для netflow-потоков резолвятся через внешний RIPEstat API (`backend/src/services/traffic-flow-ripe.ts`): лимит 30 новых префиксов/мин, очередь на 90, кэш в PG `flow_ip_meta`. Новые IP «дозревают» с задержкой, IPv6 не покрывается (кэш индексируется только по IPv4). Локальные mmdb-базы дают мгновенный синхронный lookup всех IP без внешних вызовов.
|
||||
|
||||
Решения (подтверждены):
|
||||
- Источник — **MaxMind GeoLite2 через P3TERX-зеркало**: `https://github.com/P3TERX/GeoLite.mmdb/raw/download/GeoLite2-Country.mmdb` и `.../GeoLite2-ASN.mmdb`. Без регистрации, ключей и tar-распаковки. Точность по стране у GeoLite2 и IPinfo паритетная (<1% ошибок у обоих, arXiv 2026); выбран P3TERX за надёжность зеркала (5.2k звёзд) и преемственность: текущий RIPE-путь и так читает GeoLite (`maxmind-geo-lite`), история в кэше остаётся консистентной.
|
||||
- **RIPEstat остаётся fallback** (до первой загрузки баз / если lookup не дал результата).
|
||||
- City-базу не качаем (lat/lng фронтенд не использует).
|
||||
|
||||
## Изменения
|
||||
|
||||
### 1. Зависимость
|
||||
- `npm install -w mikrotik-manager-backend maxmind` — sync-чтение mmdb, встроенные TS-типы, без транзитивных зависимостей, Node 22 ок.
|
||||
|
||||
### 2. Новый сервис `backend/src/services/traffic-flow-geoip.ts`
|
||||
(по конвенциям окружения traffic-flow-*: контракт → маршрут → сервис, без БД-логики в маршрутах)
|
||||
- Каталог: `backend/storage/geoip/` (конвенция `storage/backups`), файлы `GeoLite2-Country.mmdb`, `GeoLite2-ASN.mmdb`.
|
||||
- `initGeoip()` — открыть ридеры best-effort при старте (из `index.ts` рядом с `startTrafficFlowListener`), независимо от настроек автообновления: файлы есть — работают.
|
||||
- `lookupGeoip(ip): FlowIpMeta | null` — синхронно: `country.iso_code` (fallback `registered_country.iso_code`) с валидацией `isIsoCountry`, ASN = `autonomous_system_number`, holder = `autonomous_system_organization`; приватные IP → negative-запись как в RIPE (`isNonPublicIp`); в PG не пишем (lookup и так быстрый). IPv6 поддержан ридером.
|
||||
- `resolveFlowIp(ip)` — фасад: `lookupGeoip(ip) ?? lookupRipeCached(ip)`; главный экспорт для потребителей.
|
||||
- `geoipStatus()` — loaded, даты сборки баз (метаданные mmdb). Тест-хук `setGeoipReadersForTests`. Смена ридеров после обновления — атомарная замена ссылок.
|
||||
|
||||
### 3. Коллектор `backend/src/services/geoip-update-collector.ts`
|
||||
`collectGeoipUpdateOnce()` по образцу `certificate-renew-collector.ts`:
|
||||
1. Conditional GET с ETag/If-None-Match из настроек → 304 = skip (фолбэк-сравнение: размер/содержимое).
|
||||
2. Скачивание в `*.tmp` через глобальный `fetch` + AbortController с таймаутом (внешний HTTP из service-слоя — по правилу fastify-backend-drizzle).
|
||||
3. Валидация: открыть ридер из tmp-файла, пробой 8.8.8.8 (страна US, ASN 15169).
|
||||
4. `fs.rename` атомарная подмена, старый файл → `*.prev` (откат, если новый ридер не открылся).
|
||||
5. Перезагрузка ридеров, статус в настройках; snapshot для `scheduler_runs` (checked/downloaded/skipped/bytes/error).
|
||||
|
||||
### 4. Планировщик (`backend/src/services/scheduler.ts`)
|
||||
- `JOB_KEYS` += `geoip_update`; case в `runSchedulerJobBody`; блок в `refreshScheduler()` по образцу `certificates_renew`: интервал `Math.max(6ч, updateIntervalSec*1000)`, по умолчанию 7 дней (upstream обновляется еженедельно) + немедленный первый запуск при включённой настройке.
|
||||
|
||||
### 5. Схема и миграция
|
||||
- `backend/src/db/schema.ts`: singleton `geoip_settings` — `enabled` (default true), `updateIntervalSec` (default 604800), `lastCheckAt`, `lastSuccessAt`, `lastError`, `countryBuildAt`, `asnBuildAt`, `etagsJson` (jsonb), `createdAt/updatedAt`.
|
||||
- Миграция: `npm run db:generate` → файл в `backend/drizzle/`.
|
||||
|
||||
### 6. API + контракты
|
||||
- `packages/contracts/src/geoip.ts`: zod-схемы настроек/статуса (все входы — Zod, по правилам проекта).
|
||||
- Новый `backend/src/routes/geoip.ts`, регистрация в `index.ts` с prefix `/api`:
|
||||
- `GET /api/geoip` — настройки + статус (ready, даты сборки, последняя проверка/ошибка);
|
||||
- `PUT /api/geoip` — сохранить настройки, затем `refreshScheduler()`;
|
||||
- `POST /api/geoip/update` — запустить загрузку сейчас (409, если уже идёт; флаг-гард как в коллекторах).
|
||||
|
||||
### 7. Интеграция в пайплайн (geoip-first, RIPE-fallback)
|
||||
- `traffic-flow-engine.ts` (`queueParsedFlows`, ~строка 336): `lookupRipeCached` → `resolveFlowIp`. Логика misses не меняется: при готовом mmdb публичные IP (v4+v6) резолвятся сразу, очередь RIPE пустеет; до скачивания баз — прежнее поведение.
|
||||
- Остальные вызовы `lookupRipeCached` → `resolveFlowIp` (grep: как минимум `traffic-flow-analytics.ts` ~258–271).
|
||||
- `classifyFlowDst`/бренды не трогаем: holder из mmdb (org name) встаёт в существующие `HOLDER_BRANDS`-регулярки как есть.
|
||||
|
||||
### 8. Frontend (по next-shadcn-production / ui-guardian: только переиспользование)
|
||||
- Секция «GeoIP-базы (GeoLite2)» внутри существующей `components/traffic/netflow-settings-panel.tsx`: статус (готово/не скачано, даты сборки Country/ASN, последняя проверка, ошибка), тумблер автообновления, интервал, кнопка «Обновить сейчас» с индикатором. Только уже используемые в панели примитивы (Switch/Button/поля) — никаких новых визуальных паттернов и Card-shell. API-клиент через существующие http-хелперы.
|
||||
|
||||
### 9. Хаускипинг, тесты, проверки
|
||||
- `backend/.gitignore`: `storage/geoip/`.
|
||||
- Тесты `backend/src/services/traffic-flow-geoip.test.ts` + скрипт `test:geoip` (по образцу `test:traffic-flow`): приоритет фасада (geoip hit → RIPE не зовётся; miss → fallback), negative на приватных IP, фильтрация EU/ZZ через `isIsoCountry`, коллектор с мокнутым fetch (304-skip, битый файл → подмены нет, `.prev` сохранён), dims по стране/ASN с засеянным ридером.
|
||||
- Проверки после реализации (обязательно по правилам): типы/сборка бэка (`npm run build -w mikrotik-manager-backend`), типы фронта при правке UI (`npx tsc --noEmit`), `npm run test:geoip` и `test:traffic-flow`; предупреждения не игнорировать.
|
||||
- Коммит: `feat(netflow): <subject по-русски>` — новая пользовательская фича (мгновенные страна/ASN в потоках), по commit-messages-ru.
|
||||
- README: короткий раздел о GeoIP; примечание, что в Docker `storage/geoip` ephemeral без тома — базы перекачаются после пересоздания контейнера (~17 МБ); при желании смонтировать volume.
|
||||
|
||||
## Что это даёт
|
||||
Страна и ASN появляются у потока мгновенно при ingest (включая IPv6), без ограничения скорости RIPE; dims `country`/`asn` в `flow_daily_dims`, аналитика (карта, топы, monthly) становятся полными сразу. Внешняя зависимость от stat.ripe.net остаётся только как fallback до первой загрузки баз.
|
||||
@@ -128,7 +128,7 @@ sequenceDiagram
|
||||
| Зависимость | Реализация |
|
||||
|-------------|------------|
|
||||
| PostgreSQL | Контейнер `mmapp-postgres` (`postgres:18-alpine`). `DATABASE_URL=postgres://mmapp:…@postgres:5432/mmapp`. |
|
||||
| SQLite (ETL) | Файл `mikrotik.db` на томе `/app/data` (`DATABASE_PATH=/app/data/mikrotik.db`). При первом старте, если PG пустой, backend сам импортирует данные и ставит маркер. Повторный старт не копирует заново. |
|
||||
| SQLite (ETL) | Файл `mikrotik.db` на томе `/app/data` (`DATABASE_PATH=/app/data/mikrotik.db`). При старте, пока нет маркера `data_migration.sqlite_imported_at`, backend импортирует sqlite в PG (`ON CONFLICT` / upsert). После успешного импорта повтор не копирует заново. Том PG18: `mmapp-pgdata:/var/lib/postgresql` (не `.../data`). |
|
||||
| Docker socket | Только у контейнера updater: `/var/run/docker.sock` — доступ к Docker API хоста (управление контейнерами, pull). |
|
||||
|
||||
## Локальная разработка
|
||||
@@ -190,6 +190,14 @@ npm run build -w @mmapp/contracts
|
||||
npm --prefix backend run db:migrate-from-sqlite
|
||||
```
|
||||
|
||||
### GeoIP-базы GeoLite2 (страны и ASN для NetFlow)
|
||||
|
||||
Backend держит локальные mmdb-базы MaxMind GeoLite2 (Country + ASN) в `backend/storage/geoip/` и скачивает их с зеркала [P3TERX/GeoLite.mmdb](https://github.com/P3TERX/GeoLite.mmdb) — без регистрации и ключей. Lookup страны/ASN потока при ingest становится мгновенным (включая IPv6) и не упирается в лимиты RIPEstat; пока базы не скачаны или lookup промахнулся, работает прежний RIPE-fallback.
|
||||
|
||||
Управление — секция «GeoIP-базы (GeoLite2)» в настройках NetFlow (страница «Сбор данных»): автообновление (по умолчанию проверка раз в 7 дней, upstream обновляется еженедельно), статус сборки баз и кнопка «Обновить сейчас». Джоба планировщика — `geoip_update`. Атрибуция: данные MaxMind GeoLite2, CC BY-SA 4.0.
|
||||
|
||||
Примечание для Docker: каталог `storage/geoip` внутри контейнера ephemeral — без смонтированного volume базы (~17 МБ) перекачаются после пересоздания контейнера. Каталог переопределяется переменной `GEOIP_DIR`.
|
||||
|
||||
## CI/CD (Gitea Actions)
|
||||
|
||||
Файл: `.gitea/workflows/docker.yml` (имя workflow: **Docker images**).
|
||||
|
||||
+309
-583
File diff suppressed because it is too large
Load Diff
@@ -11,6 +11,7 @@ import { OpsPanel } from "@/components/ops-panel"
|
||||
import { DataPageCard } from "@/components/data-page-card"
|
||||
import { DataPageToolbar } from "@/components/data-page-toolbar"
|
||||
import { CertificatesDataGrid } from "@/components/data-grids/certificates-data-grid"
|
||||
import { CertificateRenewSettingsPanel } from "@/components/certificates/certificate-renew-settings"
|
||||
import { KpiStatGrid } from "@/components/reui-kit/kpi-stat-grid"
|
||||
import { ServerRailLayout, ServerRailMobileButton } from "@/components/server-rail-layout"
|
||||
import { ALL_SERVERS_ID, type ServerTileItem } from "@/components/server-tile-rail"
|
||||
@@ -243,7 +244,7 @@ function CertPartReference() {
|
||||
return (
|
||||
<OpsPanel
|
||||
title="RouterOS 7 · /certificate — справка CLI"
|
||||
description="RouterOS 7.22+ · публичные LE для Cloudflare через backend DNS-01, не через /certificate add-acme на устройстве."
|
||||
description="RouterOS 7 умеет обновлять Let's Encrypt сам. Этот CLI — справка; автообновление MM включается панелью выше."
|
||||
contentClassName="px-5 py-4"
|
||||
>
|
||||
<div className="grid grid-cols-1 sm:grid-cols-3 gap-4 text-xs font-mono">
|
||||
@@ -715,6 +716,8 @@ export default function CertificatesPage() {
|
||||
|
||||
<CertPartKpi displayCerts={scopedCerts} expiring={expiring} expired={expired} />
|
||||
|
||||
<CertificateRenewSettingsPanel backendUrl={backendUrl} liveReady={liveReady} />
|
||||
|
||||
{liveReady && (
|
||||
<CertPartAcmeSettings
|
||||
acmeDirectoryUrl={acmeDirectoryUrl}
|
||||
|
||||
+260
-851
File diff suppressed because it is too large
Load Diff
@@ -34,6 +34,7 @@ import {
|
||||
type InternetPathRunSnapshot,
|
||||
type CertificatesRenewRunSnapshot,
|
||||
type BackupsRunSnapshot,
|
||||
type GeoipUpdateRunSnapshot,
|
||||
type PingRunSnapshot,
|
||||
type ResourcesRunSnapshot,
|
||||
type SchedulerRunSnapshot,
|
||||
@@ -340,6 +341,41 @@ function SnapshotTables({ snap }: { snap: SchedulerRunSnapshot }) {
|
||||
</div>
|
||||
)
|
||||
}
|
||||
if (snap.job === "geoip_update") {
|
||||
const g = snap as GeoipUpdateRunSnapshot
|
||||
return (
|
||||
<div className="flex flex-col gap-3">
|
||||
{g.skipped ? (
|
||||
<p className="text-xs text-amber-600 dark:text-amber-400">Прогон пропущен: обновление уже выполнялось или задача отключена.</p>
|
||||
) : null}
|
||||
<dl className="grid grid-cols-2 gap-3 text-xs sm:grid-cols-4">
|
||||
<div>
|
||||
<dt className="text-muted-foreground">Баз проверено</dt>
|
||||
<dd className="font-mono font-medium">{g.checked}</dd>
|
||||
</div>
|
||||
<div>
|
||||
<dt className="text-muted-foreground">Скачано</dt>
|
||||
<dd className="font-mono font-medium">{g.downloaded}</dd>
|
||||
</div>
|
||||
<div>
|
||||
<dt className="text-muted-foreground">Без изменений</dt>
|
||||
<dd className="font-mono font-medium">{g.skippedUnchanged}</dd>
|
||||
</div>
|
||||
<div>
|
||||
<dt className="text-muted-foreground">Объём, МБ</dt>
|
||||
<dd className="font-mono font-medium">{(g.bytes / 1024 / 1024).toFixed(1)}</dd>
|
||||
</div>
|
||||
</dl>
|
||||
{g.errors.length ? (
|
||||
<div className="rounded-md border border-amber-500/40 bg-amber-500/10 px-3 py-2 text-xs text-amber-800 dark:text-amber-200 flex flex-col gap-1">
|
||||
{g.errors.map((e, i) => (
|
||||
<p key={i} className="break-words">{e}</p>
|
||||
))}
|
||||
</div>
|
||||
) : null}
|
||||
</div>
|
||||
)
|
||||
}
|
||||
if (snap.job === "alert_engine") {
|
||||
const a = snap as AlertEngineRunSnapshot
|
||||
return (
|
||||
@@ -1127,7 +1163,11 @@ export default function DataCollectionPage() {
|
||||
onChange={(e) => setRenewBeforeDaysDraft(e.target.value)}
|
||||
className="h-8 text-sm"
|
||||
inputMode="numeric"
|
||||
disabled={!draftCertRenewEnabled}
|
||||
/>
|
||||
<p className="text-[11px] text-muted-foreground mt-1">
|
||||
Вкл/выкл автообновления MM — также на странице «Сертификаты». Не включайте вместе со встроенным ACME RouterOS.
|
||||
</p>
|
||||
</div>
|
||||
</div>
|
||||
<p className="text-[11px] text-muted-foreground leading-snug">
|
||||
|
||||
@@ -67,6 +67,7 @@ import {
|
||||
CableIcon, CopyIcon, ActivityIcon, ExternalLinkIcon,
|
||||
} from "lucide-react"
|
||||
import { cn } from "@/lib/utils"
|
||||
import { formatServicePathLabel, formatServicePathTitle } from "@/lib/format-service-path-label"
|
||||
import Link from "next/link"
|
||||
import { Flag } from "@/components/flag"
|
||||
|
||||
@@ -820,9 +821,15 @@ function ServicePathList({
|
||||
{paths.map((p) => {
|
||||
const rowKey = servicePathKey(p)
|
||||
const via = servers.find((s) => s.id === p.viaId)
|
||||
const viaLabel = via?.site || p.viaName
|
||||
const en = servers.find((s) => s.id === p.enId)
|
||||
const svc = services.find((s) => s.id === p.serviceId)
|
||||
const mid = viaMode === "via" ? viaLabel : (svc?.label ?? p.serviceId)
|
||||
const label = formatServicePathLabel(p, viaMode, {
|
||||
viaName: via?.name,
|
||||
viaSite: via?.site,
|
||||
enName: en?.name,
|
||||
serviceLabel: svc?.label,
|
||||
})
|
||||
const title = formatServicePathTitle(label, svc?.label ?? p.serviceId)
|
||||
const active = Boolean(
|
||||
highlight
|
||||
&& highlight.viaId === p.viaId
|
||||
@@ -833,13 +840,14 @@ function ServicePathList({
|
||||
<button
|
||||
key={rowKey}
|
||||
type="button"
|
||||
title={title}
|
||||
onClick={() => onToggle(p)}
|
||||
className={cn(
|
||||
"flex items-center justify-between gap-2 rounded-md px-2 py-1.5 text-left text-xs transition-colors",
|
||||
active ? "bg-cyan-500/15 ring-1 ring-cyan-500/40" : "hover:bg-muted/50",
|
||||
)}
|
||||
>
|
||||
<span className="font-mono truncate min-w-0">{p.clientName} · {mid}</span>
|
||||
<span className="font-mono truncate min-w-0">{label}</span>
|
||||
<span className="font-mono text-emerald-400 tabular-nums shrink-0">
|
||||
{formatNetflowRate({ bytes: p.bytes, bps: p.bps, bpsFwd: p.bps, bpsRev: 0 })}
|
||||
</span>
|
||||
@@ -1216,6 +1224,9 @@ export default function NetworkMapPage() {
|
||||
// ── Interaction ─────────────────────────────────────────────────────────────
|
||||
const [selected, setSelected] = useState<Server | null>(null)
|
||||
const [selectedService, setSelectedService] = useState<FlowMapService | null>(null)
|
||||
const liveSelectedService = selectedService
|
||||
? (mapServices.find((s) => s.id === selectedService.id) ?? selectedService)
|
||||
: null
|
||||
const [highlightedPath, setHighlightedPath] = useState<{ viaId: string; enId: string; serviceId: string } | null>(null)
|
||||
const [selWanIdx, setSelWanIdx] = useState<number | null>(null)
|
||||
const [hoveredId, setHoveredId] = useState<string | null>(null)
|
||||
@@ -2702,16 +2713,16 @@ export default function NetworkMapPage() {
|
||||
})()}
|
||||
</div>
|
||||
</>
|
||||
) : selectedService ? (
|
||||
) : liveSelectedService ? (
|
||||
<>
|
||||
<div className="flex items-start gap-2 px-4 py-3 border-b">
|
||||
<div className="mt-0.5">
|
||||
<ServiceBrandIcon label={selectedService.label} size={22} />
|
||||
<ServiceBrandIcon label={liveSelectedService.label} size={22} />
|
||||
</div>
|
||||
<div className="flex-1 min-w-0">
|
||||
<p className="font-mono font-semibold text-sm truncate">{selectedService.label}</p>
|
||||
<p className="font-mono font-semibold text-sm truncate">{liveSelectedService.label}</p>
|
||||
<p className="text-xs text-muted-foreground mt-0.5">
|
||||
Конечный сервис · {selectedService.category}
|
||||
Конечный сервис · {liveSelectedService.category}
|
||||
</p>
|
||||
</div>
|
||||
<button
|
||||
@@ -2726,15 +2737,15 @@ export default function NetworkMapPage() {
|
||||
<div className="flex flex-col gap-0">
|
||||
<div className="flex items-center justify-between py-2 border-b border-border/50">
|
||||
<span className="text-xs text-muted-foreground">Доля окна</span>
|
||||
<span className="text-xs font-mono font-medium text-cyan-400">{serviceSharePct(selectedService.share)}</span>
|
||||
<span className="text-xs font-mono font-medium text-cyan-400">{serviceSharePct(liveSelectedService.share)}</span>
|
||||
</div>
|
||||
<div className="flex items-center justify-between py-2 border-b border-border/50">
|
||||
<span className="text-xs text-muted-foreground">Скорость</span>
|
||||
<span className="text-xs font-mono font-medium">
|
||||
{formatNetflowRate({
|
||||
bytes: selectedService.bytes,
|
||||
bps: selectedService.bps,
|
||||
bpsFwd: selectedService.bps,
|
||||
bytes: liveSelectedService.bytes,
|
||||
bps: liveSelectedService.bps,
|
||||
bpsFwd: liveSelectedService.bps,
|
||||
bpsRev: 0,
|
||||
})}
|
||||
</span>
|
||||
@@ -2742,34 +2753,34 @@ export default function NetworkMapPage() {
|
||||
</div>
|
||||
<div>
|
||||
<p className="text-xs font-semibold text-muted-foreground uppercase tracking-wider mb-2">Выход</p>
|
||||
<div className="flex flex-col gap-1.5">
|
||||
{visibleServiceEdges.filter((e) => e.toId === selectedService.id).map((e) => {
|
||||
<div className="flex flex-col gap-3">
|
||||
{visibleServiceEdges.filter((e) => e.toId === liveSelectedService.id).map((e) => {
|
||||
const src = mapServers.find((s) => s.id === e.fromId)
|
||||
const enPaths = mapServicePaths
|
||||
.filter((p) => p.serviceId === liveSelectedService.id && p.enId === e.fromId)
|
||||
.slice()
|
||||
.sort((a, b) => b.bps - a.bps)
|
||||
return (
|
||||
<div key={`${e.fromId}|${e.toId}`} className="flex items-center justify-between text-xs">
|
||||
<span className="font-mono truncate">{src?.name ?? e.fromId}</span>
|
||||
<span className="font-mono text-emerald-400 tabular-nums">
|
||||
{formatNetflowRate({ bytes: e.bytes, bps: e.bps, bpsFwd: e.bpsFwd, bpsRev: e.bpsRev })}
|
||||
</span>
|
||||
<div key={`${e.fromId}|${e.toId}`} className="flex flex-col gap-1.5">
|
||||
<div className="flex items-center justify-between text-xs">
|
||||
<span className="font-mono truncate">{src?.name ?? e.fromId}</span>
|
||||
<span className="font-mono text-emerald-400 tabular-nums">
|
||||
{formatNetflowRate({ bytes: e.bytes, bps: e.bps, bpsFwd: e.bpsFwd, bpsRev: e.bpsRev })}
|
||||
</span>
|
||||
</div>
|
||||
<ServicePathList
|
||||
paths={enPaths}
|
||||
servers={mapServers}
|
||||
services={mapServices}
|
||||
highlight={highlightedPath}
|
||||
viaMode="via"
|
||||
onToggle={togglePathHighlight}
|
||||
/>
|
||||
</div>
|
||||
)
|
||||
})}
|
||||
</div>
|
||||
</div>
|
||||
<div>
|
||||
<p className="text-xs font-semibold text-muted-foreground uppercase tracking-wider mb-2">Пути</p>
|
||||
<ServicePathList
|
||||
paths={mapServicePaths
|
||||
.filter((p) => p.serviceId === selectedService.id)
|
||||
.slice()
|
||||
.sort((a, b) => b.bps - a.bps)}
|
||||
servers={mapServers}
|
||||
services={mapServices}
|
||||
highlight={highlightedPath}
|
||||
viaMode="via"
|
||||
onToggle={togglePathHighlight}
|
||||
/>
|
||||
</div>
|
||||
</div>
|
||||
</>
|
||||
) : selected ? (
|
||||
@@ -3002,7 +3013,11 @@ export default function NetworkMapPage() {
|
||||
<p className="text-xs font-semibold text-muted-foreground uppercase tracking-wider mb-2">Пути</p>
|
||||
<ServicePathList
|
||||
paths={mapServicePaths
|
||||
.filter((p) => p.viaId === selected.id || p.enId === selected.id)
|
||||
.filter((p) => (
|
||||
selected.type === "exit-node"
|
||||
? p.enId === selected.id
|
||||
: p.viaId === selected.id
|
||||
))
|
||||
.slice()
|
||||
.sort((a, b) => b.bps - a.bps)}
|
||||
servers={mapServers}
|
||||
|
||||
@@ -0,0 +1,522 @@
|
||||
"use client"
|
||||
|
||||
import { useCallback, useEffect, useMemo, useState } from "react"
|
||||
import { useRouter, useSearchParams } from "next/navigation"
|
||||
import {
|
||||
ActivityIcon,
|
||||
DatabaseIcon,
|
||||
GaugeIcon,
|
||||
ServerIcon,
|
||||
UsersIcon,
|
||||
} from "lucide-react"
|
||||
import { PageHeader } from "@/components/page-header"
|
||||
import { KpiStatGrid } from "@/components/reui-kit/kpi-stat-grid"
|
||||
import { DataPageCard } from "@/components/data-page-card"
|
||||
import { DataPageToolbar } from "@/components/data-page-toolbar"
|
||||
import { SegmentedControl } from "@/components/form-kit"
|
||||
import { EmptyState } from "@/components/empty-state"
|
||||
import { PeriodSelector, rangeForPreset, type DateRangeYmd } from "@/components/statistics/period-selector"
|
||||
import { StatisticsVolumeChart } from "@/components/statistics/statistics-volume-chart"
|
||||
import { DimensionSelect, PivotDimSelect } from "@/components/statistics/dimension-select"
|
||||
import { SliceChips } from "@/components/statistics/slice-chips"
|
||||
import { BreakdownDashboard } from "@/components/statistics/breakdown-dashboard"
|
||||
import { StatisticsPivotGrid } from "@/components/statistics/statistics-pivot-grid"
|
||||
import {
|
||||
StatisticsBreakdownDataGrid,
|
||||
type StatisticsSliceKind,
|
||||
} from "@/components/data-grids/statistics-breakdown-data-grid"
|
||||
import { Alert, AlertDescription, AlertTitle } from "@/components/reui/alert"
|
||||
import type { Filter } from "@/components/reui/filters"
|
||||
import { STATISTICS_FILTER_FIELDS } from "@/lib/data-filters/statistics-filter-fields"
|
||||
import {
|
||||
isStatisticsPivotDim,
|
||||
isStatisticsSliceKind,
|
||||
STATISTICS_DIMS,
|
||||
} from "@/lib/statistics-dims"
|
||||
import { useDataSource } from "@/lib/data-source"
|
||||
import { fmtBps, formatBytes } from "@/lib/fmt-rate"
|
||||
import {
|
||||
getStatistics,
|
||||
getStatisticsPivot,
|
||||
STATISTICS_UNBOUND_USER_ID,
|
||||
type StatisticsDto,
|
||||
type StatisticsPivotDto,
|
||||
type StatisticsQuery,
|
||||
} from "@/shared/api/statistics"
|
||||
import type { StatisticsBreakdownRow, StatisticsPivotDim } from "@mmapp/contracts/statistics"
|
||||
|
||||
/**
|
||||
* BI-куб трафика: критерий → остальные разрезы + pivot.
|
||||
* Preview: https://reui.io/preview/base/dashboard-1 · https://reui.io/preview/base/stats-12
|
||||
* · https://reui.io/preview/base/data-grid-filtering-2 · https://reui.io/preview/base/solution-analytics-8
|
||||
* · https://reui.io/docs/components/base/frame · https://reui.io/docs/components/base/data-grid
|
||||
*/
|
||||
|
||||
const EMPTY: StatisticsDto = {
|
||||
from: "",
|
||||
to: "",
|
||||
grain: "day",
|
||||
kpis: {
|
||||
bytes: 0,
|
||||
packets: 0,
|
||||
avgBps: 0,
|
||||
users: 0,
|
||||
servers: 0,
|
||||
ifaces: 0,
|
||||
topCountry: "",
|
||||
topService: "",
|
||||
},
|
||||
series: [],
|
||||
users: [],
|
||||
servers: [],
|
||||
interfaces: [],
|
||||
countries: [],
|
||||
services: [],
|
||||
asns: [],
|
||||
}
|
||||
|
||||
const EMPTY_PIVOT: StatisticsPivotDto = {
|
||||
rowDim: "country",
|
||||
colDim: "service",
|
||||
metric: "bytes",
|
||||
columns: [],
|
||||
rows: [],
|
||||
otherBytes: 0,
|
||||
}
|
||||
|
||||
interface CubeSlices {
|
||||
country?: string
|
||||
service?: string
|
||||
asn?: string
|
||||
serverId?: string
|
||||
userId?: string
|
||||
iface?: string
|
||||
}
|
||||
|
||||
const SLICE_KEYS = ["country", "service", "asn", "serverId", "userId", "iface"] as const
|
||||
|
||||
function readRange(sp: URLSearchParams): DateRangeYmd {
|
||||
const from = sp.get("from")
|
||||
const to = sp.get("to")
|
||||
if (from && to && from <= to) return { from, to }
|
||||
return rangeForPreset("7d")
|
||||
}
|
||||
|
||||
function readDim(sp: URLSearchParams): StatisticsSliceKind {
|
||||
const t = sp.get("dim") ?? sp.get("tab")
|
||||
return t && isStatisticsSliceKind(t) ? t : "users"
|
||||
}
|
||||
|
||||
function readView(sp: URLSearchParams): "explore" | "pivot" {
|
||||
return sp.get("view") === "pivot" ? "pivot" : "explore"
|
||||
}
|
||||
|
||||
function readPivotDim(sp: URLSearchParams, key: string, fallback: StatisticsPivotDim): StatisticsPivotDim {
|
||||
const v = sp.get(key)
|
||||
return v && isStatisticsPivotDim(v) ? v : fallback
|
||||
}
|
||||
|
||||
function readSlices(sp: URLSearchParams): CubeSlices {
|
||||
const next: CubeSlices = {}
|
||||
for (const key of SLICE_KEYS) {
|
||||
const v = sp.get(key)?.trim()
|
||||
if (v) next[key] = v
|
||||
}
|
||||
return next
|
||||
}
|
||||
|
||||
function slicesToFilters(slices: CubeSlices): Filter[] {
|
||||
return SLICE_KEYS.flatMap((key) => {
|
||||
const val = slices[key]
|
||||
if (!val) return []
|
||||
return [{ id: key, field: key, operator: "is", values: [val] }]
|
||||
})
|
||||
}
|
||||
|
||||
function filtersToSlices(filters: Filter[]): CubeSlices {
|
||||
const next: CubeSlices = {}
|
||||
for (const f of filters) {
|
||||
const raw = String(f.values[0] ?? "").trim()
|
||||
if (!raw) continue
|
||||
if (f.field === "country") next.country = raw.toUpperCase().slice(0, 2)
|
||||
else if (f.field === "service") next.service = raw
|
||||
else if (f.field === "asn") next.asn = raw.replace(/[^\d]/g, "")
|
||||
else if (f.field === "serverId") next.serverId = raw
|
||||
else if (f.field === "userId") next.userId = raw
|
||||
else if (f.field === "iface") next.iface = raw
|
||||
}
|
||||
return next
|
||||
}
|
||||
|
||||
function toQuery(range: DateRangeYmd, slices: CubeSlices): StatisticsQuery {
|
||||
const serverId = slices.serverId ? Number(slices.serverId) : undefined
|
||||
const asn = slices.asn != null && slices.asn !== "" ? Number(slices.asn) : undefined
|
||||
return {
|
||||
from: range.from,
|
||||
to: range.to,
|
||||
serverId: Number.isFinite(serverId) && (serverId ?? 0) > 0 ? serverId : undefined,
|
||||
userId: slices.userId,
|
||||
iface: slices.iface,
|
||||
country: slices.country && slices.country.length === 2 ? slices.country : undefined,
|
||||
service: slices.service,
|
||||
asn: Number.isFinite(asn) ? asn : undefined,
|
||||
}
|
||||
}
|
||||
|
||||
function selectedIdForKind(kind: StatisticsSliceKind, slices: CubeSlices): string | undefined {
|
||||
if (kind === "users") return slices.userId
|
||||
if (kind === "servers") return slices.serverId
|
||||
if (kind === "countries") return slices.country
|
||||
if (kind === "services") return slices.service
|
||||
if (kind === "asns") return slices.asn
|
||||
if (kind === "interfaces" && slices.serverId && slices.iface) {
|
||||
return `${slices.serverId}:${slices.iface}`
|
||||
}
|
||||
if (kind === "interfaces") return slices.iface
|
||||
return undefined
|
||||
}
|
||||
|
||||
function rowsForKind(data: StatisticsDto, kind: StatisticsSliceKind) {
|
||||
if (kind === "users") return data.users
|
||||
if (kind === "servers") return data.servers
|
||||
if (kind === "interfaces") return data.interfaces
|
||||
if (kind === "countries") return data.countries
|
||||
if (kind === "services") return data.services
|
||||
return data.asns
|
||||
}
|
||||
|
||||
function hasAnySlice(slices: CubeSlices): boolean {
|
||||
return SLICE_KEYS.some((k) => Boolean(slices[k]))
|
||||
}
|
||||
|
||||
function hiddenKinds(slices: CubeSlices): Set<StatisticsSliceKind> {
|
||||
const hidden = new Set<StatisticsSliceKind>()
|
||||
if (slices.userId) hidden.add("users")
|
||||
if (slices.serverId) hidden.add("servers")
|
||||
if (slices.iface) hidden.add("interfaces")
|
||||
if (slices.country) hidden.add("countries")
|
||||
if (slices.service) hidden.add("services")
|
||||
if (slices.asn) hidden.add("asns")
|
||||
return hidden
|
||||
}
|
||||
|
||||
function applyDimValue(slices: CubeSlices, kind: StatisticsSliceKind, rowId: string): CubeSlices {
|
||||
const next: CubeSlices = { ...slices }
|
||||
if (kind === "users") {
|
||||
if (rowId === STATISTICS_UNBOUND_USER_ID) return next
|
||||
if (next.userId === rowId) delete next.userId
|
||||
else next.userId = rowId
|
||||
} else if (kind === "servers") {
|
||||
if (next.serverId === rowId) delete next.serverId
|
||||
else next.serverId = rowId
|
||||
} else if (kind === "countries") {
|
||||
if (next.country === rowId) delete next.country
|
||||
else next.country = rowId
|
||||
} else if (kind === "services") {
|
||||
if (next.service === rowId) delete next.service
|
||||
else next.service = rowId
|
||||
} else if (kind === "asns") {
|
||||
if (next.asn === rowId) delete next.asn
|
||||
else next.asn = rowId
|
||||
} else {
|
||||
const colon = rowId.indexOf(":")
|
||||
const sid = colon >= 0 ? rowId.slice(0, colon) : undefined
|
||||
const iface = colon >= 0 ? rowId.slice(colon + 1) : rowId
|
||||
if (next.iface === iface && next.serverId === sid) {
|
||||
delete next.iface
|
||||
delete next.serverId
|
||||
} else {
|
||||
next.iface = iface
|
||||
if (sid) next.serverId = sid
|
||||
}
|
||||
}
|
||||
return next
|
||||
}
|
||||
|
||||
function applyPivotDim(slices: CubeSlices, dim: StatisticsPivotDim, id: string): CubeSlices {
|
||||
const kind = STATISTICS_DIMS.find((d) => d.pivot === dim)?.id ?? "users"
|
||||
return applyDimValue(slices, kind, id)
|
||||
}
|
||||
|
||||
function chipList(slices: CubeSlices): { key: string; label: string }[] {
|
||||
const chips: { key: string; label: string }[] = []
|
||||
if (slices.country) chips.push({ key: "country", label: `страна ${slices.country}` })
|
||||
if (slices.service) chips.push({ key: "service", label: `сервис ${slices.service}` })
|
||||
if (slices.asn) chips.push({ key: "asn", label: `ASN ${slices.asn}` })
|
||||
if (slices.serverId) chips.push({ key: "serverId", label: `сервер ${slices.serverId}` })
|
||||
if (slices.userId) chips.push({ key: "userId", label: `пользователь ${slices.userId}` })
|
||||
if (slices.iface) chips.push({ key: "iface", label: `iface ${slices.iface}` })
|
||||
return chips
|
||||
}
|
||||
|
||||
export default function StatisticsPage() {
|
||||
const router = useRouter()
|
||||
const searchParams = useSearchParams()
|
||||
const { mode, backendUrl, prefsHydrated } = useDataSource()
|
||||
const isLive = mode === "live"
|
||||
|
||||
const range = useMemo(() => readRange(searchParams), [searchParams])
|
||||
const slices = useMemo(() => readSlices(searchParams), [searchParams])
|
||||
const filters = useMemo(() => slicesToFilters(slices), [slices])
|
||||
const dim = useMemo(() => readDim(searchParams), [searchParams])
|
||||
const view = useMemo(() => readView(searchParams), [searchParams])
|
||||
const pivotRow = useMemo(() => readPivotDim(searchParams, "pivotRow", "country"), [searchParams])
|
||||
const pivotCol = useMemo(() => readPivotDim(searchParams, "pivotCol", "service"), [searchParams])
|
||||
|
||||
const [data, setData] = useState<StatisticsDto>(EMPTY)
|
||||
const [pivot, setPivot] = useState<StatisticsPivotDto>(EMPTY_PIVOT)
|
||||
const [loading, setLoading] = useState(false)
|
||||
const [error, setError] = useState<string | null>(null)
|
||||
|
||||
const replaceParams = useCallback(
|
||||
(patch: Record<string, string | undefined>) => {
|
||||
const sp = new URLSearchParams(searchParams.toString())
|
||||
for (const [k, v] of Object.entries(patch)) {
|
||||
if (v) sp.set(k, v)
|
||||
else sp.delete(k)
|
||||
}
|
||||
const qs = sp.toString()
|
||||
router.replace(qs ? `/statistics?${qs}` : "/statistics")
|
||||
},
|
||||
[router, searchParams],
|
||||
)
|
||||
|
||||
const setRange = useCallback(
|
||||
(next: DateRangeYmd) => {
|
||||
replaceParams({ from: next.from, to: next.to })
|
||||
},
|
||||
[replaceParams],
|
||||
)
|
||||
|
||||
const setSlices = useCallback(
|
||||
(next: CubeSlices) => {
|
||||
replaceParams({
|
||||
country: next.country,
|
||||
service: next.service,
|
||||
asn: next.asn,
|
||||
serverId: next.serverId,
|
||||
userId: next.userId,
|
||||
iface: next.iface,
|
||||
})
|
||||
},
|
||||
[replaceParams],
|
||||
)
|
||||
|
||||
useEffect(() => {
|
||||
if (!prefsHydrated || !isLive) return
|
||||
let cancelled = false
|
||||
void (async () => {
|
||||
setLoading(true)
|
||||
setError(null)
|
||||
try {
|
||||
const query = toQuery(range, slices)
|
||||
const dto = await getStatistics(backendUrl, query)
|
||||
if (!cancelled) setData(dto)
|
||||
if (view === "pivot" && pivotRow !== pivotCol) {
|
||||
const matrix = await getStatisticsPivot(backendUrl, {
|
||||
...query,
|
||||
row: pivotRow,
|
||||
col: pivotCol,
|
||||
metric: "bytes",
|
||||
})
|
||||
if (!cancelled) setPivot(matrix)
|
||||
}
|
||||
} catch (e: unknown) {
|
||||
if (!cancelled) {
|
||||
setData(EMPTY)
|
||||
setPivot(EMPTY_PIVOT)
|
||||
setError(e instanceof Error ? e.message : "Не удалось загрузить статистику")
|
||||
}
|
||||
} finally {
|
||||
if (!cancelled) setLoading(false)
|
||||
}
|
||||
})()
|
||||
return () => {
|
||||
cancelled = true
|
||||
}
|
||||
}, [backendUrl, isLive, prefsHydrated, range, slices, view, pivotRow, pivotCol])
|
||||
|
||||
const viewData = isLive ? data : EMPTY
|
||||
const sliced = hasAnySlice(slices)
|
||||
const emptyCube = !isLive || (!loading && viewData.kpis.bytes === 0)
|
||||
|
||||
function handleRowClick(kind: StatisticsSliceKind, row: StatisticsBreakdownRow) {
|
||||
if (kind === "users" && row.id === STATISTICS_UNBOUND_USER_ID) return
|
||||
setSlices(applyDimValue(slices, kind, row.id))
|
||||
}
|
||||
|
||||
function handlePivotCell(rowId: string, colId: string) {
|
||||
if (rowId === "__other__" || colId === "__other__") return
|
||||
let next = applyPivotDim(slices, pivotRow, rowId)
|
||||
next = applyPivotDim(next, pivotCol, colId)
|
||||
replaceParams({
|
||||
country: next.country,
|
||||
service: next.service,
|
||||
asn: next.asn,
|
||||
serverId: next.serverId,
|
||||
userId: next.userId,
|
||||
iface: next.iface,
|
||||
view: "explore",
|
||||
})
|
||||
}
|
||||
|
||||
const kpis = viewData.kpis
|
||||
const chips = chipList(slices)
|
||||
const countLabel =
|
||||
view === "pivot"
|
||||
? `${pivot.rows.length} × ${pivot.columns.length}`
|
||||
: sliced
|
||||
? `${STATISTICS_DIMS.filter((d) => !hiddenKinds(slices).has(d.id)).length} разрезов`
|
||||
: `${rowsForKind(viewData, dim).length} строк`
|
||||
|
||||
return (
|
||||
<div className="flex h-full flex-col">
|
||||
<PageHeader
|
||||
crumbs={[{ label: "Обзор", href: "/dashboard" }, { label: "Статистика" }]}
|
||||
actions={<PeriodSelector range={range} onChange={setRange} />}
|
||||
/>
|
||||
|
||||
<div className="flex flex-1 flex-col gap-4 overflow-y-auto px-4 py-4 md:gap-6 md:px-6 md:py-5">
|
||||
{!isLive ? (
|
||||
<Alert>
|
||||
<AlertTitle>Живые данные выключены</AlertTitle>
|
||||
<AlertDescription>
|
||||
Куб статистики строится из IPFIX. Переключитесь на живой источник, чтобы увидеть отчёт.
|
||||
</AlertDescription>
|
||||
</Alert>
|
||||
) : null}
|
||||
|
||||
{error ? (
|
||||
<Alert variant="destructive">
|
||||
<AlertTitle>Ошибка загрузки</AlertTitle>
|
||||
<AlertDescription>{error}</AlertDescription>
|
||||
</Alert>
|
||||
) : null}
|
||||
|
||||
<KpiStatGrid
|
||||
aria-label="Сводка трафика"
|
||||
isLoading={loading}
|
||||
skeletonCount={5}
|
||||
items={[
|
||||
{
|
||||
id: "bytes",
|
||||
label: "Объём",
|
||||
value: formatBytes(kpis.bytes),
|
||||
hint: kpis.topCountry ? `топ: ${kpis.topCountry}` : undefined,
|
||||
icon: <DatabaseIcon />,
|
||||
iconClassName: "text-muted-foreground",
|
||||
},
|
||||
{
|
||||
id: "packets",
|
||||
label: "Пакеты",
|
||||
value: kpis.packets.toLocaleString("ru-RU"),
|
||||
hint: kpis.topService ? `топ: ${kpis.topService}` : undefined,
|
||||
icon: <ActivityIcon />,
|
||||
iconClassName: "text-muted-foreground",
|
||||
},
|
||||
{
|
||||
id: "bps",
|
||||
label: "Средний bitrate",
|
||||
value: fmtBps(kpis.avgBps),
|
||||
icon: <GaugeIcon />,
|
||||
iconClassName: "text-muted-foreground",
|
||||
},
|
||||
{
|
||||
id: "users",
|
||||
label: "Пользователи",
|
||||
value: String(kpis.users),
|
||||
icon: <UsersIcon />,
|
||||
iconClassName: "text-muted-foreground",
|
||||
},
|
||||
{
|
||||
id: "servers",
|
||||
label: "Серверы",
|
||||
value: String(kpis.servers),
|
||||
hint: kpis.ifaces ? `${kpis.ifaces} iface` : undefined,
|
||||
icon: <ServerIcon />,
|
||||
iconClassName: "text-muted-foreground",
|
||||
},
|
||||
]}
|
||||
/>
|
||||
|
||||
<StatisticsVolumeChart series={viewData.series} grain={viewData.grain} />
|
||||
|
||||
<DataPageCard>
|
||||
<DataPageToolbar
|
||||
leading={
|
||||
<div className="flex flex-wrap items-center gap-3">
|
||||
<SegmentedControl
|
||||
value={view}
|
||||
onChange={(next) => replaceParams({ view: next === "pivot" ? "pivot" : "explore" })}
|
||||
options={[
|
||||
{ value: "explore", label: "Разрез" },
|
||||
{ value: "pivot", label: "Сводка" },
|
||||
]}
|
||||
/>
|
||||
{view === "explore" && !sliced ? (
|
||||
<DimensionSelect
|
||||
label="Критерий"
|
||||
value={dim}
|
||||
onChange={(next) => replaceParams({ dim: next })}
|
||||
/>
|
||||
) : null}
|
||||
{view === "pivot" ? (
|
||||
<>
|
||||
<PivotDimSelect
|
||||
label="Строки"
|
||||
value={pivotRow}
|
||||
exclude={pivotCol}
|
||||
onChange={(next) => replaceParams({ pivotRow: next })}
|
||||
/>
|
||||
<PivotDimSelect
|
||||
label="Колонки"
|
||||
value={pivotCol}
|
||||
exclude={pivotRow}
|
||||
onChange={(next) => replaceParams({ pivotCol: next })}
|
||||
/>
|
||||
</>
|
||||
) : null}
|
||||
</div>
|
||||
}
|
||||
filters={filters}
|
||||
onFiltersChange={(next) => setSlices(filtersToSlices(next))}
|
||||
filterFields={STATISTICS_FILTER_FIELDS}
|
||||
countLabel={countLabel}
|
||||
/>
|
||||
<SliceChips
|
||||
chips={chips}
|
||||
onRemove={(key) => {
|
||||
const next = { ...slices }
|
||||
delete next[key as keyof CubeSlices]
|
||||
setSlices(next)
|
||||
}}
|
||||
/>
|
||||
{emptyCube ? (
|
||||
<EmptyState
|
||||
title="Нет данных куба"
|
||||
description="За выбранный период нет IPFIX-фактов. Куб заполняется с момента деплоя, без бэкфилла за год."
|
||||
/>
|
||||
) : view === "pivot" ? (
|
||||
<StatisticsPivotGrid data={isLive ? pivot : EMPTY_PIVOT} onCellClick={handlePivotCell} isLoading={loading} />
|
||||
) : sliced ? (
|
||||
<BreakdownDashboard
|
||||
data={viewData}
|
||||
hidden={hiddenKinds(slices)}
|
||||
selectedIdFor={(kind) => selectedIdForKind(kind, slices)}
|
||||
onRowClick={handleRowClick}
|
||||
isLoading={loading}
|
||||
/>
|
||||
) : (
|
||||
<StatisticsBreakdownDataGrid
|
||||
rows={rowsForKind(viewData, dim)}
|
||||
kind={dim}
|
||||
selectedId={selectedIdForKind(dim, slices)}
|
||||
onRowClick={(row) => handleRowClick(dim, row)}
|
||||
isLoading={loading}
|
||||
/>
|
||||
)}
|
||||
</DataPageCard>
|
||||
</div>
|
||||
</div>
|
||||
)
|
||||
}
|
||||
@@ -952,10 +952,6 @@ export default function TrafficPage() {
|
||||
setFlowAnalytics(null)
|
||||
return
|
||||
}
|
||||
if (range === "5m") {
|
||||
setFlowAnalytics(null)
|
||||
return
|
||||
}
|
||||
if (range === "30d") {
|
||||
const month = new Date().toISOString().slice(0, 7)
|
||||
void getFlowMonthly(backendUrl, {
|
||||
@@ -1128,7 +1124,7 @@ export default function TrafficPage() {
|
||||
const ingestLine = flowIngestLine(flowStats)
|
||||
const collectorAlive = Boolean(flowStats?.listenerBound || flowStats?.packetsReceived)
|
||||
const flowError = liveError
|
||||
|| (flowLiveError && !(collectorAlive && /live HTTP 500/.test(flowLiveError)) ? flowLiveError : null)
|
||||
|| flowLiveError
|
||||
|| (displayedFlow?.degraded ? "Коллектор перегружен: упрощённая аналитика" : null)
|
||||
|
||||
const flowKpiItems = [
|
||||
|
||||
@@ -5,3 +5,4 @@ dist/
|
||||
*.db-wal
|
||||
.env
|
||||
storage/backups/
|
||||
storage/geoip/
|
||||
|
||||
@@ -0,0 +1,620 @@
|
||||
-- Compact PostgreSQL 18 schema: wipe all app tables (not schema_migrations),
|
||||
-- recreate with smaller time-series rows. SQLite is re-imported after this.
|
||||
|
||||
DO $$
|
||||
DECLARE
|
||||
r record;
|
||||
BEGIN
|
||||
FOR r IN
|
||||
SELECT tablename
|
||||
FROM pg_tables
|
||||
WHERE schemaname = 'public'
|
||||
AND tablename <> 'schema_migrations'
|
||||
LOOP
|
||||
EXECUTE format('DROP TABLE IF EXISTS public.%I CASCADE', r.tablename);
|
||||
END LOOP;
|
||||
END $$;
|
||||
|
||||
CREATE TABLE IF NOT EXISTS schema_migrations (
|
||||
id TEXT PRIMARY KEY,
|
||||
applied_at TIMESTAMPTZ NOT NULL DEFAULT now()
|
||||
);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS servers (
|
||||
id BIGINT GENERATED BY DEFAULT AS IDENTITY PRIMARY KEY,
|
||||
name TEXT NOT NULL DEFAULT '',
|
||||
host TEXT NOT NULL,
|
||||
port INTEGER NOT NULL DEFAULT 443,
|
||||
username TEXT NOT NULL DEFAULT 'admin',
|
||||
password TEXT NOT NULL DEFAULT '',
|
||||
use_ssl BOOLEAN NOT NULL DEFAULT TRUE,
|
||||
verify_ssl BOOLEAN NOT NULL DEFAULT FALSE,
|
||||
type TEXT NOT NULL DEFAULT 'home-router'
|
||||
CHECK (type IN ('jump-host', 'exit-node', 'home-router')),
|
||||
site TEXT NOT NULL DEFAULT '',
|
||||
country TEXT NOT NULL DEFAULT '',
|
||||
asn TEXT NOT NULL DEFAULT '',
|
||||
comment TEXT NOT NULL DEFAULT '',
|
||||
enabled BOOLEAN NOT NULL DEFAULT TRUE,
|
||||
lan_subnet TEXT NOT NULL DEFAULT '',
|
||||
wan_uplinks JSONB COMPRESSION lz4 NOT NULL DEFAULT '[]'::jsonb,
|
||||
mgmt_tunnel_ip TEXT NOT NULL DEFAULT '',
|
||||
created_at TIMESTAMPTZ NOT NULL DEFAULT now(),
|
||||
updated_at TIMESTAMPTZ NOT NULL DEFAULT now()
|
||||
);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS traffic_settings (
|
||||
id BIGINT PRIMARY KEY CHECK (id = 1),
|
||||
enabled BOOLEAN NOT NULL DEFAULT TRUE,
|
||||
interval_sec INTEGER NOT NULL DEFAULT 30,
|
||||
retention_days INTEGER NOT NULL DEFAULT 14,
|
||||
last_collected_at TIMESTAMPTZ,
|
||||
last_duration_ms INTEGER,
|
||||
last_error TEXT,
|
||||
created_at TIMESTAMPTZ NOT NULL DEFAULT now(),
|
||||
updated_at TIMESTAMPTZ NOT NULL DEFAULT now()
|
||||
);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS servers_api_ping_settings (
|
||||
id BIGINT PRIMARY KEY CHECK (id = 1),
|
||||
enabled BOOLEAN NOT NULL DEFAULT FALSE,
|
||||
interval_sec INTEGER NOT NULL DEFAULT 120,
|
||||
last_collected_at TIMESTAMPTZ,
|
||||
last_duration_ms INTEGER,
|
||||
last_error TEXT,
|
||||
created_at TIMESTAMPTZ NOT NULL DEFAULT now(),
|
||||
updated_at TIMESTAMPTZ NOT NULL DEFAULT now()
|
||||
);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS traffic_flow_settings (
|
||||
id BIGINT PRIMARY KEY CHECK (id = 1),
|
||||
enabled BOOLEAN NOT NULL DEFAULT FALSE,
|
||||
collector_ip TEXT NOT NULL DEFAULT '10.255.254.1',
|
||||
flow_listen_port INTEGER NOT NULL DEFAULT 4739,
|
||||
wg_listen_port INTEGER NOT NULL DEFAULT 51821,
|
||||
prefix TEXT NOT NULL DEFAULT '10.255.254.0/24',
|
||||
public_endpoint TEXT NOT NULL DEFAULT '',
|
||||
host_public_key TEXT NOT NULL DEFAULT '',
|
||||
host_private_key TEXT NOT NULL DEFAULT '',
|
||||
hub_server_id BIGINT,
|
||||
retention_hours INTEGER NOT NULL DEFAULT 24,
|
||||
top_n INTEGER NOT NULL DEFAULT 200,
|
||||
map_service_min_share_pct DOUBLE PRECISION NOT NULL DEFAULT 5,
|
||||
last_datagram_at TIMESTAMPTZ,
|
||||
last_exporter_ip TEXT,
|
||||
last_error TEXT,
|
||||
packets_received BIGINT NOT NULL DEFAULT 0,
|
||||
peers_json JSONB COMPRESSION lz4 NOT NULL DEFAULT '[]'::jsonb,
|
||||
created_at TIMESTAMPTZ NOT NULL DEFAULT now(),
|
||||
updated_at TIMESTAMPTZ NOT NULL DEFAULT now()
|
||||
);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS uptime_settings (
|
||||
id BIGINT PRIMARY KEY CHECK (id = 1),
|
||||
enabled BOOLEAN NOT NULL DEFAULT TRUE,
|
||||
resources_enabled BOOLEAN NOT NULL DEFAULT TRUE,
|
||||
ping_enabled BOOLEAN NOT NULL DEFAULT TRUE,
|
||||
speed_enabled BOOLEAN NOT NULL DEFAULT TRUE,
|
||||
interval_sec INTEGER NOT NULL DEFAULT 15,
|
||||
probe_interval_sec INTEGER NOT NULL DEFAULT 15,
|
||||
speed_interval_sec INTEGER NOT NULL DEFAULT 60,
|
||||
retention_days INTEGER NOT NULL DEFAULT 14,
|
||||
last_collected_at TIMESTAMPTZ,
|
||||
last_duration_ms INTEGER,
|
||||
last_error TEXT,
|
||||
created_at TIMESTAMPTZ NOT NULL DEFAULT now(),
|
||||
updated_at TIMESTAMPTZ NOT NULL DEFAULT now()
|
||||
);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS evobgp_settings (
|
||||
id BIGINT PRIMARY KEY CHECK (id = 1),
|
||||
base_url TEXT NOT NULL DEFAULT '',
|
||||
api_key TEXT NOT NULL DEFAULT '',
|
||||
enabled BOOLEAN NOT NULL DEFAULT FALSE,
|
||||
updated_at TIMESTAMPTZ NOT NULL DEFAULT now()
|
||||
);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS alert_telegram_settings (
|
||||
id BIGINT PRIMARY KEY CHECK (id = 1),
|
||||
bot_token TEXT NOT NULL DEFAULT '',
|
||||
chat_id TEXT NOT NULL DEFAULT '',
|
||||
message_thread_id INTEGER,
|
||||
updated_at TIMESTAMPTZ NOT NULL DEFAULT now()
|
||||
);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS acme_settings (
|
||||
id BIGINT PRIMARY KEY CHECK (id = 1),
|
||||
directory_url TEXT NOT NULL DEFAULT 'https://acme-v02.api.letsencrypt.org/directory',
|
||||
cloudflare_api_token TEXT NOT NULL DEFAULT '',
|
||||
default_zone_id TEXT NOT NULL DEFAULT '',
|
||||
account_private_key TEXT NOT NULL DEFAULT '',
|
||||
updated_at TIMESTAMPTZ NOT NULL DEFAULT now()
|
||||
);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS certificate_renew_settings (
|
||||
id BIGINT PRIMARY KEY CHECK (id = 1),
|
||||
enabled BOOLEAN NOT NULL DEFAULT TRUE,
|
||||
interval_sec INTEGER NOT NULL DEFAULT 21600,
|
||||
renew_before_days INTEGER NOT NULL DEFAULT 30,
|
||||
last_collected_at TIMESTAMPTZ,
|
||||
last_duration_ms INTEGER,
|
||||
last_error TEXT,
|
||||
updated_at TIMESTAMPTZ NOT NULL DEFAULT now()
|
||||
);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS backup_schedule_settings (
|
||||
id BIGINT PRIMARY KEY CHECK (id = 1),
|
||||
enabled BOOLEAN NOT NULL DEFAULT TRUE,
|
||||
frequency TEXT NOT NULL DEFAULT 'daily' CHECK (frequency IN ('daily', 'weekly', 'monthly')),
|
||||
hour INTEGER NOT NULL DEFAULT 3,
|
||||
minute INTEGER NOT NULL DEFAULT 0,
|
||||
week_day INTEGER NOT NULL DEFAULT 0,
|
||||
month_day INTEGER NOT NULL DEFAULT 1,
|
||||
keep_count INTEGER NOT NULL DEFAULT 7,
|
||||
format TEXT NOT NULL DEFAULT 'rsc' CHECK (format IN ('rsc', 'backup')),
|
||||
server_ids_json JSONB COMPRESSION lz4 NOT NULL DEFAULT '[]'::jsonb,
|
||||
last_run_at TIMESTAMPTZ,
|
||||
last_duration_ms INTEGER,
|
||||
last_error TEXT,
|
||||
updated_at TIMESTAMPTZ NOT NULL DEFAULT now()
|
||||
);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS internet_path_settings (
|
||||
id BIGINT PRIMARY KEY CHECK (id = 1),
|
||||
enabled BOOLEAN NOT NULL DEFAULT TRUE,
|
||||
interval_sec INTEGER NOT NULL DEFAULT 300,
|
||||
retention_days INTEGER NOT NULL DEFAULT 14,
|
||||
last_collected_at TIMESTAMPTZ,
|
||||
last_duration_ms INTEGER,
|
||||
last_error TEXT,
|
||||
created_at TIMESTAMPTZ NOT NULL DEFAULT now(),
|
||||
updated_at TIMESTAMPTZ NOT NULL DEFAULT now()
|
||||
);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS alert_engine_cursor (
|
||||
id BIGINT PRIMARY KEY CHECK (id = 1),
|
||||
last_source_finished_at TIMESTAMPTZ,
|
||||
updated_at TIMESTAMPTZ NOT NULL DEFAULT now()
|
||||
);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS data_migration (
|
||||
id INTEGER PRIMARY KEY CHECK (id = 1),
|
||||
sqlite_imported_at TIMESTAMPTZ,
|
||||
sqlite_path TEXT,
|
||||
sqlite_sha256 TEXT,
|
||||
report_json JSONB COMPRESSION lz4
|
||||
);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS filter_rules (
|
||||
id BIGINT GENERATED BY DEFAULT AS IDENTITY PRIMARY KEY,
|
||||
server_id BIGINT NOT NULL REFERENCES servers(id) ON DELETE CASCADE,
|
||||
sort_order INTEGER NOT NULL DEFAULT 0,
|
||||
community TEXT NOT NULL,
|
||||
community_name TEXT,
|
||||
action TEXT NOT NULL DEFAULT 'route' CHECK (action IN ('route', 'blackhole')),
|
||||
gateway TEXT NOT NULL DEFAULT '',
|
||||
gateway_tunnel_id TEXT NOT NULL DEFAULT '',
|
||||
description TEXT NOT NULL DEFAULT '',
|
||||
created_at TIMESTAMPTZ NOT NULL DEFAULT now(),
|
||||
updated_at TIMESTAMPTZ NOT NULL DEFAULT now()
|
||||
);
|
||||
CREATE INDEX IF NOT EXISTS idx_filter_rules_server_sort ON filter_rules(server_id, sort_order);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS recursive_routes (
|
||||
id BIGINT GENERATED BY DEFAULT AS IDENTITY PRIMARY KEY,
|
||||
server_id BIGINT NOT NULL REFERENCES servers(id) ON DELETE CASCADE,
|
||||
sort_order INTEGER NOT NULL DEFAULT 0,
|
||||
dst_address TEXT NOT NULL,
|
||||
gateway TEXT NOT NULL,
|
||||
distance INTEGER NOT NULL DEFAULT 1,
|
||||
scope INTEGER,
|
||||
target_scope INTEGER,
|
||||
routing_table TEXT NOT NULL DEFAULT 'main',
|
||||
check_gateway TEXT NOT NULL DEFAULT '',
|
||||
country TEXT NOT NULL DEFAULT '',
|
||||
comment TEXT NOT NULL DEFAULT '',
|
||||
disabled BOOLEAN NOT NULL DEFAULT FALSE,
|
||||
created_at TIMESTAMPTZ NOT NULL DEFAULT now(),
|
||||
updated_at TIMESTAMPTZ NOT NULL DEFAULT now()
|
||||
);
|
||||
CREATE INDEX IF NOT EXISTS idx_recursive_routes_server_sort ON recursive_routes(server_id, sort_order);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS server_snapshots (
|
||||
id BIGINT GENERATED BY DEFAULT AS IDENTITY,
|
||||
server_id BIGINT NOT NULL REFERENCES servers(id) ON DELETE CASCADE,
|
||||
polled_at TIMESTAMPTZ NOT NULL,
|
||||
status TEXT NOT NULL CHECK (status IN ('online', 'offline')),
|
||||
latency_ms DOUBLE PRECISION,
|
||||
ros_version TEXT,
|
||||
board_name TEXT,
|
||||
uptime TEXT,
|
||||
cpu_load INTEGER,
|
||||
free_memory BIGINT,
|
||||
total_memory BIGINT,
|
||||
identity_name TEXT,
|
||||
raw_interfaces JSONB COMPRESSION lz4,
|
||||
raw_ip_addresses JSONB COMPRESSION lz4,
|
||||
PRIMARY KEY (id, polled_at)
|
||||
) PARTITION BY RANGE (polled_at);
|
||||
CREATE INDEX IF NOT EXISTS idx_server_snapshots_server_time ON server_snapshots(server_id, polled_at);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS traffic_samples (
|
||||
server_id BIGINT NOT NULL REFERENCES servers(id) ON DELETE CASCADE,
|
||||
interface_name TEXT NOT NULL,
|
||||
peer_public_key TEXT NOT NULL DEFAULT '',
|
||||
sampled_at TIMESTAMPTZ NOT NULL,
|
||||
rx_bytes BIGINT NOT NULL DEFAULT 0,
|
||||
tx_bytes BIGINT NOT NULL DEFAULT 0,
|
||||
rx_bps BIGINT NOT NULL DEFAULT 0,
|
||||
tx_bps BIGINT NOT NULL DEFAULT 0,
|
||||
flags SMALLINT NOT NULL DEFAULT 0,
|
||||
PRIMARY KEY (server_id, sampled_at, interface_name, peer_public_key)
|
||||
) PARTITION BY RANGE (sampled_at);
|
||||
CREATE INDEX IF NOT EXISTS idx_traffic_samples_server_iface_time ON traffic_samples(server_id, interface_name, sampled_at);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS servers_rest_ping_samples (
|
||||
server_id BIGINT NOT NULL REFERENCES servers(id) ON DELETE CASCADE,
|
||||
sampled_at TIMESTAMPTZ NOT NULL,
|
||||
ok BOOLEAN NOT NULL,
|
||||
latency_ms INTEGER,
|
||||
error TEXT,
|
||||
PRIMARY KEY (server_id, sampled_at)
|
||||
) PARTITION BY RANGE (sampled_at);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS flow_buckets (
|
||||
server_id BIGINT NOT NULL REFERENCES servers(id) ON DELETE CASCADE,
|
||||
bucket_at TIMESTAMPTZ NOT NULL,
|
||||
src INET NOT NULL,
|
||||
dst INET NOT NULL,
|
||||
proto SMALLINT NOT NULL DEFAULT 0,
|
||||
src_port INTEGER NOT NULL DEFAULT 0,
|
||||
dst_port INTEGER NOT NULL DEFAULT 0,
|
||||
bytes BIGINT NOT NULL DEFAULT 0,
|
||||
packets BIGINT NOT NULL DEFAULT 0,
|
||||
in_iface TEXT NOT NULL DEFAULT '',
|
||||
out_iface TEXT NOT NULL DEFAULT '',
|
||||
next_hop INET,
|
||||
flow_start_ms BIGINT NOT NULL DEFAULT 0,
|
||||
flow_end_ms BIGINT NOT NULL DEFAULT 0,
|
||||
PRIMARY KEY (server_id, bucket_at, src, dst, proto, src_port, dst_port, in_iface)
|
||||
) PARTITION BY RANGE (bucket_at);
|
||||
CREATE INDEX IF NOT EXISTS idx_flow_buckets_server_time ON flow_buckets(server_id, bucket_at);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS flow_minute_stats (
|
||||
server_id BIGINT NOT NULL REFERENCES servers(id) ON DELETE CASCADE,
|
||||
bucket_at TIMESTAMPTZ NOT NULL,
|
||||
bytes BIGINT NOT NULL DEFAULT 0,
|
||||
packets BIGINT NOT NULL DEFAULT 0,
|
||||
unique_src INTEGER NOT NULL DEFAULT 0,
|
||||
unique_dst INTEGER NOT NULL DEFAULT 0,
|
||||
conversations INTEGER NOT NULL DEFAULT 0,
|
||||
PRIMARY KEY (server_id, bucket_at)
|
||||
) PARTITION BY RANGE (bucket_at);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS flow_minute_dims (
|
||||
server_id BIGINT NOT NULL REFERENCES servers(id) ON DELETE CASCADE,
|
||||
bucket_at TIMESTAMPTZ NOT NULL,
|
||||
dim TEXT NOT NULL,
|
||||
key TEXT NOT NULL,
|
||||
bytes BIGINT NOT NULL DEFAULT 0,
|
||||
packets BIGINT NOT NULL DEFAULT 0,
|
||||
PRIMARY KEY (server_id, bucket_at, dim, key)
|
||||
) PARTITION BY RANGE (bucket_at);
|
||||
CREATE INDEX IF NOT EXISTS idx_flow_minute_dims_time ON flow_minute_dims(bucket_at, dim);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS flow_daily_dims (
|
||||
server_id BIGINT NOT NULL REFERENCES servers(id) ON DELETE CASCADE,
|
||||
day DATE NOT NULL,
|
||||
dim TEXT NOT NULL,
|
||||
key TEXT NOT NULL,
|
||||
bytes BIGINT NOT NULL DEFAULT 0,
|
||||
packets BIGINT NOT NULL DEFAULT 0,
|
||||
PRIMARY KEY (server_id, day, dim, key)
|
||||
) PARTITION BY RANGE (day);
|
||||
CREATE INDEX IF NOT EXISTS idx_flow_daily_dims_day ON flow_daily_dims(day, dim);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS flow_ip_meta (
|
||||
prefix TEXT PRIMARY KEY,
|
||||
asn INTEGER NOT NULL DEFAULT 0,
|
||||
country TEXT NOT NULL DEFAULT '',
|
||||
lat DOUBLE PRECISION,
|
||||
lng DOUBLE PRECISION,
|
||||
holder TEXT NOT NULL DEFAULT '',
|
||||
ok INTEGER NOT NULL DEFAULT 1,
|
||||
fetched_at TIMESTAMPTZ NOT NULL
|
||||
);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS flow_asn_meta (
|
||||
asn INTEGER PRIMARY KEY,
|
||||
holder TEXT NOT NULL DEFAULT '',
|
||||
fetched_at TIMESTAMPTZ NOT NULL
|
||||
);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS uptime_probes (
|
||||
id TEXT PRIMARY KEY,
|
||||
src_server_id BIGINT NOT NULL REFERENCES servers(id) ON DELETE CASCADE,
|
||||
src_interface TEXT NOT NULL DEFAULT '',
|
||||
name TEXT NOT NULL,
|
||||
target TEXT NOT NULL,
|
||||
probe_filter TEXT NOT NULL DEFAULT '—',
|
||||
enabled BOOLEAN NOT NULL DEFAULT TRUE,
|
||||
interval_sec INTEGER NOT NULL DEFAULT 0,
|
||||
show_on_dashboard BOOLEAN NOT NULL DEFAULT FALSE,
|
||||
sort_order INTEGER NOT NULL DEFAULT 0,
|
||||
created_at TIMESTAMPTZ NOT NULL DEFAULT now(),
|
||||
updated_at TIMESTAMPTZ NOT NULL DEFAULT now()
|
||||
);
|
||||
CREATE INDEX IF NOT EXISTS idx_uptime_probes_server_sort ON uptime_probes(src_server_id, sort_order);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS uptime_probe_samples (
|
||||
probe_id TEXT NOT NULL REFERENCES uptime_probes(id) ON DELETE CASCADE,
|
||||
sampled_at TIMESTAMPTZ NOT NULL,
|
||||
rtt_ms INTEGER,
|
||||
loss_pct INTEGER NOT NULL DEFAULT 0,
|
||||
status TEXT NOT NULL DEFAULT 'down' CHECK (status IN ('up', 'warn', 'down')),
|
||||
PRIMARY KEY (probe_id, sampled_at)
|
||||
) PARTITION BY RANGE (sampled_at);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS uptime_resource_samples (
|
||||
server_id BIGINT NOT NULL REFERENCES servers(id) ON DELETE CASCADE,
|
||||
sampled_at TIMESTAMPTZ NOT NULL,
|
||||
status TEXT NOT NULL DEFAULT 'offline' CHECK (status IN ('online', 'offline')),
|
||||
cpu_load INTEGER NOT NULL DEFAULT 0,
|
||||
free_memory BIGINT NOT NULL DEFAULT 0,
|
||||
total_memory BIGINT NOT NULL DEFAULT 0,
|
||||
free_hdd_space BIGINT NOT NULL DEFAULT 0,
|
||||
total_hdd_space BIGINT NOT NULL DEFAULT 0,
|
||||
uptime_seconds BIGINT NOT NULL DEFAULT 0,
|
||||
PRIMARY KEY (server_id, sampled_at)
|
||||
) PARTITION BY RANGE (sampled_at);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS uptime_speed_probes (
|
||||
id TEXT PRIMARY KEY,
|
||||
src_server_id BIGINT NOT NULL REFERENCES servers(id) ON DELETE CASCADE,
|
||||
dst_server_id BIGINT NOT NULL REFERENCES servers(id) ON DELETE CASCADE,
|
||||
src_interface TEXT NOT NULL DEFAULT '',
|
||||
dst_interface TEXT NOT NULL DEFAULT '',
|
||||
protocol TEXT NOT NULL DEFAULT 'tcp' CHECK (protocol IN ('tcp', 'udp')),
|
||||
direction TEXT NOT NULL DEFAULT 'both' CHECK (direction IN ('transmit', 'receive', 'both')),
|
||||
duration_sec INTEGER NOT NULL DEFAULT 10,
|
||||
enabled BOOLEAN NOT NULL DEFAULT TRUE,
|
||||
last_run_at TIMESTAMPTZ,
|
||||
last_tx_avg_mbps DOUBLE PRECISION,
|
||||
last_rx_avg_mbps DOUBLE PRECISION,
|
||||
last_status TEXT CHECK (last_status IN ('done', 'error')),
|
||||
last_error TEXT,
|
||||
last_ping_rtt_ms INTEGER,
|
||||
last_ping_loss_pct INTEGER,
|
||||
last_ping_at TIMESTAMPTZ,
|
||||
last_ping_error TEXT,
|
||||
sort_order INTEGER NOT NULL DEFAULT 0,
|
||||
created_at TIMESTAMPTZ NOT NULL DEFAULT now(),
|
||||
updated_at TIMESTAMPTZ NOT NULL DEFAULT now()
|
||||
);
|
||||
CREATE INDEX IF NOT EXISTS idx_uptime_speed_probes_src_sort ON uptime_speed_probes(src_server_id, sort_order);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS uptime_speed_test_runs (
|
||||
id TEXT PRIMARY KEY,
|
||||
probe_id TEXT,
|
||||
src_server_id BIGINT NOT NULL REFERENCES servers(id) ON DELETE CASCADE,
|
||||
dst_server_id BIGINT NOT NULL REFERENCES servers(id) ON DELETE CASCADE,
|
||||
src_interface TEXT NOT NULL DEFAULT '',
|
||||
dst_interface TEXT NOT NULL DEFAULT '',
|
||||
src_address TEXT,
|
||||
dst_address TEXT,
|
||||
src_interface_address TEXT,
|
||||
dst_interface_address TEXT,
|
||||
protocol TEXT NOT NULL DEFAULT 'tcp' CHECK (protocol IN ('tcp', 'udp')),
|
||||
direction TEXT NOT NULL DEFAULT 'both' CHECK (direction IN ('transmit', 'receive', 'both')),
|
||||
duration_sec INTEGER NOT NULL DEFAULT 10,
|
||||
tx_avg_mbps DOUBLE PRECISION,
|
||||
rx_avg_mbps DOUBLE PRECISION,
|
||||
ping_rtt_ms INTEGER,
|
||||
ping_loss_pct INTEGER,
|
||||
ping_error TEXT,
|
||||
status TEXT NOT NULL DEFAULT 'done' CHECK (status IN ('done', 'error')),
|
||||
error TEXT,
|
||||
created_at TIMESTAMPTZ NOT NULL DEFAULT now()
|
||||
);
|
||||
CREATE INDEX IF NOT EXISTS idx_uptime_speed_test_runs_created_at ON uptime_speed_test_runs(created_at);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS certificate_issue_jobs (
|
||||
id TEXT PRIMARY KEY,
|
||||
status TEXT NOT NULL DEFAULT 'queued' CHECK (status IN ('queued', 'running', 'done', 'failed')),
|
||||
step TEXT NOT NULL DEFAULT 'queued',
|
||||
source TEXT NOT NULL DEFAULT 'manual' CHECK (source IN ('manual', 'scheduler')),
|
||||
server_id BIGINT NOT NULL REFERENCES servers(id) ON DELETE CASCADE,
|
||||
cert_name TEXT NOT NULL,
|
||||
domain_names JSONB COMPRESSION lz4 NOT NULL,
|
||||
key_type TEXT NOT NULL DEFAULT 'rsa2048',
|
||||
trust_store TEXT NOT NULL DEFAULT 'www,api',
|
||||
requested_at TIMESTAMPTZ NOT NULL DEFAULT now(),
|
||||
started_at TIMESTAMPTZ,
|
||||
finished_at TIMESTAMPTZ,
|
||||
error TEXT
|
||||
);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS backup_entries (
|
||||
id TEXT PRIMARY KEY,
|
||||
server_id BIGINT REFERENCES servers(id) ON DELETE SET NULL,
|
||||
server_name TEXT NOT NULL,
|
||||
filename TEXT NOT NULL,
|
||||
size_bytes BIGINT NOT NULL,
|
||||
kind TEXT NOT NULL DEFAULT 'manual' CHECK (kind IN ('manual', 'auto')),
|
||||
notes TEXT,
|
||||
created_at TIMESTAMPTZ NOT NULL
|
||||
);
|
||||
CREATE UNIQUE INDEX IF NOT EXISTS idx_backup_entries_filename ON backup_entries(filename);
|
||||
CREATE INDEX IF NOT EXISTS idx_backup_entries_server_created ON backup_entries(server_id, created_at);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS alert_groups (
|
||||
id TEXT PRIMARY KEY,
|
||||
name TEXT NOT NULL,
|
||||
combine_mode TEXT NOT NULL DEFAULT 'any' CHECK (combine_mode IN ('any', 'all')),
|
||||
enabled BOOLEAN NOT NULL DEFAULT TRUE,
|
||||
cooldown_override TEXT,
|
||||
created_at TIMESTAMPTZ NOT NULL DEFAULT now(),
|
||||
updated_at TIMESTAMPTZ NOT NULL DEFAULT now()
|
||||
);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS alert_rules (
|
||||
id TEXT PRIMARY KEY,
|
||||
name TEXT NOT NULL,
|
||||
type TEXT NOT NULL,
|
||||
target TEXT NOT NULL,
|
||||
condition TEXT NOT NULL,
|
||||
severity TEXT NOT NULL CHECK (severity IN ('critical', 'warning', 'info')),
|
||||
enabled BOOLEAN NOT NULL DEFAULT TRUE,
|
||||
cooldown TEXT NOT NULL DEFAULT '5м',
|
||||
rule_chat_id TEXT NOT NULL DEFAULT '',
|
||||
confirm_stability_sec INTEGER,
|
||||
recovery_mode TEXT NOT NULL DEFAULT 'always' CHECK (recovery_mode IN ('always', 'never', 'conditional')),
|
||||
recovery_stability_sec INTEGER,
|
||||
group_id TEXT REFERENCES alert_groups(id) ON DELETE SET NULL,
|
||||
created_at TIMESTAMPTZ NOT NULL DEFAULT now(),
|
||||
updated_at TIMESTAMPTZ NOT NULL DEFAULT now()
|
||||
);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS alert_rule_targets (
|
||||
id TEXT PRIMARY KEY,
|
||||
rule_id TEXT NOT NULL REFERENCES alert_rules(id) ON DELETE CASCADE,
|
||||
target TEXT NOT NULL,
|
||||
sort_index INTEGER NOT NULL DEFAULT 0
|
||||
);
|
||||
CREATE INDEX IF NOT EXISTS idx_alert_rule_targets_rule ON alert_rule_targets(rule_id);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS alert_rule_conditions (
|
||||
id TEXT PRIMARY KEY,
|
||||
rule_id TEXT NOT NULL REFERENCES alert_rules(id) ON DELETE CASCADE,
|
||||
condition_line TEXT NOT NULL,
|
||||
sort_index INTEGER NOT NULL DEFAULT 0
|
||||
);
|
||||
CREATE INDEX IF NOT EXISTS idx_alert_rule_conditions_rule ON alert_rule_conditions(rule_id);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS alert_engine_state (
|
||||
scope_key TEXT PRIMARY KEY,
|
||||
last_fired_at TEXT NOT NULL DEFAULT '',
|
||||
last_payload_hash TEXT
|
||||
);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS alert_engine_prev_live (
|
||||
kind TEXT PRIMARY KEY CHECK (kind IN ('gre', 'bgp')),
|
||||
payload_json JSONB COMPRESSION lz4 NOT NULL DEFAULT '{}'::jsonb,
|
||||
updated_at TIMESTAMPTZ NOT NULL DEFAULT now()
|
||||
);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS alert_engine_confirm_pending (
|
||||
rule_id TEXT PRIMARY KEY,
|
||||
payload_hash TEXT NOT NULL,
|
||||
since_at TIMESTAMPTZ NOT NULL
|
||||
);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS alert_history (
|
||||
id TEXT PRIMARY KEY,
|
||||
rule_id TEXT,
|
||||
group_id TEXT,
|
||||
rule_name TEXT NOT NULL,
|
||||
severity TEXT NOT NULL CHECK (severity IN ('critical', 'warning', 'info')),
|
||||
message TEXT NOT NULL,
|
||||
sent_ok BOOLEAN NOT NULL DEFAULT TRUE,
|
||||
fired_at TIMESTAMPTZ NOT NULL
|
||||
);
|
||||
CREATE INDEX IF NOT EXISTS idx_alert_history_fired_at ON alert_history(fired_at);
|
||||
CREATE INDEX IF NOT EXISTS idx_alert_history_rule_id ON alert_history(rule_id);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS alert_outbox (
|
||||
id TEXT PRIMARY KEY,
|
||||
dedupe_key TEXT NOT NULL,
|
||||
channel TEXT NOT NULL DEFAULT 'telegram' CHECK (channel IN ('telegram')),
|
||||
status TEXT NOT NULL DEFAULT 'pending' CHECK (status IN ('pending', 'sent', 'failed')),
|
||||
retry_count INTEGER NOT NULL DEFAULT 0,
|
||||
max_retries INTEGER NOT NULL DEFAULT 3,
|
||||
next_attempt_at TIMESTAMPTZ NOT NULL,
|
||||
payload_json JSONB COMPRESSION lz4 NOT NULL,
|
||||
created_at TIMESTAMPTZ NOT NULL DEFAULT now(),
|
||||
sent_at TIMESTAMPTZ,
|
||||
last_error TEXT
|
||||
);
|
||||
CREATE UNIQUE INDEX IF NOT EXISTS idx_alert_outbox_dedupe ON alert_outbox(dedupe_key);
|
||||
CREATE INDEX IF NOT EXISTS idx_alert_outbox_status_next_attempt ON alert_outbox(status, next_attempt_at);
|
||||
CREATE INDEX IF NOT EXISTS idx_alert_outbox_pending_next ON alert_outbox(next_attempt_at) WHERE status = 'pending';
|
||||
|
||||
CREATE TABLE IF NOT EXISTS scheduler_runs (
|
||||
id TEXT PRIMARY KEY,
|
||||
job_key TEXT NOT NULL,
|
||||
started_at TIMESTAMPTZ NOT NULL,
|
||||
finished_at TIMESTAMPTZ NOT NULL,
|
||||
status TEXT NOT NULL CHECK (status IN ('ok', 'error')),
|
||||
error TEXT,
|
||||
duration_ms INTEGER NOT NULL DEFAULT 0,
|
||||
result_json JSONB COMPRESSION lz4
|
||||
);
|
||||
CREATE INDEX IF NOT EXISTS idx_scheduler_runs_job_time ON scheduler_runs(job_key, started_at);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS events (
|
||||
id TEXT PRIMARY KEY,
|
||||
created_at TIMESTAMPTZ NOT NULL,
|
||||
level TEXT NOT NULL CHECK (level IN ('critical', 'warning', 'info')),
|
||||
event_type TEXT NOT NULL,
|
||||
source_module TEXT NOT NULL,
|
||||
title TEXT NOT NULL,
|
||||
message TEXT NOT NULL,
|
||||
entity_type TEXT,
|
||||
entity_id TEXT,
|
||||
payload_json JSONB COMPRESSION lz4
|
||||
);
|
||||
CREATE INDEX IF NOT EXISTS idx_events_created_at ON events(created_at DESC);
|
||||
CREATE INDEX IF NOT EXISTS idx_events_level_created_at ON events(level, created_at DESC);
|
||||
CREATE INDEX IF NOT EXISTS idx_events_source_created_at ON events(source_module, created_at DESC);
|
||||
CREATE INDEX IF NOT EXISTS idx_events_event_type_created_at ON events(event_type, created_at DESC);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS app_users (
|
||||
id TEXT PRIMARY KEY,
|
||||
name TEXT NOT NULL DEFAULT '',
|
||||
login TEXT NOT NULL UNIQUE,
|
||||
email TEXT NOT NULL DEFAULT '',
|
||||
role TEXT NOT NULL DEFAULT 'viewer' CHECK (role IN ('admin', 'operator', 'viewer')),
|
||||
active BOOLEAN NOT NULL DEFAULT TRUE,
|
||||
avatar TEXT NOT NULL DEFAULT '',
|
||||
last_seen TIMESTAMPTZ,
|
||||
sections_json JSONB COMPRESSION lz4 NOT NULL DEFAULT '[]'::jsonb,
|
||||
servers_json JSONB COMPRESSION lz4 NOT NULL DEFAULT '[]'::jsonb,
|
||||
created_at TIMESTAMPTZ NOT NULL DEFAULT now(),
|
||||
updated_at TIMESTAMPTZ NOT NULL DEFAULT now()
|
||||
);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS user_interface_bindings (
|
||||
id TEXT PRIMARY KEY,
|
||||
user_id TEXT NOT NULL REFERENCES app_users(id) ON DELETE CASCADE,
|
||||
server_id BIGINT NOT NULL REFERENCES servers(id) ON DELETE CASCADE,
|
||||
interface_name TEXT NOT NULL,
|
||||
interface_type TEXT NOT NULL DEFAULT 'other' CHECK (interface_type IN ('ether', 'gre', 'wg', 'other')),
|
||||
peer_public_key TEXT NOT NULL DEFAULT '',
|
||||
peer_name TEXT NOT NULL DEFAULT '',
|
||||
comment TEXT NOT NULL DEFAULT '',
|
||||
created_at TIMESTAMPTZ NOT NULL DEFAULT now(),
|
||||
updated_at TIMESTAMPTZ NOT NULL DEFAULT now(),
|
||||
UNIQUE (server_id, interface_name, peer_public_key)
|
||||
);
|
||||
CREATE INDEX IF NOT EXISTS idx_user_iface_bind_user ON user_interface_bindings(user_id);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS internet_path_snapshots (
|
||||
id BIGINT GENERATED BY DEFAULT AS IDENTITY,
|
||||
sampled_at TIMESTAMPTZ NOT NULL,
|
||||
payload_json JSONB COMPRESSION lz4 NOT NULL,
|
||||
PRIMARY KEY (id, sampled_at)
|
||||
) PARTITION BY RANGE (sampled_at);
|
||||
CREATE INDEX IF NOT EXISTS idx_internet_path_snapshots_sampled ON internet_path_snapshots(sampled_at);
|
||||
|
||||
INSERT INTO traffic_settings (id) VALUES (1) ON CONFLICT (id) DO NOTHING;
|
||||
INSERT INTO traffic_flow_settings (id) VALUES (1) ON CONFLICT (id) DO NOTHING;
|
||||
INSERT INTO uptime_settings (id) VALUES (1) ON CONFLICT (id) DO NOTHING;
|
||||
INSERT INTO evobgp_settings (id) VALUES (1) ON CONFLICT (id) DO NOTHING;
|
||||
INSERT INTO servers_api_ping_settings (id) VALUES (1) ON CONFLICT (id) DO NOTHING;
|
||||
INSERT INTO internet_path_settings (id) VALUES (1) ON CONFLICT (id) DO NOTHING;
|
||||
INSERT INTO alert_telegram_settings (id) VALUES (1) ON CONFLICT (id) DO NOTHING;
|
||||
INSERT INTO acme_settings (id) VALUES (1) ON CONFLICT (id) DO NOTHING;
|
||||
INSERT INTO certificate_renew_settings (id) VALUES (1) ON CONFLICT (id) DO NOTHING;
|
||||
INSERT INTO backup_schedule_settings (id) VALUES (1) ON CONFLICT (id) DO NOTHING;
|
||||
INSERT INTO alert_engine_cursor (id) VALUES (1) ON CONFLICT (id) DO NOTHING;
|
||||
INSERT INTO data_migration (id) VALUES (1) ON CONFLICT (id) DO NOTHING;
|
||||
@@ -0,0 +1,44 @@
|
||||
-- S3-compatible storage for RouterOS backups
|
||||
|
||||
CREATE TABLE IF NOT EXISTS backup_storage_settings (
|
||||
id BIGINT PRIMARY KEY CHECK (id = 1),
|
||||
provider TEXT NOT NULL DEFAULT 'local' CHECK (provider IN ('local', 's3')),
|
||||
s3_endpoint TEXT NOT NULL DEFAULT '',
|
||||
s3_region TEXT NOT NULL DEFAULT 'us-east-1',
|
||||
s3_bucket TEXT NOT NULL DEFAULT '',
|
||||
s3_prefix TEXT NOT NULL DEFAULT 'mikrotik',
|
||||
s3_access_key_id TEXT NOT NULL DEFAULT '',
|
||||
s3_secret_access_key TEXT NOT NULL DEFAULT '',
|
||||
s3_force_path_style BOOLEAN NOT NULL DEFAULT TRUE,
|
||||
keep_local_copy BOOLEAN NOT NULL DEFAULT TRUE,
|
||||
last_test_at TIMESTAMPTZ,
|
||||
last_test_error TEXT,
|
||||
updated_at TIMESTAMPTZ NOT NULL DEFAULT now()
|
||||
);
|
||||
|
||||
INSERT INTO backup_storage_settings (id)
|
||||
VALUES (1)
|
||||
ON CONFLICT (id) DO NOTHING;
|
||||
|
||||
ALTER TABLE backup_entries
|
||||
ADD COLUMN IF NOT EXISTS storage TEXT NOT NULL DEFAULT 'local';
|
||||
|
||||
ALTER TABLE backup_entries
|
||||
ADD COLUMN IF NOT EXISTS s3_key TEXT;
|
||||
|
||||
ALTER TABLE backup_entries
|
||||
ADD COLUMN IF NOT EXISTS s3_etag TEXT;
|
||||
|
||||
ALTER TABLE backup_entries
|
||||
ADD COLUMN IF NOT EXISTS upload_error TEXT;
|
||||
|
||||
DO $$
|
||||
BEGIN
|
||||
IF NOT EXISTS (
|
||||
SELECT 1 FROM pg_constraint WHERE conname = 'backup_entries_storage_check'
|
||||
) THEN
|
||||
ALTER TABLE backup_entries
|
||||
ADD CONSTRAINT backup_entries_storage_check
|
||||
CHECK (storage IN ('local', 's3', 'both'));
|
||||
END IF;
|
||||
END $$;
|
||||
@@ -0,0 +1,19 @@
|
||||
-- GeoLite2 mmdb (страна/ASN для netflow): настройки автообновления зеркала P3TERX
|
||||
|
||||
CREATE TABLE IF NOT EXISTS geoip_settings (
|
||||
id BIGINT PRIMARY KEY CHECK (id = 1),
|
||||
enabled BOOLEAN NOT NULL DEFAULT TRUE,
|
||||
update_interval_sec INTEGER NOT NULL DEFAULT 604800,
|
||||
last_check_at TIMESTAMPTZ,
|
||||
last_success_at TIMESTAMPTZ,
|
||||
last_error TEXT,
|
||||
country_build_at TIMESTAMPTZ,
|
||||
asn_build_at TIMESTAMPTZ,
|
||||
etags_json JSONB NOT NULL DEFAULT '{}'::jsonb,
|
||||
created_at TIMESTAMPTZ NOT NULL DEFAULT now(),
|
||||
updated_at TIMESTAMPTZ NOT NULL DEFAULT now()
|
||||
);
|
||||
|
||||
INSERT INTO geoip_settings (id)
|
||||
VALUES (1)
|
||||
ON CONFLICT (id) DO NOTHING;
|
||||
@@ -0,0 +1,28 @@
|
||||
-- Statistics cube: hour + daily facts (server × iface × country × service × ASN).
|
||||
-- Compact types. FILLFACTOR/autovacuum нельзя на partitioned parent (PG 42809) —
|
||||
-- задаются на листовых партициях в ensurePartitionFor.
|
||||
-- Retention: DROP partitions only (see PARTITION_SPECS).
|
||||
|
||||
CREATE TABLE IF NOT EXISTS flow_hour_facts (
|
||||
server_id BIGINT NOT NULL REFERENCES servers(id) ON DELETE CASCADE,
|
||||
bucket_at TIMESTAMPTZ NOT NULL,
|
||||
iface TEXT NOT NULL,
|
||||
country CHAR(2) NOT NULL,
|
||||
service TEXT NOT NULL,
|
||||
asn INTEGER NOT NULL,
|
||||
bytes BIGINT NOT NULL DEFAULT 0,
|
||||
packets BIGINT NOT NULL DEFAULT 0,
|
||||
PRIMARY KEY (server_id, bucket_at, iface, country, service, asn)
|
||||
) PARTITION BY RANGE (bucket_at);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS flow_daily_facts (
|
||||
server_id BIGINT NOT NULL REFERENCES servers(id) ON DELETE CASCADE,
|
||||
day DATE NOT NULL,
|
||||
iface TEXT NOT NULL,
|
||||
country CHAR(2) NOT NULL,
|
||||
service TEXT NOT NULL,
|
||||
asn INTEGER NOT NULL,
|
||||
bytes BIGINT NOT NULL DEFAULT 0,
|
||||
packets BIGINT NOT NULL DEFAULT 0,
|
||||
PRIMARY KEY (server_id, day, iface, country, service, asn)
|
||||
) PARTITION BY RANGE (day);
|
||||
@@ -15,12 +15,15 @@
|
||||
"test:auth": "tsx src/lib/permissions.test.ts && tsx src/plugins/auth.smoke.test.ts",
|
||||
"test:wireguard": "npx tsx src/services/wireguard-config.test.ts",
|
||||
"test:traffic-rate": "tsx src/services/traffic-rate.test.ts",
|
||||
"test:traffic-flow": "tsx src/services/traffic-flow-parse.test.ts && tsx src/services/traffic-flow-map-exporter.test.ts && tsx src/services/traffic-flow-ifaces.test.ts && tsx src/services/traffic-flow-dedup.test.ts && tsx src/services/traffic-flow-planes.test.ts && tsx src/services/traffic-flow-ip.test.ts && tsx src/services/traffic-flow-classify.test.ts && tsx src/services/traffic-flow-ripe.test.ts && tsx src/services/traffic-flow-brands.test.ts && tsx src/services/traffic-flow-ingest.test.ts && tsx src/services/traffic-flow-analytics.test.ts && tsx src/services/traffic-flow-map-hops.test.ts && tsx src/services/traffic-flow-purge.test.ts",
|
||||
"test:traffic-flow": "tsx src/services/traffic-flow-parse.test.ts && tsx src/services/traffic-flow-map-exporter.test.ts && tsx src/services/traffic-flow-ifaces.test.ts && tsx src/services/traffic-flow-ifindex.test.ts && tsx src/services/traffic-flow-dedup.test.ts && tsx src/services/traffic-flow-planes.test.ts && tsx src/services/traffic-flow-ip.test.ts && tsx src/services/traffic-flow-classify.test.ts && tsx src/services/traffic-flow-ripe.test.ts && tsx src/services/traffic-flow-brands.test.ts && tsx src/services/traffic-flow-ingest.test.ts && tsx src/services/traffic-flow-analytics.test.ts && tsx src/services/traffic-flow-map-hops.test.ts && tsx src/services/traffic-flow-purge.test.ts && tsx src/services/traffic-flow-geoip.test.ts && tsx src/services/traffic-flow-facts.test.ts && tsx src/services/statistics-aggregate.test.ts",
|
||||
"test:users": "tsx src/modules/users/iface-type.test.ts && tsx src/modules/users/bindings.test.ts",
|
||||
"test:pg": "tsx src/db/sql-bind.test.ts && tsx src/db/pg-schema.test.ts",
|
||||
"test": "npm run test:alert-engine && npm run test:auth && npm run test:wireguard && npm run test:traffic-rate && npm run test:traffic-flow && npm run test:users && npm run test:pg"
|
||||
"test:pg": "tsx src/db/sql-bind.test.ts && tsx src/db/sqlite-json.test.ts && tsx src/db/traffic-flags.test.ts && tsx src/db/pg-schema.test.ts",
|
||||
"test:backups": "tsx src/services/s3-backup-client.test.ts",
|
||||
"test": "npm run test:alert-engine && npm run test:auth && npm run test:wireguard && npm run test:traffic-rate && npm run test:traffic-flow && npm run test:users && npm run test:pg && npm run test:backups",
|
||||
"test:geoip": "tsx src/services/traffic-flow-geoip.test.ts"
|
||||
},
|
||||
"dependencies": {
|
||||
"@aws-sdk/client-s3": "^3.888.0",
|
||||
"@fastify/cors": "^11.2.0",
|
||||
"@fastify/jwt": "^10.2.2",
|
||||
"@fastify/type-provider-zod": "^1.0.0",
|
||||
@@ -31,6 +34,7 @@
|
||||
"drizzle-orm": "^0.45.2",
|
||||
"fastify": "^5.8.5",
|
||||
"fastify-plugin": "^5.1.0",
|
||||
"maxmind": "^5.0.7",
|
||||
"pg": "^8.23.0",
|
||||
"undici": "^8.1.0",
|
||||
"zod": "^4.4.1"
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
import { pool } from "./index.js"
|
||||
import { applySqlMigrations } from "./migrate.js"
|
||||
import { dropExpiredPartitions, ensurePartitionsAround } from "./partitions.js"
|
||||
import { importSqliteToPostgres, shouldImportSqlite } from "./sqlite-import.js"
|
||||
import { importSqliteToPostgres, shouldImportSqlite, sqliteFileLooksPresent } from "./sqlite-import.js"
|
||||
import { env } from "../config.js"
|
||||
|
||||
const ETL_LOCK = 8723101
|
||||
@@ -19,6 +19,15 @@ export async function initDatabase(): Promise<void> {
|
||||
console.log(
|
||||
`SQLite → PostgreSQL: готово за ${report.durationMs}ms, таблиц ${Object.keys(report.tables).length}`,
|
||||
)
|
||||
} else {
|
||||
const marker = await pool.query<{ sqlite_imported_at: string | null }>(
|
||||
`SELECT sqlite_imported_at FROM data_migration WHERE id = 1`,
|
||||
)
|
||||
if (!marker.rows[0]?.sqlite_imported_at && !sqliteFileLooksPresent(env.DATABASE_PATH)) {
|
||||
console.warn(
|
||||
`SQLite → PostgreSQL: файл ${env.DATABASE_PATH} не найден, база после wipe остаётся пустой (defaults settings)`,
|
||||
)
|
||||
}
|
||||
}
|
||||
} finally {
|
||||
try {
|
||||
|
||||
+23
-12
@@ -1,9 +1,9 @@
|
||||
import { readFileSync } from "node:fs"
|
||||
import { readdirSync, readFileSync } from "node:fs"
|
||||
import { dirname, join } from "node:path"
|
||||
import { fileURLToPath } from "node:url"
|
||||
import type { Pool } from "pg"
|
||||
|
||||
const MIGRATION_ID = "0000_postgresql"
|
||||
const FIRST_MIGRATION = "0000_postgresql.sql"
|
||||
|
||||
function migrationsDir(): string {
|
||||
const here = dirname(fileURLToPath(import.meta.url))
|
||||
@@ -14,7 +14,7 @@ function migrationsDir(): string {
|
||||
]
|
||||
for (const dir of candidates) {
|
||||
try {
|
||||
readFileSync(join(dir, `${MIGRATION_ID}.sql`), "utf8")
|
||||
readFileSync(join(dir, FIRST_MIGRATION), "utf8")
|
||||
return dir
|
||||
} catch {
|
||||
/* try next */
|
||||
@@ -23,6 +23,10 @@ function migrationsDir(): string {
|
||||
throw new Error("Не найден backend/drizzle/0000_postgresql.sql")
|
||||
}
|
||||
|
||||
function migrationId(file: string): string {
|
||||
return file.replace(/\.sql$/i, "")
|
||||
}
|
||||
|
||||
export async function applySqlMigrations(pool: Pool): Promise<void> {
|
||||
await pool.query(`
|
||||
CREATE TABLE IF NOT EXISTS schema_migrations (
|
||||
@@ -30,14 +34,21 @@ export async function applySqlMigrations(pool: Pool): Promise<void> {
|
||||
applied_at TIMESTAMPTZ NOT NULL DEFAULT now()
|
||||
)
|
||||
`)
|
||||
const { rows } = await pool.query<{ id: string }>(
|
||||
`SELECT id FROM schema_migrations WHERE id = $1`,
|
||||
[MIGRATION_ID],
|
||||
const dir = migrationsDir()
|
||||
const files = readdirSync(dir)
|
||||
.filter((f) => /^\d{4}_.+\.sql$/i.test(f))
|
||||
.sort((a, b) => a.localeCompare(b))
|
||||
const applied = new Set(
|
||||
(await pool.query<{ id: string }>(`SELECT id FROM schema_migrations`)).rows.map((r) => r.id),
|
||||
)
|
||||
if (rows.length > 0) return
|
||||
const sql = readFileSync(join(migrationsDir(), `${MIGRATION_ID}.sql`), "utf8")
|
||||
await pool.query(sql)
|
||||
await pool.query(`INSERT INTO schema_migrations (id) VALUES ($1) ON CONFLICT (id) DO NOTHING`, [
|
||||
MIGRATION_ID,
|
||||
])
|
||||
for (const file of files) {
|
||||
const id = migrationId(file)
|
||||
if (applied.has(id)) continue
|
||||
const sql = readFileSync(join(dir, file), "utf8")
|
||||
await pool.query(sql)
|
||||
await pool.query(
|
||||
`INSERT INTO schema_migrations (id) VALUES ($1) ON CONFLICT (id) DO NOTHING`,
|
||||
[id],
|
||||
)
|
||||
}
|
||||
}
|
||||
|
||||
@@ -8,11 +8,19 @@ export interface PartitionSpec {
|
||||
keepDays: number
|
||||
}
|
||||
|
||||
/** Leaf-only: PG forbids storage params on partitioned parents (SQLSTATE 42809). */
|
||||
const FACT_LEAF_STORAGE =
|
||||
"fillfactor = 70, autovacuum_vacuum_scale_factor = 0.05, autovacuum_vacuum_cost_limit = 2000"
|
||||
|
||||
const FACT_PARENTS = new Set(["flow_hour_facts", "flow_daily_facts"])
|
||||
|
||||
export const PARTITION_SPECS: PartitionSpec[] = [
|
||||
{ parent: "flow_buckets", kind: "day", keepDays: 4 },
|
||||
{ parent: "flow_minute_stats", kind: "day", keepDays: 4 },
|
||||
{ parent: "flow_minute_dims", kind: "day", keepDays: 4 },
|
||||
{ parent: "flow_daily_dims", kind: "month", keepDays: 420 },
|
||||
{ parent: "flow_hour_facts", kind: "day", keepDays: 3 },
|
||||
{ parent: "flow_daily_facts", kind: "month", keepDays: 420 },
|
||||
{ parent: "traffic_samples", kind: "week", keepDays: 21 },
|
||||
{ parent: "servers_rest_ping_samples", kind: "week", keepDays: 35 },
|
||||
{ parent: "uptime_probe_samples", kind: "week", keepDays: 21 },
|
||||
@@ -21,6 +29,37 @@ export const PARTITION_SPECS: PartitionSpec[] = [
|
||||
{ parent: "internet_path_snapshots", kind: "week", keepDays: 21 },
|
||||
]
|
||||
|
||||
const WEEK_SLACK_DAYS = 7
|
||||
|
||||
async function resolveKeepDays(pool: Pool): Promise<Map<string, number>> {
|
||||
const map = new Map(PARTITION_SPECS.map((s) => [s.parent, s.keepDays]))
|
||||
try {
|
||||
const traffic = await pool.query<{ retention_days: number }>(
|
||||
`SELECT retention_days FROM traffic_settings WHERE id = 1`,
|
||||
)
|
||||
const td = Number(traffic.rows[0]?.retention_days)
|
||||
if (Number.isFinite(td) && td > 0) map.set("traffic_samples", td + WEEK_SLACK_DAYS)
|
||||
|
||||
const uptime = await pool.query<{ retention_days: number }>(
|
||||
`SELECT retention_days FROM uptime_settings WHERE id = 1`,
|
||||
)
|
||||
const ud = Number(uptime.rows[0]?.retention_days)
|
||||
if (Number.isFinite(ud) && ud > 0) {
|
||||
map.set("uptime_probe_samples", ud + WEEK_SLACK_DAYS)
|
||||
map.set("uptime_resource_samples", ud + WEEK_SLACK_DAYS)
|
||||
}
|
||||
|
||||
const path = await pool.query<{ retention_days: number }>(
|
||||
`SELECT retention_days FROM internet_path_settings WHERE id = 1`,
|
||||
)
|
||||
const pd = Number(path.rows[0]?.retention_days)
|
||||
if (Number.isFinite(pd) && pd > 0) map.set("internet_path_snapshots", pd + WEEK_SLACK_DAYS)
|
||||
} catch {
|
||||
/* settings may be absent mid-migration */
|
||||
}
|
||||
return map
|
||||
}
|
||||
|
||||
function utcDate(d: Date): Date {
|
||||
return new Date(Date.UTC(d.getUTCFullYear(), d.getUTCMonth(), d.getUTCDate()))
|
||||
}
|
||||
@@ -79,13 +118,35 @@ export async function ensurePartitionFor(
|
||||
await pool.query(
|
||||
`CREATE TABLE IF NOT EXISTS ${name} PARTITION OF ${parent} FOR VALUES FROM ('${from}') TO ('${to}')`,
|
||||
)
|
||||
if (FACT_PARENTS.has(parent)) {
|
||||
await pool.query(`ALTER TABLE ${name} SET (${FACT_LEAF_STORAGE})`)
|
||||
}
|
||||
return name
|
||||
}
|
||||
|
||||
export async function ensurePartitionsBetween(
|
||||
pool: Pool,
|
||||
parent: string,
|
||||
kind: PartitionKind,
|
||||
from: Date,
|
||||
to: Date,
|
||||
): Promise<void> {
|
||||
const start = from.getTime() <= to.getTime() ? from : to
|
||||
const end = from.getTime() <= to.getTime() ? to : from
|
||||
for (let t = new Date(start.getTime()); t <= end; ) {
|
||||
await ensurePartitionFor(pool, parent, kind, t)
|
||||
if (kind === "day") t = addUtcDays(t, 1)
|
||||
else if (kind === "week") t = addUtcDays(t, 7)
|
||||
else t = new Date(Date.UTC(t.getUTCFullYear(), t.getUTCMonth() + 1, 1))
|
||||
}
|
||||
}
|
||||
|
||||
export async function ensurePartitionsAround(pool: Pool, around = new Date()): Promise<void> {
|
||||
const keepDays = await resolveKeepDays(pool)
|
||||
for (const spec of PARTITION_SPECS) {
|
||||
const keep = keepDays.get(spec.parent) ?? spec.keepDays
|
||||
const daysAhead = spec.kind === "month" ? 40 : spec.kind === "week" ? 21 : 8
|
||||
const start = addUtcDays(around, -spec.keepDays)
|
||||
const start = addUtcDays(around, -keep)
|
||||
const end = addUtcDays(around, daysAhead)
|
||||
for (let t = new Date(start.getTime()); t < end; ) {
|
||||
await ensurePartitionFor(pool, spec.parent, spec.kind, t)
|
||||
@@ -97,8 +158,10 @@ export async function ensurePartitionsAround(pool: Pool, around = new Date()): P
|
||||
}
|
||||
|
||||
export async function dropExpiredPartitions(pool: Pool, around = new Date()): Promise<void> {
|
||||
const keepDays = await resolveKeepDays(pool)
|
||||
for (const spec of PARTITION_SPECS) {
|
||||
const cutoff = addUtcDays(around, -spec.keepDays)
|
||||
const keep = keepDays.get(spec.parent) ?? spec.keepDays
|
||||
const cutoff = addUtcDays(around, -keep)
|
||||
const { rows } = await pool.query<{ relname: string }>(
|
||||
`SELECT c.relname
|
||||
FROM pg_inherits i
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
import assert from "node:assert/strict"
|
||||
import { withPgOrSkip } from "../test/pg.js"
|
||||
import { dbQuery, pool } from "./index.js"
|
||||
import { applySqlMigrations } from "./migrate.js"
|
||||
import { ensurePartitionFor } from "./partitions.js"
|
||||
|
||||
if (!(await withPgOrSkip())) {
|
||||
@@ -53,4 +54,120 @@ if (!(await withPgOrSkip())) {
|
||||
await dbQuery(`DELETE FROM alert_outbox WHERE dedupe_key = 'pg-dedupe-key'`)
|
||||
}
|
||||
|
||||
{
|
||||
const peers = [{ endpoint: "msk-gw02.rtnt.top:13232", publicKey: "x" }]
|
||||
await dbQuery(
|
||||
`INSERT INTO alert_outbox (id, dedupe_key, payload_json, next_attempt_at)
|
||||
VALUES ('pg-json-arr', 'pg-json-arr', $1::jsonb, now())`,
|
||||
[JSON.stringify(peers)],
|
||||
)
|
||||
const { rows } = await dbQuery<{ payload_json: unknown }>(
|
||||
`SELECT payload_json FROM alert_outbox WHERE id = 'pg-json-arr'`,
|
||||
)
|
||||
assert.equal(Array.isArray(rows[0]?.payload_json), true)
|
||||
await dbQuery(`DELETE FROM alert_outbox WHERE id = 'pg-json-arr'`)
|
||||
|
||||
let arrayAsPgArrayFailed = false
|
||||
try {
|
||||
await dbQuery(
|
||||
`INSERT INTO alert_outbox (id, dedupe_key, payload_json, next_attempt_at)
|
||||
VALUES ('pg-json-bad', 'pg-json-bad', $1, now())`,
|
||||
[peers],
|
||||
)
|
||||
} catch (err) {
|
||||
arrayAsPgArrayFailed = err instanceof Error && /json|22P02/i.test(err.message)
|
||||
}
|
||||
await dbQuery(`DELETE FROM alert_outbox WHERE id = 'pg-json-bad'`).catch(() => undefined)
|
||||
assert.equal(arrayAsPgArrayFailed, true, "JS array must not be bound as jsonb without stringify")
|
||||
}
|
||||
|
||||
{
|
||||
const { rows } = await dbQuery<{ attname: string }>(`
|
||||
SELECT a.attname
|
||||
FROM pg_index i
|
||||
JOIN unnest(i.indkey) WITH ORDINALITY AS k(attnum, ord) ON true
|
||||
JOIN pg_attribute a ON a.attrelid = i.indrelid AND a.attnum = k.attnum
|
||||
WHERE i.indrelid = 'traffic_samples'::regclass AND i.indisprimary
|
||||
ORDER BY k.ord
|
||||
`)
|
||||
assert.deepEqual(
|
||||
rows.map((r) => r.attname),
|
||||
["server_id", "sampled_at", "interface_name", "peer_public_key"],
|
||||
"traffic_samples PK без id",
|
||||
)
|
||||
}
|
||||
|
||||
{
|
||||
const { rows } = await dbQuery<{ column_name: string }>(`
|
||||
SELECT column_name FROM information_schema.columns
|
||||
WHERE table_schema = 'public' AND table_name = 'traffic_samples'
|
||||
`)
|
||||
const cols = new Set(rows.map((r) => r.column_name))
|
||||
assert.equal(cols.has("id"), false)
|
||||
assert.equal(cols.has("running"), false)
|
||||
assert.equal(cols.has("disabled"), false)
|
||||
assert.equal(cols.has("flags"), true)
|
||||
}
|
||||
|
||||
{
|
||||
const { rows } = await dbQuery<{ column_name: string; udt_name: string }>(`
|
||||
SELECT column_name, udt_name
|
||||
FROM information_schema.columns
|
||||
WHERE table_schema = 'public' AND table_name = 'flow_buckets'
|
||||
AND column_name IN ('src', 'dst', 'next_hop', 'proto')
|
||||
`)
|
||||
const by = Object.fromEntries(rows.map((r) => [r.column_name, r.udt_name]))
|
||||
assert.equal(by.src, "inet")
|
||||
assert.equal(by.dst, "inet")
|
||||
assert.equal(by.next_hop, "inet")
|
||||
assert.equal(by.proto, "int2")
|
||||
}
|
||||
|
||||
{
|
||||
const { rows } = await dbQuery<{ indexdef: string }>(`
|
||||
SELECT indexdef FROM pg_indexes
|
||||
WHERE schemaname = 'public' AND tablename = 'traffic_samples'
|
||||
`)
|
||||
const defs = rows.map((r) => r.indexdef.toLowerCase())
|
||||
assert.equal(defs.some((d) => d.includes("using brin")), false, "нет BRIN на traffic_samples")
|
||||
assert.equal(
|
||||
defs.filter((d) => d.includes("idx_traffic_samples_server_iface_time")).length,
|
||||
1,
|
||||
"один btree (server_id, interface_name, sampled_at)",
|
||||
)
|
||||
assert.equal(defs.some((d) => d.includes("idx_traffic_samples_server_time")), false)
|
||||
}
|
||||
|
||||
{
|
||||
const { rows } = await dbQuery<{ column_name: string }>(`
|
||||
SELECT column_name FROM information_schema.columns
|
||||
WHERE table_schema = 'public' AND table_name = 'uptime_resource_samples'
|
||||
`)
|
||||
const cols = new Set(rows.map((r) => r.column_name))
|
||||
assert.equal(cols.has("board_name"), false)
|
||||
assert.equal(cols.has("ros_version"), false)
|
||||
}
|
||||
|
||||
{
|
||||
const mig = await dbQuery<{ id: string }>(
|
||||
`SELECT id FROM schema_migrations WHERE id = '0002_compact_schema'`,
|
||||
)
|
||||
assert.equal(mig.rows.length, 1, "0002 применена")
|
||||
|
||||
await dbQuery(`INSERT INTO servers (name, host) VALUES ('pg-wipe-idempotent', '127.0.0.1')`)
|
||||
await applySqlMigrations(pool)
|
||||
const still = await dbQuery<{ n: string }>(
|
||||
`SELECT COUNT(*)::text AS n FROM servers WHERE name = 'pg-wipe-idempotent'`,
|
||||
)
|
||||
assert.equal(still.rows[0]?.n, "1", "повторный applySqlMigrations не wipe")
|
||||
await dbQuery(`DELETE FROM servers WHERE name = 'pg-wipe-idempotent'`)
|
||||
}
|
||||
|
||||
{
|
||||
const marker = await dbQuery<{ sqlite_imported_at: string | null }>(
|
||||
`SELECT sqlite_imported_at FROM data_migration WHERE id = 1`,
|
||||
)
|
||||
assert.ok(marker.rows[0], "data_migration singleton после wipe")
|
||||
}
|
||||
|
||||
console.log("pg-schema.test.ts: ok")
|
||||
|
||||
+75
-20
@@ -5,10 +5,12 @@ import {
|
||||
date,
|
||||
doublePrecision,
|
||||
index,
|
||||
inet,
|
||||
integer,
|
||||
jsonb,
|
||||
pgTable,
|
||||
primaryKey,
|
||||
smallint,
|
||||
text,
|
||||
timestamp,
|
||||
uniqueIndex,
|
||||
@@ -135,15 +137,13 @@ export const serversApiPingSettings = pgTable("servers_api_ping_settings", {
|
||||
})
|
||||
|
||||
export const serversRestPingSamples = pgTable("servers_rest_ping_samples", {
|
||||
id: idIdentity(),
|
||||
serverId: intPkRef().references(() => servers.id, { onDelete: "cascade" }),
|
||||
sampledAt: ts("sampled_at").notNull(),
|
||||
ok: boolean("ok").notNull(),
|
||||
latencyMs: integer("latency_ms"),
|
||||
error: text("error"),
|
||||
}, (t) => [
|
||||
primaryKey({ columns: [t.id, t.sampledAt] }),
|
||||
index("idx_servers_rest_ping_samples_server_id").on(t.serverId, t.sampledAt),
|
||||
primaryKey({ columns: [t.serverId, t.sampledAt] }),
|
||||
])
|
||||
|
||||
export const trafficFlowSettings = pgTable("traffic_flow_settings", {
|
||||
@@ -208,19 +208,49 @@ export const flowDailyDims = pgTable("flow_daily_dims", {
|
||||
index("idx_flow_daily_dims_day").on(t.day, t.dim),
|
||||
])
|
||||
|
||||
/** Hour-grain traffic cube for statistics (≤48h). No secondary indexes. */
|
||||
export const flowHourFacts = pgTable("flow_hour_facts", {
|
||||
serverId: bigint("server_id", { mode: "number" }).notNull()
|
||||
.references(() => servers.id, { onDelete: "cascade" }),
|
||||
bucketAt: ts("bucket_at").notNull(),
|
||||
iface: text("iface").notNull(),
|
||||
country: text("country").notNull(),
|
||||
service: text("service").notNull(),
|
||||
asn: integer("asn").notNull(),
|
||||
bytes: bigint("bytes", { mode: "number" }).notNull().default(0),
|
||||
packets: bigint("packets", { mode: "number" }).notNull().default(0),
|
||||
}, (t) => [
|
||||
primaryKey({ columns: [t.serverId, t.bucketAt, t.iface, t.country, t.service, t.asn] }),
|
||||
])
|
||||
|
||||
/** Daily-grain traffic cube for statistics (long window). No secondary indexes. */
|
||||
export const flowDailyFacts = pgTable("flow_daily_facts", {
|
||||
serverId: bigint("server_id", { mode: "number" }).notNull()
|
||||
.references(() => servers.id, { onDelete: "cascade" }),
|
||||
day: date("day", { mode: "string" }).notNull(),
|
||||
iface: text("iface").notNull(),
|
||||
country: text("country").notNull(),
|
||||
service: text("service").notNull(),
|
||||
asn: integer("asn").notNull(),
|
||||
bytes: bigint("bytes", { mode: "number" }).notNull().default(0),
|
||||
packets: bigint("packets", { mode: "number" }).notNull().default(0),
|
||||
}, (t) => [
|
||||
primaryKey({ columns: [t.serverId, t.day, t.iface, t.country, t.service, t.asn] }),
|
||||
])
|
||||
|
||||
export const flowBuckets = pgTable("flow_buckets", {
|
||||
serverId: intPkRef().references(() => servers.id, { onDelete: "cascade" }),
|
||||
bucketAt: ts("bucket_at").notNull(),
|
||||
src: text("src").notNull(),
|
||||
dst: text("dst").notNull(),
|
||||
proto: integer("proto").notNull().default(0),
|
||||
src: inet("src").notNull(),
|
||||
dst: inet("dst").notNull(),
|
||||
proto: smallint("proto").notNull().default(0),
|
||||
srcPort: integer("src_port").notNull().default(0),
|
||||
dstPort: integer("dst_port").notNull().default(0),
|
||||
bytes: bigint("bytes", { mode: "number" }).notNull().default(0),
|
||||
packets: bigint("packets", { mode: "number" }).notNull().default(0),
|
||||
inIface: text("in_iface").notNull().default(""),
|
||||
outIface: text("out_iface").notNull().default(""),
|
||||
nextHop: text("next_hop").notNull().default(""),
|
||||
nextHop: inet("next_hop"),
|
||||
flowStartMs: bigint("flow_start_ms", { mode: "number" }).notNull().default(0),
|
||||
flowEndMs: bigint("flow_end_ms", { mode: "number" }).notNull().default(0),
|
||||
}, (t) => [
|
||||
@@ -231,6 +261,20 @@ export const flowBuckets = pgTable("flow_buckets", {
|
||||
index("idx_flow_buckets_server_time").on(t.serverId, t.bucketAt),
|
||||
])
|
||||
|
||||
export const geoipSettings = pgTable("geoip_settings", {
|
||||
id: idSingleton(),
|
||||
enabled: boolean("enabled").notNull().default(true),
|
||||
updateIntervalSec: integer("update_interval_sec").notNull().default(604800),
|
||||
lastCheckAt: ts("last_check_at"),
|
||||
lastSuccessAt: ts("last_success_at"),
|
||||
lastError: text("last_error"),
|
||||
countryBuildAt: ts("country_build_at"),
|
||||
asnBuildAt: ts("asn_build_at"),
|
||||
etagsJson: jsonb("etags_json").notNull().default(sql`'{}'::jsonb`),
|
||||
createdAt: ts("created_at").notNull().defaultNow(),
|
||||
updatedAt: ts("updated_at").notNull().defaultNow(),
|
||||
})
|
||||
|
||||
export const flowIpMeta = pgTable("flow_ip_meta", {
|
||||
prefix: text("prefix").primaryKey(),
|
||||
asn: integer("asn").notNull().default(0),
|
||||
@@ -249,7 +293,6 @@ export const flowAsnMeta = pgTable("flow_asn_meta", {
|
||||
})
|
||||
|
||||
export const trafficSamples = pgTable("traffic_samples", {
|
||||
id: idIdentity(),
|
||||
serverId: intPkRef().references(() => servers.id, { onDelete: "cascade" }),
|
||||
interfaceName: text("interface_name").notNull(),
|
||||
peerPublicKey: text("peer_public_key").notNull().default(""),
|
||||
@@ -258,11 +301,9 @@ export const trafficSamples = pgTable("traffic_samples", {
|
||||
txBytes: bigint("tx_bytes", { mode: "number" }).notNull().default(0),
|
||||
rxBps: bigint("rx_bps", { mode: "number" }).notNull().default(0),
|
||||
txBps: bigint("tx_bps", { mode: "number" }).notNull().default(0),
|
||||
running: boolean("running").notNull().default(false),
|
||||
disabled: boolean("disabled").notNull().default(false),
|
||||
flags: smallint("flags").notNull().default(0),
|
||||
}, (t) => [
|
||||
primaryKey({ columns: [t.id, t.sampledAt] }),
|
||||
index("idx_traffic_samples_server_time").on(t.serverId, t.sampledAt),
|
||||
primaryKey({ columns: [t.serverId, t.sampledAt, t.interfaceName, t.peerPublicKey] }),
|
||||
index("idx_traffic_samples_server_iface_time").on(t.serverId, t.interfaceName, t.sampledAt),
|
||||
])
|
||||
|
||||
@@ -302,19 +343,16 @@ export const uptimeProbes = pgTable("uptime_probes", {
|
||||
])
|
||||
|
||||
export const uptimeProbeSamples = pgTable("uptime_probe_samples", {
|
||||
id: idIdentity(),
|
||||
probeId: text("probe_id").notNull().references(() => uptimeProbes.id, { onDelete: "cascade" }),
|
||||
sampledAt: ts("sampled_at").notNull(),
|
||||
rttMs: integer("rtt_ms"),
|
||||
lossPct: integer("loss_pct").notNull().default(0),
|
||||
status: text("status", { enum: ["up", "warn", "down"] }).notNull().default("down"),
|
||||
}, (t) => [
|
||||
primaryKey({ columns: [t.id, t.sampledAt] }),
|
||||
index("idx_uptime_probe_samples_probe_time").on(t.probeId, t.sampledAt),
|
||||
primaryKey({ columns: [t.probeId, t.sampledAt] }),
|
||||
])
|
||||
|
||||
export const uptimeResourceSamples = pgTable("uptime_resource_samples", {
|
||||
id: idIdentity(),
|
||||
serverId: intPkRef().references(() => servers.id, { onDelete: "cascade" }),
|
||||
sampledAt: ts("sampled_at").notNull(),
|
||||
status: text("status", { enum: ["online", "offline"] }).notNull().default("offline"),
|
||||
@@ -324,11 +362,8 @@ export const uptimeResourceSamples = pgTable("uptime_resource_samples", {
|
||||
freeHddSpace: bigint("free_hdd_space", { mode: "number" }).notNull().default(0),
|
||||
totalHddSpace: bigint("total_hdd_space", { mode: "number" }).notNull().default(0),
|
||||
uptimeSeconds: bigint("uptime_seconds", { mode: "number" }).notNull().default(0),
|
||||
boardName: text("board_name").notNull().default(""),
|
||||
rosVersion: text("ros_version").notNull().default(""),
|
||||
}, (t) => [
|
||||
primaryKey({ columns: [t.id, t.sampledAt] }),
|
||||
index("idx_uptime_resource_samples_server_time").on(t.serverId, t.sampledAt),
|
||||
primaryKey({ columns: [t.serverId, t.sampledAt] }),
|
||||
])
|
||||
|
||||
export const uptimeSpeedProbes = pgTable("uptime_speed_probes", {
|
||||
@@ -429,6 +464,22 @@ export const backupScheduleSettings = pgTable("backup_schedule_settings", {
|
||||
updatedAt: ts("updated_at").notNull().defaultNow(),
|
||||
})
|
||||
|
||||
export const backupStorageSettings = pgTable("backup_storage_settings", {
|
||||
id: idSingleton(),
|
||||
provider: text("provider", { enum: ["local", "s3"] }).notNull().default("local"),
|
||||
s3Endpoint: text("s3_endpoint").notNull().default(""),
|
||||
s3Region: text("s3_region").notNull().default("us-east-1"),
|
||||
s3Bucket: text("s3_bucket").notNull().default(""),
|
||||
s3Prefix: text("s3_prefix").notNull().default("mikrotik"),
|
||||
s3AccessKeyId: text("s3_access_key_id").notNull().default(""),
|
||||
s3SecretAccessKey: text("s3_secret_access_key").notNull().default(""),
|
||||
s3ForcePathStyle: boolean("s3_force_path_style").notNull().default(true),
|
||||
keepLocalCopy: boolean("keep_local_copy").notNull().default(true),
|
||||
lastTestAt: ts("last_test_at"),
|
||||
lastTestError: text("last_test_error"),
|
||||
updatedAt: ts("updated_at").notNull().defaultNow(),
|
||||
})
|
||||
|
||||
export const backupEntries = pgTable("backup_entries", {
|
||||
id: text("id").primaryKey(),
|
||||
serverId: bigint("server_id", { mode: "number" })
|
||||
@@ -438,6 +489,10 @@ export const backupEntries = pgTable("backup_entries", {
|
||||
sizeBytes: bigint("size_bytes", { mode: "number" }).notNull(),
|
||||
kind: text("kind", { enum: ["manual", "auto"] }).notNull().default("manual"),
|
||||
notes: text("notes"),
|
||||
storage: text("storage", { enum: ["local", "s3", "both"] }).notNull().default("local"),
|
||||
s3Key: text("s3_key"),
|
||||
s3Etag: text("s3_etag"),
|
||||
uploadError: text("upload_error"),
|
||||
createdAt: ts("created_at").notNull(),
|
||||
}, (t) => [
|
||||
uniqueIndex("idx_backup_entries_filename").on(t.filename),
|
||||
|
||||
@@ -3,7 +3,8 @@ import { existsSync, readFileSync } from "node:fs"
|
||||
import Database from "better-sqlite3"
|
||||
import type { Pool } from "pg"
|
||||
import { env } from "../config.js"
|
||||
import { ensurePartitionFor, specForParent } from "./partitions.js"
|
||||
import { ensurePartitionsBetween, specForParent } from "./partitions.js"
|
||||
import { encodeTrafficFlags } from "./traffic-flags.js"
|
||||
|
||||
export interface ImportReport {
|
||||
sqlitePath: string
|
||||
@@ -15,7 +16,9 @@ export interface ImportReport {
|
||||
|
||||
const SNAPSHOT_RETENTION_DAYS = 14
|
||||
|
||||
type ColKind = "ts" | "date" | "bool" | "json" | "json-null" | "bigint-id" | "int" | "text" | "num"
|
||||
type ColKind = "ts" | "date" | "bool" | "json" | "json-null" | "bigint-id" | "int" | "text" | "num" | "flags" | "inet"
|
||||
|
||||
const INSERT_CHUNK = 1000
|
||||
|
||||
interface TableCopy {
|
||||
table: string
|
||||
@@ -78,6 +81,12 @@ const TABLES: TableCopy[] = [
|
||||
["server_ids_json", "json"], ["last_run_at", "ts"], ["last_duration_ms", "int"],
|
||||
["last_error", "text"], ["updated_at", "ts"],
|
||||
]},
|
||||
{ table: "backup_storage_settings", upsert: true, columns: [
|
||||
["id", "int"], ["provider", "text"], ["s3_endpoint", "text"], ["s3_region", "text"],
|
||||
["s3_bucket", "text"], ["s3_prefix", "text"], ["s3_access_key_id", "text"],
|
||||
["s3_secret_access_key", "text"], ["s3_force_path_style", "bool"], ["keep_local_copy", "bool"],
|
||||
["last_test_at", "ts"], ["last_test_error", "text"], ["updated_at", "ts"],
|
||||
]},
|
||||
{ table: "internet_path_settings", upsert: true, columns: [
|
||||
["id", "int"], ["enabled", "bool"], ["interval_sec", "int"], ["retention_days", "int"],
|
||||
["last_collected_at", "ts"], ["last_duration_ms", "int"], ["last_error", "text"],
|
||||
@@ -103,18 +112,18 @@ const TABLES: TableCopy[] = [
|
||||
["free_memory", "int"], ["total_memory", "int"], ["identity_name", "text"],
|
||||
["raw_interfaces", "json-null"], ["raw_ip_addresses", "json-null"],
|
||||
]},
|
||||
{ table: "traffic_samples", identity: true, timeCol: "sampled_at", retentionDays: 14, columns: [
|
||||
["id", "int"], ["server_id", "int"], ["interface_name", "text"], ["peer_public_key", "text"],
|
||||
{ table: "traffic_samples", timeCol: "sampled_at", retentionDays: 14, columns: [
|
||||
["server_id", "int"], ["interface_name", "text"], ["peer_public_key", "text"],
|
||||
["sampled_at", "ts"], ["rx_bytes", "int"], ["tx_bytes", "int"], ["rx_bps", "int"], ["tx_bps", "int"],
|
||||
["running", "bool"], ["disabled", "bool"],
|
||||
["flags", "flags"],
|
||||
]},
|
||||
{ table: "servers_rest_ping_samples", identity: true, timeCol: "sampled_at", retentionDays: 30, columns: [
|
||||
["id", "int"], ["server_id", "int"], ["sampled_at", "ts"], ["ok", "bool"], ["latency_ms", "int"], ["error", "text"],
|
||||
{ table: "servers_rest_ping_samples", timeCol: "sampled_at", retentionDays: 30, columns: [
|
||||
["server_id", "int"], ["sampled_at", "ts"], ["ok", "bool"], ["latency_ms", "int"], ["error", "text"],
|
||||
]},
|
||||
{ table: "flow_buckets", timeCol: "bucket_at", retentionDays: 2, columns: [
|
||||
["server_id", "int"], ["bucket_at", "ts"], ["src", "text"], ["dst", "text"], ["proto", "int"],
|
||||
["server_id", "int"], ["bucket_at", "ts"], ["src", "inet"], ["dst", "inet"], ["proto", "int"],
|
||||
["src_port", "int"], ["dst_port", "int"], ["bytes", "int"], ["packets", "int"],
|
||||
["in_iface", "text"], ["out_iface", "text"], ["next_hop", "text"],
|
||||
["in_iface", "text"], ["out_iface", "text"], ["next_hop", "inet"],
|
||||
["flow_start_ms", "int"], ["flow_end_ms", "int"],
|
||||
]},
|
||||
{ table: "flow_minute_stats", timeCol: "bucket_at", retentionDays: 3, columns: [
|
||||
@@ -139,13 +148,13 @@ const TABLES: TableCopy[] = [
|
||||
["target", "text"], ["probe_filter", "text"], ["enabled", "bool"], ["interval_sec", "int"],
|
||||
["show_on_dashboard", "bool"], ["sort_order", "int"], ["created_at", "ts"], ["updated_at", "ts"],
|
||||
]},
|
||||
{ table: "uptime_probe_samples", identity: true, timeCol: "sampled_at", retentionDays: 14, columns: [
|
||||
["id", "int"], ["probe_id", "text"], ["sampled_at", "ts"], ["rtt_ms", "int"], ["loss_pct", "int"], ["status", "text"],
|
||||
{ table: "uptime_probe_samples", timeCol: "sampled_at", retentionDays: 14, columns: [
|
||||
["probe_id", "text"], ["sampled_at", "ts"], ["rtt_ms", "int"], ["loss_pct", "int"], ["status", "text"],
|
||||
]},
|
||||
{ table: "uptime_resource_samples", identity: true, timeCol: "sampled_at", retentionDays: 14, columns: [
|
||||
["id", "int"], ["server_id", "int"], ["sampled_at", "ts"], ["status", "text"], ["cpu_load", "int"],
|
||||
{ table: "uptime_resource_samples", timeCol: "sampled_at", retentionDays: 14, columns: [
|
||||
["server_id", "int"], ["sampled_at", "ts"], ["status", "text"], ["cpu_load", "int"],
|
||||
["free_memory", "int"], ["total_memory", "int"], ["free_hdd_space", "int"], ["total_hdd_space", "int"],
|
||||
["uptime_seconds", "int"], ["board_name", "text"], ["ros_version", "text"],
|
||||
["uptime_seconds", "int"],
|
||||
]},
|
||||
{ table: "uptime_speed_probes", columns: [
|
||||
["id", "text"], ["src_server_id", "int"], ["dst_server_id", "int"], ["src_interface", "text"],
|
||||
@@ -269,7 +278,20 @@ function parseJson(value: unknown, fallback: unknown): unknown {
|
||||
}
|
||||
}
|
||||
|
||||
function coerce(kind: ColKind, value: unknown, strict: boolean, rejects: string[], ctx: string): unknown {
|
||||
function parseInet(value: unknown): string | null {
|
||||
const s = String(value ?? "").trim()
|
||||
if (!s) return null
|
||||
return s
|
||||
}
|
||||
|
||||
function coerce(
|
||||
kind: ColKind,
|
||||
value: unknown,
|
||||
strict: boolean,
|
||||
rejects: string[],
|
||||
ctx: string,
|
||||
row?: Record<string, unknown>,
|
||||
): unknown {
|
||||
switch (kind) {
|
||||
case "ts":
|
||||
return parseTs(value, strict, rejects, ctx)
|
||||
@@ -278,9 +300,11 @@ function coerce(kind: ColKind, value: unknown, strict: boolean, rejects: string[
|
||||
case "bool":
|
||||
return parseBool(value)
|
||||
case "json":
|
||||
return parseJson(value, [])
|
||||
// Always JSON text. JS arrays must not go to node-pg as values — it encodes
|
||||
// them as PG arrays (`{...}`), which jsonb rejects (22P02).
|
||||
return JSON.stringify(parseJson(value, []))
|
||||
case "json-null":
|
||||
return value == null || value === "" ? null : parseJson(value, null)
|
||||
return value == null || value === "" ? null : JSON.stringify(parseJson(value, null))
|
||||
case "bigint-id": {
|
||||
const t = String(value ?? "").trim()
|
||||
if (!t) return null
|
||||
@@ -301,6 +325,10 @@ function coerce(kind: ColKind, value: unknown, strict: boolean, rejects: string[
|
||||
return Number(value)
|
||||
case "text":
|
||||
return value == null ? "" : String(value)
|
||||
case "flags":
|
||||
return encodeTrafficFlags(parseBool(row?.running), parseBool(row?.disabled))
|
||||
case "inet":
|
||||
return parseInet(value)
|
||||
default:
|
||||
return value == null ? null : String(value)
|
||||
}
|
||||
@@ -324,6 +352,35 @@ async function setval(pool: Pool, table: string): Promise<void> {
|
||||
)
|
||||
}
|
||||
|
||||
function placeholderFor(kind: ColKind, index: number): string {
|
||||
if (kind === "json" || kind === "json-null") return `$${index}::jsonb`
|
||||
if (kind === "inet") return `$${index}::inet`
|
||||
return `$${index}`
|
||||
}
|
||||
|
||||
async function precreatePartitions(
|
||||
sqlite: Database.Database,
|
||||
pool: Pool,
|
||||
spec: TableCopy,
|
||||
where: string,
|
||||
): Promise<void> {
|
||||
const part = specForParent(spec.table)
|
||||
if (!part || !spec.timeCol) return
|
||||
const bounds = sqlite.prepare(
|
||||
`SELECT MIN(${spec.timeCol}) AS a, MAX(${spec.timeCol}) AS b FROM ${spec.table}${where}`,
|
||||
).get() as { a?: unknown; b?: unknown }
|
||||
if (bounds?.a == null || bounds?.b == null) return
|
||||
const isDate = spec.columns.find((c) => c[0] === spec.timeCol)?.[1] === "date"
|
||||
const fromIso = isDate
|
||||
? `${String(bounds.a).slice(0, 10)}T00:00:00Z`
|
||||
: parseTs(bounds.a, false, [], spec.table)
|
||||
const toIso = isDate
|
||||
? `${String(bounds.b).slice(0, 10)}T00:00:00Z`
|
||||
: parseTs(bounds.b, false, [], spec.table)
|
||||
if (!fromIso || !toIso) return
|
||||
await ensurePartitionsBetween(pool, spec.table, part.kind, new Date(fromIso), new Date(toIso))
|
||||
}
|
||||
|
||||
async function copyTable(
|
||||
sqlite: Database.Database,
|
||||
pool: Pool,
|
||||
@@ -339,30 +396,32 @@ async function copyTable(
|
||||
where = ` WHERE ${spec.timeCol} >= '${cutoff.replace("T", " ").slice(0, 19)}' OR ${spec.timeCol} >= '${cutoff}'`
|
||||
}
|
||||
const total = (sqlite.prepare(`SELECT COUNT(*) AS c FROM ${spec.table}${where}`).get() as { c: number }).c
|
||||
const part = specForParent(spec.table)
|
||||
await precreatePartitions(sqlite, pool, spec, where)
|
||||
const cols = spec.columns.map(([c]) => c)
|
||||
const placeholders = cols.map((_, i) => `$${i + 1}`).join(", ")
|
||||
const conflictSql = spec.upsert
|
||||
? `ON CONFLICT (id) DO UPDATE SET ${cols.filter((c) => c !== "id").map((c) => `${c} = EXCLUDED.${c}`).join(", ")}`
|
||||
: `ON CONFLICT DO NOTHING`
|
||||
const insertSql = `INSERT INTO ${spec.table} (${cols.join(", ")}) VALUES (${placeholders}) ${conflictSql}`
|
||||
let copied = 0
|
||||
let skipped = 0
|
||||
const stmt = sqlite.prepare(`SELECT * FROM ${spec.table}${where}`)
|
||||
const batch: unknown[][] = []
|
||||
const flush = async () => {
|
||||
if (batch.length === 0) return
|
||||
const n = spec.columns.length
|
||||
const valuesSql = batch.map((_, i) =>
|
||||
`(${spec.columns.map(([, kind], j) => placeholderFor(kind, i * n + j + 1)).join(", ")})`,
|
||||
).join(", ")
|
||||
const insertSql = `INSERT INTO ${spec.table} (${cols.join(", ")}) VALUES ${valuesSql} ${conflictSql}`
|
||||
const client = await pool.connect()
|
||||
try {
|
||||
await client.query("BEGIN")
|
||||
for (const values of batch) {
|
||||
await client.query(insertSql, values)
|
||||
copied += 1
|
||||
}
|
||||
await client.query(insertSql, batch.flat())
|
||||
copied += batch.length
|
||||
await client.query("COMMIT")
|
||||
} catch (err) {
|
||||
await client.query("ROLLBACK")
|
||||
throw err
|
||||
const detail = err instanceof Error ? err.message : String(err)
|
||||
throw new Error(`${spec.table}: ${detail}`)
|
||||
} finally {
|
||||
client.release()
|
||||
batch.length = 0
|
||||
@@ -370,22 +429,15 @@ async function copyTable(
|
||||
}
|
||||
for (const row of stmt.iterate() as Iterable<Record<string, unknown>>) {
|
||||
try {
|
||||
if (part && spec.timeCol) {
|
||||
const raw = row[spec.timeCol]
|
||||
const ts = spec.columns.find((c) => c[0] === spec.timeCol)?.[1] === "date"
|
||||
? `${String(raw).slice(0, 10)}T00:00:00Z`
|
||||
: parseTs(raw, false, opts.rejects, spec.table)
|
||||
if (ts) await ensurePartitionFor(pool, spec.table, part.kind, new Date(ts))
|
||||
}
|
||||
const values = spec.columns.map(([col, kind]) =>
|
||||
coerce(kind, row[col], opts.strict, opts.rejects, `${spec.table}.${col}`),
|
||||
coerce(kind, row[col], opts.strict, opts.rejects, `${spec.table}.${col}`, row),
|
||||
)
|
||||
if (spec.table === "certificate_issue_jobs" && values[4] == null) {
|
||||
skipped += 1
|
||||
continue
|
||||
}
|
||||
batch.push(values)
|
||||
if (batch.length >= 200) await flush()
|
||||
if (batch.length >= INSERT_CHUNK) await flush()
|
||||
} catch (err) {
|
||||
skipped += 1
|
||||
const msg = `${spec.table}: ${err instanceof Error ? err.message : String(err)}`
|
||||
@@ -421,7 +473,11 @@ export async function shouldImportSqlite(pool: Pool, sqlitePath: string): Promis
|
||||
)
|
||||
if (marker.rows[0]?.sqlite_imported_at) return false
|
||||
const servers = await pool.query<{ c: string }>(`SELECT COUNT(*)::text AS c FROM servers`)
|
||||
if (Number(servers.rows[0]?.c ?? 0) > 0) return false
|
||||
if (Number(servers.rows[0]?.c ?? 0) > 0) {
|
||||
console.warn(
|
||||
"SQLite → PostgreSQL: повтор недописанного импорта (маркера нет, servers уже не пустые)",
|
||||
)
|
||||
}
|
||||
return true
|
||||
}
|
||||
|
||||
@@ -462,6 +518,7 @@ export async function importSqliteToPostgres(
|
||||
return report
|
||||
}
|
||||
for (const spec of TABLES) {
|
||||
console.log(`SQLite → PostgreSQL: таблица ${spec.table}`)
|
||||
report.tables[spec.table] = await copyTable(sqlite, pool, spec, { strict, fullHistory, rejects })
|
||||
}
|
||||
sqlite.close()
|
||||
|
||||
@@ -0,0 +1,11 @@
|
||||
import assert from "node:assert/strict"
|
||||
|
||||
/** node-pg encodes a JS array as a PostgreSQL array (`{...}`), not JSON (`[...]`). */
|
||||
{
|
||||
const peers = [{ endpoint: "msk-gw02.rtnt.top:13232" }]
|
||||
const asJson = JSON.stringify(peers)
|
||||
assert.equal(asJson.startsWith("["), true)
|
||||
assert.equal(asJson.includes("msk-gw02.rtnt.top:13232"), true)
|
||||
}
|
||||
|
||||
console.log("sqlite-json.test.ts: ok")
|
||||
@@ -0,0 +1,19 @@
|
||||
import assert from "node:assert/strict"
|
||||
import {
|
||||
decodeTrafficSampleFlags,
|
||||
encodeTrafficFlags,
|
||||
trafficFlagDisabled,
|
||||
trafficFlagRunning,
|
||||
} from "./traffic-flags.js"
|
||||
|
||||
assert.equal(encodeTrafficFlags(true, false), 1)
|
||||
assert.equal(encodeTrafficFlags(false, true), 2)
|
||||
assert.equal(encodeTrafficFlags(true, true), 3)
|
||||
assert.equal(encodeTrafficFlags(false, false), 0)
|
||||
assert.equal(trafficFlagRunning(1), true)
|
||||
assert.equal(trafficFlagDisabled(1), false)
|
||||
assert.deepEqual(decodeTrafficSampleFlags(1), { running: true, disabled: false })
|
||||
assert.deepEqual(decodeTrafficSampleFlags(0), { running: false, disabled: false })
|
||||
assert.deepEqual(decodeTrafficSampleFlags(undefined), { running: false, disabled: false })
|
||||
|
||||
console.log("traffic-flags.test.ts: ok")
|
||||
@@ -0,0 +1,24 @@
|
||||
export const TRAFFIC_FLAG_RUNNING = 1
|
||||
export const TRAFFIC_FLAG_DISABLED = 2
|
||||
|
||||
export function encodeTrafficFlags(running: boolean, disabled: boolean): number {
|
||||
return (running ? TRAFFIC_FLAG_RUNNING : 0) | (disabled ? TRAFFIC_FLAG_DISABLED : 0)
|
||||
}
|
||||
|
||||
export function trafficFlagRunning(flags: number | null | undefined): boolean {
|
||||
return ((Number(flags) || 0) & TRAFFIC_FLAG_RUNNING) !== 0
|
||||
}
|
||||
|
||||
export function trafficFlagDisabled(flags: number | null | undefined): boolean {
|
||||
return ((Number(flags) || 0) & TRAFFIC_FLAG_DISABLED) !== 0
|
||||
}
|
||||
|
||||
export function decodeTrafficSampleFlags(flags: number | null | undefined): {
|
||||
running: boolean
|
||||
disabled: boolean
|
||||
} {
|
||||
return {
|
||||
running: trafficFlagRunning(flags),
|
||||
disabled: trafficFlagDisabled(flags),
|
||||
}
|
||||
}
|
||||
@@ -14,6 +14,7 @@ import filtersRoutes from "./routes/filters.js"
|
||||
import recursiveRoutes from "./routes/recursive-routes.js"
|
||||
import trafficRoutes from "./routes/traffic.js"
|
||||
import trafficFlowRoutes from "./routes/traffic-flow.js"
|
||||
import geoipRoutes from "./routes/geoip.js"
|
||||
import serversApiPingRoutes from "./routes/servers-api-ping.js"
|
||||
import uptimeRoutes from "./routes/uptime.js"
|
||||
import networkRoutes from "./routes/network.js"
|
||||
@@ -30,8 +31,10 @@ import eventsRoutes from "./routes/events.js"
|
||||
import wireguardRoutes from "./routes/wireguard.js"
|
||||
import firewallRoutes from "./routes/firewall.js"
|
||||
import usersRoutes from "./routes/users.js"
|
||||
import statisticsRoutes from "./routes/statistics.js"
|
||||
import { refreshScheduler, stopScheduler } from "./services/scheduler.js"
|
||||
import { getFlowWorkerHealth, startTrafficFlowListener, stopTrafficFlowListener } from "./services/traffic-flow-ingest.js"
|
||||
import { initGeoip } from "./services/traffic-flow-geoip.js"
|
||||
|
||||
const eventLoopDelay = monitorEventLoopDelay({ resolution: 20 })
|
||||
eventLoopDelay.enable()
|
||||
@@ -116,6 +119,7 @@ export async function buildApp(opts?: {
|
||||
await app.register(recursiveRoutes, { prefix: "/api" })
|
||||
await app.register(trafficRoutes, { prefix: "/api" })
|
||||
await app.register(trafficFlowRoutes, { prefix: "/api" })
|
||||
await app.register(geoipRoutes, { prefix: "/api" })
|
||||
await app.register(serversApiPingRoutes, { prefix: "/api" })
|
||||
await app.register(uptimeRoutes, { prefix: "/api" })
|
||||
await app.register(networkRoutes, { prefix: "/api" })
|
||||
@@ -132,9 +136,11 @@ export async function buildApp(opts?: {
|
||||
await app.register(wireguardRoutes, { prefix: "/api" })
|
||||
await app.register(firewallRoutes, { prefix: "/api" })
|
||||
await app.register(usersRoutes, { prefix: "/api" })
|
||||
await app.register(statisticsRoutes, { prefix: "/api" })
|
||||
|
||||
if (opts?.startScheduler !== false) {
|
||||
await refreshScheduler()
|
||||
await initGeoip()
|
||||
await startTrafficFlowListener()
|
||||
app.addHook("onClose", async () => {
|
||||
stopScheduler()
|
||||
|
||||
@@ -18,7 +18,7 @@ assert.equal(
|
||||
"mm:settings:admin",
|
||||
)
|
||||
assert.equal(
|
||||
permissionForRequest("GET", "/api/traffic/servers/1/live"),
|
||||
permissionForRequest("GET", "/api/statistics"),
|
||||
"mm:traffic:read",
|
||||
)
|
||||
assert.equal(
|
||||
|
||||
@@ -107,7 +107,7 @@ const RULES: Rule[] = [
|
||||
},
|
||||
{
|
||||
methods: ["GET"],
|
||||
match: (p) => p.startsWith("/api/traffic"),
|
||||
match: (p) => p.startsWith("/api/traffic") || p.startsWith("/api/statistics"),
|
||||
permission: "mm:traffic:read",
|
||||
},
|
||||
{
|
||||
|
||||
@@ -13,6 +13,7 @@ import type {
|
||||
} from "@mmapp/contracts/users"
|
||||
import { db } from "../../../db/index.js"
|
||||
import { parseJsonArray } from "../../../db/json.js"
|
||||
import { decodeTrafficSampleFlags } from "../../../db/traffic-flags.js"
|
||||
import { servers, trafficSamples } from "../../../db/schema.js"
|
||||
import {
|
||||
createBindingRow,
|
||||
@@ -270,8 +271,7 @@ export async function listInterfaceCatalog(serverId: number): Promise<CatalogInt
|
||||
.select({
|
||||
interfaceName: trafficSamples.interfaceName,
|
||||
peerPublicKey: trafficSamples.peerPublicKey,
|
||||
running: trafficSamples.running,
|
||||
disabled: trafficSamples.disabled,
|
||||
flags: trafficSamples.flags,
|
||||
})
|
||||
.from(trafficSamples)
|
||||
.where(eq(trafficSamples.serverId, serverId)))
|
||||
@@ -281,11 +281,12 @@ export async function listInterfaceCatalog(serverId: number): Promise<CatalogInt
|
||||
for (const r of rows) {
|
||||
if (seen.has(r.interfaceName)) continue
|
||||
seen.add(r.interfaceName)
|
||||
const decoded = decodeTrafficSampleFlags(r.flags)
|
||||
ifaces.push({
|
||||
name: r.interfaceName,
|
||||
type: mapRosInterfaceType("", r.interfaceName),
|
||||
running: Boolean(r.running),
|
||||
disabled: Boolean(r.disabled),
|
||||
running: decoded.running,
|
||||
disabled: decoded.disabled,
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
@@ -1,20 +1,26 @@
|
||||
import { randomUUID } from "node:crypto"
|
||||
import { readFile } from "node:fs/promises"
|
||||
import path from "node:path"
|
||||
import { z } from "zod"
|
||||
import type { FastifyPluginAsyncZod } from "@fastify/type-provider-zod"
|
||||
import { putBackupScheduleSettingsSchema } from "@mmapp/contracts/backups"
|
||||
import {
|
||||
putBackupScheduleSettingsSchema,
|
||||
putBackupStorageSettingsSchema,
|
||||
} from "@mmapp/contracts/backups"
|
||||
import { listServersRead } from "../modules/servers/service/servers-service.js"
|
||||
import { appendEvent } from "../modules/events/service/events-service.js"
|
||||
import { refreshScheduler } from "../services/scheduler.js"
|
||||
import {
|
||||
deleteBackupRecord,
|
||||
getBackupById,
|
||||
getBackupsDir,
|
||||
getBackupScheduleSettings,
|
||||
getBackupStorageSettings,
|
||||
listBackups,
|
||||
readBackupContent,
|
||||
restoreBackupToDevice,
|
||||
runBackupForServer,
|
||||
syncBackupsFromS3,
|
||||
testBackupStorageConnection,
|
||||
updateBackupScheduleSettings,
|
||||
updateBackupStorageSettings,
|
||||
type BackupMeta,
|
||||
} from "../services/backup-service.js"
|
||||
|
||||
@@ -97,6 +103,39 @@ const backupsRoutes: FastifyPluginAsyncZod = async (app) => {
|
||||
return reply.send(result)
|
||||
})
|
||||
|
||||
app.get("/backups/storage", async (_req, reply) => {
|
||||
return reply.send(await getBackupStorageSettings())
|
||||
})
|
||||
|
||||
app.put("/backups/storage", async (req, reply) => {
|
||||
const parsed = putBackupStorageSettingsSchema.safeParse(req.body ?? {})
|
||||
if (!parsed.success) {
|
||||
return reply.status(400).send({ error: "Некорректное тело запроса", details: parsed.error.flatten() })
|
||||
}
|
||||
return reply.send(await updateBackupStorageSettings(parsed.data))
|
||||
})
|
||||
|
||||
app.post("/backups/storage/test", async (_req, reply) => {
|
||||
try {
|
||||
return reply.send(await testBackupStorageConnection())
|
||||
} catch (err) {
|
||||
return reply.status(400).send({
|
||||
error: err instanceof Error ? err.message : "Ошибка проверки S3",
|
||||
settings: await getBackupStorageSettings(),
|
||||
})
|
||||
}
|
||||
})
|
||||
|
||||
app.post("/backups/storage/sync", async (_req, reply) => {
|
||||
try {
|
||||
return reply.send(await syncBackupsFromS3())
|
||||
} catch (err) {
|
||||
return reply.status(400).send({
|
||||
error: err instanceof Error ? err.message : "Ошибка синхронизации S3",
|
||||
})
|
||||
}
|
||||
})
|
||||
|
||||
app.post("/backups/create", { schema: { body: CreateBackupBodySchema } }, async (req, reply) => {
|
||||
const inputIds = req.body.serverIds.map((x) => String(x))
|
||||
const notes = req.body.notes?.trim() || undefined
|
||||
@@ -169,12 +208,34 @@ const backupsRoutes: FastifyPluginAsyncZod = async (app) => {
|
||||
app.get("/backups/:id/download", { schema: { params: BackupIdParamSchema } }, async (req, reply) => {
|
||||
const hit = await getBackupById(req.params.id)
|
||||
if (!hit) return reply.status(404).send({ error: "Бэкап не найден" })
|
||||
const filePath = path.join(getBackupsDir(), hit.filename)
|
||||
const content = await readFile(filePath, "utf8").catch(() => null)
|
||||
if (content == null) return reply.status(404).send({ error: "Файл бэкапа не найден" })
|
||||
reply.header("Content-Type", "text/plain; charset=utf-8")
|
||||
reply.header("Content-Disposition", `attachment; filename="${hit.filename}"`)
|
||||
return reply.send(content)
|
||||
try {
|
||||
const content = await readBackupContent(hit)
|
||||
reply.header("Content-Type", "text/plain; charset=utf-8")
|
||||
reply.header("Content-Disposition", `attachment; filename="${hit.filename}"`)
|
||||
return reply.send(content)
|
||||
} catch {
|
||||
return reply.status(404).send({ error: "Файл бэкапа не найден" })
|
||||
}
|
||||
})
|
||||
|
||||
app.post("/backups/:id/restore", { schema: { params: BackupIdParamSchema } }, async (req, reply) => {
|
||||
try {
|
||||
const result = await restoreBackupToDevice(req.params.id)
|
||||
await appendEvent({
|
||||
level: "warning",
|
||||
eventType: "backups.restore",
|
||||
sourceModule: "backups",
|
||||
title: "Восстановление бэкапа",
|
||||
message: `${result.filename} → ${result.serverName}`,
|
||||
entityType: "backup",
|
||||
entityId: req.params.id,
|
||||
})
|
||||
return reply.send(result)
|
||||
} catch (err) {
|
||||
return reply.status(400).send({
|
||||
error: err instanceof Error ? err.message : "Не удалось восстановить бэкап",
|
||||
})
|
||||
}
|
||||
})
|
||||
|
||||
app.delete("/backups/:id", { schema: { params: BackupIdParamSchema } }, async (req, reply) => {
|
||||
|
||||
@@ -0,0 +1,57 @@
|
||||
import type { FastifyPluginAsyncZod } from "@fastify/type-provider-zod"
|
||||
import { geoipSettingsPatchSchema } from "@mmapp/contracts/geoip"
|
||||
import { refreshScheduler } from "../services/scheduler.js"
|
||||
import { getGeoipSettings, updateGeoipSettings } from "../services/geoip-settings.js"
|
||||
import {
|
||||
GEOIP_ASN_FILE,
|
||||
GEOIP_COUNTRY_FILE,
|
||||
geoipReadersStatus,
|
||||
initGeoip,
|
||||
} from "../services/traffic-flow-geoip.js"
|
||||
import { collectGeoipUpdateOnce, getGeoipUpdateState } from "../services/geoip-update-collector.js"
|
||||
|
||||
async function buildGeoipStatus() {
|
||||
await initGeoip()
|
||||
const readers = geoipReadersStatus()
|
||||
return {
|
||||
ready: readers.countryLoaded && readers.asnLoaded,
|
||||
countryLoaded: readers.countryLoaded,
|
||||
asnLoaded: readers.asnLoaded,
|
||||
countryFile: GEOIP_COUNTRY_FILE,
|
||||
asnFile: GEOIP_ASN_FILE,
|
||||
dir: readers.dir,
|
||||
running: getGeoipUpdateState().running,
|
||||
settings: await getGeoipSettings(),
|
||||
}
|
||||
}
|
||||
|
||||
const geoipRoutes: FastifyPluginAsyncZod = async (app) => {
|
||||
app.get("/geoip", async (_req, reply) => {
|
||||
return reply.send(await buildGeoipStatus())
|
||||
})
|
||||
|
||||
app.put("/geoip", async (req, reply) => {
|
||||
const parsed = geoipSettingsPatchSchema.safeParse(req.body ?? {})
|
||||
if (!parsed.success) {
|
||||
return reply
|
||||
.status(400)
|
||||
.send({ error: "Некорректное тело запроса", details: parsed.error.flatten() })
|
||||
}
|
||||
await updateGeoipSettings(parsed.data)
|
||||
await refreshScheduler()
|
||||
return reply.send({ ok: true, status: await buildGeoipStatus() })
|
||||
})
|
||||
|
||||
app.post("/geoip/update", async (_req, reply) => {
|
||||
try {
|
||||
const snapshot = await collectGeoipUpdateOnce({ force: true })
|
||||
return reply.send({ ok: !snapshot.fatalError && snapshot.errors.length === 0, snapshot })
|
||||
} catch (e) {
|
||||
const status = (e as { statusCode?: number }).statusCode ?? 502
|
||||
const msg = e instanceof Error ? e.message : String(e)
|
||||
return reply.status(status).send({ error: msg })
|
||||
}
|
||||
})
|
||||
}
|
||||
|
||||
export default geoipRoutes
|
||||
@@ -217,7 +217,7 @@ async function getLatestSnapshotLatencyMs(serverId: number): Promise<number> {
|
||||
return latest.length > 0 ? Math.max(1, Math.round(latest[0].latencyMs ?? 100)) : 100
|
||||
}
|
||||
|
||||
async function latestTrafficByInterface(serverId: number): Promise<Map<string, { id: number; serverId: number; disabled: boolean; sampledAt: string; interfaceName: string; peerPublicKey: string; rxBytes: number; txBytes: number; rxBps: number; txBps: number; running: boolean; }>> {
|
||||
async function latestTrafficByInterface(serverId: number): Promise<Map<string, TrafficSampleRow>> {
|
||||
const rows = await db
|
||||
.select()
|
||||
.from(trafficSamples)
|
||||
|
||||
@@ -0,0 +1,26 @@
|
||||
import type { FastifyPluginAsyncZod } from "@fastify/type-provider-zod"
|
||||
import { statisticsPivotQuerySchema, statisticsQuerySchema } from "@mmapp/contracts/statistics"
|
||||
import { getStatistics, getStatisticsPivot, pivotDimsConflict } from "../services/statistics-aggregate.js"
|
||||
|
||||
const statisticsRoutes: FastifyPluginAsyncZod = async (app) => {
|
||||
app.get("/statistics", async (req, reply) => {
|
||||
const parsed = statisticsQuerySchema.safeParse(req.query ?? {})
|
||||
if (!parsed.success) {
|
||||
return reply.status(400).send({ error: "Некорректный период или фильтры", details: parsed.error.flatten() })
|
||||
}
|
||||
return reply.send(await getStatistics(parsed.data))
|
||||
})
|
||||
|
||||
app.get("/statistics/pivot", async (req, reply) => {
|
||||
const parsed = statisticsPivotQuerySchema.safeParse(req.query ?? {})
|
||||
if (!parsed.success) {
|
||||
return reply.status(400).send({ error: "Некорректный период или измерения", details: parsed.error.flatten() })
|
||||
}
|
||||
if (pivotDimsConflict(parsed.data.row, parsed.data.col)) {
|
||||
return reply.status(400).send({ error: "Строки и колонки должны отличаться" })
|
||||
}
|
||||
return reply.send(await getStatisticsPivot(parsed.data))
|
||||
})
|
||||
}
|
||||
|
||||
export default statisticsRoutes
|
||||
@@ -275,7 +275,7 @@ const trafficFlowRoutes: FastifyPluginAsyncZod = async (app) => {
|
||||
|
||||
try {
|
||||
while (!abort.signal.aborted) {
|
||||
const payload = safeBuildLiveFlowSample(liveQuery)
|
||||
const payload = await safeBuildLiveFlowSample(liveQuery)
|
||||
writeSse(reply.raw, payload.event, payload.data)
|
||||
await sleep(LIVE_TICK_MS, abort.signal)
|
||||
}
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
import type { FastifyPluginAsyncZod } from "@fastify/type-provider-zod"
|
||||
import { asc, desc, eq } from "drizzle-orm"
|
||||
import { db } from "../db/index.js"
|
||||
import { servers, uptimeSpeedProbes, uptimeSpeedTestRuns } from "../db/schema.js"
|
||||
import { servers, serverSnapshots, uptimeSpeedProbes, uptimeSpeedTestRuns } from "../db/schema.js"
|
||||
import { MikrotikClient } from "../services/mikrotik.js"
|
||||
import { scheduleAlertEngineAfterDataCollectors } from "../services/alert-collector-hooks.js"
|
||||
import { refreshScheduler, getSchedulerStatus } from "../services/scheduler.js"
|
||||
@@ -502,7 +502,14 @@ const uptimeRoutes: FastifyPluginAsyncZod = async (app) => {
|
||||
})
|
||||
|
||||
const resources = await Promise.all(allServers.map(async (s) => {
|
||||
const rows = await readResourceSamplesSince(sinceIso, s.id)
|
||||
const [rows, snap] = await Promise.all([
|
||||
readResourceSamplesSince(sinceIso, s.id),
|
||||
db.select({ boardName: serverSnapshots.boardName })
|
||||
.from(serverSnapshots)
|
||||
.where(eq(serverSnapshots.serverId, s.id))
|
||||
.orderBy(desc(serverSnapshots.polledAt))
|
||||
.limit(1),
|
||||
])
|
||||
const { row: pick, hasData } = pickResourceDisplayRow(rows, resourceFallbackMaxGapMs)
|
||||
const cpuHistory = toSeries(rows.map((r) => r.cpuLoad), 40)
|
||||
return {
|
||||
@@ -515,7 +522,7 @@ const uptimeRoutes: FastifyPluginAsyncZod = async (app) => {
|
||||
hddUsed: Math.max(0, ((pick?.totalHddSpace ?? 0) - (pick?.freeHddSpace ?? 0)) / (1024 * 1024)),
|
||||
hddTotal: Math.max(0, (pick?.totalHddSpace ?? 0) / (1024 * 1024)),
|
||||
uptimeSeconds: pick?.uptimeSeconds ?? 0,
|
||||
boardName: pick?.boardName || "RouterBOARD",
|
||||
boardName: snap[0]?.boardName || "RouterBOARD",
|
||||
temp: undefined as number | undefined,
|
||||
}
|
||||
}))
|
||||
|
||||
@@ -1,14 +1,30 @@
|
||||
import { randomUUID } from "node:crypto"
|
||||
import { mkdir, rm, stat, writeFile } from "node:fs/promises"
|
||||
import { mkdir, rm, stat, writeFile, readFile } from "node:fs/promises"
|
||||
import path from "node:path"
|
||||
import { desc, eq } from "drizzle-orm"
|
||||
import type { BackupScheduleSettingsDto } from "@mmapp/contracts/backups"
|
||||
import type {
|
||||
BackupScheduleSettingsDto,
|
||||
BackupStorageSettingsDto,
|
||||
PutBackupStorageSettings,
|
||||
} from "@mmapp/contracts/backups"
|
||||
import { db } from "../db/index.js"
|
||||
import { parseJsonArray } from "../db/json.js"
|
||||
import { backupEntries, backupScheduleSettings } from "../db/schema.js"
|
||||
import { backupEntries, backupScheduleSettings, backupStorageSettings } from "../db/schema.js"
|
||||
import { getServerRowById } from "../modules/servers/repository/servers-repository.js"
|
||||
import { listServersRead } from "../modules/servers/service/servers-service.js"
|
||||
import { MikrotikClient } from "./mikrotik.js"
|
||||
import {
|
||||
buildS3ObjectKey,
|
||||
createS3ClientFromConfig,
|
||||
parseS3ObjectKey,
|
||||
sanitizeServerName,
|
||||
s3DeleteObject,
|
||||
s3GetObject,
|
||||
s3ListObjects,
|
||||
s3PutObject,
|
||||
s3TestConnection,
|
||||
type S3BackupConfig,
|
||||
} from "./s3-backup-client.js"
|
||||
|
||||
const SETTINGS_ID = 1
|
||||
const BACKUPS_DIR = path.resolve(process.cwd(), "storage", "backups")
|
||||
@@ -22,9 +38,14 @@ export type BackupMeta = {
|
||||
createdAt: string
|
||||
kind: "manual" | "auto"
|
||||
notes?: string
|
||||
storage: "local" | "s3" | "both"
|
||||
s3Key?: string | null
|
||||
uploadError?: string | null
|
||||
}
|
||||
|
||||
function rowToMeta(row: typeof backupEntries.$inferSelect): BackupMeta {
|
||||
type BackupRow = typeof backupEntries.$inferSelect
|
||||
|
||||
function rowToMeta(row: BackupRow): BackupMeta {
|
||||
return {
|
||||
id: row.id,
|
||||
serverId: row.serverId,
|
||||
@@ -34,6 +55,9 @@ function rowToMeta(row: typeof backupEntries.$inferSelect): BackupMeta {
|
||||
createdAt: row.createdAt,
|
||||
kind: row.kind,
|
||||
notes: row.notes ?? undefined,
|
||||
storage: row.storage ?? "local",
|
||||
s3Key: row.s3Key,
|
||||
uploadError: row.uploadError,
|
||||
}
|
||||
}
|
||||
|
||||
@@ -60,16 +84,36 @@ export async function insertBackup(meta: BackupMeta): Promise<void> {
|
||||
sizeBytes: meta.sizeBytes,
|
||||
kind: meta.kind,
|
||||
notes: meta.notes ?? null,
|
||||
storage: meta.storage,
|
||||
s3Key: meta.s3Key ?? null,
|
||||
s3Etag: null,
|
||||
uploadError: meta.uploadError ?? null,
|
||||
createdAt: meta.createdAt,
|
||||
})
|
||||
}
|
||||
|
||||
async function getBackupRow(id: string): Promise<BackupRow | undefined> {
|
||||
return (await db.select().from(backupEntries).where(eq(backupEntries.id, id)).limit(1))[0]
|
||||
}
|
||||
|
||||
export async function deleteBackupRecord(id: string): Promise<BackupMeta | null> {
|
||||
const hit = await getBackupById(id)
|
||||
if (!hit) return null
|
||||
const row = await getBackupRow(id)
|
||||
if (!row) return null
|
||||
const meta = rowToMeta(row)
|
||||
if (row.s3Key) {
|
||||
try {
|
||||
const cfg = await getS3ConfigIfEnabled()
|
||||
if (cfg) {
|
||||
const client = createS3ClientFromConfig(cfg)
|
||||
await s3DeleteObject(client, cfg.bucket, row.s3Key)
|
||||
}
|
||||
} catch {
|
||||
/* объект мог уже отсутствовать */
|
||||
}
|
||||
}
|
||||
await db.delete(backupEntries).where(eq(backupEntries.id, id))
|
||||
await rm(path.join(BACKUPS_DIR, hit.filename), { force: true })
|
||||
return hit
|
||||
await rm(path.join(BACKUPS_DIR, row.filename), { force: true })
|
||||
return meta
|
||||
}
|
||||
|
||||
function fmtTs(d = new Date()): string {
|
||||
@@ -167,6 +211,127 @@ export async function touchBackupScheduleRunMeta(patch: {
|
||||
}).where(eq(backupScheduleSettings.id, SETTINGS_ID))
|
||||
}
|
||||
|
||||
function defaultStorageRow() {
|
||||
return {
|
||||
id: SETTINGS_ID,
|
||||
provider: "local" as const,
|
||||
s3Endpoint: "",
|
||||
s3Region: "us-east-1",
|
||||
s3Bucket: "",
|
||||
s3Prefix: "mikrotik",
|
||||
s3AccessKeyId: "",
|
||||
s3SecretAccessKey: "",
|
||||
s3ForcePathStyle: true,
|
||||
keepLocalCopy: true,
|
||||
lastTestAt: null as string | null,
|
||||
lastTestError: null as string | null,
|
||||
updatedAt: new Date().toISOString(),
|
||||
}
|
||||
}
|
||||
|
||||
async function getBackupStorageSettingsRow() {
|
||||
return (await db.select().from(backupStorageSettings).where(eq(backupStorageSettings.id, SETTINGS_ID)).limit(1))[0]
|
||||
?? defaultStorageRow()
|
||||
}
|
||||
|
||||
function toStorageDto(row: Awaited<ReturnType<typeof getBackupStorageSettingsRow>>): BackupStorageSettingsDto {
|
||||
return {
|
||||
provider: row.provider,
|
||||
s3Endpoint: row.s3Endpoint,
|
||||
s3Region: row.s3Region,
|
||||
s3Bucket: row.s3Bucket,
|
||||
s3Prefix: row.s3Prefix,
|
||||
s3AccessKeyId: row.s3AccessKeyId,
|
||||
secretConfigured: Boolean(row.s3SecretAccessKey),
|
||||
s3ForcePathStyle: row.s3ForcePathStyle,
|
||||
keepLocalCopy: row.keepLocalCopy,
|
||||
lastTestAt: row.lastTestAt ?? null,
|
||||
lastTestError: row.lastTestError ?? null,
|
||||
updatedAt: row.updatedAt,
|
||||
}
|
||||
}
|
||||
|
||||
export async function getBackupStorageSettings(): Promise<BackupStorageSettingsDto> {
|
||||
return toStorageDto(await getBackupStorageSettingsRow())
|
||||
}
|
||||
|
||||
export async function updateBackupStorageSettings(
|
||||
patch: PutBackupStorageSettings,
|
||||
): Promise<BackupStorageSettingsDto> {
|
||||
const prev = await getBackupStorageSettingsRow()
|
||||
const now = new Date().toISOString()
|
||||
const secret = patch.s3SecretAccessKey
|
||||
const next = {
|
||||
provider: patch.provider ?? prev.provider,
|
||||
s3Endpoint: patch.s3Endpoint ?? prev.s3Endpoint,
|
||||
s3Region: patch.s3Region ?? prev.s3Region,
|
||||
s3Bucket: patch.s3Bucket ?? prev.s3Bucket,
|
||||
s3Prefix: patch.s3Prefix ?? prev.s3Prefix,
|
||||
s3AccessKeyId: patch.s3AccessKeyId ?? prev.s3AccessKeyId,
|
||||
s3SecretAccessKey: secret && secret.length > 0 ? secret : prev.s3SecretAccessKey,
|
||||
s3ForcePathStyle: patch.s3ForcePathStyle ?? prev.s3ForcePathStyle,
|
||||
keepLocalCopy: patch.keepLocalCopy ?? prev.keepLocalCopy,
|
||||
updatedAt: now,
|
||||
}
|
||||
if ((await db.select().from(backupStorageSettings).where(eq(backupStorageSettings.id, SETTINGS_ID)).limit(1))[0]) {
|
||||
await db.update(backupStorageSettings).set(next).where(eq(backupStorageSettings.id, SETTINGS_ID))
|
||||
} else {
|
||||
await db.insert(backupStorageSettings).values({ id: SETTINGS_ID, ...next })
|
||||
}
|
||||
return await getBackupStorageSettings()
|
||||
}
|
||||
|
||||
function rowToS3Config(row: Awaited<ReturnType<typeof getBackupStorageSettingsRow>>): S3BackupConfig | null {
|
||||
if (row.provider !== "s3") return null
|
||||
if (!row.s3Bucket.trim() || !row.s3AccessKeyId.trim() || !row.s3SecretAccessKey) return null
|
||||
return {
|
||||
endpoint: row.s3Endpoint,
|
||||
region: row.s3Region,
|
||||
bucket: row.s3Bucket.trim(),
|
||||
prefix: row.s3Prefix,
|
||||
accessKeyId: row.s3AccessKeyId,
|
||||
secretAccessKey: row.s3SecretAccessKey,
|
||||
forcePathStyle: row.s3ForcePathStyle,
|
||||
}
|
||||
}
|
||||
|
||||
async function getS3ConfigIfEnabled(): Promise<S3BackupConfig | null> {
|
||||
return rowToS3Config(await getBackupStorageSettingsRow())
|
||||
}
|
||||
|
||||
export async function testBackupStorageConnection(): Promise<BackupStorageSettingsDto> {
|
||||
const row = await getBackupStorageSettingsRow()
|
||||
const cfg = rowToS3Config(row)
|
||||
const now = new Date().toISOString()
|
||||
if (!cfg) {
|
||||
const error = row.provider === "s3"
|
||||
? "Заполните bucket, ключ доступа и секрет"
|
||||
: "S3 не выбран"
|
||||
await persistStorageTest(now, error)
|
||||
throw new Error(error)
|
||||
}
|
||||
try {
|
||||
const client = createS3ClientFromConfig(cfg)
|
||||
await s3TestConnection(client, cfg.bucket)
|
||||
await persistStorageTest(now, null)
|
||||
} catch (err) {
|
||||
const message = err instanceof Error ? err.message : String(err)
|
||||
await persistStorageTest(now, message)
|
||||
throw new Error(message)
|
||||
}
|
||||
return await getBackupStorageSettings()
|
||||
}
|
||||
|
||||
async function persistStorageTest(at: string, error: string | null) {
|
||||
const exists = (await db.select().from(backupStorageSettings).where(eq(backupStorageSettings.id, SETTINGS_ID)).limit(1))[0]
|
||||
const patch = { lastTestAt: at, lastTestError: error, updatedAt: at }
|
||||
if (exists) {
|
||||
await db.update(backupStorageSettings).set(patch).where(eq(backupStorageSettings.id, SETTINGS_ID))
|
||||
} else {
|
||||
await db.insert(backupStorageSettings).values({ id: SETTINGS_ID, ...patch })
|
||||
}
|
||||
}
|
||||
|
||||
export async function resolveBackupServerIds(settings: BackupScheduleSettingsDto): Promise<string[]> {
|
||||
const enabled = new Set((await listServersRead()).map((s) => String(s.id)))
|
||||
const requested = settings.serverIds.length > 0 ? settings.serverIds : [...enabled]
|
||||
@@ -214,6 +379,23 @@ export function isBackupDue(now: Date, settings: BackupScheduleSettingsDto, last
|
||||
return true
|
||||
}
|
||||
|
||||
async function uploadBackupToS3(params: {
|
||||
serverName: string
|
||||
filename: string
|
||||
body: string
|
||||
}): Promise<{ key: string; etag?: string } | { error: string }> {
|
||||
const cfg = await getS3ConfigIfEnabled()
|
||||
if (!cfg) return { error: "S3 не настроен" }
|
||||
try {
|
||||
const client = createS3ClientFromConfig(cfg)
|
||||
const key = buildS3ObjectKey(cfg.prefix, sanitizeServerName(params.serverName), params.filename)
|
||||
const put = await s3PutObject(client, cfg.bucket, key, params.body)
|
||||
return { key, etag: put.etag }
|
||||
} catch (err) {
|
||||
return { error: err instanceof Error ? err.message : String(err) }
|
||||
}
|
||||
}
|
||||
|
||||
export async function runBackupForServer(
|
||||
id: string,
|
||||
kind: BackupMeta["kind"],
|
||||
@@ -230,12 +412,33 @@ export async function runBackupForServer(
|
||||
const client = MikrotikClient.fromServer(row)
|
||||
const script = await client.exportConfigScript()
|
||||
const ts = fmtTs()
|
||||
const safeServer = row.name.replace(/[^a-zA-Z0-9._-]+/g, "_")
|
||||
const safeServer = sanitizeServerName(row.name)
|
||||
const filename = `${safeServer}_${ts}.rsc`
|
||||
const filePath = path.join(BACKUPS_DIR, filename)
|
||||
await ensureBackupStorage()
|
||||
await writeFile(filePath, script, "utf8")
|
||||
const st = await stat(filePath)
|
||||
const storageRow = await getBackupStorageSettingsRow()
|
||||
let storage: BackupMeta["storage"] = "local"
|
||||
let s3Key: string | null = null
|
||||
let s3Etag: string | null = null
|
||||
let uploadError: string | null = null
|
||||
|
||||
if (storageRow.provider === "s3") {
|
||||
const uploaded = await uploadBackupToS3({ serverName: row.name, filename, body: script })
|
||||
if ("key" in uploaded) {
|
||||
s3Key = uploaded.key
|
||||
s3Etag = uploaded.etag ?? null
|
||||
storage = storageRow.keepLocalCopy ? "both" : "s3"
|
||||
if (!storageRow.keepLocalCopy) {
|
||||
await rm(filePath, { force: true })
|
||||
}
|
||||
} else {
|
||||
uploadError = uploaded.error
|
||||
storage = "local"
|
||||
}
|
||||
}
|
||||
|
||||
const meta: BackupMeta = {
|
||||
id: randomUUID(),
|
||||
serverId: row.id,
|
||||
@@ -245,8 +448,24 @@ export async function runBackupForServer(
|
||||
createdAt: new Date().toISOString(),
|
||||
kind,
|
||||
notes,
|
||||
storage,
|
||||
s3Key,
|
||||
uploadError,
|
||||
}
|
||||
await insertBackup(meta)
|
||||
await db.insert(backupEntries).values({
|
||||
id: meta.id,
|
||||
serverId: meta.serverId,
|
||||
serverName: meta.serverName,
|
||||
filename: meta.filename,
|
||||
sizeBytes: meta.sizeBytes,
|
||||
kind: meta.kind,
|
||||
notes: meta.notes ?? null,
|
||||
storage: meta.storage,
|
||||
s3Key: meta.s3Key ?? null,
|
||||
s3Etag,
|
||||
uploadError: meta.uploadError ?? null,
|
||||
createdAt: meta.createdAt,
|
||||
})
|
||||
return meta
|
||||
}
|
||||
|
||||
@@ -257,12 +476,82 @@ export async function pruneBackupsForServer(serverId: string, keepCount: number)
|
||||
if (rows.length <= keepCount) return 0
|
||||
const toDelete = rows.slice(keepCount)
|
||||
for (const hit of toDelete) {
|
||||
await db.delete(backupEntries).where(eq(backupEntries.id, hit.id))
|
||||
await rm(path.join(BACKUPS_DIR, hit.filename), { force: true })
|
||||
await deleteBackupRecord(hit.id)
|
||||
}
|
||||
return toDelete.length
|
||||
}
|
||||
|
||||
export async function readBackupContent(meta: BackupMeta): Promise<Buffer> {
|
||||
if ((meta.storage === "s3" || meta.storage === "both") && meta.s3Key) {
|
||||
try {
|
||||
const cfg = await getS3ConfigIfEnabled()
|
||||
if (cfg) {
|
||||
const client = createS3ClientFromConfig(cfg)
|
||||
return await s3GetObject(client, cfg.bucket, meta.s3Key)
|
||||
}
|
||||
} catch {
|
||||
/* fallback: локальная копия, если есть */
|
||||
}
|
||||
}
|
||||
return await readFile(path.join(BACKUPS_DIR, meta.filename))
|
||||
}
|
||||
|
||||
export async function restoreBackupToDevice(id: string): Promise<{ filename: string; serverName: string }> {
|
||||
const meta = await getBackupById(id)
|
||||
if (!meta) throw new Error("Бэкап не найден")
|
||||
if (!meta.serverId) throw new Error("Сервер бэкапа удалён — восстановить нельзя")
|
||||
const row = await getServerRowById(meta.serverId)
|
||||
if (!row) throw new Error("Сервер не найден")
|
||||
const content = await readBackupContent(meta)
|
||||
const client = MikrotikClient.fromServer(row)
|
||||
const uploaded = await client.uploadTextFile(meta.filename, content.toString("utf8"), 60_000)
|
||||
await client.importUploadedFile(uploaded)
|
||||
return { filename: meta.filename, serverName: row.name }
|
||||
}
|
||||
|
||||
export async function syncBackupsFromS3(): Promise<{ imported: number; skipped: number }> {
|
||||
const cfg = await getS3ConfigIfEnabled()
|
||||
if (!cfg) throw new Error("S3 не настроен")
|
||||
const client = createS3ClientFromConfig(cfg)
|
||||
const objects = await s3ListObjects(client, cfg.bucket, cfg.prefix)
|
||||
const existing = new Set(
|
||||
(await db.select({ filename: backupEntries.filename, s3Key: backupEntries.s3Key }).from(backupEntries))
|
||||
.flatMap((row) => [row.filename, row.s3Key].filter((v): v is string => Boolean(v))),
|
||||
)
|
||||
let imported = 0
|
||||
let skipped = 0
|
||||
for (const obj of objects) {
|
||||
if (!obj.key.toLowerCase().endsWith(".rsc")) {
|
||||
skipped += 1
|
||||
continue
|
||||
}
|
||||
const parsed = parseS3ObjectKey(obj.key)
|
||||
if (existing.has(obj.key) || existing.has(parsed.filename)) {
|
||||
skipped += 1
|
||||
continue
|
||||
}
|
||||
const createdAt = obj.lastModified ?? new Date().toISOString()
|
||||
await db.insert(backupEntries).values({
|
||||
id: randomUUID(),
|
||||
serverId: null,
|
||||
serverName: parsed.serverName,
|
||||
filename: parsed.filename,
|
||||
sizeBytes: obj.size,
|
||||
kind: "auto",
|
||||
notes: "Импорт из S3",
|
||||
storage: "s3",
|
||||
s3Key: obj.key,
|
||||
s3Etag: null,
|
||||
uploadError: null,
|
||||
createdAt,
|
||||
})
|
||||
existing.add(obj.key)
|
||||
existing.add(parsed.filename)
|
||||
imported += 1
|
||||
}
|
||||
return { imported, skipped }
|
||||
}
|
||||
|
||||
export function getBackupsDir(): string {
|
||||
return BACKUPS_DIR
|
||||
}
|
||||
|
||||
@@ -113,6 +113,15 @@ export async function collectCertificatesRenewOnce(): Promise<CertificatesRenewR
|
||||
continue
|
||||
}
|
||||
|
||||
const stillOn = await getCertificateRenewSettings()
|
||||
if (!stillOn.enabled) {
|
||||
item.action = "skipped"
|
||||
item.message = "Автообновление выключено"
|
||||
snapshot.skippedTargets += 1
|
||||
snapshot.targets?.push(item)
|
||||
continue
|
||||
}
|
||||
|
||||
const jobId = randomUUID()
|
||||
await createIssueJobRecord({
|
||||
id: jobId,
|
||||
|
||||
@@ -0,0 +1,100 @@
|
||||
import { eq } from "drizzle-orm"
|
||||
import { db } from "../db/index.js"
|
||||
import { geoipSettings } from "../db/schema.js"
|
||||
import type { GeoipSettingsDto, GeoipSettingsPatch } from "@mmapp/contracts/geoip"
|
||||
|
||||
const SETTINGS_ID = 1
|
||||
const DEFAULT_INTERVAL_SEC = 604800
|
||||
|
||||
let dbEnabled = true
|
||||
|
||||
/** Тесты без PostgreSQL: геттеры отдают дефолты, touch/update — no-op. */
|
||||
export function disableGeoipDbForTests(): void {
|
||||
dbEnabled = false
|
||||
}
|
||||
|
||||
export function resetGeoipSettingsForTests(): void {
|
||||
dbEnabled = true
|
||||
}
|
||||
|
||||
type GeoipSettingsRow = typeof geoipSettings.$inferSelect
|
||||
|
||||
async function getGeoipSettingsRow(): Promise<GeoipSettingsRow | undefined> {
|
||||
if (!dbEnabled) return undefined
|
||||
return (
|
||||
(await db.select().from(geoipSettings).where(eq(geoipSettings.id, SETTINGS_ID)).limit(1))[0]
|
||||
)
|
||||
}
|
||||
|
||||
function toDto(row: GeoipSettingsRow | undefined): GeoipSettingsDto {
|
||||
return {
|
||||
enabled: row?.enabled ?? true,
|
||||
updateIntervalSec: row?.updateIntervalSec ?? DEFAULT_INTERVAL_SEC,
|
||||
lastCheckAt: row?.lastCheckAt ?? null,
|
||||
lastSuccessAt: row?.lastSuccessAt ?? null,
|
||||
lastError: row?.lastError ?? null,
|
||||
countryBuildAt: row?.countryBuildAt ?? null,
|
||||
asnBuildAt: row?.asnBuildAt ?? null,
|
||||
updatedAt: row?.updatedAt ?? new Date().toISOString(),
|
||||
}
|
||||
}
|
||||
|
||||
export async function getGeoipSettings(): Promise<GeoipSettingsDto> {
|
||||
return toDto(await getGeoipSettingsRow())
|
||||
}
|
||||
|
||||
/** ETag'и зеркала для conditional GET (ключ — имя файла базы). */
|
||||
export async function getGeoipEtags(): Promise<Record<string, string>> {
|
||||
const row = await getGeoipSettingsRow()
|
||||
if (!row) return {}
|
||||
const raw = row?.etagsJson
|
||||
if (!raw || typeof raw !== "object") return {}
|
||||
return Object.fromEntries(
|
||||
Object.entries(raw as Record<string, unknown>).filter(
|
||||
(entry): entry is [string, string] => typeof entry[1] === "string",
|
||||
),
|
||||
)
|
||||
}
|
||||
|
||||
export async function updateGeoipSettings(patch: GeoipSettingsPatch): Promise<GeoipSettingsDto> {
|
||||
const prev = await getGeoipSettingsRow()
|
||||
const next = {
|
||||
enabled: patch.enabled ?? prev?.enabled ?? true,
|
||||
updateIntervalSec: patch.updateIntervalSec ?? prev?.updateIntervalSec ?? DEFAULT_INTERVAL_SEC,
|
||||
updatedAt: new Date().toISOString(),
|
||||
}
|
||||
if (prev) {
|
||||
await db.update(geoipSettings).set(next).where(eq(geoipSettings.id, SETTINGS_ID))
|
||||
} else {
|
||||
await db.insert(geoipSettings).values({ id: SETTINGS_ID, ...next })
|
||||
}
|
||||
return getGeoipSettings()
|
||||
}
|
||||
|
||||
export async function touchGeoipRunMeta(patch: {
|
||||
lastCheckAt?: string
|
||||
lastSuccessAt?: string | null
|
||||
lastError?: string | null
|
||||
countryBuildAt?: string | null
|
||||
asnBuildAt?: string | null
|
||||
etags?: Record<string, string>
|
||||
}): Promise<void> {
|
||||
const prev = await getGeoipSettingsRow()
|
||||
const set: Partial<typeof geoipSettings.$inferInsert> = {
|
||||
updatedAt: new Date().toISOString(),
|
||||
}
|
||||
if (patch.lastCheckAt !== undefined) set.lastCheckAt = patch.lastCheckAt
|
||||
if (patch.lastSuccessAt !== undefined) set.lastSuccessAt = patch.lastSuccessAt
|
||||
if (patch.lastError !== undefined) set.lastError = patch.lastError
|
||||
if (patch.countryBuildAt !== undefined) set.countryBuildAt = patch.countryBuildAt
|
||||
if (patch.asnBuildAt !== undefined) set.asnBuildAt = patch.asnBuildAt
|
||||
if (patch.etags !== undefined) {
|
||||
const prevEtags = (prev?.etagsJson as Record<string, string> | null) ?? {}
|
||||
set.etagsJson = { ...prevEtags, ...patch.etags }
|
||||
}
|
||||
if (prev) {
|
||||
await db.update(geoipSettings).set(set).where(eq(geoipSettings.id, SETTINGS_ID))
|
||||
} else {
|
||||
await db.insert(geoipSettings).values({ id: SETTINGS_ID, ...set })
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,198 @@
|
||||
import { rename, rm, mkdir, writeFile } from "node:fs/promises"
|
||||
import path from "node:path"
|
||||
import { open, type AsnResponse, type CountryResponse } from "maxmind"
|
||||
import type { GeoipUpdateRunSnapshot } from "../types/scheduler-run-snapshot.js"
|
||||
import { SCHEDULER_RUN_SNAPSHOT_VERSION } from "../types/scheduler-run-snapshot.js"
|
||||
import { getGeoipEtags, getGeoipSettings, touchGeoipRunMeta } from "./geoip-settings.js"
|
||||
import {
|
||||
GEOIP_ASN_FILE,
|
||||
GEOIP_COUNTRY_FILE,
|
||||
geoipDir,
|
||||
reloadGeoipReaders,
|
||||
} from "./traffic-flow-geoip.js"
|
||||
|
||||
/** Зеркало GeoLite2 без регистрации и ключей (см. README: GeoIP). */
|
||||
const MIRROR_BASE = "https://github.com/P3TERX/GeoLite.mmdb/raw/download"
|
||||
const DOWNLOAD_TIMEOUT_MS = 120_000
|
||||
/** Пробный IP для валидации скачанной базы: Google DNS. */
|
||||
const PROBE_IP = "8.8.8.8"
|
||||
|
||||
type GeoipDbKind = "country" | "asn"
|
||||
|
||||
let updating = false
|
||||
let fetchImpl: typeof fetch = globalThis.fetch.bind(globalThis)
|
||||
|
||||
export function getGeoipUpdateState(): { running: boolean } {
|
||||
return { running: updating }
|
||||
}
|
||||
|
||||
async function validateCountryFile(filePath: string): Promise<string> {
|
||||
const reader = await open<CountryResponse>(filePath)
|
||||
const rec = reader.get(PROBE_IP)
|
||||
const iso = rec?.country?.iso_code ?? rec?.registered_country?.iso_code ?? ""
|
||||
if (iso !== "US") {
|
||||
throw new Error(`база Country не распознала ${PROBE_IP} как US (${iso || "нет записи"})`)
|
||||
}
|
||||
return reader.metadata.buildEpoch.toISOString()
|
||||
}
|
||||
|
||||
async function validateAsnFile(filePath: string): Promise<string> {
|
||||
const reader = await open<AsnResponse>(filePath)
|
||||
const rec = reader.get(PROBE_IP)
|
||||
const asn = rec?.autonomous_system_number ?? 0
|
||||
if (asn !== 15169) {
|
||||
throw new Error(`база ASN не распознала ${PROBE_IP} как AS15169 (${asn ? `AS${asn}` : "нет записи"})`)
|
||||
}
|
||||
return reader.metadata.buildEpoch.toISOString()
|
||||
}
|
||||
|
||||
let validateCountry = validateCountryFile
|
||||
let validateAsn = validateAsnFile
|
||||
|
||||
export function setGeoipFetchForTests(fn: typeof fetch): void {
|
||||
fetchImpl = fn
|
||||
}
|
||||
|
||||
export function setGeoipValidateForTests(opts: {
|
||||
country?: (filePath: string) => Promise<string>
|
||||
asn?: (filePath: string) => Promise<string>
|
||||
}): void {
|
||||
validateCountry = opts.country ?? validateCountryFile
|
||||
validateAsn = opts.asn ?? validateAsnFile
|
||||
}
|
||||
|
||||
export function resetGeoipUpdateForTests(): void {
|
||||
updating = false
|
||||
fetchImpl = globalThis.fetch.bind(globalThis)
|
||||
validateCountry = validateCountryFile
|
||||
validateAsn = validateAsnFile
|
||||
}
|
||||
|
||||
/**
|
||||
* Разовая проверка/доставка баз с зеркала P3TERX. Conditional GET по ETag
|
||||
* (304 = не меняем файл), валидация пробоем 8.8.8.8, атомарная подмена через rename.
|
||||
*/
|
||||
export async function collectGeoipUpdateOnce(
|
||||
opts: { force?: boolean } = {},
|
||||
): Promise<GeoipUpdateRunSnapshot> {
|
||||
const sampledAt = new Date().toISOString()
|
||||
if (updating) {
|
||||
if (opts.force) {
|
||||
throw Object.assign(new Error("Обновление GeoIP уже выполняется"), { statusCode: 409 })
|
||||
}
|
||||
return emptySnapshot(sampledAt, true)
|
||||
}
|
||||
|
||||
const settings = await getGeoipSettings()
|
||||
if (!settings.enabled && !opts.force) {
|
||||
return emptySnapshot(sampledAt, true)
|
||||
}
|
||||
|
||||
updating = true
|
||||
const snapshot: GeoipUpdateRunSnapshot = {
|
||||
v: SCHEDULER_RUN_SNAPSHOT_VERSION,
|
||||
job: "geoip_update",
|
||||
sampledAt,
|
||||
checked: 0,
|
||||
downloaded: 0,
|
||||
skippedUnchanged: 0,
|
||||
bytes: 0,
|
||||
errors: [],
|
||||
}
|
||||
|
||||
try {
|
||||
const dir = geoipDir()
|
||||
await mkdir(dir, { recursive: true })
|
||||
const storedEtags = await getGeoipEtags()
|
||||
const etags: Record<string, string> = {}
|
||||
const buildAt: Partial<Record<GeoipDbKind, string>> = {}
|
||||
|
||||
for (const kind of ["country", "asn"] as const) {
|
||||
snapshot.checked += 1
|
||||
const file = kind === "country" ? GEOIP_COUNTRY_FILE : GEOIP_ASN_FILE
|
||||
const target = path.join(dir, file)
|
||||
const tmp = `${target}.tmp`
|
||||
const prevEtag = storedEtags[file]
|
||||
try {
|
||||
const ac = new AbortController()
|
||||
const timer = setTimeout(() => ac.abort(), DOWNLOAD_TIMEOUT_MS)
|
||||
let res: Response
|
||||
try {
|
||||
res = await fetchImpl(`${MIRROR_BASE}/${file}`, {
|
||||
headers: prevEtag ? { "If-None-Match": prevEtag } : {},
|
||||
signal: ac.signal,
|
||||
})
|
||||
} finally {
|
||||
clearTimeout(timer)
|
||||
}
|
||||
if (res.status === 304) {
|
||||
snapshot.skippedUnchanged += 1
|
||||
if (prevEtag) etags[file] = prevEtag
|
||||
continue
|
||||
}
|
||||
if (!res.ok) throw new Error(`HTTP ${res.status}`)
|
||||
const etag = res.headers.get("etag") ?? ""
|
||||
const body = Buffer.from(await res.arrayBuffer())
|
||||
snapshot.bytes += body.byteLength
|
||||
await writeFile(tmp, body)
|
||||
buildAt[kind] =
|
||||
kind === "country" ? await validateCountry(tmp) : await validateAsn(tmp)
|
||||
|
||||
const prevFile = `${target}.prev`
|
||||
await rm(prevFile, { force: true })
|
||||
await rename(target, prevFile).catch(() => {
|
||||
/* текущего файла могло ещё не быть */
|
||||
})
|
||||
await rename(tmp, target)
|
||||
snapshot.downloaded += 1
|
||||
if (etag) etags[file] = etag
|
||||
} catch (e) {
|
||||
await rm(tmp, { force: true }).catch(() => {
|
||||
/* best-effort */
|
||||
})
|
||||
const message = e instanceof Error ? e.message : String(e)
|
||||
snapshot.errors.push(`${file}: ${message}`)
|
||||
}
|
||||
}
|
||||
|
||||
if (snapshot.downloaded > 0) {
|
||||
await reloadGeoipReaders()
|
||||
}
|
||||
|
||||
await touchGeoipRunMeta({
|
||||
lastCheckAt: sampledAt,
|
||||
lastSuccessAt: snapshot.errors.length ? null : sampledAt,
|
||||
lastError: snapshot.errors.length ? snapshot.errors.join("; ") : null,
|
||||
countryBuildAt: buildAt.country,
|
||||
asnBuildAt: buildAt.asn,
|
||||
etags,
|
||||
})
|
||||
return snapshot
|
||||
} catch (e) {
|
||||
const message = e instanceof Error ? e.message : String(e)
|
||||
snapshot.fatalError = message
|
||||
await touchGeoipRunMeta({
|
||||
lastCheckAt: sampledAt,
|
||||
lastError: message,
|
||||
}).catch(() => {
|
||||
/* best-effort */
|
||||
})
|
||||
return snapshot
|
||||
} finally {
|
||||
updating = false
|
||||
}
|
||||
}
|
||||
|
||||
function emptySnapshot(sampledAt: string, skipped: boolean): GeoipUpdateRunSnapshot {
|
||||
return {
|
||||
v: SCHEDULER_RUN_SNAPSHOT_VERSION,
|
||||
job: "geoip_update",
|
||||
sampledAt,
|
||||
skipped,
|
||||
checked: 0,
|
||||
downloaded: 0,
|
||||
skippedUnchanged: 0,
|
||||
bytes: 0,
|
||||
errors: [],
|
||||
}
|
||||
}
|
||||
@@ -1,4 +1,4 @@
|
||||
import { and, asc, desc, eq, lt } from "drizzle-orm"
|
||||
import { and, asc, desc, eq } from "drizzle-orm"
|
||||
import { db } from "../db/index.js"
|
||||
import {
|
||||
filterRules,
|
||||
@@ -45,11 +45,6 @@ async function getSettingsRow() {
|
||||
return (await db.select().from(internetPathSettings).where(eq(internetPathSettings.id, 1)).limit(1))[0]
|
||||
}
|
||||
|
||||
async function cleanupSnapshots(retentionDays: number) {
|
||||
const cutoff = new Date(Date.now() - retentionDays * 24 * 60 * 60 * 1000).toISOString()
|
||||
await db.delete(internetPathSnapshots).where(lt(internetPathSnapshots.sampledAt, cutoff))
|
||||
}
|
||||
|
||||
async function buildRulesets() {
|
||||
const enabled = await db.select().from(servers).where(eq(servers.enabled, true))
|
||||
const rules = await db.select().from(filterRules).orderBy(asc(filterRules.serverId), asc(filterRules.sortOrder))
|
||||
@@ -273,7 +268,6 @@ export async function collectInternetPathSnapshotOnce(): Promise<InternetPathRun
|
||||
sampledAt,
|
||||
payloadJson: payload,
|
||||
})
|
||||
await cleanupSnapshots(Math.max(1, settings.retentionDays))
|
||||
await db.update(internetPathSettings).set({
|
||||
lastCollectedAt: sampledAt,
|
||||
lastDurationMs: Date.now() - started,
|
||||
|
||||
@@ -586,6 +586,14 @@ export class MikrotikClient {
|
||||
: new Error(`Не удалось загрузить файл ${normalized} на RouterOS`)
|
||||
}
|
||||
|
||||
async importUploadedFile(fileName: string): Promise<unknown> {
|
||||
try {
|
||||
return await this.post("/import", { "file-name": fileName }, 120_000)
|
||||
} catch {
|
||||
return await this.post("/execute", { script: `/import file-name="${fileName}"` }, 120_000)
|
||||
}
|
||||
}
|
||||
|
||||
async importCertificate(params: {
|
||||
fileName: string
|
||||
name: string
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
import { eq, lt } from "drizzle-orm"
|
||||
import { db, pool } from "../db/index.js"
|
||||
import { eq } from "drizzle-orm"
|
||||
import { db, dbQuery, pool } from "../db/index.js"
|
||||
import { dropExpiredPartitions, ensurePartitionsAround } from "../db/partitions.js"
|
||||
import { servers, serverSnapshots } from "../db/schema.js"
|
||||
import type { SnapshotInsert } from "../db/schema.js"
|
||||
@@ -82,11 +82,18 @@ export async function pollServer(serverId: number): Promise<SnapshotRead> {
|
||||
.values(partialSnap as SnapshotInsert)
|
||||
.returning()
|
||||
|
||||
const cutoffIso = new Date(Date.now() - 14 * 24 * 3600_000).toISOString()
|
||||
await db.delete(serverSnapshots).where(lt(serverSnapshots.polledAt, cutoffIso))
|
||||
if (inserted) {
|
||||
await dbQuery(
|
||||
`UPDATE server_snapshots
|
||||
SET raw_interfaces = NULL, raw_ip_addresses = NULL
|
||||
WHERE server_id = $1 AND polled_at < $2
|
||||
AND (raw_interfaces IS NOT NULL OR raw_ip_addresses IS NOT NULL)`,
|
||||
[serverId, inserted.polledAt],
|
||||
)
|
||||
}
|
||||
void dropExpiredPartitions(pool).then(() => ensurePartitionsAround(pool))
|
||||
|
||||
return toSnapshotRead(inserted)
|
||||
return toSnapshotRead(inserted!)
|
||||
}
|
||||
|
||||
// ── helper ─────────────────────────────────────────────────────────────────────
|
||||
|
||||
@@ -0,0 +1,89 @@
|
||||
import assert from "node:assert/strict"
|
||||
import {
|
||||
DeleteObjectCommand,
|
||||
GetObjectCommand,
|
||||
HeadBucketCommand,
|
||||
ListObjectsV2Command,
|
||||
PutObjectCommand,
|
||||
type S3Client,
|
||||
} from "@aws-sdk/client-s3"
|
||||
import {
|
||||
buildS3ObjectKey,
|
||||
normalizeS3Prefix,
|
||||
parseS3ObjectKey,
|
||||
sanitizeServerName,
|
||||
s3DeleteObject,
|
||||
s3GetObject,
|
||||
s3ListObjects,
|
||||
s3PutObject,
|
||||
s3TestConnection,
|
||||
} from "./s3-backup-client.js"
|
||||
|
||||
assert.equal(normalizeS3Prefix("/mikrotik/backups/"), "mikrotik/backups")
|
||||
assert.equal(sanitizeServerName("MSK CHR 01"), "MSK_CHR_01")
|
||||
assert.equal(
|
||||
buildS3ObjectKey("mikrotik", "msk-chr01", "chr_2026-09-08_03-00-00.rsc"),
|
||||
"mikrotik/msk-chr01/chr_2026-09-08_03-00-00.rsc",
|
||||
)
|
||||
assert.deepEqual(
|
||||
parseS3ObjectKey("mikrotik/msk-chr01/chr_2026-09-08_03-00-00.rsc"),
|
||||
{ filename: "chr_2026-09-08_03-00-00.rsc", serverName: "msk-chr01" },
|
||||
)
|
||||
|
||||
const store = new Map<string, Buffer>()
|
||||
let lastCommand = ""
|
||||
|
||||
const fake = {
|
||||
send: async (command: { input?: Record<string, unknown> }) => {
|
||||
const name = command.constructor.name
|
||||
lastCommand = name
|
||||
const input = command.input ?? {}
|
||||
if (command instanceof HeadBucketCommand || name === "HeadBucketCommand") {
|
||||
if (input.Bucket !== "backups") throw new Error("no bucket")
|
||||
return {}
|
||||
}
|
||||
if (command instanceof PutObjectCommand || name === "PutObjectCommand") {
|
||||
const key = String(input.Key)
|
||||
const body = input.Body
|
||||
store.set(key, Buffer.isBuffer(body) ? body : Buffer.from(String(body)))
|
||||
return { ETag: '"etag-1"' }
|
||||
}
|
||||
if (command instanceof GetObjectCommand || name === "GetObjectCommand") {
|
||||
const key = String(input.Key)
|
||||
const body = store.get(key)
|
||||
if (!body) throw new Error("not found")
|
||||
return { Body: { transformToByteArray: async () => new Uint8Array(body) } }
|
||||
}
|
||||
if (command instanceof DeleteObjectCommand || name === "DeleteObjectCommand") {
|
||||
store.delete(String(input.Key))
|
||||
return {}
|
||||
}
|
||||
if (command instanceof ListObjectsV2Command || name === "ListObjectsV2Command") {
|
||||
const prefix = String(input.Prefix ?? "")
|
||||
const contents = [...store.entries()]
|
||||
.filter(([key]) => !prefix || key.startsWith(prefix))
|
||||
.map(([key, buf]) => ({ Key: key, Size: buf.length, LastModified: new Date("2026-09-08T00:00:00Z") }))
|
||||
return { Contents: contents, IsTruncated: false }
|
||||
}
|
||||
throw new Error(`unexpected command ${name}`)
|
||||
},
|
||||
} as unknown as S3Client
|
||||
|
||||
await s3TestConnection(fake, "backups")
|
||||
assert.equal(lastCommand === "HeadBucketCommand" || lastCommand.includes("Head"), true)
|
||||
|
||||
const put = await s3PutObject(fake, "backups", "mikrotik/a/file.rsc", "hello")
|
||||
assert.equal(put.etag, '"etag-1"')
|
||||
|
||||
const got = await s3GetObject(fake, "backups", "mikrotik/a/file.rsc")
|
||||
assert.equal(got.toString("utf8"), "hello")
|
||||
|
||||
const listed = await s3ListObjects(fake, "backups", "mikrotik")
|
||||
assert.equal(listed.length, 1)
|
||||
assert.equal(listed[0]?.key, "mikrotik/a/file.rsc")
|
||||
|
||||
await s3DeleteObject(fake, "backups", "mikrotik/a/file.rsc")
|
||||
const after = await s3ListObjects(fake, "backups", "mikrotik")
|
||||
assert.equal(after.length, 0)
|
||||
|
||||
console.log("s3-backup-client.test.ts: ok")
|
||||
@@ -0,0 +1,128 @@
|
||||
import {
|
||||
DeleteObjectCommand,
|
||||
GetObjectCommand,
|
||||
HeadBucketCommand,
|
||||
ListObjectsV2Command,
|
||||
PutObjectCommand,
|
||||
S3Client,
|
||||
type S3ClientConfig,
|
||||
} from "@aws-sdk/client-s3"
|
||||
|
||||
export type S3BackupConfig = {
|
||||
endpoint: string
|
||||
region: string
|
||||
bucket: string
|
||||
prefix: string
|
||||
accessKeyId: string
|
||||
secretAccessKey: string
|
||||
forcePathStyle: boolean
|
||||
}
|
||||
|
||||
export type S3ListedObject = {
|
||||
key: string
|
||||
size: number
|
||||
lastModified?: string
|
||||
}
|
||||
|
||||
export function normalizeS3Prefix(prefix: string): string {
|
||||
return prefix.trim().replace(/^\/+|\/+$/g, "")
|
||||
}
|
||||
|
||||
export function sanitizeServerName(name: string): string {
|
||||
const safe = name.replace(/[^a-zA-Z0-9._-]+/g, "_").replace(/^_+|_+$/g, "")
|
||||
return safe || "server"
|
||||
}
|
||||
|
||||
export function buildS3ObjectKey(prefix: string, serverSafe: string, filename: string): string {
|
||||
const parts = [normalizeS3Prefix(prefix), sanitizeServerName(serverSafe), filename]
|
||||
.filter((part) => part.length > 0)
|
||||
return parts.join("/")
|
||||
}
|
||||
|
||||
export function parseS3ObjectKey(key: string): { filename: string; serverName: string } {
|
||||
const parts = key.split("/").filter(Boolean)
|
||||
const filename = parts.pop() ?? key
|
||||
const folder = parts.pop() ?? ""
|
||||
const base = filename.replace(/\.(rsc|backup)$/i, "")
|
||||
const fromFilename = base.replace(/_\d{4}-\d{2}-\d{2}_\d{2}-\d{2}-\d{2}$/, "")
|
||||
return { filename, serverName: folder || fromFilename || filename }
|
||||
}
|
||||
|
||||
export function createS3ClientFromConfig(cfg: S3BackupConfig): S3Client {
|
||||
const options: S3ClientConfig = {
|
||||
region: cfg.region.trim() || "us-east-1",
|
||||
credentials: {
|
||||
accessKeyId: cfg.accessKeyId,
|
||||
secretAccessKey: cfg.secretAccessKey,
|
||||
},
|
||||
forcePathStyle: cfg.forcePathStyle,
|
||||
}
|
||||
const endpoint = cfg.endpoint.trim()
|
||||
if (endpoint) options.endpoint = endpoint
|
||||
return new S3Client(options)
|
||||
}
|
||||
|
||||
export async function s3TestConnection(client: S3Client, bucket: string): Promise<void> {
|
||||
try {
|
||||
await client.send(new HeadBucketCommand({ Bucket: bucket }))
|
||||
} catch {
|
||||
await client.send(new ListObjectsV2Command({ Bucket: bucket, MaxKeys: 1 }))
|
||||
}
|
||||
}
|
||||
|
||||
export async function s3PutObject(
|
||||
client: S3Client,
|
||||
bucket: string,
|
||||
key: string,
|
||||
body: Buffer | string,
|
||||
): Promise<{ etag?: string }> {
|
||||
const out = await client.send(new PutObjectCommand({
|
||||
Bucket: bucket,
|
||||
Key: key,
|
||||
Body: body,
|
||||
ContentType: "text/plain; charset=utf-8",
|
||||
}))
|
||||
return { etag: out.ETag }
|
||||
}
|
||||
|
||||
export async function s3GetObject(
|
||||
client: S3Client,
|
||||
bucket: string,
|
||||
key: string,
|
||||
): Promise<Buffer> {
|
||||
const out = await client.send(new GetObjectCommand({ Bucket: bucket, Key: key }))
|
||||
const bytes = await out.Body?.transformToByteArray()
|
||||
if (!bytes) throw new Error("Пустой объект S3")
|
||||
return Buffer.from(bytes)
|
||||
}
|
||||
|
||||
export async function s3DeleteObject(client: S3Client, bucket: string, key: string): Promise<void> {
|
||||
await client.send(new DeleteObjectCommand({ Bucket: bucket, Key: key }))
|
||||
}
|
||||
|
||||
export async function s3ListObjects(
|
||||
client: S3Client,
|
||||
bucket: string,
|
||||
prefix: string,
|
||||
): Promise<S3ListedObject[]> {
|
||||
const items: S3ListedObject[] = []
|
||||
let token: string | undefined
|
||||
const normalized = normalizeS3Prefix(prefix)
|
||||
do {
|
||||
const out = await client.send(new ListObjectsV2Command({
|
||||
Bucket: bucket,
|
||||
Prefix: normalized ? `${normalized}/` : undefined,
|
||||
ContinuationToken: token,
|
||||
}))
|
||||
for (const obj of out.Contents ?? []) {
|
||||
if (!obj.Key) continue
|
||||
items.push({
|
||||
key: obj.Key,
|
||||
size: obj.Size ?? 0,
|
||||
lastModified: obj.LastModified?.toISOString(),
|
||||
})
|
||||
}
|
||||
token = out.IsTruncated ? out.NextContinuationToken : undefined
|
||||
} while (token)
|
||||
return items
|
||||
}
|
||||
@@ -46,6 +46,8 @@ import { collectCertificatesRenewOnce } from "./certificate-renew-collector.js"
|
||||
import { getCertificateRenewSettings } from "./certificates-service.js"
|
||||
import { collectScheduledBackupsOnce } from "./backup-scheduler-collector.js"
|
||||
import { getBackupScheduleSettings } from "./backup-service.js"
|
||||
import { collectGeoipUpdateOnce } from "./geoip-update-collector.js"
|
||||
import { getGeoipSettings } from "./geoip-settings.js"
|
||||
import {
|
||||
endSchedulerJob,
|
||||
isSchedulerJobRunning,
|
||||
@@ -63,6 +65,7 @@ export const JOB_KEYS = [
|
||||
"gre_bgp",
|
||||
"certificates_renew",
|
||||
"backups",
|
||||
"geoip_update",
|
||||
"alert_engine",
|
||||
] as const
|
||||
export type SchedulerJobKey = (typeof JOB_KEYS)[number]
|
||||
@@ -148,6 +151,9 @@ async function runSchedulerJobBody(jobKey: SchedulerJobKey): Promise<void> {
|
||||
case "backups":
|
||||
snapshot = await collectScheduledBackupsOnce()
|
||||
break
|
||||
case "geoip_update":
|
||||
snapshot = await collectGeoipUpdateOnce()
|
||||
break
|
||||
case "alert_engine": {
|
||||
const r = await runAlertEngineOnce()
|
||||
snapshot = {
|
||||
@@ -377,6 +383,18 @@ export async function refreshScheduler(): Promise<void> {
|
||||
)
|
||||
}
|
||||
|
||||
const geoip = await getGeoipSettings()
|
||||
if (geoip.enabled) {
|
||||
const geoipMs = Math.max(6 * 3600_000, geoip.updateIntervalSec * 1000)
|
||||
void executeSchedulerJob("geoip_update").catch(() => {})
|
||||
timers.set(
|
||||
"geoip_update",
|
||||
setInterval(() => {
|
||||
void executeSchedulerJob("geoip_update").catch(() => {})
|
||||
}, geoipMs),
|
||||
)
|
||||
}
|
||||
|
||||
const alertMs = 20_000
|
||||
void executeSchedulerJob("alert_engine").catch(() => {})
|
||||
timers.set(
|
||||
@@ -409,6 +427,7 @@ export async function getSchedulerStatus() {
|
||||
const internetPath = await getInternetPathSettings()
|
||||
const certRenew = await getCertificateRenewSettings()
|
||||
const backupSchedule = await getBackupScheduleSettings()
|
||||
const geoip = await getGeoipSettings()
|
||||
|
||||
const resOn = uptime.resourcesEnabled ?? uptime.enabled
|
||||
const pingOn = uptime.pingEnabled ?? uptime.enabled
|
||||
@@ -424,6 +443,7 @@ export async function getSchedulerStatus() {
|
||||
gre_bgp: { enabled: true, intervalSec: 30 },
|
||||
certificates_renew: { enabled: certRenew.enabled, intervalSec: certRenew.intervalSec },
|
||||
backups: { enabled: backupSchedule.enabled, intervalSec: 60 },
|
||||
geoip_update: { enabled: geoip.enabled, intervalSec: geoip.updateIntervalSec },
|
||||
alert_engine: { enabled: true, intervalSec: 20 },
|
||||
}
|
||||
|
||||
|
||||
@@ -162,8 +162,6 @@ export async function collectServersRestPingOnce(): Promise<ServersRestPingRunSn
|
||||
}
|
||||
})
|
||||
|
||||
await dbQuery(`DELETE FROM servers_rest_ping_samples WHERE sampled_at < now() - interval '30 days'`)
|
||||
|
||||
await db.update(serversApiPingSettings)
|
||||
.set({
|
||||
lastCollectedAt: sampledAt,
|
||||
|
||||
@@ -0,0 +1,135 @@
|
||||
import assert from "node:assert/strict"
|
||||
import { getStatistics, getStatisticsPivot, parseStatisticsPeriod, pivotDimsConflict } from "./statistics-aggregate.js"
|
||||
import { rememberServerIfaces, resetIfaceCacheForTests } from "./traffic-flow-ifindex.js"
|
||||
import { setRefreshIfacesForTests } from "./traffic-flow-ifaces.js"
|
||||
import { withPgOrSkip } from "../test/pg.js"
|
||||
import { dbQuery } from "../db/index.js"
|
||||
import { ensurePartitionFor } from "../db/partitions.js"
|
||||
import { pool } from "../db/index.js"
|
||||
import { STATISTICS_UNBOUND_USER_ID } from "@mmapp/contracts/statistics"
|
||||
|
||||
{
|
||||
const sameDay = parseStatisticsPeriod("2026-09-10", "2026-09-10")
|
||||
assert.ok(sameDay)
|
||||
assert.equal(sameDay.fromDay, "2026-09-10")
|
||||
assert.equal(sameDay.toDayExclusive, "2026-09-11")
|
||||
assert.equal(sameDay.grain, "hour")
|
||||
const month = parseStatisticsPeriod("2026-08-01", "2026-08-31")
|
||||
assert.ok(month)
|
||||
assert.equal(month.grain, "day")
|
||||
assert.equal(month.toDayExclusive, "2026-09-01")
|
||||
assert.equal(parseStatisticsPeriod("2026-09-10", "2026-09-09"), null)
|
||||
assert.equal(pivotDimsConflict("country", "country"), true)
|
||||
assert.equal(pivotDimsConflict("country", "service"), false)
|
||||
}
|
||||
|
||||
if (!(await withPgOrSkip())) {
|
||||
console.log("statistics-aggregate.test.ts: skip")
|
||||
process.exit(0)
|
||||
}
|
||||
|
||||
const inserted = await dbQuery<{ id: number }>(`
|
||||
INSERT INTO servers (name, host) VALUES ('stats-cube', '127.0.0.1') RETURNING id
|
||||
`)
|
||||
const serverId = inserted.rows[0]?.id
|
||||
if (serverId == null) throw new Error("no server")
|
||||
|
||||
await ensurePartitionFor(pool, "flow_daily_facts", "month", new Date("2026-09-01T00:00:00Z"))
|
||||
await ensurePartitionFor(pool, "flow_hour_facts", "day", new Date("2026-09-10T00:00:00Z"))
|
||||
await dbQuery(`DELETE FROM flow_daily_facts WHERE server_id = $1`, [serverId])
|
||||
await dbQuery(`DELETE FROM flow_hour_facts WHERE server_id = $1`, [serverId])
|
||||
await dbQuery(`DELETE FROM user_interface_bindings WHERE server_id = $1`, [serverId])
|
||||
await dbQuery(`DELETE FROM app_users WHERE id = 'u-stats-1'`)
|
||||
|
||||
await dbQuery(`
|
||||
INSERT INTO app_users (id, name, login, role, active)
|
||||
VALUES ('u-stats-1', 'Клиент', 'stats-user', 'viewer', TRUE)
|
||||
ON CONFLICT (id) DO NOTHING
|
||||
`)
|
||||
await dbQuery(`
|
||||
INSERT INTO user_interface_bindings (id, user_id, server_id, interface_name, interface_type)
|
||||
VALUES ('bind-stats-1', 'u-stats-1', $1, 'gre-client', 'gre')
|
||||
`, [serverId])
|
||||
|
||||
resetIfaceCacheForTests()
|
||||
rememberServerIfaces(serverId, [{ name: "gre-client", ifindex: "2" }])
|
||||
setRefreshIfacesForTests(async () => {})
|
||||
|
||||
await dbQuery(`
|
||||
INSERT INTO flow_daily_facts (server_id, day, iface, country, service, asn, bytes, packets)
|
||||
VALUES
|
||||
($1, '2026-09-10', '2', 'US', 'https', 15169, 800, 10),
|
||||
($1, '2026-09-10', '2', 'DE', 'dns', 15133, 200, 4),
|
||||
($1, '2026-09-10', 'wan1', 'NL', 'other', 0, 70, 1)
|
||||
`, [serverId])
|
||||
|
||||
try {
|
||||
const all = await getStatistics({ from: "2026-09-01", to: "2026-09-30" })
|
||||
assert.equal(all.grain, "day")
|
||||
assert.equal(all.kpis.bytes, 1070)
|
||||
assert.equal(all.kpis.users, 1)
|
||||
assert.ok(all.countries.some((r) => r.id === "US"))
|
||||
assert.ok(all.users.some((r) => r.id === "u-stats-1"))
|
||||
const unbound = all.users.find((r) => r.id === STATISTICS_UNBOUND_USER_ID)
|
||||
assert.ok(unbound)
|
||||
assert.equal(unbound.bytes, 70)
|
||||
assert.ok(all.servers.some((r) => r.id === String(serverId)))
|
||||
const greIface = all.interfaces.find((r) => r.label.includes("gre-client"))
|
||||
assert.ok(greIface)
|
||||
assert.equal(greIface.bytes, 1000)
|
||||
assert.equal(greIface.id, `${serverId}:gre-client`)
|
||||
assert.ok(!all.interfaces.some((r) => /· (?:#)?\d+$/.test(r.label)))
|
||||
|
||||
const sliced = await getStatistics({
|
||||
from: "2026-09-01",
|
||||
to: "2026-09-30",
|
||||
country: "US",
|
||||
service: "https",
|
||||
asn: 15169,
|
||||
})
|
||||
assert.equal(sliced.kpis.bytes, 800)
|
||||
assert.equal(sliced.countries.length, 1)
|
||||
assert.equal(sliced.countries[0]?.id, "US")
|
||||
assert.ok(sliced.users.some((r) => r.id === "u-stats-1"))
|
||||
|
||||
const byUser = await getStatistics({
|
||||
from: "2026-09-01",
|
||||
to: "2026-09-30",
|
||||
userId: "u-stats-1",
|
||||
})
|
||||
assert.equal(byUser.kpis.bytes, 1000)
|
||||
|
||||
const pivot = await getStatisticsPivot({
|
||||
from: "2026-09-01",
|
||||
to: "2026-09-30",
|
||||
row: "country",
|
||||
col: "service",
|
||||
metric: "bytes",
|
||||
})
|
||||
const us = pivot.rows.find((r) => r.id === "US")
|
||||
const de = pivot.rows.find((r) => r.id === "DE")
|
||||
assert.ok(us)
|
||||
assert.ok(de)
|
||||
assert.equal(us.cells.https, 800)
|
||||
assert.equal(de.cells.dns, 200)
|
||||
|
||||
await dbQuery(`
|
||||
INSERT INTO flow_hour_facts (server_id, bucket_at, iface, country, service, asn, bytes, packets)
|
||||
VALUES ($1, '2026-09-10T10:00:00Z', '2', 'US', 'https', 15169, 40, 2)
|
||||
`, [serverId])
|
||||
const hourly = await getStatistics({
|
||||
from: "2026-09-10T00:00:00.000Z",
|
||||
to: "2026-09-10T23:00:00.000Z",
|
||||
})
|
||||
assert.equal(hourly.grain, "hour")
|
||||
assert.equal(hourly.kpis.bytes, 40)
|
||||
assert.ok(hourly.users.some((r) => r.id === "u-stats-1"))
|
||||
} finally {
|
||||
setRefreshIfacesForTests(null)
|
||||
resetIfaceCacheForTests()
|
||||
await dbQuery(`DELETE FROM flow_daily_facts WHERE server_id = $1`, [serverId])
|
||||
await dbQuery(`DELETE FROM flow_hour_facts WHERE server_id = $1`, [serverId])
|
||||
await dbQuery(`DELETE FROM servers WHERE id = $1`, [serverId])
|
||||
}
|
||||
|
||||
console.log("statistics-aggregate.test.ts: ok")
|
||||
@@ -0,0 +1,746 @@
|
||||
import { eq } from "drizzle-orm"
|
||||
import { db, dbAll } from "../db/index.js"
|
||||
import { appUsers, flowAsnMeta, servers, userInterfaceBindings } from "../db/schema.js"
|
||||
import {
|
||||
STATISTICS_UNBOUND_USER_ID,
|
||||
type StatisticsBreakdownRow,
|
||||
type StatisticsDto,
|
||||
type StatisticsPivotDim,
|
||||
type StatisticsPivotDto,
|
||||
type StatisticsPivotQuery,
|
||||
type StatisticsQuery,
|
||||
} from "@mmapp/contracts/statistics"
|
||||
import {
|
||||
collapseServerIfaceRows,
|
||||
displayFactIface,
|
||||
expandBindingIfaces,
|
||||
factIfaceAliases,
|
||||
} from "./traffic-flow-ifindex.js"
|
||||
import { refreshServerIfaces } from "./traffic-flow-ifaces.js"
|
||||
|
||||
const TOP_N = 200
|
||||
const HOUR_WINDOW_MS = 48 * 3600_000
|
||||
const PIVOT_ROW_CAP = 50
|
||||
const PIVOT_COL_CAP = 15
|
||||
const PIVOT_OTHER_ID = "__other__"
|
||||
|
||||
export interface ParsedPeriod {
|
||||
fromIso: string
|
||||
toIso: string
|
||||
fromDay: string
|
||||
toDayExclusive: string
|
||||
grain: "hour" | "day"
|
||||
windowSec: number
|
||||
}
|
||||
|
||||
function pad2(n: number): string {
|
||||
return String(n).padStart(2, "0")
|
||||
}
|
||||
|
||||
function toUtcDay(d: Date): string {
|
||||
return `${d.getUTCFullYear()}-${pad2(d.getUTCMonth() + 1)}-${pad2(d.getUTCDate())}`
|
||||
}
|
||||
|
||||
function addUtcDays(day: string, n: number): string {
|
||||
const d = new Date(`${day}T00:00:00Z`)
|
||||
d.setUTCDate(d.getUTCDate() + n)
|
||||
return toUtcDay(d)
|
||||
}
|
||||
|
||||
/** Parse from/to. Date-only `to` is inclusive (end of that UTC day). */
|
||||
export function parseStatisticsPeriod(fromRaw: string, toRaw: string): ParsedPeriod | null {
|
||||
const from = Date.parse(fromRaw.includes("T") ? fromRaw : `${fromRaw}T00:00:00Z`)
|
||||
const toHasTime = toRaw.includes("T")
|
||||
const to = Date.parse(toHasTime ? toRaw : `${toRaw}T00:00:00Z`)
|
||||
if (!Number.isFinite(from) || !Number.isFinite(to)) return null
|
||||
const fromDate = new Date(from)
|
||||
let toDate = new Date(to)
|
||||
let toDayExclusive: string
|
||||
if (toHasTime) {
|
||||
toDayExclusive = toUtcDay(toDate)
|
||||
if (toDate.getUTCHours() !== 0 || toDate.getUTCMinutes() !== 0 || toDate.getUTCSeconds() !== 0) {
|
||||
toDayExclusive = addUtcDays(toDayExclusive, 1)
|
||||
}
|
||||
} else {
|
||||
toDayExclusive = addUtcDays(toUtcDay(toDate), 1)
|
||||
toDate = new Date(`${toDayExclusive}T00:00:00Z`)
|
||||
}
|
||||
if (toDate.getTime() <= from) return null
|
||||
const windowSec = Math.max(1, Math.round((toDate.getTime() - from) / 1000))
|
||||
const grain: "hour" | "day" = toDate.getTime() - from <= HOUR_WINDOW_MS ? "hour" : "day"
|
||||
return {
|
||||
fromIso: fromDate.toISOString(),
|
||||
toIso: toDate.toISOString(),
|
||||
fromDay: toUtcDay(fromDate),
|
||||
toDayExclusive,
|
||||
grain,
|
||||
windowSec,
|
||||
}
|
||||
}
|
||||
|
||||
interface FilterCtx {
|
||||
fromIso: string
|
||||
toIso: string
|
||||
fromDay: string
|
||||
toDayExclusive: string
|
||||
serverId?: number
|
||||
iface?: string
|
||||
country?: string
|
||||
service?: string
|
||||
asn?: number
|
||||
userIfaces: Array<{ serverId: number; iface: string }> | null
|
||||
unboundOnly: boolean
|
||||
boundIfaces: Array<{ serverId: number; iface: string }>
|
||||
}
|
||||
|
||||
function ifaceFilterAliases(iface: string, serverId?: number): string[] {
|
||||
return factIfaceAliases(iface.trim(), serverId)
|
||||
}
|
||||
|
||||
function looksLikeIfIndex(iface: string): boolean {
|
||||
const raw = iface.trim()
|
||||
return /^\d+$/.test(raw) || /^#\d+$/.test(raw)
|
||||
}
|
||||
|
||||
async function warmIfaceCache(ids: Iterable<number>): Promise<void> {
|
||||
const uniq = [...new Set(ids)].filter((id) => Number.isFinite(id) && id > 0)
|
||||
if (!uniq.length) return
|
||||
await Promise.all(uniq.map((id) => refreshServerIfaces(id)))
|
||||
}
|
||||
|
||||
async function warmBindingIfaceCache(): Promise<void> {
|
||||
const rows = await db.select({ serverId: userInterfaceBindings.serverId }).from(userInterfaceBindings)
|
||||
await warmIfaceCache(rows.map((r) => r.serverId))
|
||||
}
|
||||
|
||||
function canonicalIfaceDimId(id: string): string {
|
||||
const colon = id.indexOf(":")
|
||||
if (colon < 0) return id
|
||||
const sid = Number(id.slice(0, colon))
|
||||
if (!Number.isFinite(sid)) return id
|
||||
return `${sid}:${displayFactIface(sid, id.slice(colon + 1))}`
|
||||
}
|
||||
|
||||
function factWhere(alias: string, grain: "hour" | "day", ctx: FilterCtx): { sql: string; params: unknown[] } {
|
||||
const params: unknown[] = []
|
||||
const parts: string[] = []
|
||||
if (grain === "hour") {
|
||||
params.push(ctx.fromIso, ctx.toIso)
|
||||
parts.push(`${alias}.bucket_at >= ? AND ${alias}.bucket_at < ?`)
|
||||
} else {
|
||||
params.push(ctx.fromDay, ctx.toDayExclusive)
|
||||
parts.push(`${alias}.day >= ? AND ${alias}.day < ?`)
|
||||
}
|
||||
if (ctx.serverId != null) {
|
||||
parts.push(`${alias}.server_id = ?`)
|
||||
params.push(ctx.serverId)
|
||||
}
|
||||
if (ctx.iface) {
|
||||
const aliases = ifaceFilterAliases(ctx.iface, ctx.serverId)
|
||||
if (aliases.length <= 1) {
|
||||
parts.push(`${alias}.iface = ?`)
|
||||
params.push(aliases[0] ?? ctx.iface)
|
||||
} else {
|
||||
parts.push(`${alias}.iface IN (${aliases.map(() => "?").join(", ")})`)
|
||||
params.push(...aliases)
|
||||
}
|
||||
}
|
||||
if (ctx.country) {
|
||||
parts.push(`${alias}.country = ?`)
|
||||
params.push(ctx.country.toUpperCase())
|
||||
}
|
||||
if (ctx.service) {
|
||||
parts.push(`${alias}.service = ?`)
|
||||
params.push(ctx.service)
|
||||
}
|
||||
if (ctx.asn != null) {
|
||||
parts.push(`${alias}.asn = ?`)
|
||||
params.push(ctx.asn)
|
||||
}
|
||||
if (ctx.userIfaces) {
|
||||
if (ctx.userIfaces.length === 0) {
|
||||
parts.push("FALSE")
|
||||
} else {
|
||||
const tuples = ctx.userIfaces.map(() => "(?, ?)").join(", ")
|
||||
parts.push(`(${alias}.server_id, ${alias}.iface) IN (${tuples})`)
|
||||
for (const u of ctx.userIfaces) {
|
||||
params.push(u.serverId, u.iface)
|
||||
}
|
||||
}
|
||||
}
|
||||
if (ctx.unboundOnly) {
|
||||
if (ctx.boundIfaces.length === 0) {
|
||||
/* весь трафик без привязок */
|
||||
} else {
|
||||
const tuples = ctx.boundIfaces.map(() => "(?, ?)").join(", ")
|
||||
parts.push(`(${alias}.server_id, ${alias}.iface) NOT IN (${tuples})`)
|
||||
for (const u of ctx.boundIfaces) {
|
||||
params.push(u.serverId, u.iface)
|
||||
}
|
||||
}
|
||||
}
|
||||
return { sql: parts.join(" AND "), params }
|
||||
}
|
||||
|
||||
function emptyDto(period: ParsedPeriod): StatisticsDto {
|
||||
return {
|
||||
from: period.fromIso,
|
||||
to: period.toIso,
|
||||
grain: period.grain,
|
||||
kpis: {
|
||||
bytes: 0,
|
||||
packets: 0,
|
||||
avgBps: 0,
|
||||
users: 0,
|
||||
servers: 0,
|
||||
ifaces: 0,
|
||||
topCountry: "—",
|
||||
topService: "—",
|
||||
},
|
||||
series: [],
|
||||
users: [],
|
||||
servers: [],
|
||||
interfaces: [],
|
||||
countries: [],
|
||||
services: [],
|
||||
asns: [],
|
||||
}
|
||||
}
|
||||
|
||||
function toBreakdown(
|
||||
rows: Array<{ id: string; label: string; bytes: number; packets: number }>,
|
||||
totalBytes: number,
|
||||
windowSec: number,
|
||||
): StatisticsBreakdownRow[] {
|
||||
const denom = totalBytes || 1
|
||||
return rows
|
||||
.sort((a, b) => b.bytes - a.bytes)
|
||||
.slice(0, TOP_N)
|
||||
.map((r) => ({
|
||||
id: r.id,
|
||||
label: r.label,
|
||||
bytes: r.bytes,
|
||||
packets: r.packets,
|
||||
bps: (r.bytes * 8) / windowSec,
|
||||
percent: (r.bytes / denom) * 100,
|
||||
}))
|
||||
}
|
||||
|
||||
interface UserBindTuple {
|
||||
userId: string
|
||||
serverId: number
|
||||
iface: string
|
||||
}
|
||||
|
||||
async function loadBindUserTuples(): Promise<UserBindTuple[]> {
|
||||
const binds = await db.select().from(userInterfaceBindings)
|
||||
const seen = new Set<string>()
|
||||
const out: UserBindTuple[] = []
|
||||
for (const b of binds) {
|
||||
for (const iface of factIfaceAliases(b.interfaceName, b.serverId)) {
|
||||
const k = `${b.userId}\0${b.serverId}\0${iface}`
|
||||
if (seen.has(k)) continue
|
||||
seen.add(k)
|
||||
out.push({ userId: b.userId, serverId: b.serverId, iface })
|
||||
}
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
function uniqueBoundIfaces(tuples: UserBindTuple[]): Array<{ serverId: number; iface: string }> {
|
||||
const seen = new Set<string>()
|
||||
const out: Array<{ serverId: number; iface: string }> = []
|
||||
for (const t of tuples) {
|
||||
const k = `${t.serverId}\0${t.iface}`
|
||||
if (seen.has(k)) continue
|
||||
seen.add(k)
|
||||
out.push({ serverId: t.serverId, iface: t.iface })
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
async function resolveUserIfaces(userId?: string): Promise<Array<{ serverId: number; iface: string }> | null> {
|
||||
if (!userId || userId === STATISTICS_UNBOUND_USER_ID) return null
|
||||
const binds = await db.select().from(userInterfaceBindings).where(eq(userInterfaceBindings.userId, userId))
|
||||
return expandBindingIfaces(binds.map((b) => ({ serverId: b.serverId, iface: b.interfaceName })))
|
||||
}
|
||||
|
||||
function userBindJoinSql(tuples: UserBindTuple[]): { sql: string; params: unknown[] } {
|
||||
const values = tuples.map(() => "(?::text, ?::int, ?::text)").join(", ")
|
||||
const params = tuples.flatMap((t) => [t.userId, t.serverId, t.iface])
|
||||
return {
|
||||
sql: `JOIN (VALUES ${values}) AS b(user_id, server_id, iface) ON b.server_id = f.server_id AND b.iface = f.iface`,
|
||||
params,
|
||||
}
|
||||
}
|
||||
|
||||
async function buildFilterCtx(query: StatisticsQuery, period: ParsedPeriod): Promise<FilterCtx | null> {
|
||||
const bindTuples = await loadBindUserTuples()
|
||||
const boundIfaces = uniqueBoundIfaces(bindTuples)
|
||||
const unboundOnly = query.userId === STATISTICS_UNBOUND_USER_ID
|
||||
const userIfaces = unboundOnly ? null : await resolveUserIfaces(query.userId)
|
||||
if (userIfaces && userIfaces.length === 0) return null
|
||||
return {
|
||||
...period,
|
||||
serverId: query.serverId,
|
||||
iface: query.iface,
|
||||
country: query.country,
|
||||
service: query.service,
|
||||
asn: query.asn,
|
||||
userIfaces,
|
||||
unboundOnly,
|
||||
boundIfaces,
|
||||
}
|
||||
}
|
||||
|
||||
export async function getStatistics(query: StatisticsQuery): Promise<StatisticsDto> {
|
||||
const period = parseStatisticsPeriod(query.from, query.to)
|
||||
if (!period) return emptyDto({
|
||||
fromIso: query.from,
|
||||
toIso: query.to,
|
||||
fromDay: query.from.slice(0, 10),
|
||||
toDayExclusive: query.to.slice(0, 10),
|
||||
grain: "day",
|
||||
windowSec: 1,
|
||||
})
|
||||
|
||||
await warmBindingIfaceCache()
|
||||
if (query.serverId) await warmIfaceCache([query.serverId])
|
||||
const bindTuples = await loadBindUserTuples()
|
||||
const ctx = await buildFilterCtx(query, period)
|
||||
if (!ctx) return emptyDto(period)
|
||||
|
||||
const table = period.grain === "hour" ? "flow_hour_facts" : "flow_daily_facts"
|
||||
const timeCol = period.grain === "hour" ? "bucket_at" : "day"
|
||||
const where = factWhere("f", period.grain, ctx)
|
||||
|
||||
const totals = await dbAll<{ bytes: number; packets: number; servers: number }>(`
|
||||
SELECT
|
||||
COALESCE(SUM(f.bytes), 0) AS bytes,
|
||||
COALESCE(SUM(f.packets), 0) AS packets,
|
||||
COUNT(DISTINCT f.server_id)::int AS servers
|
||||
FROM ${table} f
|
||||
WHERE ${where.sql}
|
||||
`, where.params)
|
||||
|
||||
const bytes = Number(totals[0]?.bytes) || 0
|
||||
const packets = Number(totals[0]?.packets) || 0
|
||||
const serverCount = Number(totals[0]?.servers) || 0
|
||||
|
||||
const seriesRows = await dbAll<{ t: string; bytes: number }>(`
|
||||
SELECT ${timeCol}::text AS t, SUM(f.bytes) AS bytes
|
||||
FROM ${table} f
|
||||
WHERE ${where.sql}
|
||||
GROUP BY ${timeCol}
|
||||
ORDER BY ${timeCol}
|
||||
`, where.params)
|
||||
|
||||
const countryRows = await dbAll<{ id: string; bytes: number; packets: number }>(`
|
||||
SELECT f.country AS id, SUM(f.bytes) AS bytes, SUM(f.packets) AS packets
|
||||
FROM ${table} f
|
||||
WHERE ${where.sql}
|
||||
GROUP BY f.country
|
||||
`, where.params)
|
||||
|
||||
const serviceRows = await dbAll<{ id: string; bytes: number; packets: number }>(`
|
||||
SELECT f.service AS id, SUM(f.bytes) AS bytes, SUM(f.packets) AS packets
|
||||
FROM ${table} f
|
||||
WHERE ${where.sql}
|
||||
GROUP BY f.service
|
||||
`, where.params)
|
||||
|
||||
const asnRows = await dbAll<{ id: number; bytes: number; packets: number }>(`
|
||||
SELECT f.asn AS id, SUM(f.bytes) AS bytes, SUM(f.packets) AS packets
|
||||
FROM ${table} f
|
||||
WHERE ${where.sql}
|
||||
GROUP BY f.asn
|
||||
`, where.params)
|
||||
|
||||
const serverRows = await dbAll<{ id: number; bytes: number; packets: number }>(`
|
||||
SELECT f.server_id AS id, SUM(f.bytes) AS bytes, SUM(f.packets) AS packets
|
||||
FROM ${table} f
|
||||
WHERE ${where.sql}
|
||||
GROUP BY f.server_id
|
||||
`, where.params)
|
||||
|
||||
const ifaceRowsRaw = await dbAll<{ serverId: number; iface: string; bytes: number; packets: number }>(`
|
||||
SELECT f.server_id AS "serverId", f.iface AS iface, SUM(f.bytes) AS bytes, SUM(f.packets) AS packets
|
||||
FROM ${table} f
|
||||
WHERE ${where.sql}
|
||||
GROUP BY f.server_id, f.iface
|
||||
`, where.params)
|
||||
await warmIfaceCache(ifaceRowsRaw.filter((r) => looksLikeIfIndex(r.iface)).map((r) => r.serverId))
|
||||
const ifaceRows = collapseServerIfaceRows(ifaceRowsRaw)
|
||||
const ifaceCount = ifaceRows.length
|
||||
|
||||
let userRows: Array<{ id: string; bytes: number; packets: number }> = []
|
||||
if (bindTuples.length && !ctx.unboundOnly) {
|
||||
const join = userBindJoinSql(bindTuples)
|
||||
userRows = await dbAll<{ id: string; bytes: number; packets: number }>(`
|
||||
SELECT b.user_id AS id, SUM(f.bytes) AS bytes, SUM(f.packets) AS packets
|
||||
FROM ${table} f
|
||||
${join.sql}
|
||||
WHERE ${where.sql}
|
||||
GROUP BY b.user_id
|
||||
`, [...join.params, ...where.params])
|
||||
}
|
||||
|
||||
const serverNames = new Map<number, string>()
|
||||
const allServers = await db.select({ id: servers.id, name: servers.name, host: servers.host }).from(servers)
|
||||
for (const s of allServers) serverNames.set(s.id, s.name || s.host)
|
||||
|
||||
const userNames = new Map<string, string>()
|
||||
const allUsers = await db.select({ id: appUsers.id, name: appUsers.name, login: appUsers.login }).from(appUsers)
|
||||
for (const u of allUsers) userNames.set(u.id, u.name || u.login)
|
||||
|
||||
const asnHolders = new Map<number, string>()
|
||||
const asnMeta = await db.select({ asn: flowAsnMeta.asn, holder: flowAsnMeta.holder }).from(flowAsnMeta)
|
||||
for (const a of asnMeta) asnHolders.set(a.asn, a.holder)
|
||||
|
||||
const countries = toBreakdown(
|
||||
countryRows.map((r) => ({
|
||||
id: r.id,
|
||||
label: r.id === "XX" ? "Неизвестно" : r.id,
|
||||
bytes: Number(r.bytes) || 0,
|
||||
packets: Number(r.packets) || 0,
|
||||
})),
|
||||
bytes,
|
||||
period.windowSec,
|
||||
)
|
||||
const services = toBreakdown(
|
||||
serviceRows.map((r) => ({
|
||||
id: r.id,
|
||||
label: r.id,
|
||||
bytes: Number(r.bytes) || 0,
|
||||
packets: Number(r.packets) || 0,
|
||||
})),
|
||||
bytes,
|
||||
period.windowSec,
|
||||
)
|
||||
const asns = toBreakdown(
|
||||
asnRows.map((r) => {
|
||||
const id = Number(r.id) || 0
|
||||
const holder = asnHolders.get(id)
|
||||
return {
|
||||
id: String(id),
|
||||
label: id === 0 ? "other" : holder ? `AS${id} · ${holder}` : `AS${id}`,
|
||||
bytes: Number(r.bytes) || 0,
|
||||
packets: Number(r.packets) || 0,
|
||||
}
|
||||
}),
|
||||
bytes,
|
||||
period.windowSec,
|
||||
)
|
||||
const serverBreakdown = toBreakdown(
|
||||
serverRows.map((r) => ({
|
||||
id: String(r.id),
|
||||
label: serverNames.get(r.id) || String(r.id),
|
||||
bytes: Number(r.bytes) || 0,
|
||||
packets: Number(r.packets) || 0,
|
||||
})),
|
||||
bytes,
|
||||
period.windowSec,
|
||||
)
|
||||
const interfaces = toBreakdown(
|
||||
ifaceRows.map((r) => ({
|
||||
id: `${r.serverId}:${r.iface}`,
|
||||
label: `${serverNames.get(r.serverId) || r.serverId} · ${r.iface}`,
|
||||
bytes: Number(r.bytes) || 0,
|
||||
packets: Number(r.packets) || 0,
|
||||
})),
|
||||
bytes,
|
||||
period.windowSec,
|
||||
)
|
||||
const matchedUsers = toBreakdown(
|
||||
userRows.map((r) => ({
|
||||
id: r.id,
|
||||
label: userNames.get(r.id) || r.id,
|
||||
bytes: Number(r.bytes) || 0,
|
||||
packets: Number(r.packets) || 0,
|
||||
})),
|
||||
bytes,
|
||||
period.windowSec,
|
||||
)
|
||||
|
||||
const users = [...matchedUsers]
|
||||
if (!ctx.unboundOnly && !ctx.userIfaces) {
|
||||
let unboundBytes = 0
|
||||
let unboundPackets = 0
|
||||
if (ctx.boundIfaces.length === 0) {
|
||||
unboundBytes = bytes
|
||||
unboundPackets = packets
|
||||
} else {
|
||||
const tuples = ctx.boundIfaces.map(() => "(?, ?)").join(", ")
|
||||
const unboundParams = [...where.params]
|
||||
for (const u of ctx.boundIfaces) unboundParams.push(u.serverId, u.iface)
|
||||
const unboundRows = await dbAll<{ bytes: number; packets: number }>(`
|
||||
SELECT COALESCE(SUM(f.bytes), 0) AS bytes, COALESCE(SUM(f.packets), 0) AS packets
|
||||
FROM ${table} f
|
||||
WHERE ${where.sql}
|
||||
AND (f.server_id, f.iface) NOT IN (${tuples})
|
||||
`, unboundParams)
|
||||
unboundBytes = Number(unboundRows[0]?.bytes) || 0
|
||||
unboundPackets = Number(unboundRows[0]?.packets) || 0
|
||||
}
|
||||
if (unboundBytes > 0) {
|
||||
const denom = bytes || 1
|
||||
users.push({
|
||||
id: STATISTICS_UNBOUND_USER_ID,
|
||||
label: "Без привязки",
|
||||
bytes: unboundBytes,
|
||||
packets: unboundPackets,
|
||||
bps: (unboundBytes * 8) / period.windowSec,
|
||||
percent: (unboundBytes / denom) * 100,
|
||||
})
|
||||
users.sort((a, b) => b.bytes - a.bytes)
|
||||
}
|
||||
}
|
||||
|
||||
return {
|
||||
from: period.fromIso,
|
||||
to: period.toIso,
|
||||
grain: period.grain,
|
||||
kpis: {
|
||||
bytes,
|
||||
packets,
|
||||
avgBps: (bytes * 8) / period.windowSec,
|
||||
users: matchedUsers.length,
|
||||
servers: serverCount,
|
||||
ifaces: ifaceCount,
|
||||
topCountry: countries[0]?.label || "—",
|
||||
topService: services[0]?.label || "—",
|
||||
},
|
||||
series: seriesRows.map((r) => ({ t: r.t, bytes: Number(r.bytes) || 0 })),
|
||||
users,
|
||||
servers: serverBreakdown,
|
||||
interfaces,
|
||||
countries,
|
||||
services,
|
||||
asns,
|
||||
}
|
||||
}
|
||||
|
||||
function dimSql(dim: StatisticsPivotDim, factAlias: string, bindAlias: string): string {
|
||||
if (dim === "country") return `${factAlias}.country`
|
||||
if (dim === "service") return `${factAlias}.service`
|
||||
if (dim === "asn") return `${factAlias}.asn::text`
|
||||
if (dim === "server") return `${factAlias}.server_id::text`
|
||||
if (dim === "iface") return `(${factAlias}.server_id::text || ':' || ${factAlias}.iface)`
|
||||
return `${bindAlias}.user_id`
|
||||
}
|
||||
|
||||
function emptyPivot(query: StatisticsPivotQuery): StatisticsPivotDto {
|
||||
return {
|
||||
rowDim: query.row,
|
||||
colDim: query.col,
|
||||
metric: query.metric,
|
||||
columns: [],
|
||||
rows: [],
|
||||
otherBytes: 0,
|
||||
}
|
||||
}
|
||||
|
||||
export function pivotDimsConflict(row: StatisticsPivotDim, col: StatisticsPivotDim): boolean {
|
||||
return row === col
|
||||
}
|
||||
|
||||
export async function getStatisticsPivot(query: StatisticsPivotQuery): Promise<StatisticsPivotDto> {
|
||||
if (pivotDimsConflict(query.row, query.col)) return emptyPivot(query)
|
||||
const period = parseStatisticsPeriod(query.from, query.to)
|
||||
if (!period) return emptyPivot(query)
|
||||
await warmBindingIfaceCache()
|
||||
if (query.serverId) await warmIfaceCache([query.serverId])
|
||||
const bindTuples = await loadBindUserTuples()
|
||||
const ctx = await buildFilterCtx(query, period)
|
||||
if (!ctx) return emptyPivot(query)
|
||||
const needsUser = query.row === "user" || query.col === "user"
|
||||
if (needsUser && bindTuples.length === 0) return emptyPivot(query)
|
||||
|
||||
const table = period.grain === "hour" ? "flow_hour_facts" : "flow_daily_facts"
|
||||
const where = factWhere("f", period.grain, ctx)
|
||||
const rowExpr = dimSql(query.row, "f", "b")
|
||||
const colExpr = dimSql(query.col, "f", "b")
|
||||
const join = needsUser ? userBindJoinSql(bindTuples) : { sql: "", params: [] as unknown[] }
|
||||
|
||||
const raw = await dbAll<{ row_id: string; col_id: string; bytes: number; packets: number }>(`
|
||||
SELECT ${rowExpr} AS row_id, ${colExpr} AS col_id,
|
||||
SUM(f.bytes) AS bytes, SUM(f.packets) AS packets
|
||||
FROM ${table} f
|
||||
${join.sql}
|
||||
WHERE ${where.sql}
|
||||
GROUP BY 1, 2
|
||||
`, [...join.params, ...where.params])
|
||||
|
||||
if (query.row === "iface" || query.col === "iface") {
|
||||
const ifaceServerIds: number[] = []
|
||||
for (const r of raw) {
|
||||
for (const dim of [query.row, query.col] as const) {
|
||||
if (dim !== "iface") continue
|
||||
const id = dim === query.row ? String(r.row_id ?? "") : String(r.col_id ?? "")
|
||||
const colon = id.indexOf(":")
|
||||
if (colon < 0) continue
|
||||
const sid = Number(id.slice(0, colon))
|
||||
if (looksLikeIfIndex(id.slice(colon + 1)) && Number.isFinite(sid)) ifaceServerIds.push(sid)
|
||||
}
|
||||
}
|
||||
await warmIfaceCache(ifaceServerIds)
|
||||
for (const r of raw) {
|
||||
if (query.row === "iface") r.row_id = canonicalIfaceDimId(String(r.row_id ?? ""))
|
||||
if (query.col === "iface") r.col_id = canonicalIfaceDimId(String(r.col_id ?? ""))
|
||||
}
|
||||
}
|
||||
|
||||
const metric = query.metric
|
||||
type Acc = { bytes: number; packets: number }
|
||||
const cell = new Map<string, Map<string, Acc>>()
|
||||
const colTotals = new Map<string, number>()
|
||||
for (const r of raw) {
|
||||
const rid = String(r.row_id ?? "")
|
||||
const cid = String(r.col_id ?? "")
|
||||
const acc: Acc = { bytes: Number(r.bytes) || 0, packets: Number(r.packets) || 0 }
|
||||
const val = metric === "packets" ? acc.packets : acc.bytes
|
||||
let rowMap = cell.get(rid)
|
||||
if (!rowMap) {
|
||||
rowMap = new Map()
|
||||
cell.set(rid, rowMap)
|
||||
}
|
||||
const prev = rowMap.get(cid)
|
||||
if (prev) {
|
||||
prev.bytes += acc.bytes
|
||||
prev.packets += acc.packets
|
||||
} else {
|
||||
rowMap.set(cid, acc)
|
||||
}
|
||||
colTotals.set(cid, (colTotals.get(cid) ?? 0) + val)
|
||||
}
|
||||
|
||||
const topCols = [...colTotals.entries()]
|
||||
.sort((a, b) => b[1] - a[1])
|
||||
.slice(0, PIVOT_COL_CAP)
|
||||
.map(([id]) => id)
|
||||
const topColSet = new Set(topCols)
|
||||
const folded = new Map<string, Map<string, number>>()
|
||||
const foldedColTotals = new Map<string, number>()
|
||||
let otherBytes = 0
|
||||
for (const [rid, cols] of cell) {
|
||||
const rowMap = new Map<string, number>()
|
||||
for (const [cid, acc] of cols) {
|
||||
const val = metric === "packets" ? acc.packets : acc.bytes
|
||||
const dest = topColSet.has(cid) ? cid : PIVOT_OTHER_ID
|
||||
if (dest === PIVOT_OTHER_ID) otherBytes += val
|
||||
rowMap.set(dest, (rowMap.get(dest) ?? 0) + val)
|
||||
foldedColTotals.set(dest, (foldedColTotals.get(dest) ?? 0) + val)
|
||||
}
|
||||
folded.set(rid, rowMap)
|
||||
}
|
||||
|
||||
const rowTotals = new Map<string, number>()
|
||||
for (const [rid, cols] of folded) {
|
||||
let t = 0
|
||||
for (const v of cols.values()) t += v
|
||||
rowTotals.set(rid, t)
|
||||
}
|
||||
const topRows = [...rowTotals.entries()]
|
||||
.sort((a, b) => b[1] - a[1])
|
||||
.slice(0, PIVOT_ROW_CAP)
|
||||
.map(([id]) => id)
|
||||
const topRowSet = new Set(topRows)
|
||||
const finalRows = new Map<string, Map<string, number>>()
|
||||
const finalRowTotals = new Map<string, number>()
|
||||
for (const [rid, cols] of folded) {
|
||||
const dest = topRowSet.has(rid) ? rid : PIVOT_OTHER_ID
|
||||
if (dest === PIVOT_OTHER_ID) {
|
||||
for (const [cid, v] of cols) {
|
||||
if (cid !== PIVOT_OTHER_ID) otherBytes += v
|
||||
}
|
||||
}
|
||||
let rowMap = finalRows.get(dest)
|
||||
if (!rowMap) {
|
||||
rowMap = new Map()
|
||||
finalRows.set(dest, rowMap)
|
||||
}
|
||||
for (const [cid, v] of cols) {
|
||||
rowMap.set(cid, (rowMap.get(cid) ?? 0) + v)
|
||||
}
|
||||
}
|
||||
for (const [rid, cols] of finalRows) {
|
||||
let t = 0
|
||||
for (const v of cols.values()) t += v
|
||||
finalRowTotals.set(rid, t)
|
||||
}
|
||||
|
||||
const colIds = [...topCols]
|
||||
if (foldedColTotals.has(PIVOT_OTHER_ID)) colIds.push(PIVOT_OTHER_ID)
|
||||
const rowIds = [...topRows]
|
||||
if (finalRows.has(PIVOT_OTHER_ID) && !topRowSet.has(PIVOT_OTHER_ID)) rowIds.push(PIVOT_OTHER_ID)
|
||||
|
||||
const labels = await loadPivotLabels(query.row, query.col, rowIds, colIds)
|
||||
|
||||
return {
|
||||
rowDim: query.row,
|
||||
colDim: query.col,
|
||||
metric,
|
||||
columns: colIds.map((id) => ({
|
||||
id,
|
||||
label: labels.col.get(id) ?? (id === PIVOT_OTHER_ID ? "Прочие" : id),
|
||||
total: foldedColTotals.get(id) ?? 0,
|
||||
})),
|
||||
rows: rowIds.map((id) => {
|
||||
const cols = finalRows.get(id) ?? new Map()
|
||||
const cells: Record<string, number> = {}
|
||||
for (const cid of colIds) cells[cid] = cols.get(cid) ?? 0
|
||||
return {
|
||||
id,
|
||||
label: labels.row.get(id) ?? (id === PIVOT_OTHER_ID ? "Прочие" : id),
|
||||
total: finalRowTotals.get(id) ?? 0,
|
||||
cells,
|
||||
}
|
||||
}),
|
||||
otherBytes,
|
||||
}
|
||||
}
|
||||
|
||||
async function loadPivotLabels(
|
||||
rowDim: StatisticsPivotDim,
|
||||
colDim: StatisticsPivotDim,
|
||||
rowIds: string[],
|
||||
colIds: string[],
|
||||
): Promise<{ row: Map<string, string>; col: Map<string, string> }> {
|
||||
const serverNames = new Map<string, string>()
|
||||
const allServers = await db.select({ id: servers.id, name: servers.name, host: servers.host }).from(servers)
|
||||
for (const s of allServers) serverNames.set(String(s.id), s.name || s.host)
|
||||
const userNames = new Map<string, string>()
|
||||
const allUsers = await db.select({ id: appUsers.id, name: appUsers.name, login: appUsers.login }).from(appUsers)
|
||||
for (const u of allUsers) userNames.set(u.id, u.name || u.login)
|
||||
const asnHolders = new Map<string, string>()
|
||||
const asnMeta = await db.select({ asn: flowAsnMeta.asn, holder: flowAsnMeta.holder }).from(flowAsnMeta)
|
||||
for (const a of asnMeta) asnHolders.set(String(a.asn), a.holder)
|
||||
|
||||
function label(dim: StatisticsPivotDim, id: string): string {
|
||||
if (id === PIVOT_OTHER_ID) return "Прочие"
|
||||
if (dim === "country") return id === "XX" ? "Неизвестно" : id
|
||||
if (dim === "server") return serverNames.get(id) || id
|
||||
if (dim === "user") return userNames.get(id) || id
|
||||
if (dim === "asn") {
|
||||
if (id === "0") return "other"
|
||||
const holder = asnHolders.get(id)
|
||||
return holder ? `AS${id} · ${holder}` : `AS${id}`
|
||||
}
|
||||
if (dim === "iface") {
|
||||
const colon = id.indexOf(":")
|
||||
if (colon < 0) return id
|
||||
const sid = id.slice(0, colon)
|
||||
const iface = id.slice(colon + 1)
|
||||
const sidNum = Number(sid)
|
||||
const name = Number.isFinite(sidNum) ? displayFactIface(sidNum, iface) : iface
|
||||
return `${serverNames.get(sid) || sid} · ${name}`
|
||||
}
|
||||
return id
|
||||
}
|
||||
|
||||
const row = new Map<string, string>()
|
||||
const col = new Map<string, string>()
|
||||
for (const id of rowIds) row.set(id, label(rowDim, id))
|
||||
for (const id of colIds) col.set(id, label(colDim, id))
|
||||
return { row, col }
|
||||
}
|
||||
@@ -1,6 +1,7 @@
|
||||
import { and, asc, desc, eq, gte, lt } from "drizzle-orm"
|
||||
import { and, asc, desc, eq, gte } from "drizzle-orm"
|
||||
import { db } from "../db/index.js"
|
||||
import { servers, trafficSamples, trafficSettings } from "../db/schema.js"
|
||||
import { decodeTrafficSampleFlags, encodeTrafficFlags } from "../db/traffic-flags.js"
|
||||
import type { TrafficRunSnapshot } from "../types/scheduler-run-snapshot.js"
|
||||
import { SCHEDULER_RUN_SNAPSHOT_VERSION } from "../types/scheduler-run-snapshot.js"
|
||||
import { MikrotikClient } from "./mikrotik.js"
|
||||
@@ -79,12 +80,6 @@ async function getSettingsRow() {
|
||||
return (await db.select().from(trafficSettings).where(eq(trafficSettings.id, 1)).limit(1))[0]
|
||||
}
|
||||
|
||||
async function cleanupOldSamples(retentionDays: number) {
|
||||
const cutoff = new Date(Date.now() - retentionDays * 24 * 60 * 60 * 1000).toISOString()
|
||||
await db.delete(trafficSamples)
|
||||
.where(lt(trafficSamples.sampledAt, cutoff))
|
||||
}
|
||||
|
||||
async function readPreviousWave(serverId: number): Promise<Map<string, { rxBytes: number; txBytes: number; sampledAt: string }>> {
|
||||
const last = (await db
|
||||
.select({ sampledAt: trafficSamples.sampledAt })
|
||||
@@ -164,8 +159,7 @@ export async function collectTrafficOnce(): Promise<TrafficRunSnapshot> {
|
||||
txBytes,
|
||||
rxBps,
|
||||
txBps,
|
||||
running,
|
||||
disabled,
|
||||
flags: encodeTrafficFlags(running, disabled),
|
||||
}
|
||||
})
|
||||
try {
|
||||
@@ -194,8 +188,7 @@ export async function collectTrafficOnce(): Promise<TrafficRunSnapshot> {
|
||||
txBytes,
|
||||
rxBps,
|
||||
txBps,
|
||||
running,
|
||||
disabled,
|
||||
flags: encodeTrafficFlags(running, disabled),
|
||||
})
|
||||
}
|
||||
} catch {
|
||||
@@ -224,7 +217,6 @@ export async function collectTrafficOnce(): Promise<TrafficRunSnapshot> {
|
||||
}
|
||||
}
|
||||
|
||||
await cleanupOldSamples(Math.max(1, settings.retentionDays))
|
||||
await db.update(trafficSettings).set({
|
||||
lastCollectedAt: now,
|
||||
lastDurationMs: Date.now() - startedAt,
|
||||
@@ -286,11 +278,12 @@ export async function updateTrafficSettings(patch: {
|
||||
}
|
||||
|
||||
export async function readServerSamplesInRange(serverId: number, sinceIso: string) {
|
||||
return await db.select()
|
||||
const rows = await db.select()
|
||||
.from(trafficSamples)
|
||||
.where(and(
|
||||
eq(trafficSamples.serverId, serverId),
|
||||
gte(trafficSamples.sampledAt, sinceIso),
|
||||
))
|
||||
.orderBy(asc(trafficSamples.sampledAt))
|
||||
return rows.map((r) => ({ ...r, ...decodeTrafficSampleFlags(r.flags) }))
|
||||
}
|
||||
|
||||
@@ -16,6 +16,29 @@ import {
|
||||
seedRipeCacheForTests,
|
||||
} from "./traffic-flow-ripe.js"
|
||||
|
||||
{
|
||||
const liveErr = await formatLiveSseFromBuilder(() => {
|
||||
throw new Error("SQLITE_BUSY")
|
||||
})
|
||||
assert.equal(liveErr.event, "error")
|
||||
assert.equal((liveErr.data as { error: string }).error, "SQLITE_BUSY")
|
||||
const liveOk = await formatLiveSseFromBuilder(() => ({ ok: true }))
|
||||
assert.equal(liveOk.event, "sample")
|
||||
const liveAsync = await formatLiveSseFromBuilder(async () => ({
|
||||
uniqueSrc: 3,
|
||||
destinations: [{ id: "8.8.8.8", label: "8.8.8.8", bytes: 1, packets: 1, bps: 1, percent: 100 }],
|
||||
}))
|
||||
assert.equal(liveAsync.event, "sample")
|
||||
assert.notEqual(JSON.stringify(liveAsync.data), "{}")
|
||||
assert.equal((liveAsync.data as { uniqueSrc: number }).uniqueSrc, 3)
|
||||
assert.ok(Array.isArray((liveAsync.data as { destinations: unknown[] }).destinations))
|
||||
const liveReject = await formatLiveSseFromBuilder(async () => {
|
||||
throw new Error("pg down")
|
||||
})
|
||||
assert.equal(liveReject.event, "error")
|
||||
assert.equal((liveReject.data as { error: string }).error, "pg down")
|
||||
}
|
||||
|
||||
if (!(await withPgOrSkip())) {
|
||||
console.log("traffic-flow-analytics.test.ts: skip")
|
||||
process.exit(0)
|
||||
@@ -248,13 +271,6 @@ try {
|
||||
assert.equal(degraded.degraded, true)
|
||||
assert.equal(degraded.conversationsList.length, 0)
|
||||
assert.ok((degraded.bytes ?? 0) >= 12_000)
|
||||
const liveErr = formatLiveSseFromBuilder(() => {
|
||||
throw new Error("SQLITE_BUSY")
|
||||
})
|
||||
assert.equal(liveErr.event, "error")
|
||||
assert.equal((liveErr.data as { error: string }).error, "SQLITE_BUSY")
|
||||
const liveOk = formatLiveSseFromBuilder(() => ({ ok: true }))
|
||||
assert.equal(liveOk.event, "sample")
|
||||
const exporters = await listFlowExporters(5)
|
||||
const clients = await listFlowClients(5)
|
||||
assert.ok(Array.isArray(exporters.exporters))
|
||||
|
||||
@@ -26,7 +26,8 @@ import { resolveIfaceName } from "./traffic-flow-ifaces.js"
|
||||
import { getTrafficFlowSettingsRow, listHostPeers } from "./traffic-flow-settings.js"
|
||||
import { applicationName, flowRowMatchesFilter } from "./traffic-flow-apps.js"
|
||||
import { dedupFlowRowsMaxBytes, flowTupleKey } from "./traffic-flow-dedup.js"
|
||||
import { enqueueRipeMisses, lookupRipeCached } from "./traffic-flow-ripe.js"
|
||||
import { enqueueRipeMisses } from "./traffic-flow-ripe.js"
|
||||
import { resolveFlowIp } from "./traffic-flow-geoip.js"
|
||||
import { classifyFlowDst, refreshFlowCatalogInBackground } from "./traffic-flow-classify.js"
|
||||
import { isIsoCountry } from "./traffic-flow-brands.js"
|
||||
import { classifyFlowPlane, flowBps, shouldKeepPlane } from "./traffic-flow-planes.js"
|
||||
@@ -255,7 +256,7 @@ async function buildFlowAnalyticsUncached(q: FlowAnalyticsQuery): Promise<FlowAn
|
||||
const peer = pickInternetPeer(r.src, r.dst, r.srcPort, r.dstPort)
|
||||
peers.add(peer)
|
||||
const app = applicationName(r.proto, r.dstPort, r.srcPort)
|
||||
const ripe = lookupRipeCached(peer)
|
||||
const ripe = resolveFlowIp(peer)
|
||||
const classified = classifyFlowDst(peer, r.proto, r.dstPort, r.srcPort, ripe)
|
||||
bump(applications, app, r.bytes, r.packets)
|
||||
bump(protocols, protoName(r.proto), r.bytes, r.packets)
|
||||
@@ -599,9 +600,12 @@ export async function listFlowClients(minutes: number): Promise<{ clients: { id:
|
||||
return { clients }
|
||||
}
|
||||
|
||||
export function formatLiveSseFromBuilder(build: () => unknown): { event: "sample" | "error"; data: unknown } {
|
||||
export async function formatLiveSseFromBuilder(
|
||||
build: () => unknown | Promise<unknown>,
|
||||
): Promise<{ event: "sample" | "error"; data: unknown }> {
|
||||
try {
|
||||
return { event: "sample", data: build() }
|
||||
const data = await build()
|
||||
return { event: "sample", data }
|
||||
} catch (err) {
|
||||
const message = err instanceof Error ? err.message : String(err)
|
||||
return { event: "error", data: { error: message } }
|
||||
@@ -613,10 +617,10 @@ export function isFlowAnalyticsDegraded(): boolean {
|
||||
return health.pendingSize >= LIVE_DEGRADED_PENDING
|
||||
}
|
||||
|
||||
export function safeBuildLiveFlowSample(q: Omit<FlowAnalyticsQuery, "minutes" | "skipHeavy">): {
|
||||
export async function safeBuildLiveFlowSample(q: Omit<FlowAnalyticsQuery, "minutes" | "skipHeavy">): Promise<{
|
||||
event: "sample" | "error"
|
||||
data: unknown
|
||||
} {
|
||||
}> {
|
||||
return formatLiveSseFromBuilder(async () => {
|
||||
const skipHeavy = isFlowAnalyticsDegraded()
|
||||
return await buildFlowAnalytics({ ...q, minutes: LIVE_ANALYTICS_MINUTES, skipHeavy })
|
||||
|
||||
@@ -1,11 +1,14 @@
|
||||
import assert from "node:assert/strict"
|
||||
import {
|
||||
brandByAsn,
|
||||
brandByHolder,
|
||||
countryFromHolder,
|
||||
isSteamGamePort,
|
||||
lookupBrand,
|
||||
OTHER_SERVICE,
|
||||
isNamedInternetService,
|
||||
mapServiceNodeId,
|
||||
resolveFlowBrand,
|
||||
resolveRipeCountry,
|
||||
} from "./traffic-flow-brands.js"
|
||||
|
||||
@@ -39,4 +42,35 @@ assert.equal(isNamedInternetService("DNS", "DNS"), false)
|
||||
assert.equal(mapServiceNodeId("AWS"), "svc:aws")
|
||||
assert.equal(mapServiceNodeId("Cloudflare"), "svc:cloudflare")
|
||||
|
||||
assert.equal(brandByAsn(714)?.service, "Apple")
|
||||
assert.equal(brandByAsn(714)?.category, "CDN")
|
||||
assert.equal(brandByAsn(36459)?.service, "GitHub")
|
||||
assert.equal(brandByAsn(395701)?.service, "Epic")
|
||||
assert.equal(brandByAsn(6507)?.service, "Riot")
|
||||
assert.equal(brandByAsn(33353)?.service, "PlayStation")
|
||||
assert.equal(brandByAsn(14061)?.service, "DigitalOcean")
|
||||
assert.equal(brandByAsn(24940)?.service, "Hetzner")
|
||||
assert.equal(brandByAsn(8403)?.service, "Spotify")
|
||||
assert.equal(brandByAsn(13414)?.service, "X")
|
||||
assert.equal(brandByAsn(47541)?.service, "VK")
|
||||
assert.equal(brandByAsn(47764)?.service, "VK")
|
||||
assert.equal(brandByAsn(30103)?.service, "Zoom")
|
||||
assert.equal(brandByAsn(19281)?.service, "Quad9")
|
||||
assert.equal(brandByAsn(9059)?.service, "AWS")
|
||||
assert.equal(brandByAsn(396982)?.service, "Google")
|
||||
assert.equal(brandByAsn(400645)?.service, "ChatGPT")
|
||||
|
||||
assert.equal(brandByHolder("VALVE-CORPORATION")?.service, "Steam")
|
||||
assert.equal(brandByHolder("OpenAI, LLC")?.service, "ChatGPT")
|
||||
assert.equal(brandByHolder("YouTube LLC")?.service, "YouTube")
|
||||
assert.equal(brandByHolder("AMAZON-AES - Amazon.com, Inc."), null)
|
||||
|
||||
assert.equal(isSteamGamePort(17, 27015, 50000), true)
|
||||
assert.equal(isSteamGamePort(6, 443, 50000), false)
|
||||
|
||||
assert.equal(resolveFlowBrand("104.18.35.51", 32590, "VALVE-CORPORATION", 6, 443, 1)?.service, "Cloudflare")
|
||||
assert.equal(resolveFlowBrand("203.0.113.9", 32590, "", 17, 27015, 50000)?.service, "Steam")
|
||||
assert.equal(resolveRipeCountry("", 9059, ""), "IE")
|
||||
assert.equal(resolveRipeCountry("", 24940, ""), "DE")
|
||||
|
||||
console.log("traffic-flow-brands.test.ts: ok")
|
||||
|
||||
@@ -7,59 +7,162 @@ export interface BrandHit {
|
||||
category: string
|
||||
}
|
||||
|
||||
const CDN = { category: "CDN" } as const
|
||||
const WEB = { category: "Веб" } as const
|
||||
const VIDEO = { category: "Видео / стриминг" } as const
|
||||
const GAMES = { category: "Игры" } as const
|
||||
const VOICE = { category: "Голос" } as const
|
||||
const AI = { category: "ИИ" } as const
|
||||
const DNS = { category: "DNS" } as const
|
||||
|
||||
const CLOUDFLARE: BrandHit = { service: "Cloudflare", ...CDN }
|
||||
const FASTLY: BrandHit = { service: "Fastly", ...CDN }
|
||||
const AKAMAI: BrandHit = { service: "Akamai", ...CDN }
|
||||
const AWS: BrandHit = { service: "AWS", ...CDN }
|
||||
const MICROSOFT: BrandHit = { service: "Microsoft", ...CDN }
|
||||
const YANDEX: BrandHit = { service: "Yandex", ...CDN }
|
||||
const APPLE: BrandHit = { service: "Apple", ...CDN }
|
||||
const DIGITALOCEAN: BrandHit = { service: "DigitalOcean", ...CDN }
|
||||
const HETZNER: BrandHit = { service: "Hetzner", ...CDN }
|
||||
const OVH: BrandHit = { service: "OVH", ...CDN }
|
||||
const ORACLE: BrandHit = { service: "Oracle", ...CDN }
|
||||
const LINODE: BrandHit = { service: "Linode", ...CDN }
|
||||
const VULTR: BrandHit = { service: "Vultr", ...CDN }
|
||||
const SCALEWAY: BrandHit = { service: "Scaleway", ...CDN }
|
||||
const IBM_CLOUD: BrandHit = { service: "IBM Cloud", ...CDN }
|
||||
const ALIBABA: BrandHit = { service: "Alibaba", ...CDN }
|
||||
const TENCENT: BrandHit = { service: "Tencent", ...CDN }
|
||||
const GCORE: BrandHit = { service: "G-Core", ...CDN }
|
||||
const CDN77: BrandHit = { service: "CDN77", ...CDN }
|
||||
const SELECTEL: BrandHit = { service: "Selectel", ...CDN }
|
||||
const TIMEWEB: BrandHit = { service: "Timeweb", ...CDN }
|
||||
const BEGET: BrandHit = { service: "Beget", ...CDN }
|
||||
const DDOS_GUARD: BrandHit = { service: "DDoS-Guard", ...CDN }
|
||||
const META: BrandHit = { service: "Meta", ...CDN }
|
||||
|
||||
const GOOGLE: BrandHit = { service: "Google", ...WEB }
|
||||
const GITHUB: BrandHit = { service: "GitHub", ...WEB }
|
||||
const GITLAB: BrandHit = { service: "GitLab", ...WEB }
|
||||
const X: BrandHit = { service: "X", ...WEB }
|
||||
const LINKEDIN: BrandHit = { service: "LinkedIn", ...WEB }
|
||||
const VK: BrandHit = { service: "VK", ...WEB }
|
||||
const REDDIT: BrandHit = { service: "Reddit", ...WEB }
|
||||
const DROPBOX: BrandHit = { service: "Dropbox", ...WEB }
|
||||
const SNAP: BrandHit = { service: "Snap", ...WEB }
|
||||
const WIKIPEDIA: BrandHit = { service: "Wikipedia", ...WEB }
|
||||
const PAYPAL: BrandHit = { service: "PayPal", ...WEB }
|
||||
const SALESFORCE: BrandHit = { service: "Salesforce", ...WEB }
|
||||
|
||||
const YOUTUBE: BrandHit = { service: "YouTube", ...VIDEO }
|
||||
const NETFLIX: BrandHit = { service: "Netflix", ...VIDEO }
|
||||
const TWITCH: BrandHit = { service: "Twitch", ...VIDEO }
|
||||
const TIKTOK: BrandHit = { service: "TikTok", ...VIDEO }
|
||||
const SPOTIFY: BrandHit = { service: "Spotify", ...VIDEO }
|
||||
|
||||
const STEAM: BrandHit = { service: "Steam", ...GAMES }
|
||||
const BLIZZARD: BrandHit = { service: "Blizzard", ...GAMES }
|
||||
const EPIC: BrandHit = { service: "Epic", ...GAMES }
|
||||
const RIOT: BrandHit = { service: "Riot", ...GAMES }
|
||||
const PLAYSTATION: BrandHit = { service: "PlayStation", ...GAMES }
|
||||
const ROBLOX: BrandHit = { service: "Roblox", ...GAMES }
|
||||
const UBISOFT: BrandHit = { service: "Ubisoft", ...GAMES }
|
||||
|
||||
const DISCORD: BrandHit = { service: "Discord", ...VOICE }
|
||||
const TELEGRAM: BrandHit = { service: "Telegram", ...VOICE }
|
||||
const ZOOM: BrandHit = { service: "Zoom", ...VOICE }
|
||||
|
||||
const CHATGPT: BrandHit = { service: "ChatGPT", ...AI }
|
||||
|
||||
const QUAD9: BrandHit = { service: "Quad9", ...DNS }
|
||||
const OPENDNS: BrandHit = { service: "OpenDNS", ...DNS }
|
||||
|
||||
function brandEntries(hit: BrandHit, asns: number[]): Array<[number, BrandHit]> {
|
||||
return asns.map((asn) => [asn, hit])
|
||||
}
|
||||
|
||||
function hqEntries(cc: string, asns: number[]): Array<[number, string]> {
|
||||
return asns.map((asn) => [asn, cc])
|
||||
}
|
||||
|
||||
const ASN_BRANDS = new Map<number, BrandHit>([
|
||||
[13335, { service: "Cloudflare", category: "CDN" }],
|
||||
[209242, { service: "Cloudflare", category: "CDN" }],
|
||||
[54113, { service: "Fastly", category: "CDN" }],
|
||||
[20940, { service: "Akamai", category: "CDN" }],
|
||||
[16509, { service: "AWS", category: "CDN" }],
|
||||
[14618, { service: "AWS", category: "CDN" }],
|
||||
[8075, { service: "Microsoft", category: "CDN" }],
|
||||
[13238, { service: "Yandex", category: "CDN" }],
|
||||
[32590, { service: "Steam", category: "Игры" }],
|
||||
[57976, { service: "Blizzard", category: "Игры" }],
|
||||
[2906, { service: "Netflix", category: "Видео / стриминг" }],
|
||||
[40027, { service: "Netflix", category: "Видео / стриминг" }],
|
||||
[15169, { service: "Google", category: "Веб" }],
|
||||
[36040, { service: "YouTube", category: "Видео / стриминг" }],
|
||||
[46489, { service: "Twitch", category: "Видео / стриминг" }],
|
||||
[401115, { service: "ChatGPT", category: "ИИ" }],
|
||||
[49544, { service: "Discord", category: "Голос" }],
|
||||
[62041, { service: "Telegram", category: "Голос" }],
|
||||
[59930, { service: "Telegram", category: "Голос" }],
|
||||
[211157, { service: "Telegram", category: "Голос" }],
|
||||
[32934, { service: "Meta", category: "CDN" }],
|
||||
[396986, { service: "TikTok", category: "Видео / стриминг" }],
|
||||
...brandEntries(CLOUDFLARE, [13335, 209242]),
|
||||
...brandEntries(FASTLY, [54113]),
|
||||
...brandEntries(AKAMAI, [20940, 16625, 32787, 35994, 16702, 24319]),
|
||||
...brandEntries(AWS, [16509, 14618, 8987, 7224, 9059]),
|
||||
...brandEntries(MICROSOFT, [8075, 8068, 8069, 8070]),
|
||||
...brandEntries(YANDEX, [13238]),
|
||||
...brandEntries(APPLE, [714, 6185]),
|
||||
...brandEntries(DIGITALOCEAN, [14061]),
|
||||
...brandEntries(HETZNER, [24940, 213230]),
|
||||
...brandEntries(OVH, [16276]),
|
||||
...brandEntries(ORACLE, [31898]),
|
||||
...brandEntries(LINODE, [63949]),
|
||||
...brandEntries(VULTR, [20473]),
|
||||
...brandEntries(SCALEWAY, [12876]),
|
||||
...brandEntries(IBM_CLOUD, [36351]),
|
||||
...brandEntries(ALIBABA, [45102]),
|
||||
...brandEntries(TENCENT, [132203]),
|
||||
...brandEntries(GCORE, [199524]),
|
||||
...brandEntries(CDN77, [60068]),
|
||||
...brandEntries(SELECTEL, [50340, 49505]),
|
||||
...brandEntries(TIMEWEB, [9123]),
|
||||
...brandEntries(BEGET, [198610]),
|
||||
...brandEntries(DDOS_GUARD, [57724]),
|
||||
...brandEntries(META, [32934, 63293, 54115]),
|
||||
...brandEntries(GOOGLE, [15169, 396982]),
|
||||
...brandEntries(GITHUB, [36459]),
|
||||
...brandEntries(GITLAB, [54876]),
|
||||
...brandEntries(X, [13414]),
|
||||
...brandEntries(LINKEDIN, [14413, 40793]),
|
||||
...brandEntries(VK, [47541, 47764]),
|
||||
...brandEntries(REDDIT, [394706]),
|
||||
...brandEntries(DROPBOX, [19679]),
|
||||
...brandEntries(SNAP, [19750]),
|
||||
...brandEntries(WIKIPEDIA, [14907]),
|
||||
...brandEntries(PAYPAL, [17012, 26101]),
|
||||
...brandEntries(SALESFORCE, [14340]),
|
||||
...brandEntries(YOUTUBE, [36040, 43515]),
|
||||
...brandEntries(NETFLIX, [2906, 40027]),
|
||||
...brandEntries(TWITCH, [46489]),
|
||||
...brandEntries(TIKTOK, [396986, 138699]),
|
||||
...brandEntries(SPOTIFY, [8403, 34081]),
|
||||
...brandEntries(STEAM, [32590]),
|
||||
...brandEntries(BLIZZARD, [57976]),
|
||||
...brandEntries(EPIC, [395701]),
|
||||
...brandEntries(RIOT, [6507, 62830]),
|
||||
...brandEntries(PLAYSTATION, [33353]),
|
||||
...brandEntries(ROBLOX, [22697]),
|
||||
...brandEntries(UBISOFT, [197922]),
|
||||
...brandEntries(DISCORD, [49544, 394141]),
|
||||
...brandEntries(TELEGRAM, [62041, 59930, 211157]),
|
||||
...brandEntries(ZOOM, [30103]),
|
||||
...brandEntries(CHATGPT, [401115, 400645]),
|
||||
...brandEntries(QUAD9, [19281]),
|
||||
...brandEntries(OPENDNS, [36692]),
|
||||
])
|
||||
|
||||
const ASN_HQ_COUNTRY = new Map<number, string>([
|
||||
[13335, "US"],
|
||||
[209242, "US"],
|
||||
[54113, "US"],
|
||||
[20940, "US"],
|
||||
[16509, "US"],
|
||||
[14618, "US"],
|
||||
[8075, "US"],
|
||||
[15169, "US"],
|
||||
[32590, "US"],
|
||||
[57976, "US"],
|
||||
[2906, "US"],
|
||||
[40027, "US"],
|
||||
[36040, "US"],
|
||||
[46489, "US"],
|
||||
[401115, "US"],
|
||||
[49544, "US"],
|
||||
[32934, "US"],
|
||||
[13238, "RU"],
|
||||
[62041, "NL"],
|
||||
[59930, "NL"],
|
||||
[211157, "NL"],
|
||||
...hqEntries("US", [
|
||||
13335, 209242, 54113, 20940, 16625, 32787, 35994, 16702, 24319,
|
||||
16509, 14618, 8987, 7224, 8075, 8068, 8069, 8070, 15169, 396982,
|
||||
32590, 57976, 2906, 40027, 36040, 43515, 46489, 401115, 400645,
|
||||
49544, 394141, 32934, 63293, 54115, 714, 6185, 36459, 54876, 14061,
|
||||
31898, 63949, 20473, 36351, 13414, 14413, 40793, 394706, 19679, 19750,
|
||||
14907, 17012, 26101, 14340, 30103, 36692, 395701, 6507, 62830, 33353, 22697,
|
||||
]),
|
||||
...hqEntries("IE", [9059]),
|
||||
...hqEntries("SG", [138699]),
|
||||
...hqEntries("DE", [24940, 213230]),
|
||||
...hqEntries("FR", [16276, 12876, 197922]),
|
||||
...hqEntries("CN", [45102, 132203]),
|
||||
...hqEntries("LU", [199524]),
|
||||
...hqEntries("CZ", [60068]),
|
||||
...hqEntries("RU", [13238, 50340, 49505, 9123, 198610, 57724, 47541, 47764]),
|
||||
...hqEntries("SE", [8403, 34081]),
|
||||
...hqEntries("NL", [62041, 59930, 211157]),
|
||||
...hqEntries("CH", [19281]),
|
||||
])
|
||||
|
||||
const GOOGLE: BrandHit = { service: "Google", category: "Веб" }
|
||||
const CLOUDFLARE: BrandHit = { service: "Cloudflare", category: "CDN" }
|
||||
const YOUTUBE: BrandHit = { service: "YouTube", category: "Видео / стриминг" }
|
||||
|
||||
const CIDR_BRANDS: Array<{ cidr: string; prefixLen: number; hit: BrandHit }> = [
|
||||
{ cidr: "104.16.0.0/13", prefixLen: 13, hit: CLOUDFLARE },
|
||||
{ cidr: "104.24.0.0/14", prefixLen: 14, hit: CLOUDFLARE },
|
||||
@@ -75,8 +178,25 @@ const CIDR_BRANDS: Array<{ cidr: string; prefixLen: number; hit: BrandHit }> = [
|
||||
{ cidr: "208.117.224.0/19", prefixLen: 19, hit: YOUTUBE },
|
||||
].sort((a, b) => b.prefixLen - a.prefixLen)
|
||||
|
||||
const HOLDER_BRANDS: Array<{ re: RegExp; hit: BrandHit }> = [
|
||||
{ re: /youtube/i, hit: YOUTUBE },
|
||||
{ re: /valve|\bsteam\b/i, hit: STEAM },
|
||||
{ re: /blizzard|battle.?net/i, hit: BLIZZARD },
|
||||
{ re: /openai/i, hit: CHATGPT },
|
||||
{ re: /riot games/i, hit: RIOT },
|
||||
{ re: /epic games/i, hit: EPIC },
|
||||
{ re: /\bapple\b/i, hit: APPLE },
|
||||
{ re: /github/i, hit: GITHUB },
|
||||
{ re: /spotify/i, hit: SPOTIFY },
|
||||
{ re: /twitter|\bx corp\b/i, hit: X },
|
||||
{ re: /dropbox/i, hit: DROPBOX },
|
||||
{ re: /akamai/i, hit: AKAMAI },
|
||||
]
|
||||
|
||||
const NON_ISO = new Set(["EU", "AP", "ZZ", "XX", "A1", "A2", "O1"])
|
||||
|
||||
const STEAM_ASN = 32590
|
||||
|
||||
export function isIsoCountry(code: string): boolean {
|
||||
const c = String(code ?? "").trim().toUpperCase()
|
||||
return /^[A-Z]{2}$/.test(c) && !NON_ISO.has(c)
|
||||
@@ -114,10 +234,51 @@ export function brandByCidr(ip: string): BrandHit | null {
|
||||
return null
|
||||
}
|
||||
|
||||
export function brandByHolder(holder: string): BrandHit | null {
|
||||
const h = String(holder ?? "").trim()
|
||||
if (!h) return null
|
||||
for (const row of HOLDER_BRANDS) {
|
||||
if (row.re.test(h)) return row.hit
|
||||
}
|
||||
return null
|
||||
}
|
||||
|
||||
/** Игровые порты Steam — только вместе с AS32590, никогда :80/:443. */
|
||||
export function isSteamGamePort(proto: number, dstPort: number, srcPort: number): boolean {
|
||||
if (proto !== 6 && proto !== 17) return false
|
||||
const port = dstPort || srcPort
|
||||
if (!port || port === 80 || port === 443) return false
|
||||
if (port === 4380 || port === 3478) return true
|
||||
return port >= 27000 && port <= 27100
|
||||
}
|
||||
|
||||
export function lookupBrand(ip: string, asn: number): BrandHit | null {
|
||||
return brandByCidr(ip) || brandByAsn(asn)
|
||||
}
|
||||
|
||||
/**
|
||||
* Cloudflare CIDR бьёт holder (витрина на CF не становится Steam).
|
||||
* Holder (YouTube и др.) бьёт остальные CIDR/ASN.
|
||||
* Порты Steam — только AS32590 и не выше Cloudflare CIDR.
|
||||
*/
|
||||
export function resolveFlowBrand(
|
||||
ip: string,
|
||||
asn: number,
|
||||
holder: string,
|
||||
proto = 0,
|
||||
dstPort = 0,
|
||||
srcPort = 0,
|
||||
): BrandHit | null {
|
||||
const cidrBrand = brandByCidr(ip)
|
||||
if (cidrBrand?.service === "Cloudflare") return cidrBrand
|
||||
const holderBrand = brandByHolder(holder)
|
||||
if (holderBrand) return holderBrand
|
||||
const fromLookup = cidrBrand || brandByAsn(asn)
|
||||
if (fromLookup) return fromLookup
|
||||
if (asn === STEAM_ASN && isSteamGamePort(proto, dstPort, srcPort)) return STEAM
|
||||
return null
|
||||
}
|
||||
|
||||
const SKIP_MAP_SERVICES = new Set([
|
||||
OTHER_SERVICE,
|
||||
"GRE",
|
||||
|
||||
@@ -53,6 +53,71 @@ const youtube = classifyFlowDst("173.194.160.163", 6, 443, 1, {
|
||||
assert.equal(youtube.service, "YouTube")
|
||||
assert.equal(youtube.category, "Видео / стриминг")
|
||||
|
||||
const valve = classifyFlowDst("203.0.113.40", 17, 27015, 50000, {
|
||||
prefix: "203.0.113.0/24",
|
||||
asn: 64501,
|
||||
country: "US",
|
||||
lat: null,
|
||||
lng: null,
|
||||
holder: "VALVE-CORPORATION",
|
||||
ok: true,
|
||||
fetchedAt: Date.now(),
|
||||
})
|
||||
assert.equal(valve.service, "Steam")
|
||||
assert.equal(valve.category, "Игры")
|
||||
|
||||
const openaiHolder = classifyFlowDst("203.0.113.41", 6, 443, 1, {
|
||||
prefix: "203.0.113.0/24",
|
||||
asn: 64502,
|
||||
country: "US",
|
||||
lat: null,
|
||||
lng: null,
|
||||
holder: "OPENAI, US",
|
||||
ok: true,
|
||||
fetchedAt: Date.now(),
|
||||
})
|
||||
assert.equal(openaiHolder.service, "ChatGPT")
|
||||
assert.equal(openaiHolder.category, "ИИ")
|
||||
|
||||
const cfNotSteam = classifyFlowDst("104.18.35.51", 6, 443, 1, {
|
||||
prefix: "104.18.0.0/16",
|
||||
asn: 32590,
|
||||
country: "US",
|
||||
lat: null,
|
||||
lng: null,
|
||||
holder: "VALVE-CORPORATION",
|
||||
ok: true,
|
||||
fetchedAt: Date.now(),
|
||||
})
|
||||
assert.equal(cfNotSteam.service, "Cloudflare")
|
||||
assert.notEqual(cfNotSteam.service, "Steam")
|
||||
|
||||
const awsIeu = classifyFlowDst("203.0.113.42", 6, 443, 1, {
|
||||
prefix: "203.0.113.0/24",
|
||||
asn: 9059,
|
||||
country: "IE",
|
||||
lat: null,
|
||||
lng: null,
|
||||
holder: "AMAZON-02",
|
||||
ok: true,
|
||||
fetchedAt: Date.now(),
|
||||
})
|
||||
assert.equal(awsIeu.service, "AWS")
|
||||
assert.equal(awsIeu.category, "CDN")
|
||||
|
||||
const googleCloud = classifyFlowDst("203.0.113.43", 6, 443, 1, {
|
||||
prefix: "203.0.113.0/24",
|
||||
asn: 396982,
|
||||
country: "US",
|
||||
lat: null,
|
||||
lng: null,
|
||||
holder: "GOOGLE-CLOUD",
|
||||
ok: true,
|
||||
fetchedAt: Date.now(),
|
||||
})
|
||||
assert.equal(googleCloud.service, "Google")
|
||||
assert.equal(googleCloud.category, "Веб")
|
||||
|
||||
const gre = classifyFlowDst("198.51.100.1", 47, 0, 0, null)
|
||||
assert.equal(gre.service, "GRE")
|
||||
assert.equal(gre.category, "Туннель")
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
import { lookupBrand, OTHER_SERVICE } from "./traffic-flow-brands.js"
|
||||
import { OTHER_SERVICE, resolveFlowBrand } from "./traffic-flow-brands.js"
|
||||
import { db } from "../db/index.js"
|
||||
import { evobgpSettings } from "../db/schema.js"
|
||||
import { ipInCidrV4, parseCidrV4 } from "./traffic-flow-ip.js"
|
||||
@@ -47,12 +47,13 @@ export function seedFlowCatalogForTests(input: {
|
||||
|
||||
export function categoryFromPurpose(purpose: string, proto: number, dstPort: number, srcPort: number): string {
|
||||
const p = purpose.toLowerCase()
|
||||
if (/gaming|steam|epic|riot/.test(p)) return "Игры"
|
||||
if (/streaming|youtube|netflix|twitch|video/.test(p)) return "Видео / стриминг"
|
||||
if (/cdn|cloudflare|akamai|fastly/.test(p)) return "CDN"
|
||||
if (/gaming|steam|epic|riot|playstation|roblox|ubisoft/.test(p)) return "Игры"
|
||||
if (/streaming|youtube|netflix|twitch|video|spotify/.test(p)) return "Видео / стриминг"
|
||||
if (/cdn|cloudflare|akamai|fastly|hetzner|ovh|apple/.test(p)) return "CDN"
|
||||
if (/voip|discord|zoom/.test(p)) return "Голос"
|
||||
if (/openai|chatgpt|\bai\b/.test(p)) return "ИИ"
|
||||
if (/веб|web|google/.test(p)) return "Веб"
|
||||
if (/quad9|opendns/.test(p)) return "DNS"
|
||||
if (/веб|web|google|github|paypal|vk|linkedin/.test(p)) return "Веб"
|
||||
const app = applicationName(proto, dstPort, srcPort)
|
||||
if (app === "DNS" || app === "SSH" || app === "BGP") return app
|
||||
if (app === "GRE" || app === "ESP" || app === "WireGuard") return "Туннель"
|
||||
@@ -79,10 +80,7 @@ export function classifyFlowDst(
|
||||
if (app === "WireGuard") return { service: "WireGuard", category: "Туннель" }
|
||||
const hit = matchCidr(dst)
|
||||
const holder = ripe?.holder ?? ""
|
||||
const youtubeHolder = /youtube/i.test(holder)
|
||||
const brand = youtubeHolder
|
||||
? { service: "YouTube", category: "Видео / стриминг" }
|
||||
: lookupBrand(dst, ripe?.asn ?? 0)
|
||||
const brand = resolveFlowBrand(dst, ripe?.asn ?? 0, holder, proto, dstPort, srcPort)
|
||||
const asnName = ripe?.asn ? asnPurpose.get(ripe.asn) : undefined
|
||||
const service = (hit?.purpose || brand?.service || asnName || OTHER_SERVICE).trim() || OTHER_SERVICE
|
||||
const category = hit
|
||||
|
||||
@@ -60,7 +60,7 @@ function stopListener(): void {
|
||||
clearInterval(flushTimer)
|
||||
flushTimer = null
|
||||
}
|
||||
void flushPending().catch((e) => {
|
||||
void flushPending({ force: true }).catch((e) => {
|
||||
setEngineError(e instanceof Error ? e.message : String(e))
|
||||
})
|
||||
if (socket) {
|
||||
|
||||
@@ -1,5 +1,10 @@
|
||||
import assert from "node:assert/strict"
|
||||
import { dedupFlowRowsMaxBytes, flowTupleKey } from "./traffic-flow-dedup.js"
|
||||
import {
|
||||
dedupFlowRowsAcrossExporters,
|
||||
dedupFlowRowsMaxBytes,
|
||||
flowConversationKey,
|
||||
flowTupleKey,
|
||||
} from "./traffic-flow-dedup.js"
|
||||
|
||||
const a = {
|
||||
serverId: 7,
|
||||
@@ -22,4 +27,13 @@ assert.equal(flowTupleKey(a), flowTupleKey(b))
|
||||
const sameIface = dedupFlowRowsMaxBytes([a, { ...a, bytes: 3_000, packets: 2 }])
|
||||
assert.equal(sameIface[0]?.bytes, 15_000)
|
||||
|
||||
const jh = { ...a, serverId: 7, bytes: 9_000 }
|
||||
const en = { ...a, serverId: 9, bytes: 11_000, inIface: "1" }
|
||||
assert.equal(flowConversationKey(jh), flowConversationKey(en))
|
||||
assert.notEqual(flowTupleKey(jh), flowTupleKey(en))
|
||||
const across = dedupFlowRowsAcrossExporters([en, jh], (x, y) => (x.serverId === 7 ? x : y))
|
||||
assert.equal(across.length, 1)
|
||||
assert.equal(across[0]?.serverId, 7)
|
||||
assert.equal(across[0]?.bytes, 9_000)
|
||||
|
||||
console.log("traffic-flow-dedup.test.ts: ok")
|
||||
|
||||
@@ -14,6 +14,32 @@ export function flowTupleKey(r: Pick<FlowTupleRow, "serverId" | "src" | "dst" |
|
||||
return `${r.serverId}|${r.src}|${r.dst}|${r.proto}|${r.srcPort}|${r.dstPort}`
|
||||
}
|
||||
|
||||
/** Один разговор на всех экспортёрах (JH+EN), без serverId. */
|
||||
export function flowConversationKey(r: Pick<FlowTupleRow, "src" | "dst" | "proto" | "srcPort" | "dstPort">): string {
|
||||
return `${r.src}|${r.dst}|${r.proto}|${r.srcPort}|${r.dstPort}`
|
||||
}
|
||||
|
||||
/**
|
||||
* Схлопнуть копии одного 5-tuple с разных серверов.
|
||||
* `prefer` выбирает ряд (клиент на JH важнее голого EN).
|
||||
*/
|
||||
export function dedupFlowRowsAcrossExporters<T extends FlowTupleRow>(
|
||||
rows: T[],
|
||||
prefer: (a: T, b: T) => T,
|
||||
): T[] {
|
||||
const byConv = new Map<string, T>()
|
||||
for (const row of rows) {
|
||||
const key = flowConversationKey(row)
|
||||
const prev = byConv.get(key)
|
||||
if (!prev) {
|
||||
byConv.set(key, row)
|
||||
continue
|
||||
}
|
||||
byConv.set(key, prefer(prev, row))
|
||||
}
|
||||
return [...byConv.values()]
|
||||
}
|
||||
|
||||
function ifaceKey(r: FlowTupleRow): string {
|
||||
return `${flowTupleKey(r)}|${r.inIface}`
|
||||
}
|
||||
|
||||
@@ -5,13 +5,24 @@ import { classifyFlowPlaneLite } from "./traffic-flow-planes.js"
|
||||
import { pickServerIdForExporter, type OverlayPeerRef } from "./traffic-flow-map-exporter.js"
|
||||
import { applicationName } from "./traffic-flow-apps.js"
|
||||
import { classifyFlowDst } from "./traffic-flow-classify.js"
|
||||
import { enqueueRipeMisses, lookupRipeCached, pruneRipeSqlite } from "./traffic-flow-ripe.js"
|
||||
import { enqueueRipeMisses, pruneRipeSqlite } from "./traffic-flow-ripe.js"
|
||||
import { resolveFlowIp } from "./traffic-flow-geoip.js"
|
||||
import { invalidateTrafficFlowSettingsCache } from "./traffic-flow-settings.js"
|
||||
import { isIsoCountry } from "./traffic-flow-brands.js"
|
||||
import { maybeRefreshIfaces } from "./traffic-flow-ifaces.js"
|
||||
import { canonicalFactIface } from "./traffic-flow-ifindex.js"
|
||||
import { pickInternetPeer } from "./traffic-flow-ip.js"
|
||||
import {
|
||||
bumpFlowFact,
|
||||
factsPendingSize,
|
||||
flushFlowFacts,
|
||||
hourBucketIso,
|
||||
resetFactsForTests,
|
||||
} from "./traffic-flow-facts.js"
|
||||
|
||||
export const TICK_MS = 2_000
|
||||
export const PERSIST_MS = 10_000
|
||||
export const STATS_PERSIST_MS = 15_000
|
||||
export const RING_LEN = 60
|
||||
export const MAX_PENDING = 50_000
|
||||
export const DAILY_ASN_TOP = 500
|
||||
@@ -48,6 +59,66 @@ export interface PendingFlowRow {
|
||||
flowEndMs: number
|
||||
}
|
||||
|
||||
function inetOrNull(value: string | null | undefined): string | null {
|
||||
const s = String(value ?? "").trim()
|
||||
return s.length > 0 ? s : null
|
||||
}
|
||||
|
||||
export function isValidFlowInet(value: string): boolean {
|
||||
const s = value.trim()
|
||||
if (!s) return false
|
||||
const v4 = /^(\d{1,3})\.(\d{1,3})\.(\d{1,3})\.(\d{1,3})$/.exec(s)
|
||||
if (v4) {
|
||||
return v4.slice(1).every((octet) => {
|
||||
const n = Number(octet)
|
||||
return Number.isInteger(n) && n >= 0 && n <= 255
|
||||
})
|
||||
}
|
||||
if (!s.includes(":")) return false
|
||||
if (!/^[0-9a-fA-F:]+$/.test(s)) return false
|
||||
const parts = s.split(":")
|
||||
if (parts.length < 3 || parts.length > 8) return false
|
||||
return parts.every((p) => p.length <= 4)
|
||||
}
|
||||
|
||||
function clampProto(n: number): number {
|
||||
if (!Number.isFinite(n)) return 0
|
||||
return Math.max(0, Math.min(255, Math.trunc(n)))
|
||||
}
|
||||
|
||||
function sanitizeFlowRow(r: PendingFlowRow): PendingFlowRow | null {
|
||||
const src = (r.src || "").trim() || "0.0.0.0"
|
||||
const dst = (r.dst || "").trim() || "0.0.0.0"
|
||||
if (!isValidFlowInet(src) || !isValidFlowInet(dst)) return null
|
||||
const next = inetOrNull(r.nextHop)
|
||||
return {
|
||||
...r,
|
||||
src,
|
||||
dst,
|
||||
nextHop: next && isValidFlowInet(next) ? next : "",
|
||||
proto: clampProto(r.proto),
|
||||
}
|
||||
}
|
||||
|
||||
function flowUpsertParams(r: PendingFlowRow) {
|
||||
return {
|
||||
serverId: r.serverId,
|
||||
bucketAt: r.bucketAt,
|
||||
src: r.src,
|
||||
dst: r.dst,
|
||||
proto: r.proto,
|
||||
srcPort: r.srcPort,
|
||||
dstPort: r.dstPort,
|
||||
bytes: r.bytes,
|
||||
packets: r.packets,
|
||||
inIface: r.inIface,
|
||||
outIface: r.outIface,
|
||||
nextHop: inetOrNull(r.nextHop),
|
||||
flowStartMs: r.flowStartMs,
|
||||
flowEndMs: r.flowEndMs,
|
||||
}
|
||||
}
|
||||
|
||||
export interface EngineStats {
|
||||
packetsReceived: number
|
||||
lastExporterIp: string | null
|
||||
@@ -105,6 +176,9 @@ let rowsStored = 0
|
||||
let lastFlushUsedTransaction = false
|
||||
let lastPruneAt = 0
|
||||
let lastPassiveCheckpointAt = 0
|
||||
let lastPersistAt = 0
|
||||
let lastFlushedMinute = ""
|
||||
let lastStatsPersistAt = 0
|
||||
let dataEpoch = 0
|
||||
let lastPersistedStats: {
|
||||
packetsReceived: number
|
||||
@@ -262,13 +336,14 @@ export function getEngineStats(): EngineStats {
|
||||
export function queueParsedFlows(serverId: number, flows: ParsedFlowInput[]): void {
|
||||
if (flows.length) bumpDataEpoch()
|
||||
const bucketAt = minuteBucketIso()
|
||||
const hourAt = hourBucketIso()
|
||||
const ripeMisses: string[] = []
|
||||
for (const raw of flows) {
|
||||
const flow = normalizeParsedFlow(raw)
|
||||
addToTick(serverId, flow, flow.bytes)
|
||||
bumpRollup(serverId, bucketAt, flow, flow.bytes, flow.packets)
|
||||
const peer = pickInternetPeer(flow.src, flow.dst, flow.srcPort, flow.dstPort)
|
||||
const ripe = lookupRipeCached(peer)
|
||||
const ripe = resolveFlowIp(peer)
|
||||
if (peer && !ripe) ripeMisses.push(peer)
|
||||
const classified = classifyFlowDst(peer, flow.proto, flow.dstPort, flow.srcPort, ripe)
|
||||
const app = applicationName(flow.proto, flow.dstPort, flow.srcPort)
|
||||
@@ -283,6 +358,16 @@ export function queueParsedFlows(serverId: number, flows: ParsedFlowInput[]): vo
|
||||
bumpDim(serverId, bucketAt, "service", classified.service, flow.bytes, flow.packets)
|
||||
if (country) bumpDim(serverId, bucketAt, "country", country, flow.bytes, flow.packets)
|
||||
bumpDim(serverId, bucketAt, "asn", asnKey, flow.bytes, flow.packets)
|
||||
bumpFlowFact({
|
||||
serverId,
|
||||
bucketAt: hourAt,
|
||||
iface: canonicalFactIface(serverId, flow.inIface),
|
||||
country: country || "XX",
|
||||
service: classified.service,
|
||||
asn: ripe?.ok && ripe.asn ? ripe.asn : 0,
|
||||
bytes: flow.bytes,
|
||||
packets: flow.packets,
|
||||
})
|
||||
|
||||
const key = pendingKey(serverId, bucketAt, flow)
|
||||
const prev = pending.get(key)
|
||||
@@ -449,7 +534,7 @@ export function applyRingSnapshot(rows: Array<{ key: string; inBps: number[]; ou
|
||||
}
|
||||
}
|
||||
|
||||
async function persistListenerStats(): Promise<boolean> {
|
||||
async function persistListenerStats(force = false): Promise<boolean> {
|
||||
if (
|
||||
lastPersistedStats
|
||||
&& lastPersistedStats.packetsReceived === packetsReceived
|
||||
@@ -459,6 +544,12 @@ async function persistListenerStats(): Promise<boolean> {
|
||||
) {
|
||||
return false
|
||||
}
|
||||
const errorChanged = lastPersistedStats?.lastError !== lastError
|
||||
const exporterChanged = lastPersistedStats?.lastExporterIp !== lastExporterIp
|
||||
const now = Date.now()
|
||||
if (!force && !errorChanged && !exporterChanged && now - lastStatsPersistAt < STATS_PERSIST_MS) {
|
||||
return false
|
||||
}
|
||||
await dbQuery(`
|
||||
UPDATE traffic_flow_settings
|
||||
SET packets_received = @packetsReceived,
|
||||
@@ -480,6 +571,7 @@ async function persistListenerStats(): Promise<boolean> {
|
||||
lastExporterIp,
|
||||
lastError,
|
||||
}
|
||||
lastStatsPersistAt = now
|
||||
invalidateTrafficFlowSettingsCache()
|
||||
return true
|
||||
}
|
||||
@@ -644,20 +736,70 @@ function topNPending(rows: PendingFlowRow[]): PendingFlowRow[] {
|
||||
return out
|
||||
}
|
||||
|
||||
export async function flushPending(): Promise<void> {
|
||||
pruneRecent()
|
||||
rollFlowRings()
|
||||
await persistListenerStats()
|
||||
if (pending.size === 0 && minuteRollup.size === 0 && minuteDims.size === 0) {
|
||||
await pruneStored()
|
||||
lastFlushUsedTransaction = false
|
||||
return
|
||||
}
|
||||
const rows = topNPending([...pending.values()].map(toPendingRow))
|
||||
pending.clear()
|
||||
for (const row of rows) mergeInto(recent, row)
|
||||
function persistDue(force: boolean, hasWork: boolean): boolean {
|
||||
if (force) return true
|
||||
if (!hasWork) return false
|
||||
if (minuteBucketIso() !== lastFlushedMinute) return true
|
||||
return Date.now() - lastPersistAt >= PERSIST_MS
|
||||
}
|
||||
|
||||
const upsertSql = `
|
||||
async function upsertFlowBucketsBatch(rows: PendingFlowRow[]): Promise<void> {
|
||||
if (rows.length === 0) return
|
||||
const days = new Set(rows.map((r) => r.bucketAt))
|
||||
for (const bucketAt of days) await ensureParentPartition("flow_buckets", bucketAt)
|
||||
await pool.query({
|
||||
text: `
|
||||
INSERT INTO flow_buckets (
|
||||
server_id, bucket_at, src, dst, proto, src_port, dst_port, bytes, packets, in_iface, out_iface, next_hop, flow_start_ms, flow_end_ms
|
||||
)
|
||||
SELECT *
|
||||
FROM UNNEST(
|
||||
$1::bigint[],
|
||||
$2::timestamptz[],
|
||||
$3::inet[],
|
||||
$4::inet[],
|
||||
$5::smallint[],
|
||||
$6::int[],
|
||||
$7::int[],
|
||||
$8::bigint[],
|
||||
$9::bigint[],
|
||||
$10::text[],
|
||||
$11::text[],
|
||||
$12::inet[],
|
||||
$13::bigint[],
|
||||
$14::bigint[]
|
||||
) AS t(server_id, bucket_at, src, dst, proto, src_port, dst_port, bytes, packets, in_iface, out_iface, next_hop, flow_start_ms, flow_end_ms)
|
||||
ON CONFLICT (server_id, bucket_at, src, dst, proto, src_port, dst_port, in_iface)
|
||||
DO UPDATE SET
|
||||
bytes = flow_buckets.bytes + excluded.bytes,
|
||||
packets = flow_buckets.packets + excluded.packets,
|
||||
out_iface = CASE WHEN excluded.out_iface != '' THEN excluded.out_iface ELSE flow_buckets.out_iface END,
|
||||
next_hop = COALESCE(excluded.next_hop, flow_buckets.next_hop),
|
||||
flow_start_ms = CASE
|
||||
WHEN excluded.flow_start_ms > 0 AND (flow_buckets.flow_start_ms = 0 OR excluded.flow_start_ms < flow_buckets.flow_start_ms)
|
||||
THEN excluded.flow_start_ms ELSE flow_buckets.flow_start_ms END,
|
||||
flow_end_ms = GREATEST(flow_buckets.flow_end_ms, excluded.flow_end_ms)
|
||||
`,
|
||||
values: [
|
||||
rows.map((r) => r.serverId),
|
||||
rows.map((r) => r.bucketAt),
|
||||
rows.map((r) => r.src),
|
||||
rows.map((r) => r.dst),
|
||||
rows.map((r) => r.proto),
|
||||
rows.map((r) => r.srcPort),
|
||||
rows.map((r) => r.dstPort),
|
||||
rows.map((r) => r.bytes),
|
||||
rows.map((r) => r.packets),
|
||||
rows.map((r) => r.inIface),
|
||||
rows.map((r) => r.outIface),
|
||||
rows.map((r) => inetOrNull(r.nextHop)),
|
||||
rows.map((r) => r.flowStartMs),
|
||||
rows.map((r) => r.flowEndMs),
|
||||
],
|
||||
})
|
||||
}
|
||||
|
||||
const FLOW_UPSERT_SQL = `
|
||||
INSERT INTO flow_buckets (
|
||||
server_id, bucket_at, src, dst, proto, src_port, dst_port, bytes, packets, in_iface, out_iface, next_hop, flow_start_ms, flow_end_ms
|
||||
) VALUES (
|
||||
@@ -668,61 +810,93 @@ export async function flushPending(): Promise<void> {
|
||||
bytes = flow_buckets.bytes + excluded.bytes,
|
||||
packets = flow_buckets.packets + excluded.packets,
|
||||
out_iface = CASE WHEN excluded.out_iface != '' THEN excluded.out_iface ELSE flow_buckets.out_iface END,
|
||||
next_hop = CASE WHEN excluded.next_hop != '' THEN excluded.next_hop ELSE flow_buckets.next_hop END,
|
||||
next_hop = COALESCE(excluded.next_hop, flow_buckets.next_hop),
|
||||
flow_start_ms = CASE
|
||||
WHEN excluded.flow_start_ms > 0 AND (flow_buckets.flow_start_ms = 0 OR excluded.flow_start_ms < flow_buckets.flow_start_ms)
|
||||
THEN excluded.flow_start_ms ELSE flow_buckets.flow_start_ms END,
|
||||
flow_end_ms = GREATEST(flow_buckets.flow_end_ms, excluded.flow_end_ms)
|
||||
`
|
||||
lastFlushUsedTransaction = false
|
||||
|
||||
async function upsertFlowBuckets(rows: PendingFlowRow[]): Promise<number> {
|
||||
if (rows.length === 0) return 0
|
||||
try {
|
||||
for (const r of rows) {
|
||||
await ensureParentPartition("flow_buckets", r.bucketAt)
|
||||
await dbQuery(upsertSql, {
|
||||
serverId: r.serverId,
|
||||
bucketAt: r.bucketAt,
|
||||
src: r.src,
|
||||
dst: r.dst,
|
||||
proto: r.proto,
|
||||
srcPort: r.srcPort,
|
||||
dstPort: r.dstPort,
|
||||
bytes: r.bytes,
|
||||
packets: r.packets,
|
||||
inIface: r.inIface,
|
||||
outIface: r.outIface,
|
||||
nextHop: r.nextHop,
|
||||
flowStartMs: r.flowStartMs,
|
||||
flowEndMs: r.flowEndMs,
|
||||
})
|
||||
}
|
||||
lastFlushUsedTransaction = true
|
||||
rowsStored += rows.length
|
||||
bumpDataEpoch()
|
||||
} catch {
|
||||
await upsertFlowBucketsBatch(rows)
|
||||
return rows.length
|
||||
} catch (err) {
|
||||
const message = err instanceof Error ? err.message : String(err)
|
||||
setEngineError(`flow_buckets: ${message}`)
|
||||
let stored = 0
|
||||
for (const r of rows) {
|
||||
try {
|
||||
await ensureParentPartition("flow_buckets", r.bucketAt)
|
||||
await dbQuery(upsertSql, {
|
||||
serverId: r.serverId,
|
||||
bucketAt: r.bucketAt,
|
||||
src: r.src,
|
||||
dst: r.dst,
|
||||
proto: r.proto,
|
||||
srcPort: r.srcPort,
|
||||
dstPort: r.dstPort,
|
||||
bytes: r.bytes,
|
||||
packets: r.packets,
|
||||
inIface: r.inIface,
|
||||
outIface: r.outIface,
|
||||
nextHop: r.nextHop,
|
||||
flowStartMs: r.flowStartMs,
|
||||
flowEndMs: r.flowEndMs,
|
||||
})
|
||||
rowsStored += 1
|
||||
} catch {
|
||||
/* ignore single-row failures */
|
||||
await dbQuery(FLOW_UPSERT_SQL, flowUpsertParams(r))
|
||||
stored += 1
|
||||
} catch (rowErr) {
|
||||
if (stored === 0 && lastError.startsWith("flow_buckets:")) {
|
||||
const rowMsg = rowErr instanceof Error ? rowErr.message : String(rowErr)
|
||||
setEngineError(`flow_buckets: ${rowMsg}`)
|
||||
}
|
||||
}
|
||||
}
|
||||
return stored
|
||||
}
|
||||
}
|
||||
|
||||
export async function flushPending(opts?: { force?: boolean }): Promise<void> {
|
||||
pruneRecent()
|
||||
rollFlowRings()
|
||||
const force = Boolean(opts?.force)
|
||||
const hasWork = pending.size > 0 || minuteRollup.size > 0 || minuteDims.size > 0 || factsPendingSize() > 0
|
||||
const due = persistDue(force, hasWork)
|
||||
try {
|
||||
await persistListenerStats(force)
|
||||
} catch {
|
||||
/* settings row may be absent in unit tests */
|
||||
}
|
||||
|
||||
if (!hasWork) {
|
||||
if (force) {
|
||||
try {
|
||||
await pruneStored()
|
||||
} catch {
|
||||
/* prune best-effort */
|
||||
}
|
||||
}
|
||||
lastFlushUsedTransaction = false
|
||||
return
|
||||
}
|
||||
if (!due) {
|
||||
lastFlushUsedTransaction = false
|
||||
return
|
||||
}
|
||||
|
||||
const sanitized: PendingFlowRow[] = []
|
||||
let skippedInet = 0
|
||||
for (const row of topNPending([...pending.values()].map(toPendingRow))) {
|
||||
const clean = sanitizeFlowRow(row)
|
||||
if (!clean) {
|
||||
skippedInet += 1
|
||||
continue
|
||||
}
|
||||
sanitized.push(clean)
|
||||
}
|
||||
pending.clear()
|
||||
for (const row of sanitized) mergeInto(recent, row)
|
||||
if (skippedInet > 0) {
|
||||
setEngineError(`flow_buckets: пропуск ${skippedInet} строк с невалидным IP`)
|
||||
}
|
||||
|
||||
lastFlushUsedTransaction = false
|
||||
lastPersistAt = Date.now()
|
||||
lastFlushedMinute = minuteBucketIso()
|
||||
try {
|
||||
const stored = await upsertFlowBuckets(sanitized)
|
||||
lastFlushUsedTransaction = stored === sanitized.length
|
||||
rowsStored += stored
|
||||
if (stored > 0) bumpDataEpoch()
|
||||
} catch (err) {
|
||||
const message = err instanceof Error ? err.message : String(err)
|
||||
setEngineError(`flow_buckets: ${message}`)
|
||||
}
|
||||
try {
|
||||
await upsertMinuteAndDaily()
|
||||
@@ -730,7 +904,16 @@ export async function flushPending(): Promise<void> {
|
||||
} catch {
|
||||
/* rollup best-effort */
|
||||
}
|
||||
await pruneStored()
|
||||
try {
|
||||
await flushFlowFacts()
|
||||
} catch {
|
||||
/* statistics cube best-effort */
|
||||
}
|
||||
try {
|
||||
await pruneStored()
|
||||
} catch {
|
||||
/* prune best-effort */
|
||||
}
|
||||
}
|
||||
|
||||
export function lastFlushUsedTransactionForTests(): boolean {
|
||||
@@ -738,7 +921,7 @@ export function lastFlushUsedTransactionForTests(): boolean {
|
||||
}
|
||||
|
||||
export async function flushPendingForTests(): Promise<void> {
|
||||
await flushPending()
|
||||
await flushPending({ force: true })
|
||||
}
|
||||
|
||||
export function onEngineTick(): void {
|
||||
@@ -757,6 +940,7 @@ export function resetEngineForTests(): void {
|
||||
rings.clear()
|
||||
minuteRollup.clear()
|
||||
minuteDims.clear()
|
||||
resetFactsForTests()
|
||||
packetsReceived = 0
|
||||
lastExporterIp = null
|
||||
lastError = ""
|
||||
@@ -766,6 +950,9 @@ export function resetEngineForTests(): void {
|
||||
lastFlushUsedTransaction = false
|
||||
lastPruneAt = 0
|
||||
lastPassiveCheckpointAt = Date.now()
|
||||
lastPersistAt = 0
|
||||
lastFlushedMinute = ""
|
||||
lastStatsPersistAt = 0
|
||||
lastPersistedStats = null
|
||||
bumpDataEpoch()
|
||||
pendingCap = MAX_PENDING
|
||||
@@ -778,3 +965,25 @@ export function pendingSizeForTests(): number {
|
||||
export function droppedForTests(): number {
|
||||
return dropped
|
||||
}
|
||||
|
||||
/** Снимок минутных dims (dim → key → bytes) для тестов обогащения потоков. */
|
||||
export function minuteDimsSnapshotForTests(): Map<string, Map<string, { bytes: number; packets: number }>> {
|
||||
const out = new Map<string, Map<string, { bytes: number; packets: number }>>()
|
||||
for (const [k, acc] of minuteDims) {
|
||||
// dimKey: serverId\0bucketAt\0dim\0key
|
||||
const parts = k.split("\0")
|
||||
const dim = parts[2] ?? ""
|
||||
const key = parts.slice(3).join("\0")
|
||||
let byKey = out.get(dim)
|
||||
if (!byKey) {
|
||||
byKey = new Map()
|
||||
out.set(dim, byKey)
|
||||
}
|
||||
const prev = byKey.get(key)
|
||||
byKey.set(key, {
|
||||
bytes: (prev?.bytes ?? 0) + acc.bytes,
|
||||
packets: (prev?.packets ?? 0) + acc.packets,
|
||||
})
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
@@ -0,0 +1,71 @@
|
||||
import assert from "node:assert/strict"
|
||||
import {
|
||||
bumpFlowFact,
|
||||
collectCappedFacts,
|
||||
FACT_ASN_TOP,
|
||||
FACT_TUPLE_CAP,
|
||||
resetFactsForTests,
|
||||
} from "./traffic-flow-facts.js"
|
||||
|
||||
resetFactsForTests()
|
||||
bumpFlowFact({
|
||||
serverId: 1,
|
||||
bucketAt: "2026-09-10T10:00:00.000Z",
|
||||
iface: "ether1",
|
||||
country: "US",
|
||||
service: "https",
|
||||
asn: 15169,
|
||||
bytes: 100,
|
||||
packets: 2,
|
||||
})
|
||||
bumpFlowFact({
|
||||
serverId: 1,
|
||||
bucketAt: "2026-09-10T10:00:00.000Z",
|
||||
iface: "ether1",
|
||||
country: "us",
|
||||
service: "https",
|
||||
asn: 15169,
|
||||
bytes: 50,
|
||||
packets: 1,
|
||||
})
|
||||
const merged = collectCappedFacts()
|
||||
assert.equal(merged.length, 1)
|
||||
assert.equal(merged[0]?.bytes, 150)
|
||||
assert.equal(merged[0]?.country, "US")
|
||||
assert.equal(merged[0]?.asn, 15169)
|
||||
|
||||
resetFactsForTests()
|
||||
for (let i = 1; i <= FACT_ASN_TOP + 20; i++) {
|
||||
bumpFlowFact({
|
||||
serverId: 2,
|
||||
bucketAt: "2026-09-10T11:00:00.000Z",
|
||||
iface: "ether1",
|
||||
country: "DE",
|
||||
service: "https",
|
||||
asn: i,
|
||||
bytes: FACT_ASN_TOP + 21 - i,
|
||||
packets: 1,
|
||||
})
|
||||
}
|
||||
const cappedAsn = collectCappedFacts()
|
||||
const asns = new Set(cappedAsn.map((r) => r.asn))
|
||||
assert.ok(asns.has(0))
|
||||
assert.ok(asns.size <= FACT_ASN_TOP + 1)
|
||||
|
||||
resetFactsForTests()
|
||||
for (let i = 0; i < FACT_TUPLE_CAP + 30; i++) {
|
||||
bumpFlowFact({
|
||||
serverId: 3,
|
||||
bucketAt: "2026-09-10T12:00:00.000Z",
|
||||
iface: `ether${i % 3}`,
|
||||
country: "NL",
|
||||
service: `svc-${i}`,
|
||||
asn: 1,
|
||||
bytes: 10,
|
||||
packets: 1,
|
||||
})
|
||||
}
|
||||
const cappedTuples = collectCappedFacts()
|
||||
assert.ok(cappedTuples.length <= FACT_TUPLE_CAP + 3)
|
||||
|
||||
console.log("traffic-flow-facts.test.ts: ok")
|
||||
@@ -0,0 +1,285 @@
|
||||
import { pool } from "../db/index.js"
|
||||
import { ensurePartitionFor, specForParent } from "../db/partitions.js"
|
||||
|
||||
export const FACT_ASN_TOP = 200
|
||||
export const FACT_TUPLE_CAP = 8000
|
||||
export const FACT_SERVICE_MAX_LEN = 64
|
||||
export const UNKNOWN_COUNTRY = "XX"
|
||||
export const OTHER_SERVICE = "other"
|
||||
export const UNKNOWN_IFACE = "__unknown__"
|
||||
|
||||
export interface FactAcc {
|
||||
bytes: number
|
||||
packets: number
|
||||
}
|
||||
|
||||
export interface FactRow {
|
||||
serverId: number
|
||||
bucketAt: string
|
||||
iface: string
|
||||
country: string
|
||||
service: string
|
||||
asn: number
|
||||
bytes: number
|
||||
packets: number
|
||||
}
|
||||
|
||||
const hourFacts = new Map<string, FactAcc>()
|
||||
const ensuredParts = new Set<string>()
|
||||
|
||||
export function hourBucketIso(at = Date.now()): string {
|
||||
const d = new Date(at)
|
||||
d.setMinutes(0, 0, 0)
|
||||
return d.toISOString()
|
||||
}
|
||||
|
||||
export function normalizeFactCountry(raw: string): string {
|
||||
const iso = raw.trim().toUpperCase()
|
||||
if (/^[A-Z]{2}$/.test(iso)) return iso
|
||||
return UNKNOWN_COUNTRY
|
||||
}
|
||||
|
||||
export function normalizeFactService(raw: string): string {
|
||||
const s = raw.trim().slice(0, FACT_SERVICE_MAX_LEN)
|
||||
return s || OTHER_SERVICE
|
||||
}
|
||||
|
||||
export function normalizeFactIface(raw: string): string {
|
||||
return raw.trim() || UNKNOWN_IFACE
|
||||
}
|
||||
|
||||
export function normalizeFactAsn(raw: number): number {
|
||||
if (!Number.isFinite(raw) || raw <= 0) return 0
|
||||
return Math.trunc(raw)
|
||||
}
|
||||
|
||||
function factKey(
|
||||
serverId: number,
|
||||
bucketAt: string,
|
||||
iface: string,
|
||||
country: string,
|
||||
service: string,
|
||||
asn: number,
|
||||
): string {
|
||||
return `${serverId}\0${bucketAt}\0${iface}\0${country}\0${service}\0${asn}`
|
||||
}
|
||||
|
||||
function parseFactKey(k: string, acc: FactAcc): FactRow | null {
|
||||
const parts = k.split("\0")
|
||||
if (parts.length !== 6) return null
|
||||
const serverId = Number(parts[0])
|
||||
const asn = Number(parts[5])
|
||||
if (!Number.isFinite(serverId) || !Number.isFinite(asn)) return null
|
||||
return {
|
||||
serverId,
|
||||
bucketAt: parts[1] ?? "",
|
||||
iface: parts[2] ?? UNKNOWN_IFACE,
|
||||
country: parts[3] ?? UNKNOWN_COUNTRY,
|
||||
service: parts[4] ?? OTHER_SERVICE,
|
||||
asn,
|
||||
bytes: acc.bytes,
|
||||
packets: acc.packets,
|
||||
}
|
||||
}
|
||||
|
||||
export function bumpFlowFact(row: {
|
||||
serverId: number
|
||||
bucketAt: string
|
||||
iface: string
|
||||
country: string
|
||||
service: string
|
||||
asn: number
|
||||
bytes: number
|
||||
packets: number
|
||||
}): void {
|
||||
const iface = normalizeFactIface(row.iface)
|
||||
const country = normalizeFactCountry(row.country)
|
||||
const service = normalizeFactService(row.service)
|
||||
const asn = normalizeFactAsn(row.asn)
|
||||
const k = factKey(row.serverId, row.bucketAt, iface, country, service, asn)
|
||||
const prev = hourFacts.get(k)
|
||||
if (prev) {
|
||||
prev.bytes += row.bytes
|
||||
prev.packets += row.packets
|
||||
return
|
||||
}
|
||||
hourFacts.set(k, { bytes: row.bytes, packets: row.packets })
|
||||
}
|
||||
|
||||
function groupKey(row: FactRow): string {
|
||||
return `${row.serverId}\0${row.bucketAt}`
|
||||
}
|
||||
|
||||
function mergeRow(map: Map<string, FactRow>, row: FactRow): void {
|
||||
const k = factKey(row.serverId, row.bucketAt, row.iface, row.country, row.service, row.asn)
|
||||
const prev = map.get(k)
|
||||
if (prev) {
|
||||
prev.bytes += row.bytes
|
||||
prev.packets += row.packets
|
||||
return
|
||||
}
|
||||
map.set(k, { ...row })
|
||||
}
|
||||
|
||||
/** Cap ASN tail and tuple count per server×hour before persist. */
|
||||
export function collectCappedFacts(): FactRow[] {
|
||||
const parsed: FactRow[] = []
|
||||
for (const [k, acc] of hourFacts) {
|
||||
const row = parseFactKey(k, acc)
|
||||
if (row) parsed.push(row)
|
||||
}
|
||||
hourFacts.clear()
|
||||
|
||||
const groups = new Map<string, FactRow[]>()
|
||||
for (const row of parsed) {
|
||||
const g = groupKey(row)
|
||||
const list = groups.get(g) ?? []
|
||||
list.push(row)
|
||||
groups.set(g, list)
|
||||
}
|
||||
|
||||
const out = new Map<string, FactRow>()
|
||||
for (const list of groups.values()) {
|
||||
const byAsn = new Map<number, number>()
|
||||
for (const row of list) {
|
||||
byAsn.set(row.asn, (byAsn.get(row.asn) ?? 0) + row.bytes)
|
||||
}
|
||||
const asnKeep = new Set(
|
||||
[...byAsn.entries()]
|
||||
.sort((a, b) => b[1] - a[1])
|
||||
.slice(0, FACT_ASN_TOP)
|
||||
.map(([asn]) => asn),
|
||||
)
|
||||
const afterAsn: FactRow[] = []
|
||||
for (const row of list) {
|
||||
if (asnKeep.has(row.asn) || row.asn === 0) {
|
||||
afterAsn.push(row)
|
||||
continue
|
||||
}
|
||||
afterAsn.push({ ...row, asn: 0 })
|
||||
}
|
||||
const collapsed = new Map<string, FactRow>()
|
||||
for (const row of afterAsn) mergeRow(collapsed, row)
|
||||
const tuples = [...collapsed.values()].sort((a, b) => b.bytes - a.bytes)
|
||||
const keep = tuples.slice(0, FACT_TUPLE_CAP)
|
||||
const tail = tuples.slice(FACT_TUPLE_CAP)
|
||||
for (const row of keep) mergeRow(out, row)
|
||||
for (const row of tail) {
|
||||
mergeRow(out, {
|
||||
...row,
|
||||
country: UNKNOWN_COUNTRY,
|
||||
service: OTHER_SERVICE,
|
||||
asn: 0,
|
||||
})
|
||||
}
|
||||
}
|
||||
return [...out.values()]
|
||||
}
|
||||
|
||||
async function ensureParentPartition(parent: string, ts: string): Promise<void> {
|
||||
const spec = specForParent(parent)
|
||||
if (!spec) return
|
||||
const iso = ts.length === 10 ? `${ts}T00:00:00Z` : ts
|
||||
const key = `${parent}:${iso.slice(0, 10)}`
|
||||
if (ensuredParts.has(key)) return
|
||||
await ensurePartitionFor(pool, parent, spec.kind, new Date(iso))
|
||||
ensuredParts.add(key)
|
||||
}
|
||||
|
||||
function dayKey(bucketAt: string): string {
|
||||
return bucketAt.slice(0, 10)
|
||||
}
|
||||
|
||||
export async function flushFlowFacts(): Promise<number> {
|
||||
const rows = collectCappedFacts()
|
||||
if (rows.length === 0) return 0
|
||||
const hours = new Set(rows.map((r) => r.bucketAt))
|
||||
const days = new Set(rows.map((r) => dayKey(r.bucketAt)))
|
||||
for (const h of hours) await ensureParentPartition("flow_hour_facts", h)
|
||||
for (const d of days) await ensureParentPartition("flow_daily_facts", d)
|
||||
|
||||
await pool.query({
|
||||
text: `
|
||||
INSERT INTO flow_hour_facts (
|
||||
server_id, bucket_at, iface, country, service, asn, bytes, packets
|
||||
)
|
||||
SELECT *
|
||||
FROM UNNEST(
|
||||
$1::bigint[],
|
||||
$2::timestamptz[],
|
||||
$3::text[],
|
||||
$4::char(2)[],
|
||||
$5::text[],
|
||||
$6::int[],
|
||||
$7::bigint[],
|
||||
$8::bigint[]
|
||||
) AS t(server_id, bucket_at, iface, country, service, asn, bytes, packets)
|
||||
ON CONFLICT (server_id, bucket_at, iface, country, service, asn)
|
||||
DO UPDATE SET
|
||||
bytes = flow_hour_facts.bytes + excluded.bytes,
|
||||
packets = flow_hour_facts.packets + excluded.packets
|
||||
`,
|
||||
values: [
|
||||
rows.map((r) => r.serverId),
|
||||
rows.map((r) => r.bucketAt),
|
||||
rows.map((r) => r.iface),
|
||||
rows.map((r) => r.country),
|
||||
rows.map((r) => r.service),
|
||||
rows.map((r) => r.asn),
|
||||
rows.map((r) => r.bytes),
|
||||
rows.map((r) => r.packets),
|
||||
],
|
||||
})
|
||||
|
||||
await pool.query({
|
||||
text: `
|
||||
INSERT INTO flow_daily_facts (
|
||||
server_id, day, iface, country, service, asn, bytes, packets
|
||||
)
|
||||
SELECT *
|
||||
FROM UNNEST(
|
||||
$1::bigint[],
|
||||
$2::date[],
|
||||
$3::text[],
|
||||
$4::char(2)[],
|
||||
$5::text[],
|
||||
$6::int[],
|
||||
$7::bigint[],
|
||||
$8::bigint[]
|
||||
) AS t(server_id, day, iface, country, service, asn, bytes, packets)
|
||||
ON CONFLICT (server_id, day, iface, country, service, asn)
|
||||
DO UPDATE SET
|
||||
bytes = flow_daily_facts.bytes + excluded.bytes,
|
||||
packets = flow_daily_facts.packets + excluded.packets
|
||||
`,
|
||||
values: [
|
||||
rows.map((r) => r.serverId),
|
||||
rows.map((r) => dayKey(r.bucketAt)),
|
||||
rows.map((r) => r.iface),
|
||||
rows.map((r) => r.country),
|
||||
rows.map((r) => r.service),
|
||||
rows.map((r) => r.asn),
|
||||
rows.map((r) => r.bytes),
|
||||
rows.map((r) => r.packets),
|
||||
],
|
||||
})
|
||||
return rows.length
|
||||
}
|
||||
|
||||
export function factsPendingSize(): number {
|
||||
return hourFacts.size
|
||||
}
|
||||
|
||||
export function resetFactsForTests(): void {
|
||||
hourFacts.clear()
|
||||
ensuredParts.clear()
|
||||
}
|
||||
|
||||
export function factsSnapshotForTests(): FactRow[] {
|
||||
const parsed: FactRow[] = []
|
||||
for (const [k, acc] of hourFacts) {
|
||||
const row = parseFactKey(k, acc)
|
||||
if (row) parsed.push(row)
|
||||
}
|
||||
return parsed
|
||||
}
|
||||
@@ -0,0 +1,186 @@
|
||||
import assert from "node:assert/strict"
|
||||
import { existsSync, mkdtempSync, readFileSync, rmSync, writeFileSync } from "node:fs"
|
||||
import { tmpdir } from "node:os"
|
||||
import path from "node:path"
|
||||
import type { AsnResponse, CountryResponse, Reader } from "maxmind"
|
||||
import {
|
||||
disableRipeEnqueueForTests,
|
||||
disableRipePersistForTests,
|
||||
resetRipeCacheForTests,
|
||||
seedRipeCacheForTests,
|
||||
} from "./traffic-flow-ripe.js"
|
||||
import {
|
||||
lookupGeoip,
|
||||
resetGeoipForTests,
|
||||
resolveFlowIp,
|
||||
setGeoipReadersForTests,
|
||||
} from "./traffic-flow-geoip.js"
|
||||
import {
|
||||
resetEngineForTests,
|
||||
ingestParsedFlowsForServerForTests,
|
||||
minuteDimsSnapshotForTests,
|
||||
} from "./traffic-flow-engine.js"
|
||||
import { classifyFlowDst } from "./traffic-flow-classify.js"
|
||||
import { disableGeoipDbForTests } from "./geoip-settings.js"
|
||||
import {
|
||||
collectGeoipUpdateOnce,
|
||||
resetGeoipUpdateForTests,
|
||||
setGeoipFetchForTests,
|
||||
setGeoipValidateForTests,
|
||||
} from "./geoip-update-collector.js"
|
||||
|
||||
disableRipePersistForTests()
|
||||
disableRipeEnqueueForTests()
|
||||
resetRipeCacheForTests()
|
||||
resetGeoipForTests()
|
||||
|
||||
// ── lookupGeoip: приватные IP → negative без ридеров ──────────────────────────
|
||||
assert.equal(lookupGeoip("10.1.1.8")?.ok, false)
|
||||
assert.equal(lookupGeoip("192.168.0.1")?.prefix, "192.168.0.1/32")
|
||||
assert.equal(lookupGeoip("100.64.1.2")?.ok, false)
|
||||
assert.equal(lookupGeoip("fe80::1")?.prefix, "fe80::1/128")
|
||||
|
||||
// ── без ридеров публичный IP → null, resolveFlowIp уходит в RIPE-кэш ─────────
|
||||
assert.equal(lookupGeoip("1.2.3.10"), null)
|
||||
seedRipeCacheForTests({
|
||||
prefix: "1.2.3.0/24",
|
||||
asn: 64500,
|
||||
country: "NL",
|
||||
lat: null,
|
||||
lng: null,
|
||||
holder: "TEST",
|
||||
ok: true,
|
||||
fetchedAt: Date.now(),
|
||||
})
|
||||
assert.equal(resolveFlowIp("1.2.3.10")?.country, "NL")
|
||||
assert.equal(resolveFlowIp("1.2.3.10")?.asn, 64500)
|
||||
|
||||
// ── fake-ридеры: geoip приоритетнее RIPE ──────────────────────────────────────
|
||||
function fakeCountryReader(byIp: Record<string, string>): Reader<CountryResponse> {
|
||||
return {
|
||||
get(ip: string) {
|
||||
const iso = byIp[ip]
|
||||
return iso ? ({ country: { iso_code: iso } } as CountryResponse) : null
|
||||
},
|
||||
metadata: { buildEpoch: new Date("2026-09-02T00:00:00Z") },
|
||||
} as unknown as Reader<CountryResponse>
|
||||
}
|
||||
|
||||
function fakeAsnReader(byIp: Record<string, { asn: number; org: string }>): Reader<AsnResponse> {
|
||||
return {
|
||||
get(ip: string) {
|
||||
const hit = byIp[ip]
|
||||
return hit
|
||||
? ({ autonomous_system_number: hit.asn, autonomous_system_organization: hit.org } as AsnResponse)
|
||||
: null
|
||||
},
|
||||
metadata: { buildEpoch: new Date("2026-09-02T00:00:00Z") },
|
||||
} as unknown as Reader<AsnResponse>
|
||||
}
|
||||
|
||||
setGeoipReadersForTests({
|
||||
country: fakeCountryReader({ "8.8.8.8": "US", "6.6.6.6": "EU" }),
|
||||
asn: fakeAsnReader({
|
||||
"8.8.8.8": { asn: 15169, org: "GOOGLE" },
|
||||
"6.6.6.6": { asn: 15169, org: "GOOGLE" },
|
||||
}),
|
||||
})
|
||||
|
||||
const hit = resolveFlowIp("8.8.8.8")
|
||||
assert.equal(hit?.country, "US")
|
||||
assert.equal(hit?.asn, 15169)
|
||||
assert.equal(hit?.holder, "GOOGLE")
|
||||
assert.equal(hit?.ok, true)
|
||||
|
||||
// 1.2.3.10 в fake-ридерах нет — по-прежнему из RIPE-кэша
|
||||
assert.equal(resolveFlowIp("1.2.3.10")?.asn, 64500)
|
||||
|
||||
// EU не ISO-страна: отфильтрована, страна выведена из ASN (HQ Google → US)
|
||||
assert.equal(lookupGeoip("6.6.6.6")?.country, "US")
|
||||
|
||||
// geoip-мета совместима с classifyFlowDst (бренд по ASN 15169)
|
||||
const classified = classifyFlowDst("8.8.8.8", 6, 443, 51504, hit)
|
||||
assert.equal(classified.service, "Google")
|
||||
|
||||
// ── движок: dims country/asn наполняются из geoip-ридеров ────────────────────
|
||||
resetEngineForTests()
|
||||
ingestParsedFlowsForServerForTests(1, [{
|
||||
src: "192.168.88.10",
|
||||
dst: "8.8.8.8",
|
||||
proto: 6,
|
||||
srcPort: 51504,
|
||||
dstPort: 443,
|
||||
bytes: 1000,
|
||||
packets: 10,
|
||||
inIface: "wg-flow",
|
||||
outIface: "",
|
||||
nextHop: "",
|
||||
flowStartMs: 0,
|
||||
flowEndMs: 0,
|
||||
natSrc: "",
|
||||
natDst: "",
|
||||
}])
|
||||
const dims = minuteDimsSnapshotForTests()
|
||||
assert.equal(dims.get("country")?.get("US")?.bytes, 1000)
|
||||
assert.equal(dims.get("asn")?.get("15169")?.bytes, 1000)
|
||||
|
||||
// ── коллектор: 304 → обе базы без изменений ───────────────────────────────────
|
||||
disableGeoipDbForTests()
|
||||
resetGeoipUpdateForTests()
|
||||
const geoipDir = mkdtempSync(path.join(tmpdir(), "mm-geoip-test-"))
|
||||
process.env.GEOIP_DIR = geoipDir
|
||||
|
||||
setGeoipFetchForTests(async () => new Response(null, { status: 304 }))
|
||||
let snap = await collectGeoipUpdateOnce({ force: true })
|
||||
assert.equal(snap.skippedUnchanged, 2)
|
||||
assert.equal(snap.downloaded, 0)
|
||||
assert.equal(existsSync(path.join(geoipDir, "GeoLite2-Country.mmdb")), false)
|
||||
|
||||
// ── коллектор: 200 + валидация ok → подмена, старый файл в .prev ─────────────
|
||||
const countryPath = path.join(geoipDir, "GeoLite2-Country.mmdb")
|
||||
const asnPath = path.join(geoipDir, "GeoLite2-ASN.mmdb")
|
||||
writeFileSync(countryPath, "old-country")
|
||||
|
||||
setGeoipFetchForTests(async () =>
|
||||
new Response(new Uint8Array([1, 2, 3]), { status: 200, headers: { etag: '"v1"' } }))
|
||||
setGeoipValidateForTests({
|
||||
country: async (p) => {
|
||||
assert.ok(p.endsWith(".tmp"), "валидация должна идти по tmp-файлу")
|
||||
return "2026-09-08T00:00:00.000Z"
|
||||
},
|
||||
asn: async () => "2026-09-08T00:00:00.000Z",
|
||||
})
|
||||
snap = await collectGeoipUpdateOnce({ force: true })
|
||||
assert.equal(snap.downloaded, 2)
|
||||
assert.equal(snap.errors.length, 0)
|
||||
assert.deepEqual(readFileSync(countryPath), Buffer.from([1, 2, 3]))
|
||||
assert.equal(readFileSync(`${countryPath}.prev`, "utf8"), "old-country")
|
||||
assert.equal(existsSync(`${asnPath}.prev`), false, "prev у asn не бывает при первой загрузке")
|
||||
assert.equal(existsSync(`${countryPath}.tmp`), false)
|
||||
|
||||
// ── коллектор: битая база → подмены нет, старый файл цел, tmp удалён ─────────
|
||||
writeFileSync(asnPath, "good-asn")
|
||||
setGeoipFetchForTests(async () =>
|
||||
new Response(new Uint8Array([9, 9]), { status: 200 }))
|
||||
setGeoipValidateForTests({
|
||||
country: async () => {
|
||||
throw new Error("битая база")
|
||||
},
|
||||
asn: async () => {
|
||||
throw new Error("битая база")
|
||||
},
|
||||
})
|
||||
snap = await collectGeoipUpdateOnce({ force: true })
|
||||
assert.equal(snap.downloaded, 0)
|
||||
assert.equal(snap.errors.length, 2)
|
||||
assert.deepEqual(readFileSync(countryPath), Buffer.from([1, 2, 3]), "country не тронута")
|
||||
assert.equal(readFileSync(asnPath, "utf8"), "good-asn", "asn не тронут")
|
||||
assert.equal(existsSync(`${countryPath}.tmp`), false)
|
||||
assert.equal(existsSync(`${asnPath}.tmp`), false)
|
||||
|
||||
rmSync(geoipDir, { recursive: true, force: true })
|
||||
delete process.env.GEOIP_DIR
|
||||
resetGeoipUpdateForTests()
|
||||
resetGeoipForTests()
|
||||
|
||||
console.log("traffic-flow-geoip.test.ts: ok")
|
||||
@@ -0,0 +1,162 @@
|
||||
import { existsSync } from "node:fs"
|
||||
import path from "node:path"
|
||||
import { open, type AsnResponse, type CountryResponse, type Reader } from "maxmind"
|
||||
import { isNonPublicIp } from "./traffic-flow-ip.js"
|
||||
import { isIsoCountry, resolveRipeCountry } from "./traffic-flow-brands.js"
|
||||
import { lookupRipeCached, type FlowIpMeta } from "./traffic-flow-ripe.js"
|
||||
|
||||
export const GEOIP_COUNTRY_FILE = "GeoLite2-Country.mmdb"
|
||||
export const GEOIP_ASN_FILE = "GeoLite2-ASN.mmdb"
|
||||
|
||||
/** Каталог баз: `storage/geoip` рядом со storage/backups; переопределяется GEOIP_DIR. */
|
||||
export function geoipDir(): string {
|
||||
return path.resolve(process.env.GEOIP_DIR ?? path.join(process.cwd(), "storage", "geoip"))
|
||||
}
|
||||
|
||||
export function geoipCountryPath(): string {
|
||||
return path.join(geoipDir(), GEOIP_COUNTRY_FILE)
|
||||
}
|
||||
|
||||
export function geoipAsnPath(): string {
|
||||
return path.join(geoipDir(), GEOIP_ASN_FILE)
|
||||
}
|
||||
|
||||
export interface GeoipReaders {
|
||||
country: Reader<CountryResponse> | null
|
||||
asn: Reader<AsnResponse> | null
|
||||
}
|
||||
|
||||
let readers: GeoipReaders = { country: null, asn: null }
|
||||
let initPromise: Promise<GeoipReaders> | null = null
|
||||
|
||||
/** Открывает оба файла best-effort: отсутствующий/битый файл не мешает второму. */
|
||||
export async function openGeoipReaders(dir = geoipDir()): Promise<GeoipReaders> {
|
||||
const next: GeoipReaders = { country: null, asn: null }
|
||||
if (existsSync(path.join(dir, GEOIP_COUNTRY_FILE))) {
|
||||
try {
|
||||
next.country = await open<CountryResponse>(path.join(dir, GEOIP_COUNTRY_FILE))
|
||||
} catch {
|
||||
/* битый файл — работаем без country */
|
||||
}
|
||||
}
|
||||
if (existsSync(path.join(dir, GEOIP_ASN_FILE))) {
|
||||
try {
|
||||
next.asn = await open<AsnResponse>(path.join(dir, GEOIP_ASN_FILE))
|
||||
} catch {
|
||||
/* битый файл — работаем без ASN */
|
||||
}
|
||||
}
|
||||
return next
|
||||
}
|
||||
|
||||
/** Открывает ридеры при старте; файлы есть — работают, нет — lookup уходит в RIPE-fallback. */
|
||||
export async function initGeoip(): Promise<GeoipReaders> {
|
||||
if (!initPromise) {
|
||||
initPromise = openGeoipReaders().then((next) => {
|
||||
readers = next
|
||||
return next
|
||||
})
|
||||
}
|
||||
return initPromise
|
||||
}
|
||||
|
||||
/** Переоткрывает ридеры после обновления файлов (атомарная замена ссылок). */
|
||||
export async function reloadGeoipReaders(): Promise<GeoipReaders> {
|
||||
const next = await openGeoipReaders()
|
||||
readers = next
|
||||
initPromise = Promise.resolve(next)
|
||||
return next
|
||||
}
|
||||
|
||||
export function setGeoipReadersForTests(next: Partial<GeoipReaders>): void {
|
||||
readers = { country: next.country ?? null, asn: next.asn ?? null }
|
||||
}
|
||||
|
||||
export function resetGeoipForTests(): void {
|
||||
readers = { country: null, asn: null }
|
||||
initPromise = null
|
||||
}
|
||||
|
||||
function negativeMeta(ip: string): FlowIpMeta {
|
||||
const v6 = ip.includes(":")
|
||||
return {
|
||||
prefix: `${ip}/${v6 ? 128 : 32}`,
|
||||
asn: 0,
|
||||
country: "—",
|
||||
lat: null,
|
||||
lng: null,
|
||||
holder: "",
|
||||
ok: false,
|
||||
fetchedAt: Date.now(),
|
||||
}
|
||||
}
|
||||
|
||||
function safeCountryIso(reader: Reader<CountryResponse>, ip: string): string {
|
||||
try {
|
||||
const rec = reader.get(ip)
|
||||
return rec?.country?.iso_code ?? rec?.registered_country?.iso_code ?? ""
|
||||
} catch {
|
||||
return ""
|
||||
}
|
||||
}
|
||||
|
||||
function safeAsn(reader: Reader<AsnResponse>, ip: string): { asn: number; holder: string } {
|
||||
try {
|
||||
const rec = reader.get(ip)
|
||||
return {
|
||||
asn: rec?.autonomous_system_number ?? 0,
|
||||
holder: rec?.autonomous_system_organization ?? "",
|
||||
}
|
||||
} catch {
|
||||
return { asn: 0, holder: "" }
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Синхронный lookup по локальным GeoLite2. Возвращает FlowIpMeta в семантике RIPE-кэша
|
||||
* (ok=true когда есть страна или ASN; null — данных нет, пусть пробует RIPE).
|
||||
*/
|
||||
export function lookupGeoip(ip: string): FlowIpMeta | null {
|
||||
const trimmed = String(ip ?? "").trim()
|
||||
if (!trimmed) return null
|
||||
if (isNonPublicIp(trimmed)) return negativeMeta(trimmed)
|
||||
const { country: countryReader, asn: asnReader } = readers
|
||||
if (!countryReader && !asnReader) return null
|
||||
const iso = countryReader ? safeCountryIso(countryReader, trimmed) : ""
|
||||
const country = iso && isIsoCountry(iso) ? iso : ""
|
||||
const { asn, holder } = asnReader ? safeAsn(asnReader, trimmed) : { asn: 0, holder: "" }
|
||||
if (!asn && !country) return null
|
||||
return {
|
||||
prefix: `${trimmed}/${trimmed.includes(":") ? 128 : 32}`,
|
||||
asn,
|
||||
country: resolveRipeCountry(country, asn, holder) || "—",
|
||||
lat: null,
|
||||
lng: null,
|
||||
holder,
|
||||
ok: true,
|
||||
fetchedAt: Date.now(),
|
||||
}
|
||||
}
|
||||
|
||||
/** Главный вход для потребителей пайплайна: локальные базы первыми, RIPE-кэш fallback. */
|
||||
export function resolveFlowIp(ip: string): FlowIpMeta | null {
|
||||
return lookupGeoip(ip) ?? lookupRipeCached(ip)
|
||||
}
|
||||
|
||||
export interface GeoipReadersStatus {
|
||||
countryLoaded: boolean
|
||||
asnLoaded: boolean
|
||||
countryBuildAt: string | null
|
||||
asnBuildAt: string | null
|
||||
dir: string
|
||||
}
|
||||
|
||||
export function geoipReadersStatus(): GeoipReadersStatus {
|
||||
return {
|
||||
countryLoaded: Boolean(readers.country),
|
||||
asnLoaded: Boolean(readers.asn),
|
||||
countryBuildAt: readers.country?.metadata.buildEpoch.toISOString() ?? null,
|
||||
asnBuildAt: readers.asn?.metadata.buildEpoch.toISOString() ?? null,
|
||||
dir: geoipDir(),
|
||||
}
|
||||
}
|
||||
@@ -22,8 +22,9 @@ rememberServerIfaces(7, [
|
||||
{ ".id": "*A", name: "wg-flow" },
|
||||
{ ".id": "*D", name: "bridge" },
|
||||
])
|
||||
assert.equal(resolveIfaceName(7, "2").name, "ether1")
|
||||
assert.equal(resolveIfaceName(7, "10").name, "wg-flow")
|
||||
assert.equal(resolveIfaceName(7, "2").name, "ether1")
|
||||
assert.equal(resolveIfaceName(7, "#2").name, "ether1")
|
||||
assert.equal(resolveIfaceName(7, "10").name, "wg-flow")
|
||||
assert.equal(resolveIfaceName(7, "13").name, "bridge")
|
||||
assert.equal(resolveIfaceName(7, "0").name, "—")
|
||||
assert.equal(resolveIfaceName(7, "ether1").name, "ether1")
|
||||
|
||||
@@ -0,0 +1,58 @@
|
||||
import assert from "node:assert/strict"
|
||||
import {
|
||||
bindingIfaceAliases,
|
||||
bindingIfaceAliasesAllServers,
|
||||
canonicalFactIface,
|
||||
collapseServerIfaceRows,
|
||||
displayFactIface,
|
||||
expandBindingIfaces,
|
||||
factIfaceAliases,
|
||||
rememberServerIfaces,
|
||||
resetIfaceCacheForTests,
|
||||
resolveIfaceName,
|
||||
} from "./traffic-flow-ifindex.js"
|
||||
|
||||
resetIfaceCacheForTests()
|
||||
assert.equal(canonicalFactIface(1, "2"), "2")
|
||||
assert.deepEqual(bindingIfaceAliases(1, "gre-client"), ["gre-client"])
|
||||
|
||||
rememberServerIfaces(1, [{ name: "gre-client", ifindex: "2" }])
|
||||
assert.equal(canonicalFactIface(1, "2"), "gre-client")
|
||||
assert.equal(canonicalFactIface(1, "gre-client"), "gre-client")
|
||||
assert.equal(canonicalFactIface(1, "9"), "9")
|
||||
assert.equal(resolveIfaceName(1, "9").name, "#9")
|
||||
assert.equal(resolveIfaceName(1, "2").name, "gre-client")
|
||||
assert.equal(resolveIfaceName(1, "#2").name, "gre-client")
|
||||
assert.equal(displayFactIface(1, "2"), "gre-client")
|
||||
|
||||
const aliases = bindingIfaceAliases(1, "gre-client")
|
||||
assert.ok(aliases.includes("gre-client"))
|
||||
assert.ok(aliases.includes("2"))
|
||||
assert.ok(aliases.includes("#2"))
|
||||
|
||||
const fromIndex = factIfaceAliases("2", 1)
|
||||
assert.ok(fromIndex.includes("gre-client"))
|
||||
assert.ok(fromIndex.includes("2"))
|
||||
assert.ok(fromIndex.includes("#2"))
|
||||
|
||||
const all = bindingIfaceAliasesAllServers("gre-client")
|
||||
assert.ok(all.includes("2"))
|
||||
|
||||
const expanded = expandBindingIfaces([{ serverId: 1, iface: "gre-client" }])
|
||||
assert.ok(expanded.some((x) => x.iface === "2"))
|
||||
assert.ok(expanded.some((x) => x.iface === "gre-client"))
|
||||
|
||||
const collapsed = collapseServerIfaceRows([
|
||||
{ serverId: 1, iface: "2", bytes: 10, packets: 1 },
|
||||
{ serverId: 1, iface: "gre-client", bytes: 5, packets: 2 },
|
||||
{ serverId: 1, iface: "wan1", bytes: 3, packets: 1 },
|
||||
])
|
||||
assert.equal(collapsed.length, 2)
|
||||
const gre = collapsed.find((r) => r.iface === "gre-client")
|
||||
assert.ok(gre)
|
||||
assert.equal(gre.bytes, 15)
|
||||
assert.equal(gre.packets, 3)
|
||||
assert.ok(collapsed.some((r) => r.iface === "wan1"))
|
||||
|
||||
resetIfaceCacheForTests()
|
||||
console.log("traffic-flow-ifindex.test.ts: ok")
|
||||
@@ -36,12 +36,129 @@ export function rememberServerIfaces(serverId: number, rows: RosIfaceIndexRow[])
|
||||
|
||||
export function resolveIfaceName(serverId: number, indexOrName: string): { name: string; index: string } {
|
||||
const trimmed = String(indexOrName ?? "").trim()
|
||||
if (!trimmed || trimmed === "0") return { name: "—", index: trimmed }
|
||||
if (!/^\d+$/.test(trimmed)) return { name: trimmed, index: "" }
|
||||
const idx = Number(trimmed)
|
||||
const name = cache.get(serverId)?.get(idx)
|
||||
if (name) return { name, index: trimmed }
|
||||
return { name: `#${trimmed}`, index: trimmed }
|
||||
const asIndex = trimmed.startsWith("#") && /^\d+$/.test(trimmed.slice(1)) ? trimmed.slice(1) : trimmed
|
||||
if (!asIndex || asIndex === "0") return { name: "—", index: asIndex }
|
||||
if (!/^\d+$/.test(asIndex)) return { name: trimmed, index: "" }
|
||||
const name = cache.get(serverId)?.get(Number(asIndex))
|
||||
if (name) return { name, index: asIndex }
|
||||
return { name: `#${asIndex}`, index: asIndex }
|
||||
}
|
||||
|
||||
/** Имя iface для факта куба: ifIndex→имя, без `#13` при пустом кэше. */
|
||||
export function canonicalFactIface(serverId: number, inIface: string): string {
|
||||
const trimmed = String(inIface ?? "").trim()
|
||||
if (!trimmed) return trimmed
|
||||
if (!/^\d+$/.test(trimmed)) return trimmed
|
||||
const name = cache.get(serverId)?.get(Number(trimmed))
|
||||
return name || trimmed
|
||||
}
|
||||
|
||||
function numericIfaceIndex(iface: string): string | null {
|
||||
const raw = String(iface ?? "").trim()
|
||||
if (/^\d+$/.test(raw)) return raw
|
||||
if (raw.startsWith("#") && /^\d+$/.test(raw.slice(1))) return raw.slice(1)
|
||||
return null
|
||||
}
|
||||
|
||||
/** Имя для UI: ifIndex → RouterOS name; `0` → «—»; miss → `#n`. */
|
||||
export function displayFactIface(serverId: number, iface: string): string {
|
||||
return resolveIfaceName(serverId, iface).name
|
||||
}
|
||||
|
||||
/** Склеить факты `2` + `ether1` в одну строку после резолва ifIndex. */
|
||||
export function collapseServerIfaceRows(
|
||||
rows: Array<{ serverId: number; iface: string; bytes: number; packets: number }>,
|
||||
): Array<{ serverId: number; iface: string; bytes: number; packets: number }> {
|
||||
const acc = new Map<string, { serverId: number; iface: string; bytes: number; packets: number }>()
|
||||
for (const r of rows) {
|
||||
const name = displayFactIface(r.serverId, r.iface)
|
||||
const k = `${r.serverId}\0${name}`
|
||||
const prev = acc.get(k)
|
||||
const bytes = Number(r.bytes) || 0
|
||||
const packets = Number(r.packets) || 0
|
||||
if (prev) {
|
||||
prev.bytes += bytes
|
||||
prev.packets += packets
|
||||
} else {
|
||||
acc.set(k, { serverId: r.serverId, iface: name, bytes, packets })
|
||||
}
|
||||
}
|
||||
return [...acc.values()]
|
||||
}
|
||||
|
||||
/** Ключи факта для фильтра: имя, ifIndex и `#n`. */
|
||||
export function factIfaceAliases(iface: string, serverId?: number): string[] {
|
||||
const raw = String(iface ?? "").trim()
|
||||
if (!raw) return []
|
||||
const out = new Set<string>([raw])
|
||||
const idx = numericIfaceIndex(raw)
|
||||
if (idx) {
|
||||
out.add(idx)
|
||||
out.add(`#${idx}`)
|
||||
const n = Number(idx)
|
||||
if (serverId != null) {
|
||||
const name = cache.get(serverId)?.get(n)
|
||||
if (name) out.add(name)
|
||||
} else {
|
||||
for (const map of cache.values()) {
|
||||
const name = map.get(n)
|
||||
if (name) out.add(name)
|
||||
}
|
||||
}
|
||||
}
|
||||
if (serverId != null) {
|
||||
for (const a of bindingIfaceAliases(serverId, raw)) out.add(a)
|
||||
} else {
|
||||
for (const a of bindingIfaceAliasesAllServers(raw)) out.add(a)
|
||||
}
|
||||
return [...out]
|
||||
}
|
||||
|
||||
/** Имя + ifIndex + `#n` — тот же матч, что карта `/traffic`. */
|
||||
export function bindingIfaceAliases(serverId: number, interfaceName: string): string[] {
|
||||
const name = String(interfaceName ?? "").trim()
|
||||
if (!name) return []
|
||||
const out = new Set<string>([name])
|
||||
const map = cache.get(serverId)
|
||||
const idx = numericIfaceIndex(name)
|
||||
const canonical = (idx && map?.get(Number(idx))) || name
|
||||
out.add(canonical)
|
||||
if (idx) {
|
||||
out.add(idx)
|
||||
out.add(`#${idx}`)
|
||||
}
|
||||
if (!map) return [...out]
|
||||
for (const [i, n] of map) {
|
||||
if (n !== canonical && n !== name) continue
|
||||
out.add(String(i))
|
||||
out.add(`#${i}`)
|
||||
}
|
||||
return [...out]
|
||||
}
|
||||
|
||||
export function bindingIfaceAliasesAllServers(interfaceName: string): string[] {
|
||||
const name = String(interfaceName ?? "").trim()
|
||||
const out = new Set<string>(name ? [name] : [])
|
||||
for (const serverId of cache.keys()) {
|
||||
for (const alias of bindingIfaceAliases(serverId, name)) out.add(alias)
|
||||
}
|
||||
return [...out]
|
||||
}
|
||||
|
||||
export function expandBindingIfaces(
|
||||
binds: Array<{ serverId: number; iface: string }>,
|
||||
): Array<{ serverId: number; iface: string }> {
|
||||
const seen = new Set<string>()
|
||||
const out: Array<{ serverId: number; iface: string }> = []
|
||||
for (const b of binds) {
|
||||
for (const iface of bindingIfaceAliases(b.serverId, b.iface)) {
|
||||
const k = `${b.serverId}\0${iface}`
|
||||
if (seen.has(k)) continue
|
||||
seen.add(k)
|
||||
out.push({ serverId: b.serverId, iface })
|
||||
}
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
export function ifaceCacheHas(serverId: number): boolean {
|
||||
|
||||
@@ -21,7 +21,7 @@ import {
|
||||
setWantListenForTests,
|
||||
simulateWorkerExitForTests,
|
||||
} from "./traffic-flow-ingest.js"
|
||||
import { configureEngine, droppedForTests, pendingSizeForTests } from "./traffic-flow-engine.js"
|
||||
import { configureEngine, droppedForTests, getEngineStats, isValidFlowInet, pendingSizeForTests } from "./traffic-flow-engine.js"
|
||||
import { dbQuery } from "../db/index.js"
|
||||
import { withPgOrSkip } from "../test/pg.js"
|
||||
|
||||
@@ -69,6 +69,27 @@ assert.equal(droppedForTests(), 3)
|
||||
assert.equal(peekPendingFlows().length, 3)
|
||||
setPendingCapForTests(null)
|
||||
|
||||
assert.equal(isValidFlowInet("10.0.0.1"), true)
|
||||
assert.equal(isValidFlowInet("8.8.8.8"), true)
|
||||
assert.equal(isValidFlowInet("0:0:0:0:0:0:0:1"), true)
|
||||
assert.equal(isValidFlowInet("not-an-ip"), false)
|
||||
assert.equal(isValidFlowInet("999.1.1.1"), false)
|
||||
|
||||
resetFlowRingsForTests()
|
||||
ingestParsedFlowsForServerForTests(1, [{
|
||||
src: "not-an-ip",
|
||||
dst: "8.8.8.8",
|
||||
proto: 6,
|
||||
srcPort: 1,
|
||||
dstPort: 443,
|
||||
bytes: 10,
|
||||
packets: 1,
|
||||
inIface: "2",
|
||||
outIface: "",
|
||||
}])
|
||||
await flushPendingForTests()
|
||||
assert.match(getEngineStats().lastError, /невалидн/)
|
||||
|
||||
resetFlowRingsForTests()
|
||||
configureEngine({ topN: 20 })
|
||||
const talkers = Array.from({ length: 25 }, (_, i) => ({
|
||||
|
||||
@@ -267,7 +267,7 @@ export async function startTrafficFlowListener() {
|
||||
export function stopTrafficFlowListener() {
|
||||
wantListen = false
|
||||
stopWorkerProcess()
|
||||
void flushPending().catch(() => { /* ignore */ })
|
||||
void flushPending({ force: true }).catch(() => { /* ignore */ })
|
||||
state = { bound: false, address: null }
|
||||
}
|
||||
|
||||
@@ -425,7 +425,7 @@ export async function ingestParsedFlowsForTests(exporterIp: string, flows: Parse
|
||||
if (serverId == null) return
|
||||
queueParsedFlows(serverId, flows)
|
||||
rollFlowRings()
|
||||
await flushPending()
|
||||
await flushPending({ force: true })
|
||||
}
|
||||
|
||||
export function ingestParsedFlowsForServerForTests(serverId: number, flows: ParsedFlowInput[]) {
|
||||
@@ -444,7 +444,7 @@ export function lastFlushUsedTransactionForTests(): boolean {
|
||||
}
|
||||
|
||||
export async function flushPendingForTests(): Promise<void> {
|
||||
await flushPending()
|
||||
await flushPending({ force: true })
|
||||
}
|
||||
|
||||
async function tableCount(name: string): Promise<number> {
|
||||
@@ -467,11 +467,15 @@ export async function purgeTrafficFlowStore(): Promise<FlowPurgeDto> {
|
||||
minuteStats: await tableCount("flow_minute_stats"),
|
||||
minuteDims: await tableCount("flow_minute_dims"),
|
||||
dailyDims: await tableCount("flow_daily_dims"),
|
||||
hourFacts: await tableCount("flow_hour_facts"),
|
||||
dailyFacts: await tableCount("flow_daily_facts"),
|
||||
}
|
||||
await dbQuery(`DELETE FROM flow_buckets`)
|
||||
await dbQuery(`DELETE FROM flow_minute_stats`)
|
||||
await dbQuery(`DELETE FROM flow_minute_dims`)
|
||||
await dbQuery(`DELETE FROM flow_daily_dims`)
|
||||
await dbQuery(`DELETE FROM flow_hour_facts`)
|
||||
await dbQuery(`DELETE FROM flow_daily_facts`)
|
||||
await resetFlowIngestCounters()
|
||||
await dropExpiredPartitions(pool)
|
||||
try {
|
||||
|
||||
@@ -4,7 +4,13 @@ import {
|
||||
ingestParsedFlowsForServerForTests,
|
||||
resetFlowRingsForTests,
|
||||
} from "./traffic-flow-ingest.js"
|
||||
import { buildFlowMapHops, resetFlowMapHopsCacheForTests } from "./traffic-flow-map-hops.js"
|
||||
import {
|
||||
buildFlowMapHops,
|
||||
MAP_SERVICE_MIN_NODES,
|
||||
MAP_SERVICE_NODE_CAP,
|
||||
pickMapServices,
|
||||
resetFlowMapHopsCacheForTests,
|
||||
} from "./traffic-flow-map-hops.js"
|
||||
import { withPgOrSkip } from "../test/pg.js"
|
||||
import { seedFlowTopologyForTests, type FlowTopology } from "./traffic-flow-topology.js"
|
||||
import { disableCatalogFetchForTests, resetFlowCatalogForTests } from "./traffic-flow-classify.js"
|
||||
@@ -15,6 +21,55 @@ import {
|
||||
seedRipeCacheForTests,
|
||||
} from "./traffic-flow-ripe.js"
|
||||
|
||||
{
|
||||
const googleOnly = pickMapServices(
|
||||
[{ id: "svc:google", label: "Google", category: "Веб", bytes: 400, bps: 0, share: 1 }],
|
||||
5,
|
||||
)
|
||||
assert.equal(googleOnly.length, 1)
|
||||
assert.equal(googleOnly[0]?.share, 1)
|
||||
|
||||
const twoNamed = pickMapServices(
|
||||
[
|
||||
{ id: "svc:google", label: "Google", category: "Веб", bytes: 400, bps: 0, share: 0.5 },
|
||||
{ id: "svc:cloudflare", label: "Cloudflare", category: "CDN", bytes: 400, bps: 0, share: 0.5 },
|
||||
],
|
||||
5,
|
||||
)
|
||||
assert.equal(twoNamed.length, 2)
|
||||
|
||||
const tinyTail = pickMapServices(
|
||||
[
|
||||
{ id: "svc:google", label: "Google", category: "Веб", bytes: 9000, bps: 0, share: 0.9 },
|
||||
...Array.from({ length: 9 }, (_, i) => ({
|
||||
id: `svc:t${i}`,
|
||||
label: `T${i}`,
|
||||
category: "Веб",
|
||||
bytes: 100,
|
||||
bps: 0,
|
||||
share: 0.01,
|
||||
})),
|
||||
],
|
||||
5,
|
||||
)
|
||||
assert.equal(tinyTail.length, MAP_SERVICE_MIN_NODES)
|
||||
assert.equal(tinyTail.at(-1)?.id, "svc:t6")
|
||||
|
||||
const allOff = pickMapServices(
|
||||
Array.from({ length: 25 }, (_, i) => ({
|
||||
id: `svc:n${i}`,
|
||||
label: `N${i}`,
|
||||
category: "Веб",
|
||||
bytes: 25 - i,
|
||||
bps: 0,
|
||||
share: 0.04,
|
||||
})),
|
||||
0,
|
||||
)
|
||||
assert.equal(allOff.length, MAP_SERVICE_NODE_CAP)
|
||||
console.log("traffic-flow-map-hops.test.ts: pickMapServices ok")
|
||||
}
|
||||
|
||||
if (!(await withPgOrSkip())) {
|
||||
console.log("traffic-flow-map-hops.test.ts: skip")
|
||||
process.exit(0)
|
||||
@@ -178,6 +233,19 @@ function googleRipe() {
|
||||
})
|
||||
}
|
||||
|
||||
function seedRipeAsn(ip: string, asn: number, holder: string) {
|
||||
seedRipeCacheForTests({
|
||||
prefix: `${ip}/32`,
|
||||
asn,
|
||||
country: "US",
|
||||
lat: 37.4,
|
||||
lng: -122.1,
|
||||
holder,
|
||||
ok: true,
|
||||
fetchedAt: Date.now(),
|
||||
})
|
||||
}
|
||||
|
||||
function payloadFlow(dst: string, bytes: number) {
|
||||
return {
|
||||
src: "10.100.1.17",
|
||||
@@ -241,10 +309,106 @@ try {
|
||||
resetFlowMapHopsCacheForTests()
|
||||
const four = await buildFlowMapHops({ minutes: 5, minSharePct: 5 })
|
||||
assert.equal(four.totalBytes, 10_000)
|
||||
assert.ok(!(four.services ?? []).some((s) => s.id === "svc:google"), "Google < 5% hidden")
|
||||
const googleFour = four.services?.find((s) => s.id === "svc:google")
|
||||
assert.ok(googleFour, "единственный бренд виден при 4% от окна")
|
||||
assert.ok(googleFour.share >= 0.99, "доля среди брендов ≈ 1")
|
||||
resetFlowMapHopsCacheForTests()
|
||||
const off = await buildFlowMapHops({ minutes: 5, minSharePct: 0 })
|
||||
assert.ok(off.services?.some((s) => s.id === "svc:google"), "порог 0 показывает Google 4%")
|
||||
assert.ok(off.services?.some((s) => s.id === "svc:google"), "порог 0 показывает Google")
|
||||
} finally {
|
||||
resetFlowRingsForTests()
|
||||
resetIfaceCacheForTests()
|
||||
resetRipeCacheForTests()
|
||||
}
|
||||
|
||||
resetFlowRingsForTests()
|
||||
resetIfaceCacheForTests()
|
||||
resetRipeCacheForTests()
|
||||
disableRipeEnqueueForTests()
|
||||
seedFlowTopologyForTests(topo)
|
||||
rememberServerIfaces(7, [
|
||||
{ ".id": "*2", name: "gre-client" },
|
||||
{ ".id": "*3", name: "gre-jh-en" },
|
||||
])
|
||||
ingestParsedFlowsForServerForTests(7, [
|
||||
payloadFlow("8.8.8.8", 400),
|
||||
payloadFlow("104.18.35.51", 400),
|
||||
payloadFlow("203.0.113.50", 9200),
|
||||
])
|
||||
try {
|
||||
resetFlowMapHopsCacheForTests()
|
||||
const two = await buildFlowMapHops({ minutes: 5, minSharePct: 5 })
|
||||
const googleTwo = two.services?.find((s) => s.id === "svc:google")
|
||||
const cfTwo = two.services?.find((s) => s.id === "svc:cloudflare")
|
||||
assert.ok(googleTwo, "Google среди брендов")
|
||||
assert.ok(cfTwo, "Cloudflare среди брендов")
|
||||
assert.ok(googleTwo.share >= 0.05)
|
||||
assert.ok(cfTwo.share >= 0.05)
|
||||
} finally {
|
||||
resetFlowRingsForTests()
|
||||
resetIfaceCacheForTests()
|
||||
resetRipeCacheForTests()
|
||||
}
|
||||
|
||||
resetFlowRingsForTests()
|
||||
resetIfaceCacheForTests()
|
||||
resetRipeCacheForTests()
|
||||
disableRipeEnqueueForTests()
|
||||
seedFlowTopologyForTests(topo)
|
||||
rememberServerIfaces(7, [
|
||||
{ ".id": "*2", name: "gre-client" },
|
||||
{ ".id": "*3", name: "gre-jh-en" },
|
||||
])
|
||||
seedRipeAsn("162.254.192.71", 32590, "VALVE-CORP")
|
||||
ingestParsedFlowsForServerForTests(7, [
|
||||
payloadFlow("162.254.192.71", 2000),
|
||||
])
|
||||
try {
|
||||
resetFlowMapHopsCacheForTests()
|
||||
const steam = await buildFlowMapHops({ minutes: 5, minSharePct: 5 })
|
||||
assert.ok(steam.services?.some((s) => s.id === "svc:steam"), "Steam AS32590 на карте")
|
||||
} finally {
|
||||
resetFlowRingsForTests()
|
||||
resetIfaceCacheForTests()
|
||||
resetRipeCacheForTests()
|
||||
}
|
||||
|
||||
const smallBrands: Array<{ ip: string; asn: number; holder: string; bytes: number; id: string }> = [
|
||||
{ ip: "203.0.113.1", asn: 714, holder: "APPLE-ENGINEERING", bytes: 400, id: "svc:apple" },
|
||||
{ ip: "203.0.113.2", asn: 36459, holder: "GITHUB", bytes: 390, id: "svc:github" },
|
||||
{ ip: "203.0.113.3", asn: 54876, holder: "GITLAB", bytes: 380, id: "svc:gitlab" },
|
||||
{ ip: "203.0.113.4", asn: 8403, holder: "SPOTIFY", bytes: 370, id: "svc:spotify" },
|
||||
{ ip: "203.0.113.5", asn: 13414, holder: "TWITTER", bytes: 360, id: "svc:x" },
|
||||
{ ip: "203.0.113.6", asn: 47541, holder: "VKONTAKTE", bytes: 350, id: "svc:vk" },
|
||||
{ ip: "203.0.113.7", asn: 30103, holder: "ZOOM", bytes: 340, id: "svc:zoom" },
|
||||
{ ip: "203.0.113.8", asn: 395701, holder: "EPIC-GAMES", bytes: 330, id: "svc:epic" },
|
||||
{ ip: "203.0.113.9", asn: 6507, holder: "RIOT-GAMES", bytes: 320, id: "svc:riot" },
|
||||
]
|
||||
|
||||
resetFlowRingsForTests()
|
||||
resetIfaceCacheForTests()
|
||||
resetRipeCacheForTests()
|
||||
disableRipeEnqueueForTests()
|
||||
seedFlowTopologyForTests(topo)
|
||||
rememberServerIfaces(7, [
|
||||
{ ".id": "*2", name: "gre-client" },
|
||||
{ ".id": "*3", name: "gre-jh-en" },
|
||||
])
|
||||
googleRipe()
|
||||
for (const b of smallBrands) seedRipeAsn(b.ip, b.asn, b.holder)
|
||||
ingestParsedFlowsForServerForTests(7, [
|
||||
payloadFlow("8.8.8.8", 5000),
|
||||
...smallBrands.map((b) => payloadFlow(b.ip, b.bytes)),
|
||||
])
|
||||
try {
|
||||
resetFlowMapHopsCacheForTests()
|
||||
const top = await buildFlowMapHops({ minutes: 5, minSharePct: 5 })
|
||||
const ids = new Set((top.services ?? []).map((s) => s.id))
|
||||
assert.equal(top.services?.length, MAP_SERVICE_MIN_NODES, "топ-8 брендов на карте")
|
||||
assert.ok(ids.has("svc:google"))
|
||||
for (const b of smallBrands.slice(0, 7)) assert.ok(ids.has(b.id), b.id)
|
||||
assert.ok(!ids.has("svc:epic"), "хвост ниже ранга 8 скрыт")
|
||||
assert.ok(!ids.has("svc:riot"))
|
||||
} finally {
|
||||
resetFlowRingsForTests()
|
||||
resetIfaceCacheForTests()
|
||||
@@ -420,4 +584,98 @@ try {
|
||||
resetFlowCatalogForTests()
|
||||
}
|
||||
|
||||
resetFlowRingsForTests()
|
||||
resetIfaceCacheForTests()
|
||||
resetRipeCacheForTests()
|
||||
disableRipeEnqueueForTests()
|
||||
seedFlowTopologyForTests(topo)
|
||||
rememberServerIfaces(7, [
|
||||
{ ".id": "*2", name: "gre-client" },
|
||||
{ ".id": "*3", name: "gre-jh-en" },
|
||||
])
|
||||
rememberServerIfaces(9, [
|
||||
{ ".id": "*1", name: "ether1" },
|
||||
])
|
||||
googleRipe()
|
||||
ingestParsedFlowsForServerForTests(7, [payloadFlow("8.8.8.8", 12_000)])
|
||||
ingestParsedFlowsForServerForTests(9, [{
|
||||
src: "10.100.1.17",
|
||||
dst: "8.8.8.8",
|
||||
proto: 6,
|
||||
srcPort: 51234,
|
||||
dstPort: 443,
|
||||
bytes: 12_000,
|
||||
packets: 10,
|
||||
inIface: "1",
|
||||
outIface: "1",
|
||||
nextHop: "",
|
||||
}])
|
||||
try {
|
||||
resetFlowMapHopsCacheForTests()
|
||||
const dual = await buildFlowMapHops({ minutes: 5, minSharePct: 0 })
|
||||
const gre = dual.hops.find((h) => h.kind === "gre" && h.fromId === "7" && h.toId === "9")
|
||||
assert.ok(gre, "GRE JH→EN сохранён")
|
||||
assert.equal(gre.bytes, 12_000)
|
||||
const googlePaths = (dual.servicePaths ?? []).filter((p) => p.serviceId === "svc:google")
|
||||
assert.equal(googlePaths.length, 1, "один путь без копии EN")
|
||||
assert.equal(googlePaths[0]?.clientId, "u1")
|
||||
assert.equal(googlePaths[0]?.viaId, "7")
|
||||
const googleEdge = dual.serviceEdges?.find((e) => e.toId === "svc:google" && e.fromId === "9")
|
||||
assert.ok(googleEdge)
|
||||
assert.equal(googleEdge.bytes, 12_000)
|
||||
assert.equal(googleEdge.bps, googlePaths[0]?.bps)
|
||||
} finally {
|
||||
seedFlowTopologyForTests(null)
|
||||
resetFlowRingsForTests()
|
||||
resetIfaceCacheForTests()
|
||||
resetRipeCacheForTests()
|
||||
resetFlowCatalogForTests()
|
||||
}
|
||||
|
||||
resetFlowRingsForTests()
|
||||
resetIfaceCacheForTests()
|
||||
resetRipeCacheForTests()
|
||||
disableRipeEnqueueForTests()
|
||||
seedFlowTopologyForTests(topo)
|
||||
rememberServerIfaces(7, [
|
||||
{ ".id": "*2", name: "gre-client" },
|
||||
{ ".id": "*3", name: "gre-jh-en" },
|
||||
])
|
||||
googleRipe()
|
||||
ingestParsedFlowsForServerForTests(7, [
|
||||
payloadFlow("8.8.8.8", 8_000),
|
||||
{
|
||||
src: "8.8.8.8",
|
||||
dst: "10.100.1.17",
|
||||
proto: 6,
|
||||
srcPort: 443,
|
||||
dstPort: 51234,
|
||||
bytes: 4_000,
|
||||
packets: 8,
|
||||
inIface: "3",
|
||||
outIface: "2",
|
||||
nextHop: "",
|
||||
},
|
||||
])
|
||||
try {
|
||||
resetFlowMapHopsCacheForTests()
|
||||
const bothDir = await buildFlowMapHops({ minutes: 5, minSharePct: 0 })
|
||||
const gre = bothDir.hops.find((h) => h.kind === "gre" && h.fromId === "7" && h.toId === "9")
|
||||
assert.ok(gre, "GRE-hop при fwd/rev")
|
||||
const googlePaths = (bothDir.servicePaths ?? []).filter((p) => p.serviceId === "svc:google")
|
||||
assert.equal(googlePaths.length, 1, "fwd+rev — один клиент")
|
||||
assert.equal(googlePaths[0]?.clientId, "u1")
|
||||
assert.ok(!(bothDir.servicePaths ?? []).some((p) => p.serviceId === "svc:google" && p.clientId === "—"))
|
||||
const googleEdge = bothDir.serviceEdges?.find((e) => e.toId === "svc:google" && e.fromId === "9")
|
||||
assert.ok(googleEdge)
|
||||
assert.equal(googleEdge.bytes, 12_000)
|
||||
assert.equal(googleEdge.bps, googlePaths[0]?.bps)
|
||||
} finally {
|
||||
seedFlowTopologyForTests(null)
|
||||
resetFlowRingsForTests()
|
||||
resetIfaceCacheForTests()
|
||||
resetRipeCacheForTests()
|
||||
resetFlowCatalogForTests()
|
||||
}
|
||||
|
||||
console.log("traffic-flow-map-hops.test.ts: ok")
|
||||
|
||||
@@ -5,21 +5,24 @@ import { userInterfaceBindings } from "../db/schema.js"
|
||||
import { applicationName, flowRowMatchesFilter } from "./traffic-flow-apps.js"
|
||||
import {
|
||||
isNamedInternetService,
|
||||
lookupBrand,
|
||||
mapServiceNodeId,
|
||||
resolveFlowBrand,
|
||||
} from "./traffic-flow-brands.js"
|
||||
import { dedupFlowRowsMaxBytes } from "./traffic-flow-dedup.js"
|
||||
import { dedupFlowRowsAcrossExporters, dedupFlowRowsMaxBytes } from "./traffic-flow-dedup.js"
|
||||
import { getFlowListenerState, listFlowRowsForWindow } from "./traffic-flow-ingest.js"
|
||||
import { resolveIfaceName } from "./traffic-flow-ifaces.js"
|
||||
import { classifyFlowPlane, shouldKeepPlane } from "./traffic-flow-planes.js"
|
||||
import { pickInternetPeer } from "./traffic-flow-ip.js"
|
||||
import { lookupRipeCached, type FlowIpMeta } from "./traffic-flow-ripe.js"
|
||||
import { type FlowIpMeta } from "./traffic-flow-ripe.js"
|
||||
import { resolveFlowIp } from "./traffic-flow-geoip.js"
|
||||
import { getTrafficFlowSettingsRow } from "./traffic-flow-settings.js"
|
||||
import { loadFlowTopology, resolveClient, resolveEn, getServerCatalog } from "./traffic-flow-topology.js"
|
||||
import { loadFlowTopology, resolveClient, resolveEn, getServerCatalog, type FlowTopology } from "./traffic-flow-topology.js"
|
||||
import { flowDataEpoch } from "./traffic-flow-engine.js"
|
||||
|
||||
export const DEFAULT_MAP_SERVICE_MIN_SHARE_PCT = 5
|
||||
export const MAP_SERVICE_NODE_CAP = 20
|
||||
/** Минимум узлов-брендов на карте, даже если доля ниже порога. */
|
||||
export const MAP_SERVICE_MIN_NODES = 8
|
||||
const HOPS_CACHE_TTL_MS = 2000
|
||||
|
||||
export interface FlowMapHopsQuery {
|
||||
@@ -99,6 +102,15 @@ export function clampMapServiceMinSharePct(n: unknown): number {
|
||||
return Math.min(100, Math.max(0, v))
|
||||
}
|
||||
|
||||
/** Доля среди именованных брендов; порог ИЛИ топ-N, затем cap. */
|
||||
export function pickMapServices(ranked: FlowMapService[], minSharePct: number): FlowMapService[] {
|
||||
if (minSharePct <= 0) return ranked.slice(0, MAP_SERVICE_NODE_CAP)
|
||||
const minShare = minSharePct / 100
|
||||
return ranked
|
||||
.filter((s, i) => s.share >= minShare || i < MAP_SERVICE_MIN_NODES)
|
||||
.slice(0, MAP_SERVICE_NODE_CAP)
|
||||
}
|
||||
|
||||
function hopsQueryKey(q: FlowMapHopsQuery, minSharePct: number): string {
|
||||
return JSON.stringify({
|
||||
epoch: flowDataEpoch(),
|
||||
@@ -129,6 +141,16 @@ function ifaceUsable(name: string): boolean {
|
||||
return Boolean(name) && name !== "—"
|
||||
}
|
||||
|
||||
function resolveMapClient(
|
||||
topo: FlowTopology,
|
||||
serverId: number,
|
||||
inName: string,
|
||||
outName: string,
|
||||
) {
|
||||
return resolveClient(topo, serverId, inName)
|
||||
?? (ifaceUsable(outName) ? resolveClient(topo, serverId, outName) : null)
|
||||
}
|
||||
|
||||
function bump(acc: Map<string, HopAcc>, key: string, seed: Omit<HopAcc, "bytes" | "bytesFwd" | "bytesRev">, bytes: number, dir: "fwd" | "rev" | "both"): void {
|
||||
const prev = acc.get(key)
|
||||
const addFwd = dir === "fwd" || dir === "both" ? bytes : 0
|
||||
@@ -174,10 +196,7 @@ function classifyMapDstLite(
|
||||
if (proto === 47 || proto === 50) return null
|
||||
const app = applicationName(proto, dstPort, srcPort)
|
||||
if (app === "WireGuard" || app === "DNS" || app === "SSH" || app === "BGP") return null
|
||||
if (/youtube/i.test(ripe?.holder ?? "")) {
|
||||
return { service: "YouTube", category: "Видео / стриминг" }
|
||||
}
|
||||
const brand = lookupBrand(dst, ripe?.asn ?? 0)
|
||||
const brand = resolveFlowBrand(dst, ripe?.asn ?? 0, ripe?.holder ?? "", proto, dstPort, srcPort)
|
||||
if (!brand || !isNamedInternetService(brand.service, brand.category)) return null
|
||||
return brand
|
||||
}
|
||||
@@ -229,6 +248,21 @@ async function buildFlowMapHopsUncached(q: FlowMapHopsQuery, minSharePct: number
|
||||
const enIds = new Set(topo.enNodes.map((n) => n.id))
|
||||
let totalBytes = 0
|
||||
|
||||
function rowClient(r: (typeof working)[number]) {
|
||||
const inName = resolveIfaceName(r.serverId, r.inIface).name
|
||||
const outName = resolveIfaceName(r.serverId, r.outIface).name
|
||||
return resolveMapClient(topo, r.serverId, inName, outName)
|
||||
}
|
||||
|
||||
const payloadRows = wantDedup
|
||||
? dedupFlowRowsAcrossExporters(working, (a, b) => {
|
||||
const aCli = Boolean(rowClient(a))
|
||||
const bCli = Boolean(rowClient(b))
|
||||
if (aCli !== bCli) return aCli ? a : b
|
||||
return a.bytes >= b.bytes ? a : b
|
||||
})
|
||||
: working
|
||||
|
||||
for (const r of working) {
|
||||
const inRes = resolveIfaceName(r.serverId, r.inIface)
|
||||
const outRes = resolveIfaceName(r.serverId, r.outIface)
|
||||
@@ -314,10 +348,14 @@ async function buildFlowMapHopsUncached(q: FlowMapHopsQuery, minSharePct: number
|
||||
}, r.bytes, "fwd")
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
for (const r of payloadRows) {
|
||||
const inName = resolveIfaceName(r.serverId, r.inIface).name
|
||||
const outName = resolveIfaceName(r.serverId, r.outIface).name
|
||||
totalBytes += r.bytes
|
||||
const peer = pickInternetPeer(r.src, r.dst, r.srcPort, r.dstPort)
|
||||
const client = resolveClient(topo, r.serverId, inName)
|
||||
const client = resolveMapClient(topo, r.serverId, inName, outName)
|
||||
const prevDst = dstAcc.get(peer)
|
||||
if (prevDst) {
|
||||
prevDst.bytes += r.bytes
|
||||
@@ -386,7 +424,7 @@ async function buildFlowMapHopsUncached(q: FlowMapHopsQuery, minSharePct: number
|
||||
}
|
||||
|
||||
for (const [dst, acc] of dstAcc) {
|
||||
const ripe = lookupRipeCached(dst)
|
||||
const ripe = resolveFlowIp(dst)
|
||||
const classified = classifyMapDstLite(dst, acc.proto, acc.dstPort, acc.srcPort, ripe)
|
||||
if (!classified) continue
|
||||
const toId = mapServiceNodeId(classified.service)
|
||||
@@ -419,10 +457,15 @@ async function buildFlowMapHopsUncached(q: FlowMapHopsQuery, minSharePct: number
|
||||
const enName = nodeName(fromId)
|
||||
const viaName = nodeName(exporterId)
|
||||
for (const [clientId, c] of from.clients) {
|
||||
const pathKey = `${clientId}|${exporterId}|${fromId}|${toId}`
|
||||
const pathKey = `${clientId}|${fromId}|${toId}`
|
||||
const prevPath = svcPaths.get(pathKey)
|
||||
if (prevPath) {
|
||||
prevPath.bytes += c.bytes
|
||||
if (exporterId !== fromId && prevPath.viaId === fromId) {
|
||||
prevPath.viaId = exporterId
|
||||
prevPath.viaName = viaName
|
||||
}
|
||||
if (prevPath.clientName === "—" && c.name !== "—") prevPath.clientName = c.name
|
||||
} else {
|
||||
svcPaths.set(pathKey, {
|
||||
clientId,
|
||||
@@ -439,21 +482,20 @@ async function buildFlowMapHopsUncached(q: FlowMapHopsQuery, minSharePct: number
|
||||
}
|
||||
}
|
||||
|
||||
const minShare = minSharePct / 100
|
||||
let services: FlowMapService[] = [...svcTotals.entries()]
|
||||
.map(([id, s]) => ({
|
||||
id,
|
||||
label: s.label,
|
||||
category: s.category,
|
||||
bytes: s.bytes,
|
||||
bps: (s.bytes * 8) / windowSec,
|
||||
share: totalBytes > 0 ? s.bytes / totalBytes : 0,
|
||||
}))
|
||||
.sort((a, b) => b.bytes - a.bytes)
|
||||
if (minSharePct > 0) {
|
||||
services = services.filter((s) => s.share >= minShare)
|
||||
}
|
||||
services = services.slice(0, MAP_SERVICE_NODE_CAP)
|
||||
const namedBytes = [...svcTotals.values()].reduce((n, s) => n + s.bytes, 0)
|
||||
const services = pickMapServices(
|
||||
[...svcTotals.entries()]
|
||||
.map(([id, s]) => ({
|
||||
id,
|
||||
label: s.label,
|
||||
category: s.category,
|
||||
bytes: s.bytes,
|
||||
bps: (s.bytes * 8) / windowSec,
|
||||
share: namedBytes > 0 ? s.bytes / namedBytes : 0,
|
||||
}))
|
||||
.sort((a, b) => b.bytes - a.bytes),
|
||||
minSharePct,
|
||||
)
|
||||
const keepSvc = new Set(services.map((s) => s.id))
|
||||
const serviceEdges: FlowMapServiceEdge[] = [...svcEdges.values()]
|
||||
.filter((e) => keepSvc.has(e.toId))
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
import { and, asc, eq, gte, inArray, lt, max, or } from "drizzle-orm"
|
||||
import { and, asc, eq, gte, inArray, max, or } from "drizzle-orm"
|
||||
import { db } from "../db/index.js"
|
||||
import {
|
||||
servers,
|
||||
@@ -63,12 +63,6 @@ export async function getSettings() {
|
||||
return (await db.select().from(uptimeSettings).where(eq(uptimeSettings.id, 1)).limit(1))[0]
|
||||
}
|
||||
|
||||
async function cleanup(retentionDays: number) {
|
||||
const cutoff = new Date(Date.now() - retentionDays * 24 * 60 * 60 * 1000).toISOString()
|
||||
await db.delete(uptimeProbeSamples).where(lt(uptimeProbeSamples.sampledAt, cutoff))
|
||||
await db.delete(uptimeResourceSamples).where(lt(uptimeResourceSamples.sampledAt, cutoff))
|
||||
}
|
||||
|
||||
export function parsePing(results: Array<{ time?: string; status?: string; sent?: string; received?: string; "packet-loss"?: string; "avg-rtt"?: string }>) {
|
||||
const sum = [...results].reverse().find((r) =>
|
||||
r.sent != null || r.received != null || r["packet-loss"] != null || r["avg-rtt"] != null,
|
||||
@@ -166,8 +160,6 @@ export async function collectResourceSamplesOnce(): Promise<ResourcesRunSnapshot
|
||||
freeHddSpace: freeHdd,
|
||||
totalHddSpace: totalHdd,
|
||||
uptimeSeconds,
|
||||
boardName: String(resource["board-name"] ?? ""),
|
||||
rosVersion: String(resource["version"] ?? ""),
|
||||
})
|
||||
const memUsedMb = totalMem > 0 ? Math.round((totalMem - freeMem) / (1024 * 1024)) : 0
|
||||
const memTotalMb = totalMem > 0 ? Math.round(totalMem / (1024 * 1024)) : 0
|
||||
@@ -198,8 +190,6 @@ export async function collectResourceSamplesOnce(): Promise<ResourcesRunSnapshot
|
||||
freeHddSpace: 0,
|
||||
totalHddSpace: 0,
|
||||
uptimeSeconds: 0,
|
||||
boardName: "",
|
||||
rosVersion: "",
|
||||
})
|
||||
snapshot.servers.push({
|
||||
serverId: s.id,
|
||||
@@ -211,7 +201,6 @@ export async function collectResourceSamplesOnce(): Promise<ResourcesRunSnapshot
|
||||
}
|
||||
}
|
||||
|
||||
await cleanup(Math.max(1, settings.retentionDays))
|
||||
await db.update(uptimeSettings).set({
|
||||
lastCollectedAt: now,
|
||||
lastDurationMs: Date.now() - started,
|
||||
@@ -324,7 +313,6 @@ export async function collectPingProbesOnce(): Promise<PingRunSnapshot> {
|
||||
}
|
||||
}
|
||||
|
||||
await cleanup(Math.max(1, settings.retentionDays))
|
||||
await db.update(uptimeSettings).set({
|
||||
lastCollectedAt: now,
|
||||
lastDurationMs: Date.now() - started,
|
||||
@@ -399,7 +387,6 @@ export async function collectPingForProbeIds(probeIds: string[]): Promise<{ poll
|
||||
polled += 1
|
||||
}
|
||||
|
||||
await cleanup(Math.max(1, settings.retentionDays))
|
||||
return { polled }
|
||||
} finally {
|
||||
collectingPing = false
|
||||
|
||||
@@ -226,6 +226,19 @@ export interface BackupsRunSnapshot {
|
||||
fatalError?: string
|
||||
}
|
||||
|
||||
export interface GeoipUpdateRunSnapshot {
|
||||
v: typeof SCHEDULER_RUN_SNAPSHOT_VERSION
|
||||
job: "geoip_update"
|
||||
sampledAt: string
|
||||
skipped?: boolean
|
||||
fatalError?: string
|
||||
checked: number
|
||||
downloaded: number
|
||||
skippedUnchanged: number
|
||||
bytes: number
|
||||
errors: string[]
|
||||
}
|
||||
|
||||
export type SchedulerRunSnapshot =
|
||||
| TrafficRunSnapshot
|
||||
| ResourcesRunSnapshot
|
||||
@@ -236,4 +249,5 @@ export type SchedulerRunSnapshot =
|
||||
| InternetPathRunSnapshot
|
||||
| CertificatesRenewRunSnapshot
|
||||
| BackupsRunSnapshot
|
||||
| GeoipUpdateRunSnapshot
|
||||
| AlertEngineRunSnapshot
|
||||
|
||||
@@ -13,6 +13,7 @@ import {
|
||||
import {
|
||||
LayoutDashboardIcon,
|
||||
ActivityIcon,
|
||||
ChartColumnIcon,
|
||||
MapIcon,
|
||||
HeartPulseIcon,
|
||||
GlobeIcon,
|
||||
@@ -54,6 +55,7 @@ const navStructure: { label: string; items: NavItemBase[] }[] = [
|
||||
items: [
|
||||
{ title: "Дашборд", url: "/dashboard", icon: <LayoutDashboardIcon /> },
|
||||
{ title: "Трафик", url: "/traffic", icon: <ActivityIcon /> },
|
||||
{ title: "Статистика", url: "/statistics", icon: <ChartColumnIcon /> },
|
||||
{ title: "Карта сети", url: "/network-map", icon: <MapIcon /> },
|
||||
{ title: "Мониторинг", url: "/uptime", icon: <HeartPulseIcon /> },
|
||||
],
|
||||
|
||||
@@ -0,0 +1,97 @@
|
||||
"use client"
|
||||
|
||||
import {
|
||||
AlertDialog,
|
||||
AlertDialogAction,
|
||||
AlertDialogCancel,
|
||||
AlertDialogContent,
|
||||
AlertDialogDescription,
|
||||
AlertDialogFooter,
|
||||
AlertDialogHeader,
|
||||
AlertDialogMedia,
|
||||
AlertDialogTitle,
|
||||
} from "@/components/ui/alert-dialog"
|
||||
import { Alert, AlertDescription, AlertTitle } from "@/components/reui/alert"
|
||||
import type { Backup } from "@/lib/data"
|
||||
import { AlertCircleIcon, LoaderCircleIcon, TriangleAlertIcon } from "lucide-react"
|
||||
|
||||
export function BackupDeleteDialog({
|
||||
backup,
|
||||
busy,
|
||||
onConfirm,
|
||||
onCancel,
|
||||
}: {
|
||||
backup: Backup | null
|
||||
busy: boolean
|
||||
onConfirm: () => void
|
||||
onCancel: () => void
|
||||
}) {
|
||||
return (
|
||||
<AlertDialog open={Boolean(backup)} onOpenChange={(v) => { if (!v && !busy) onCancel() }}>
|
||||
<AlertDialogContent>
|
||||
<AlertDialogHeader>
|
||||
<AlertDialogMedia className="bg-destructive/10 text-destructive">
|
||||
<AlertCircleIcon />
|
||||
</AlertDialogMedia>
|
||||
<AlertDialogTitle>Удалить бэкап?</AlertDialogTitle>
|
||||
<AlertDialogDescription>
|
||||
Файл <span className="font-mono text-foreground">{backup?.filename}</span> будет удалён
|
||||
{backup?.storage === "s3" || backup?.storage === "both" ? " локально и из S3" : " с диска"}.
|
||||
</AlertDialogDescription>
|
||||
</AlertDialogHeader>
|
||||
<AlertDialogFooter>
|
||||
<AlertDialogCancel disabled={busy} onClick={onCancel}>Отмена</AlertDialogCancel>
|
||||
<AlertDialogAction variant="destructive" disabled={busy} onClick={onConfirm}>
|
||||
{busy ? <LoaderCircleIcon className="size-4 animate-spin" /> : null}
|
||||
Удалить
|
||||
</AlertDialogAction>
|
||||
</AlertDialogFooter>
|
||||
</AlertDialogContent>
|
||||
</AlertDialog>
|
||||
)
|
||||
}
|
||||
|
||||
export function BackupRestoreDialog({
|
||||
backup,
|
||||
busy,
|
||||
onConfirm,
|
||||
onCancel,
|
||||
}: {
|
||||
backup: Backup | null
|
||||
busy: boolean
|
||||
onConfirm: () => void
|
||||
onCancel: () => void
|
||||
}) {
|
||||
return (
|
||||
<AlertDialog open={Boolean(backup)} onOpenChange={(v) => { if (!v && !busy) onCancel() }}>
|
||||
<AlertDialogContent>
|
||||
<AlertDialogHeader>
|
||||
<AlertDialogMedia className="bg-warning/10 text-warning">
|
||||
<TriangleAlertIcon />
|
||||
</AlertDialogMedia>
|
||||
<AlertDialogTitle>Восстановить конфигурацию?</AlertDialogTitle>
|
||||
<AlertDialogDescription className="flex flex-col gap-3">
|
||||
<span>
|
||||
Файл <span className="font-mono text-foreground">{backup?.filename}</span> будет загружен
|
||||
на <span className="text-foreground">{backup?.server}</span> и импортирован.
|
||||
</span>
|
||||
<Alert variant="warning">
|
||||
<TriangleAlertIcon />
|
||||
<AlertTitle>Это изменит рабочую конфигурацию роутера</AlertTitle>
|
||||
<AlertDescription>
|
||||
Сессия может оборваться. Убедитесь, что выбран именно этот сервер и этот снимок.
|
||||
</AlertDescription>
|
||||
</Alert>
|
||||
</AlertDialogDescription>
|
||||
</AlertDialogHeader>
|
||||
<AlertDialogFooter>
|
||||
<AlertDialogCancel disabled={busy} onClick={onCancel}>Отмена</AlertDialogCancel>
|
||||
<AlertDialogAction disabled={busy} onClick={onConfirm}>
|
||||
{busy ? <LoaderCircleIcon className="size-4 animate-spin" /> : null}
|
||||
Восстановить
|
||||
</AlertDialogAction>
|
||||
</AlertDialogFooter>
|
||||
</AlertDialogContent>
|
||||
</AlertDialog>
|
||||
)
|
||||
}
|
||||
@@ -0,0 +1,191 @@
|
||||
"use client"
|
||||
|
||||
import type { Server } from "@/lib/data"
|
||||
import { FormField } from "@/components/form-kit"
|
||||
import { StatusBadge } from "@/components/status-badge"
|
||||
import { Button } from "@/components/ui/button"
|
||||
import { Checkbox } from "@/components/ui/checkbox"
|
||||
import { Input } from "@/components/ui/input"
|
||||
import {
|
||||
Sheet,
|
||||
SheetContent,
|
||||
SheetHeader,
|
||||
SheetTitle,
|
||||
SheetDescription,
|
||||
SheetFooter,
|
||||
SheetClose,
|
||||
} from "@/components/ui/sheet"
|
||||
import {
|
||||
Stepper,
|
||||
StepperContent,
|
||||
StepperIndicator,
|
||||
StepperItem,
|
||||
StepperNav,
|
||||
StepperPanel,
|
||||
StepperSeparator,
|
||||
StepperTitle,
|
||||
StepperTrigger,
|
||||
} from "@/components/reui/stepper"
|
||||
import { cn } from "@/lib/utils"
|
||||
|
||||
export function BackupCreateSheet({
|
||||
open,
|
||||
onOpenChange,
|
||||
step,
|
||||
onStepChange,
|
||||
servers,
|
||||
selected,
|
||||
onToggle,
|
||||
onSelectAll,
|
||||
onClear,
|
||||
notes,
|
||||
onNotesChange,
|
||||
destinationLabel,
|
||||
busy,
|
||||
onSubmit,
|
||||
}: {
|
||||
open: boolean
|
||||
onOpenChange: (open: boolean) => void
|
||||
step: number
|
||||
onStepChange: (step: number) => void
|
||||
servers: Server[]
|
||||
selected: Set<string>
|
||||
onToggle: (id: string) => void
|
||||
onSelectAll: () => void
|
||||
onClear: () => void
|
||||
notes: string
|
||||
onNotesChange: (value: string) => void
|
||||
destinationLabel: string
|
||||
busy: boolean
|
||||
onSubmit: () => void
|
||||
}) {
|
||||
return (
|
||||
<Sheet open={open} onOpenChange={(v) => { onOpenChange(v); if (!v) onStepChange(1) }}>
|
||||
<SheetContent side="right" className="w-full sm:max-w-md flex flex-col gap-0 p-0">
|
||||
<SheetHeader className="px-6 pt-6 pb-4 border-b shrink-0">
|
||||
<SheetTitle>Новый бэкап</SheetTitle>
|
||||
<SheetDescription>Снять конфигурацию вручную с выбранных серверов</SheetDescription>
|
||||
</SheetHeader>
|
||||
|
||||
<Stepper value={step} onValueChange={onStepChange} className="flex-1 flex flex-col min-h-0 px-6 py-5">
|
||||
<StepperNav className="mb-5">
|
||||
<StepperItem step={1}>
|
||||
<StepperTrigger>
|
||||
<StepperIndicator>1</StepperIndicator>
|
||||
<StepperTitle className="sr-only">Серверы</StepperTitle>
|
||||
</StepperTrigger>
|
||||
<StepperSeparator />
|
||||
</StepperItem>
|
||||
<StepperItem step={2}>
|
||||
<StepperTrigger>
|
||||
<StepperIndicator>2</StepperIndicator>
|
||||
<StepperTitle className="sr-only">Заметка</StepperTitle>
|
||||
</StepperTrigger>
|
||||
<StepperSeparator />
|
||||
</StepperItem>
|
||||
<StepperItem step={3}>
|
||||
<StepperTrigger>
|
||||
<StepperIndicator>3</StepperIndicator>
|
||||
<StepperTitle className="sr-only">Подтверждение</StepperTitle>
|
||||
</StepperTrigger>
|
||||
</StepperItem>
|
||||
</StepperNav>
|
||||
<StepperPanel className="flex-1 overflow-y-auto">
|
||||
<StepperContent value={1} className="flex flex-col gap-3">
|
||||
<div className="flex items-center justify-between mb-1">
|
||||
<p className="text-sm font-medium">Выберите серверы</p>
|
||||
<div className="flex items-center gap-2">
|
||||
<button type="button" onClick={onSelectAll} className="text-xs text-primary hover:underline">
|
||||
Все
|
||||
</button>
|
||||
<span className="text-border">·</span>
|
||||
<button
|
||||
type="button"
|
||||
onClick={onClear}
|
||||
className="text-xs text-muted-foreground hover:text-foreground hover:underline"
|
||||
>
|
||||
Сбросить
|
||||
</button>
|
||||
</div>
|
||||
</div>
|
||||
{servers.map((s) => {
|
||||
const checked = selected.has(s.id)
|
||||
return (
|
||||
<label
|
||||
key={s.id}
|
||||
className={cn(
|
||||
"flex cursor-pointer items-center gap-3 rounded-lg border p-3 text-left transition-colors",
|
||||
checked ? "border-primary/40 bg-primary/5" : "border-border hover:bg-muted/40",
|
||||
)}
|
||||
>
|
||||
<Checkbox
|
||||
checked={checked}
|
||||
onCheckedChange={() => onToggle(s.id)}
|
||||
aria-label={`Выбрать ${s.name}`}
|
||||
/>
|
||||
<div className="flex-1 min-w-0">
|
||||
<p className="text-sm font-medium">{s.name}</p>
|
||||
<div className="flex items-center gap-1.5 mt-0.5">
|
||||
<span className="text-xs font-mono text-muted-foreground">{s.host}</span>
|
||||
<StatusBadge status={s.status} />
|
||||
</div>
|
||||
</div>
|
||||
{s.status === "offline" && (
|
||||
<span className="text-xs text-muted-foreground">недоступен</span>
|
||||
)}
|
||||
</label>
|
||||
)
|
||||
})}
|
||||
</StepperContent>
|
||||
<StepperContent value={2} className="flex flex-col gap-4">
|
||||
<FormField label="Заметка">
|
||||
<Input
|
||||
placeholder="Например: перед обновлением BGP"
|
||||
value={notes}
|
||||
onChange={(e) => onNotesChange(e.target.value)}
|
||||
/>
|
||||
</FormField>
|
||||
</StepperContent>
|
||||
<StepperContent value={3} className="flex flex-col gap-3 text-sm">
|
||||
<p className="text-muted-foreground">
|
||||
Будет создан бэкап для <strong className="text-foreground">{selected.size}</strong> серверов.
|
||||
</p>
|
||||
<p className="text-muted-foreground">
|
||||
Куда сохранится: <strong className="text-foreground">{destinationLabel}</strong>
|
||||
</p>
|
||||
{notes ? <p className="text-muted-foreground">Заметка: {notes}</p> : null}
|
||||
</StepperContent>
|
||||
</StepperPanel>
|
||||
</Stepper>
|
||||
|
||||
<SheetFooter className="px-6 py-4 border-t shrink-0 flex-row gap-2">
|
||||
<SheetClose render={<Button variant="outline" className="flex-1" />}>Отмена</SheetClose>
|
||||
{step > 1 && (
|
||||
<Button type="button" variant="outline" className="flex-1" onClick={() => onStepChange(step - 1)}>
|
||||
Назад
|
||||
</Button>
|
||||
)}
|
||||
{step < 3 ? (
|
||||
<Button
|
||||
type="button"
|
||||
className="flex-1"
|
||||
disabled={step === 1 && selected.size === 0}
|
||||
onClick={() => onStepChange(step + 1)}
|
||||
>
|
||||
Далее
|
||||
</Button>
|
||||
) : (
|
||||
<Button
|
||||
type="button"
|
||||
className="flex-1"
|
||||
disabled={selected.size === 0 || busy}
|
||||
onClick={onSubmit}
|
||||
>
|
||||
Снять бэкап ({selected.size})
|
||||
</Button>
|
||||
)}
|
||||
</SheetFooter>
|
||||
</SheetContent>
|
||||
</Sheet>
|
||||
)
|
||||
}
|
||||
@@ -0,0 +1,97 @@
|
||||
"use client"
|
||||
|
||||
import type { Filter } from "@/components/reui/filters"
|
||||
import type { Backup } from "@/lib/data"
|
||||
import { applyReuiFilters } from "@/lib/data-filters/apply-reui-filters"
|
||||
import {
|
||||
BACKUP_FILTER_ACCESSORS,
|
||||
BACKUP_FILTER_FIELDS,
|
||||
} from "@/lib/data-filters/backup-filter-fields"
|
||||
import { DataPageCard } from "@/components/data-page-card"
|
||||
import { DataPageToolbar } from "@/components/data-page-toolbar"
|
||||
import { BackupsDataGrid } from "@/components/data-grids/backups-data-grid"
|
||||
import { Button } from "@/components/ui/button"
|
||||
import { PlusIcon } from "lucide-react"
|
||||
|
||||
type KindFilter = "all" | "auto" | "manual"
|
||||
|
||||
export function BackupsHistory({
|
||||
backups,
|
||||
kindFilter,
|
||||
onKindFilterChange,
|
||||
search,
|
||||
onSearchChange,
|
||||
filters,
|
||||
onFiltersChange,
|
||||
onDownload,
|
||||
onRestore,
|
||||
onDelete,
|
||||
onCreate,
|
||||
}: {
|
||||
backups: Backup[]
|
||||
kindFilter: KindFilter
|
||||
onKindFilterChange: (value: KindFilter) => void
|
||||
search: string
|
||||
onSearchChange: (value: string) => void
|
||||
filters: Filter[]
|
||||
onFiltersChange: (filters: Filter[]) => void
|
||||
onDownload: (id: string, filename: string) => void
|
||||
onRestore: (backup: Backup) => void
|
||||
onDelete: (backup: Backup) => void
|
||||
onCreate: () => void
|
||||
}) {
|
||||
const autoCount = backups.filter((b) => b.kind === "auto").length
|
||||
const manualCount = backups.filter((b) => b.kind === "manual").length
|
||||
const byKind = kindFilter === "all" ? backups : backups.filter((b) => b.kind === kindFilter)
|
||||
const q = search.trim().toLowerCase()
|
||||
const searched = q
|
||||
? byKind.filter((b) =>
|
||||
[b.filename, b.server, b.notes].some((v) => v.toLowerCase().includes(q)),
|
||||
)
|
||||
: byKind
|
||||
const filtered = applyReuiFilters(searched, filters, BACKUP_FILTER_ACCESSORS)
|
||||
const isEmptyAll = backups.length === 0
|
||||
|
||||
return (
|
||||
<DataPageCard>
|
||||
<DataPageToolbar
|
||||
segmented={{
|
||||
value: kindFilter,
|
||||
onChange: onKindFilterChange,
|
||||
options: [
|
||||
{ value: "all", label: "Все", count: backups.length },
|
||||
{ value: "auto", label: "Авто", count: autoCount },
|
||||
{ value: "manual", label: "Вручную", count: manualCount },
|
||||
],
|
||||
}}
|
||||
filters={filters}
|
||||
onFiltersChange={onFiltersChange}
|
||||
filterFields={BACKUP_FILTER_FIELDS}
|
||||
search={search}
|
||||
onSearchChange={onSearchChange}
|
||||
searchPlaceholder="Поиск по файлу, серверу, заметке…"
|
||||
countLabel={`${filtered.length} бэкапов`}
|
||||
/>
|
||||
<BackupsDataGrid
|
||||
backups={filtered}
|
||||
onDownload={onDownload}
|
||||
onRestore={onRestore}
|
||||
onDelete={onDelete}
|
||||
emptyTitle={isEmptyAll ? "Нет бэкапов" : "Ничего не найдено"}
|
||||
emptyDescription={
|
||||
isEmptyAll
|
||||
? "Создайте первый бэкап вручную или настройте расписание"
|
||||
: "Измените фильтры или поисковый запрос"
|
||||
}
|
||||
emptyAction={
|
||||
isEmptyAll ? (
|
||||
<Button type="button" size="sm" onClick={onCreate}>
|
||||
<PlusIcon className="size-4" />
|
||||
Новый бэкап
|
||||
</Button>
|
||||
) : null
|
||||
}
|
||||
/>
|
||||
</DataPageCard>
|
||||
)
|
||||
}
|
||||
@@ -0,0 +1,419 @@
|
||||
"use client"
|
||||
|
||||
import type { Server } from "@/lib/data"
|
||||
import type { BackupStorageSettingsDto } from "@mmapp/contracts/backups"
|
||||
import { Badge } from "@/components/reui/badge"
|
||||
import { OpsPanel } from "@/components/ops-panel"
|
||||
import { FormField, FormToggle, SegmentedControl } from "@/components/form-kit"
|
||||
import { StatusBadge } from "@/components/status-badge"
|
||||
import { Button } from "@/components/ui/button"
|
||||
import { Checkbox } from "@/components/ui/checkbox"
|
||||
import { Input } from "@/components/ui/input"
|
||||
import { cn } from "@/lib/utils"
|
||||
import { LoaderCircleIcon } from "lucide-react"
|
||||
|
||||
export const WEEK_DAYS = ["Пн", "Вт", "Ср", "Чт", "Пт", "Сб", "Вс"]
|
||||
|
||||
export type BackupFreq = "daily" | "weekly" | "monthly"
|
||||
export type StorageProvider = "local" | "s3"
|
||||
|
||||
export type BackupScheduleForm = {
|
||||
enabled: boolean
|
||||
frequency: BackupFreq
|
||||
hour: number
|
||||
minute: number
|
||||
weekDay: number
|
||||
monthDay: number
|
||||
keepCount: number
|
||||
format: "rsc" | "backup"
|
||||
}
|
||||
|
||||
export type BackupStorageForm = {
|
||||
provider: StorageProvider
|
||||
s3Endpoint: string
|
||||
s3Region: string
|
||||
s3Bucket: string
|
||||
s3Prefix: string
|
||||
s3AccessKeyId: string
|
||||
s3SecretAccessKey: string
|
||||
s3ForcePathStyle: boolean
|
||||
keepLocalCopy: boolean
|
||||
showPassword: boolean
|
||||
}
|
||||
|
||||
export const defaultSchedule: BackupScheduleForm = {
|
||||
enabled: true,
|
||||
frequency: "daily",
|
||||
hour: 3,
|
||||
minute: 0,
|
||||
weekDay: 0,
|
||||
monthDay: 1,
|
||||
keepCount: 7,
|
||||
format: "rsc",
|
||||
}
|
||||
|
||||
export const defaultStorageForm: BackupStorageForm = {
|
||||
provider: "local",
|
||||
s3Endpoint: "",
|
||||
s3Region: "us-east-1",
|
||||
s3Bucket: "",
|
||||
s3Prefix: "mikrotik",
|
||||
s3AccessKeyId: "",
|
||||
s3SecretAccessKey: "",
|
||||
s3ForcePathStyle: true,
|
||||
keepLocalCopy: true,
|
||||
showPassword: false,
|
||||
}
|
||||
|
||||
function storageStatus(saved: BackupStorageSettingsDto | null, form: BackupStorageForm) {
|
||||
if (form.provider === "local") {
|
||||
return { label: "Локально", variant: "secondary" as const }
|
||||
}
|
||||
if (saved?.lastTestError) {
|
||||
return { label: "Ошибка", variant: "destructive-light" as const }
|
||||
}
|
||||
if (saved?.lastTestAt && !saved.lastTestError) {
|
||||
return { label: "Connected", variant: "success-light" as const }
|
||||
}
|
||||
if (saved?.secretConfigured && saved.s3Bucket) {
|
||||
return { label: "Не проверено", variant: "warning-light" as const }
|
||||
}
|
||||
return { label: "Не настроено", variant: "secondary" as const }
|
||||
}
|
||||
|
||||
export function BackupsSettings({
|
||||
schedule,
|
||||
onScheduleChange,
|
||||
storage,
|
||||
onStorageChange,
|
||||
savedStorage,
|
||||
servers,
|
||||
selectedServers,
|
||||
onToggleServer,
|
||||
onSelectAll,
|
||||
onClearServers,
|
||||
onSave,
|
||||
onTest,
|
||||
onSync,
|
||||
saveBusy,
|
||||
testBusy,
|
||||
syncBusy,
|
||||
}: {
|
||||
schedule: BackupScheduleForm
|
||||
onScheduleChange: <K extends keyof BackupScheduleForm>(k: K, v: BackupScheduleForm[K]) => void
|
||||
storage: BackupStorageForm
|
||||
onStorageChange: <K extends keyof BackupStorageForm>(k: K, v: BackupStorageForm[K]) => void
|
||||
savedStorage: BackupStorageSettingsDto | null
|
||||
servers: Server[]
|
||||
selectedServers: Set<string>
|
||||
onToggleServer: (id: string) => void
|
||||
onSelectAll: () => void
|
||||
onClearServers: () => void
|
||||
onSave: () => void
|
||||
onTest: () => void
|
||||
onSync: () => void
|
||||
saveBusy: boolean
|
||||
testBusy: boolean
|
||||
syncBusy: boolean
|
||||
}) {
|
||||
const status = storageStatus(savedStorage, storage)
|
||||
const secretPlaceholder = savedStorage?.secretConfigured ? "•••••••• (сохранён)" : "••••••••"
|
||||
|
||||
return (
|
||||
<div className="grid grid-cols-1 lg:grid-cols-2 gap-5">
|
||||
<OpsPanel title="Расписание" description="Автоматический съём конфигурации" contentClassName="px-5 py-5 flex flex-col gap-5">
|
||||
<div className="flex items-center justify-between gap-3">
|
||||
<div>
|
||||
<p className="text-sm font-medium">Автоматический бэкап</p>
|
||||
<p className="text-xs text-muted-foreground mt-0.5">Создавать бэкапы по расписанию</p>
|
||||
</div>
|
||||
<FormToggle checked={schedule.enabled} onChange={(v) => onScheduleChange("enabled", v)} />
|
||||
</div>
|
||||
|
||||
<div className={cn("flex flex-col gap-4", !schedule.enabled && "opacity-40 pointer-events-none")}>
|
||||
<FormField label="Частота">
|
||||
<SegmentedControl
|
||||
value={schedule.frequency}
|
||||
onChange={(v) => onScheduleChange("frequency", v)}
|
||||
options={[
|
||||
{ value: "daily", label: "Ежедневно" },
|
||||
{ value: "weekly", label: "Еженедельно" },
|
||||
{ value: "monthly", label: "Ежемесячно" },
|
||||
]}
|
||||
/>
|
||||
</FormField>
|
||||
|
||||
{schedule.frequency === "weekly" && (
|
||||
<FormField label="День недели">
|
||||
<div className="flex gap-1">
|
||||
{WEEK_DAYS.map((d, i) => (
|
||||
<button
|
||||
key={d}
|
||||
type="button"
|
||||
onClick={() => onScheduleChange("weekDay", i)}
|
||||
className={cn(
|
||||
"w-9 h-9 rounded text-sm font-medium border transition-colors",
|
||||
schedule.weekDay === i
|
||||
? "bg-primary text-primary-foreground border-primary"
|
||||
: "border-border text-muted-foreground hover:text-foreground",
|
||||
)}
|
||||
>
|
||||
{d}
|
||||
</button>
|
||||
))}
|
||||
</div>
|
||||
</FormField>
|
||||
)}
|
||||
|
||||
{schedule.frequency === "monthly" && (
|
||||
<FormField label="День месяца" hint="1–28">
|
||||
<Input
|
||||
type="number"
|
||||
min={1}
|
||||
max={28}
|
||||
className="font-mono w-24"
|
||||
value={schedule.monthDay}
|
||||
onChange={(e) => onScheduleChange("monthDay", Math.min(28, Math.max(1, Number(e.target.value))))}
|
||||
/>
|
||||
</FormField>
|
||||
)}
|
||||
|
||||
<FormField label="Время запуска">
|
||||
<div className="flex items-center gap-2">
|
||||
<Input
|
||||
type="number"
|
||||
min={0}
|
||||
max={23}
|
||||
className="font-mono w-20 text-center"
|
||||
value={String(schedule.hour).padStart(2, "0")}
|
||||
onChange={(e) => onScheduleChange("hour", Math.min(23, Math.max(0, Number(e.target.value))))}
|
||||
/>
|
||||
<span className="text-muted-foreground font-mono text-lg">:</span>
|
||||
<div className="flex gap-1">
|
||||
{[0, 15, 30, 45].map((m) => (
|
||||
<button
|
||||
key={m}
|
||||
type="button"
|
||||
onClick={() => onScheduleChange("minute", m)}
|
||||
className={cn(
|
||||
"px-2.5 py-1.5 rounded text-xs font-mono border transition-colors",
|
||||
schedule.minute === m
|
||||
? "bg-primary text-primary-foreground border-primary"
|
||||
: "border-border text-muted-foreground hover:text-foreground",
|
||||
)}
|
||||
>
|
||||
{String(m).padStart(2, "0")}
|
||||
</button>
|
||||
))}
|
||||
</div>
|
||||
</div>
|
||||
</FormField>
|
||||
|
||||
<FormField label="Хранить бэкапов" hint="На каждый сервер">
|
||||
<Input
|
||||
type="number"
|
||||
min={1}
|
||||
max={90}
|
||||
className="font-mono w-24"
|
||||
value={schedule.keepCount}
|
||||
onChange={(e) => onScheduleChange("keepCount", Math.max(1, Number(e.target.value)))}
|
||||
/>
|
||||
</FormField>
|
||||
|
||||
<div className="flex items-center justify-between gap-3 rounded-lg border border-border px-3 py-3">
|
||||
<div>
|
||||
<p className="text-sm font-medium">Формат файла</p>
|
||||
<p className="text-xs text-muted-foreground mt-0.5">Снимается текстовый экспорт RouterOS</p>
|
||||
</div>
|
||||
<div className="flex items-center gap-2">
|
||||
<Badge variant="secondary" size="sm">.rsc</Badge>
|
||||
<Badge variant="warning-light" size="sm">.backup скоро</Badge>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</OpsPanel>
|
||||
|
||||
<OpsPanel
|
||||
title="Хранилище"
|
||||
description="Локальный диск приложения или S3-compatible бакет"
|
||||
headerRight={
|
||||
<Badge variant={status.variant} size="sm" radius="full">
|
||||
{status.label}
|
||||
</Badge>
|
||||
}
|
||||
contentClassName="px-5 py-5 flex flex-col gap-5"
|
||||
>
|
||||
<FormField label="Тип хранилища">
|
||||
<SegmentedControl
|
||||
value={storage.provider}
|
||||
onChange={(v) => onStorageChange("provider", v)}
|
||||
options={[
|
||||
{ value: "local", label: "Локально" },
|
||||
{ value: "s3", label: "S3" },
|
||||
]}
|
||||
/>
|
||||
</FormField>
|
||||
|
||||
{storage.provider === "local" ? (
|
||||
<p className="text-xs text-muted-foreground">
|
||||
Файлы пишутся в каталог приложения <span className="font-mono text-foreground">storage/backups</span>.
|
||||
</p>
|
||||
) : (
|
||||
<div className="flex flex-col gap-4">
|
||||
<FormField label="Endpoint" hint="Пусто для AWS. Для R2/MinIO/Selectel — полный URL">
|
||||
<Input
|
||||
className="font-mono"
|
||||
placeholder="https://s3.amazonaws.com"
|
||||
value={storage.s3Endpoint}
|
||||
onChange={(e) => onStorageChange("s3Endpoint", e.target.value)}
|
||||
/>
|
||||
</FormField>
|
||||
<div className="grid grid-cols-2 gap-3">
|
||||
<FormField label="Region">
|
||||
<Input
|
||||
className="font-mono"
|
||||
placeholder="us-east-1"
|
||||
value={storage.s3Region}
|
||||
onChange={(e) => onStorageChange("s3Region", e.target.value)}
|
||||
/>
|
||||
</FormField>
|
||||
<FormField label="Bucket" required>
|
||||
<Input
|
||||
className="font-mono"
|
||||
placeholder="mikrotik-backups"
|
||||
value={storage.s3Bucket}
|
||||
onChange={(e) => onStorageChange("s3Bucket", e.target.value)}
|
||||
/>
|
||||
</FormField>
|
||||
</div>
|
||||
<FormField label="Prefix">
|
||||
<Input
|
||||
className="font-mono"
|
||||
placeholder="mikrotik"
|
||||
value={storage.s3Prefix}
|
||||
onChange={(e) => onStorageChange("s3Prefix", e.target.value)}
|
||||
/>
|
||||
</FormField>
|
||||
<div className="grid grid-cols-2 gap-3">
|
||||
<FormField label="Access key" required>
|
||||
<Input
|
||||
className="font-mono"
|
||||
autoComplete="off"
|
||||
value={storage.s3AccessKeyId}
|
||||
onChange={(e) => onStorageChange("s3AccessKeyId", e.target.value)}
|
||||
/>
|
||||
</FormField>
|
||||
<FormField label="Secret key">
|
||||
<div className="relative">
|
||||
<Input
|
||||
type={storage.showPassword ? "text" : "password"}
|
||||
className="font-mono pr-14"
|
||||
autoComplete="new-password"
|
||||
placeholder={secretPlaceholder}
|
||||
value={storage.s3SecretAccessKey}
|
||||
onChange={(e) => onStorageChange("s3SecretAccessKey", e.target.value)}
|
||||
/>
|
||||
<button
|
||||
type="button"
|
||||
onClick={() => onStorageChange("showPassword", !storage.showPassword)}
|
||||
className="absolute right-2.5 top-1/2 -translate-y-1/2 text-xs text-muted-foreground hover:text-foreground"
|
||||
>
|
||||
{storage.showPassword ? "скрыть" : "показ"}
|
||||
</button>
|
||||
</div>
|
||||
</FormField>
|
||||
</div>
|
||||
<div className="flex items-center justify-between gap-3">
|
||||
<div>
|
||||
<p className="text-sm font-medium">Path-style</p>
|
||||
<p className="text-xs text-muted-foreground mt-0.5">Нужен для MinIO и части совместимых API</p>
|
||||
</div>
|
||||
<FormToggle checked={storage.s3ForcePathStyle} onChange={(v) => onStorageChange("s3ForcePathStyle", v)} />
|
||||
</div>
|
||||
<div className="flex items-center justify-between gap-3">
|
||||
<div>
|
||||
<p className="text-sm font-medium">Оставлять локальную копию</p>
|
||||
<p className="text-xs text-muted-foreground mt-0.5">После успешной загрузки в S3</p>
|
||||
</div>
|
||||
<FormToggle checked={storage.keepLocalCopy} onChange={(v) => onStorageChange("keepLocalCopy", v)} />
|
||||
</div>
|
||||
{savedStorage?.lastTestError ? (
|
||||
<p className="text-xs text-destructive">{savedStorage.lastTestError}</p>
|
||||
) : null}
|
||||
<div className="flex flex-wrap items-center gap-2">
|
||||
<Button type="button" variant="outline" size="sm" onClick={onTest} disabled={testBusy}>
|
||||
{testBusy ? <LoaderCircleIcon className="size-3.5 animate-spin" /> : null}
|
||||
Проверить
|
||||
</Button>
|
||||
<Button type="button" variant="outline" size="sm" onClick={onSync} disabled={syncBusy}>
|
||||
{syncBusy ? <LoaderCircleIcon className="size-3.5 animate-spin" /> : null}
|
||||
Синхронизировать из бакета
|
||||
</Button>
|
||||
</div>
|
||||
</div>
|
||||
)}
|
||||
</OpsPanel>
|
||||
|
||||
<OpsPanel
|
||||
className="lg:col-span-2"
|
||||
title="Серверы для бэкапа"
|
||||
headerRight={
|
||||
<div className="flex items-center gap-2 shrink-0">
|
||||
<button type="button" onClick={onSelectAll} className="text-xs text-primary hover:underline">
|
||||
Выбрать все
|
||||
</button>
|
||||
<span className="text-border">·</span>
|
||||
<button
|
||||
type="button"
|
||||
onClick={onClearServers}
|
||||
className="text-xs text-muted-foreground hover:text-foreground hover:underline"
|
||||
>
|
||||
Сбросить
|
||||
</button>
|
||||
</div>
|
||||
}
|
||||
contentClassName="px-5 py-5 flex flex-col gap-4"
|
||||
>
|
||||
<div className="grid grid-cols-1 sm:grid-cols-2 lg:grid-cols-3 xl:grid-cols-4 gap-2">
|
||||
{servers.map((s) => {
|
||||
const checked = selectedServers.has(s.id)
|
||||
return (
|
||||
<label
|
||||
key={s.id}
|
||||
className={cn(
|
||||
"flex cursor-pointer items-center gap-3 rounded-lg border p-3 text-left transition-colors",
|
||||
checked
|
||||
? "border-primary/40 bg-primary/5"
|
||||
: "border-border hover:border-border/80 hover:bg-muted/40",
|
||||
)}
|
||||
>
|
||||
<Checkbox
|
||||
checked={checked}
|
||||
onCheckedChange={() => onToggleServer(s.id)}
|
||||
aria-label={`Выбрать ${s.name}`}
|
||||
/>
|
||||
<div className="flex-1 min-w-0">
|
||||
<p className="text-sm font-medium truncate">{s.name}</p>
|
||||
<div className="flex items-center gap-1.5 mt-0.5">
|
||||
<span className="text-xs text-muted-foreground">{s.site}</span>
|
||||
<StatusBadge status={s.status} />
|
||||
</div>
|
||||
</div>
|
||||
</label>
|
||||
)
|
||||
})}
|
||||
</div>
|
||||
<p className="text-xs text-muted-foreground">
|
||||
Выбрано {selectedServers.size} из {servers.length} серверов
|
||||
</p>
|
||||
</OpsPanel>
|
||||
|
||||
<div className="lg:col-span-2 flex items-center gap-3">
|
||||
<Button type="button" onClick={onSave} className="gap-2" disabled={saveBusy}>
|
||||
{saveBusy ? <LoaderCircleIcon className="size-4 animate-spin" /> : null}
|
||||
Сохранить настройки
|
||||
</Button>
|
||||
</div>
|
||||
</div>
|
||||
)
|
||||
}
|
||||
@@ -0,0 +1,184 @@
|
||||
"use client"
|
||||
|
||||
import { useCallback, useEffect, useState } from "react"
|
||||
import Link from "next/link"
|
||||
import { OpsPanel } from "@/components/ops-panel"
|
||||
import { FormField, FormToggle } from "@/components/form-kit"
|
||||
import { Alert, AlertDescription, AlertTitle } from "@/components/reui/alert"
|
||||
import { Badge } from "@/components/reui/badge"
|
||||
import { Button, buttonVariants } from "@/components/ui/button"
|
||||
import { Input } from "@/components/ui/input"
|
||||
import { cn } from "@/lib/utils"
|
||||
import {
|
||||
getCertificateRenewSettings,
|
||||
putCertificateRenewSettings,
|
||||
} from "@/shared/api/certificates"
|
||||
import { toast } from "sonner"
|
||||
|
||||
/**
|
||||
* Автообновление сертификатов через MM (ACME DNS-01).
|
||||
* Preview: https://reui.io/preview/base/settings-16 · https://reui.io/preview/base/settings-3
|
||||
* Docs: https://reui.io/docs/components/base/frame · https://reui.io/docs/components/base/alert · https://reui.io/docs/components/base/badge
|
||||
*/
|
||||
export function CertificateRenewSettingsPanel({
|
||||
backendUrl,
|
||||
liveReady,
|
||||
}: {
|
||||
backendUrl: string
|
||||
liveReady: boolean
|
||||
}) {
|
||||
const [enabled, setEnabled] = useState(true)
|
||||
const [intervalDraft, setIntervalDraft] = useState("21600")
|
||||
const [daysDraft, setDaysDraft] = useState("30")
|
||||
const [lastCollectedAt, setLastCollectedAt] = useState<string | null>(null)
|
||||
const [lastError, setLastError] = useState<string | null>(null)
|
||||
const [loaded, setLoaded] = useState(false)
|
||||
const [toggleBusy, setToggleBusy] = useState(false)
|
||||
const [saveBusy, setSaveBusy] = useState(false)
|
||||
|
||||
const load = useCallback(async () => {
|
||||
if (!liveReady) return
|
||||
try {
|
||||
const s = await getCertificateRenewSettings(backendUrl)
|
||||
setEnabled(s.enabled)
|
||||
setIntervalDraft(String(s.intervalSec))
|
||||
setDaysDraft(String(s.renewBeforeDays))
|
||||
setLastCollectedAt(s.lastCollectedAt ?? null)
|
||||
setLastError(s.lastError ?? null)
|
||||
setLoaded(true)
|
||||
} catch (e) {
|
||||
setLoaded(true)
|
||||
toast.error(e instanceof Error ? e.message : "Не удалось загрузить настройки автообновления")
|
||||
}
|
||||
}, [backendUrl, liveReady])
|
||||
|
||||
useEffect(() => {
|
||||
queueMicrotask(() => {
|
||||
void load()
|
||||
})
|
||||
}, [load])
|
||||
|
||||
async function handleEnabledChange(next: boolean) {
|
||||
if (!liveReady || toggleBusy) return
|
||||
const prev = enabled
|
||||
setEnabled(next)
|
||||
setToggleBusy(true)
|
||||
try {
|
||||
const saved = await putCertificateRenewSettings(backendUrl, { enabled: next })
|
||||
setEnabled(saved.enabled)
|
||||
toast.success(next ? "Автообновление через MikrotikManager включено" : "Автообновление через MikrotikManager выключено")
|
||||
} catch (e) {
|
||||
setEnabled(prev)
|
||||
toast.error(e instanceof Error ? e.message : "Не удалось сохранить")
|
||||
} finally {
|
||||
setToggleBusy(false)
|
||||
}
|
||||
}
|
||||
|
||||
async function handleSaveSchedule() {
|
||||
if (!liveReady || saveBusy) return
|
||||
const intervalSec = Math.max(300, Number.parseInt(intervalDraft, 10) || 21600)
|
||||
const renewBeforeDays = Math.max(1, Math.min(90, Number.parseInt(daysDraft, 10) || 30))
|
||||
setSaveBusy(true)
|
||||
try {
|
||||
const saved = await putCertificateRenewSettings(backendUrl, { intervalSec, renewBeforeDays })
|
||||
setIntervalDraft(String(saved.intervalSec))
|
||||
setDaysDraft(String(saved.renewBeforeDays))
|
||||
toast.success("Расписание автообновления сохранено")
|
||||
} catch (e) {
|
||||
toast.error(e instanceof Error ? e.message : "Не удалось сохранить расписание")
|
||||
} finally {
|
||||
setSaveBusy(false)
|
||||
}
|
||||
}
|
||||
|
||||
const interactionsOff = !liveReady || toggleBusy || (liveReady && !loaded)
|
||||
|
||||
return (
|
||||
<OpsPanel
|
||||
title="Автообновление через MikrotikManager"
|
||||
description="Фоновый выпуск Let's Encrypt (Cloudflare DNS-01) для сертификатов, выпущенных из этой панели. Ручной выпуск не зависит от переключателя."
|
||||
headerRight={
|
||||
<Badge
|
||||
size="sm"
|
||||
variant={!liveReady ? "warning-light" : enabled ? "success-light" : "secondary"}
|
||||
>
|
||||
{!liveReady ? "нет backend" : enabled ? "Включено" : "Выключено"}
|
||||
</Badge>
|
||||
}
|
||||
contentClassName="px-5 py-4 flex flex-col gap-4"
|
||||
>
|
||||
<div className="flex items-center justify-between gap-3">
|
||||
<div>
|
||||
<p className="text-sm font-medium">Обновлять сертификаты из MM</p>
|
||||
<p className="text-muted-foreground mt-0.5 text-xs">
|
||||
Если ACME уже крутит RouterOS — выключите, чтобы не было двойного перевыпуска.
|
||||
</p>
|
||||
</div>
|
||||
<FormToggle checked={enabled} onChange={handleEnabledChange} disabled={interactionsOff} />
|
||||
</div>
|
||||
|
||||
{!liveReady ? (
|
||||
<Alert variant="warning">
|
||||
<AlertTitle>Нет подключения к API</AlertTitle>
|
||||
<AlertDescription>
|
||||
Переключатель станет активен, когда backend доступен. Планировщик читает тот же флаг, что и страница «Сбор данных».
|
||||
</AlertDescription>
|
||||
</Alert>
|
||||
) : enabled ? (
|
||||
<Alert variant="warning">
|
||||
<AlertTitle>Не смешивайте с ACME RouterOS</AlertTitle>
|
||||
<AlertDescription>
|
||||
MM обновляет только сертификаты, выпущенные через эту страницу. Встроенный Let's Encrypt на
|
||||
устройстве для тех же имён лучше не включать одновременно.
|
||||
</AlertDescription>
|
||||
</Alert>
|
||||
) : (
|
||||
<Alert variant="info">
|
||||
<AlertTitle>Обновление отдано RouterOS</AlertTitle>
|
||||
<AlertDescription>
|
||||
Планировщик MM больше не проверяет срок и не перевыпускает сертификаты. Ручной выпуск и импорт
|
||||
остаются доступны.
|
||||
</AlertDescription>
|
||||
</Alert>
|
||||
)}
|
||||
|
||||
<div className={cn("flex flex-col gap-4", !enabled && "pointer-events-none opacity-40")}>
|
||||
<div className="grid grid-cols-1 gap-3 sm:grid-cols-2">
|
||||
<FormField label="Интервал проверки" hint="Секунды, минимум 300">
|
||||
<Input
|
||||
inputMode="numeric"
|
||||
value={intervalDraft}
|
||||
onChange={(e) => setIntervalDraft(e.target.value)}
|
||||
disabled={!liveReady || saveBusy}
|
||||
/>
|
||||
</FormField>
|
||||
<FormField label="Обновлять за" hint="Дней до истечения, 1–90">
|
||||
<Input
|
||||
inputMode="numeric"
|
||||
value={daysDraft}
|
||||
onChange={(e) => setDaysDraft(e.target.value)}
|
||||
disabled={!liveReady || saveBusy}
|
||||
/>
|
||||
</FormField>
|
||||
</div>
|
||||
<div className="flex flex-wrap items-center gap-2">
|
||||
<Button variant="outline" size="sm" disabled={!liveReady || saveBusy || !enabled} onClick={() => void handleSaveSchedule()}>
|
||||
Сохранить расписание
|
||||
</Button>
|
||||
<Link href="/data-collection" className={cn(buttonVariants({ variant: "ghost", size: "sm" }), "h-8 text-xs")}>
|
||||
Журнал планировщика →
|
||||
</Link>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
{lastCollectedAt || lastError ? (
|
||||
<p className="text-muted-foreground text-xs">
|
||||
Последний прогон:{" "}
|
||||
{lastCollectedAt ? new Date(lastCollectedAt).toLocaleString("ru-RU") : "ещё не было"}
|
||||
{lastError ? ` · ошибка: ${lastError}` : ""}
|
||||
</p>
|
||||
) : null}
|
||||
</OpsPanel>
|
||||
)
|
||||
}
|
||||
@@ -4,7 +4,7 @@ import { useEffect, useState, useRef, useMemo } from "react"
|
||||
import { useRouter, usePathname } from "next/navigation"
|
||||
import { cn } from "@/lib/utils"
|
||||
import {
|
||||
SearchIcon, LayoutDashboardIcon, ActivityIcon, MapIcon, HeartPulseIcon,
|
||||
SearchIcon, LayoutDashboardIcon, ActivityIcon, ChartColumnIcon, MapIcon, HeartPulseIcon,
|
||||
GlobeIcon, NetworkIcon, LayersIcon, TagIcon, ServerIcon, FilterIcon,
|
||||
ShieldIcon, ShieldCheckIcon, CableIcon, BoxIcon, BadgeCheckIcon,
|
||||
HardDriveIcon, RouteIcon, GitForkIcon, GitMergeIcon, ScanLineIcon,
|
||||
@@ -27,6 +27,7 @@ const ALL_ITEMS: CommandItem[] = [
|
||||
// Обзор
|
||||
{ id: "dashboard", title: "Дашборд", group: "Обзор", url: "/dashboard", icon: <LayoutDashboardIcon />, keywords: ["главная","home","overview"] },
|
||||
{ id: "traffic", title: "Трафик", group: "Обзор", url: "/traffic", icon: <ActivityIcon />, keywords: ["bandwidth","traffic","клиенты","интерфейсы"] },
|
||||
{ id: "statistics", title: "Статистика", group: "Обзор", url: "/statistics", icon: <ChartColumnIcon />, keywords: ["stats","отчёт","куб","страны","asn","ipfix"] },
|
||||
{ id: "network-map", title: "Карта сети", group: "Обзор", url: "/network-map", icon: <MapIcon />, keywords: ["topology","топология","map"] },
|
||||
{ id: "uptime", title: "Мониторинг / Uptime", group: "Обзор", url: "/uptime", icon: <HeartPulseIcon />, keywords: ["ping","uptime","мониторинг","проверка"] },
|
||||
// Данные
|
||||
|
||||
@@ -0,0 +1,90 @@
|
||||
"use client"
|
||||
|
||||
import Link from "next/link"
|
||||
import {
|
||||
Timeline,
|
||||
TimelineContent,
|
||||
TimelineDate,
|
||||
TimelineHeader,
|
||||
TimelineIndicator,
|
||||
TimelineItem,
|
||||
TimelineSeparator,
|
||||
TimelineTitle,
|
||||
} from "@/components/reui/timeline"
|
||||
import { Alert, AlertDescription, AlertTitle } from "@/components/reui/alert"
|
||||
import { cn } from "@/lib/utils"
|
||||
import type { EventItem } from "@mmapp/contracts/events"
|
||||
|
||||
function formatEventAge(iso: string): string {
|
||||
const ts = Date.parse(iso)
|
||||
if (!Number.isFinite(ts)) return "—"
|
||||
const diffMs = Math.max(0, Date.now() - ts)
|
||||
const minutes = Math.floor(diffMs / 60_000)
|
||||
if (minutes < 1) return "сейчас"
|
||||
if (minutes < 60) return `${minutes}м`
|
||||
const hours = Math.floor(minutes / 60)
|
||||
if (hours < 24) return `${hours}ч`
|
||||
const days = Math.floor(hours / 24)
|
||||
return `${days}д`
|
||||
}
|
||||
|
||||
const LEVEL_DOT: Record<EventItem["level"], string> = {
|
||||
critical: "border-destructive bg-destructive/20 group-data-completed/timeline-item:border-destructive",
|
||||
warning: "border-warning bg-warning/20 group-data-completed/timeline-item:border-warning",
|
||||
info: "border-info bg-info/20 group-data-completed/timeline-item:border-info",
|
||||
}
|
||||
|
||||
/**
|
||||
* Compact activity timeline.
|
||||
* Preview: https://reui.io/preview/base/timeline-3
|
||||
* Docs: https://reui.io/docs/components/base/timeline
|
||||
*/
|
||||
export function DashboardEventsTimeline({
|
||||
events,
|
||||
loading,
|
||||
error,
|
||||
}: {
|
||||
events: EventItem[]
|
||||
loading?: boolean
|
||||
error?: string | null
|
||||
}) {
|
||||
if (loading && events.length === 0) {
|
||||
return <p className="text-muted-foreground px-5 py-6 text-sm">Загрузка событий…</p>
|
||||
}
|
||||
if (error && events.length === 0) {
|
||||
return (
|
||||
<div className="px-5 py-4">
|
||||
<Alert variant="destructive">
|
||||
<AlertTitle>События недоступны</AlertTitle>
|
||||
<AlertDescription>{error}</AlertDescription>
|
||||
</Alert>
|
||||
</div>
|
||||
)
|
||||
}
|
||||
if (events.length === 0) {
|
||||
return (
|
||||
<p className="text-muted-foreground px-5 py-6 text-sm">
|
||||
Событий пока нет.{" "}
|
||||
<Link href="/alerts" className="underline underline-offset-2">
|
||||
Оповещения
|
||||
</Link>
|
||||
</p>
|
||||
)
|
||||
}
|
||||
|
||||
return (
|
||||
<Timeline defaultValue={events.length} className="px-5 py-4">
|
||||
{events.map((event, index) => (
|
||||
<TimelineItem key={event.id} step={index + 1}>
|
||||
<TimelineHeader>
|
||||
<TimelineSeparator />
|
||||
<TimelineDate>{formatEventAge(event.createdAt)}</TimelineDate>
|
||||
<TimelineTitle className="text-[13px] leading-tight">{event.title}</TimelineTitle>
|
||||
<TimelineIndicator className={cn(LEVEL_DOT[event.level])} />
|
||||
</TimelineHeader>
|
||||
<TimelineContent className="text-xs leading-snug">{event.message}</TimelineContent>
|
||||
</TimelineItem>
|
||||
))}
|
||||
</Timeline>
|
||||
)
|
||||
}
|
||||
@@ -2,8 +2,6 @@
|
||||
|
||||
import { useMemo, useState } from "react"
|
||||
import { Flag } from "@/components/flag"
|
||||
import { OpsPanel } from "@/components/ops-panel"
|
||||
import { StatusBadge } from "@/components/status-badge"
|
||||
import { cn } from "@/lib/utils"
|
||||
import type { InternetPathViewModel } from "@/lib/dashboard-internet-path"
|
||||
import { Maximize2Icon, ZoomInIcon, ZoomOutIcon } from "lucide-react"
|
||||
@@ -167,7 +165,7 @@ function ServerNode({
|
||||
)
|
||||
}
|
||||
|
||||
export function InternetPathMapCard({ model }: { model: InternetPathViewModel | null }) {
|
||||
export function InternetPathMapCanvas({ model }: { model: InternetPathViewModel | null }) {
|
||||
const [zoom, setZoom] = useState(1)
|
||||
const [pan, setPan] = useState({ x: 0, y: 0 })
|
||||
const [isDragging, setIsDragging] = useState(false)
|
||||
@@ -226,22 +224,7 @@ export function InternetPathMapCard({ model }: { model: InternetPathViewModel |
|
||||
}
|
||||
|
||||
return (
|
||||
<OpsPanel
|
||||
title="Internet path map"
|
||||
description="Основной и текущий путь трафика HomeRouter → Internet"
|
||||
headerRight={
|
||||
<StatusBadge
|
||||
status={
|
||||
model?.pathState === "healthy"
|
||||
? "online"
|
||||
: model?.pathState === "failover"
|
||||
? "degraded"
|
||||
: "offline"
|
||||
}
|
||||
/>
|
||||
}
|
||||
contentClassName="px-5 pb-4"
|
||||
>
|
||||
<div className="flex flex-col gap-3">
|
||||
{!model && (
|
||||
<div className="h-[240px] rounded-md border border-dashed border-border grid place-items-center text-sm text-muted-foreground">
|
||||
Недостаточно данных для построения маршрута
|
||||
@@ -412,6 +395,6 @@ export function InternetPathMapCard({ model }: { model: InternetPathViewModel |
|
||||
)}
|
||||
</div>
|
||||
)}
|
||||
</OpsPanel>
|
||||
</div>
|
||||
)
|
||||
}
|
||||
|
||||
@@ -0,0 +1,97 @@
|
||||
"use client"
|
||||
|
||||
import { useEffect, useState } from "react"
|
||||
import Link from "next/link"
|
||||
import { ChevronDownIcon } from "lucide-react"
|
||||
import { OpsPanel } from "@/components/ops-panel"
|
||||
import { Alert, AlertDescription, AlertTitle } from "@/components/reui/alert"
|
||||
import { buttonVariants } from "@/components/ui/button"
|
||||
import {
|
||||
Collapsible,
|
||||
CollapsibleContent,
|
||||
CollapsibleTrigger,
|
||||
} from "@/components/ui/collapsible"
|
||||
import { cn } from "@/lib/utils"
|
||||
import type { InternetPathViewModel } from "@/lib/dashboard-internet-path"
|
||||
import { InternetPathMapCanvas } from "@/components/dashboard/internet-path-map"
|
||||
import { InternetPathSummary } from "@/components/dashboard/internet-path-summary"
|
||||
|
||||
const PATH_MAP_OPEN_LS = "mm:dashboard-path-map-open"
|
||||
|
||||
function readMapOpen(): boolean {
|
||||
if (typeof window === "undefined") return true
|
||||
try {
|
||||
const raw = localStorage.getItem(PATH_MAP_OPEN_LS)
|
||||
if (raw === "0") return false
|
||||
if (raw === "1") return true
|
||||
} catch {
|
||||
/* ignore */
|
||||
}
|
||||
return true
|
||||
}
|
||||
|
||||
export function InternetPathPanel({
|
||||
model,
|
||||
loading,
|
||||
error,
|
||||
}: {
|
||||
model: InternetPathViewModel | null
|
||||
loading?: boolean
|
||||
error?: string | null
|
||||
}) {
|
||||
const [open, setOpen] = useState(true)
|
||||
const [hydrated, setHydrated] = useState(false)
|
||||
|
||||
useEffect(() => {
|
||||
queueMicrotask(() => {
|
||||
setOpen(readMapOpen())
|
||||
setHydrated(true)
|
||||
})
|
||||
}, [])
|
||||
|
||||
function handleOpenChange(next: boolean) {
|
||||
setOpen(next)
|
||||
try {
|
||||
localStorage.setItem(PATH_MAP_OPEN_LS, next ? "1" : "0")
|
||||
} catch {
|
||||
/* ignore */
|
||||
}
|
||||
}
|
||||
|
||||
return (
|
||||
<OpsPanel
|
||||
title="Internet path"
|
||||
description="Home → WAN → JH → Exit"
|
||||
headerRight={
|
||||
<Link href="/network-map" className={cn(buttonVariants({ variant: "outline", size: "sm" }), "h-7 text-xs")}>
|
||||
Карта сети →
|
||||
</Link>
|
||||
}
|
||||
contentClassName="flex flex-col gap-3 px-5 pb-4"
|
||||
>
|
||||
{error ? (
|
||||
<Alert variant="destructive">
|
||||
<AlertTitle>Не удалось загрузить путь</AlertTitle>
|
||||
<AlertDescription>{error}</AlertDescription>
|
||||
</Alert>
|
||||
) : null}
|
||||
{loading && !model ? (
|
||||
<div className="h-20 animate-pulse rounded-md bg-muted/40" />
|
||||
) : (
|
||||
<InternetPathSummary model={model} />
|
||||
)}
|
||||
|
||||
<Collapsible open={hydrated ? open : true} onOpenChange={handleOpenChange}>
|
||||
<CollapsibleTrigger
|
||||
className={cn(buttonVariants({ variant: "ghost", size: "sm" }), "h-7 w-fit gap-1.5 text-xs")}
|
||||
>
|
||||
<ChevronDownIcon className={cn("size-3.5 transition-transform", open && "rotate-180")} />
|
||||
{open ? "Скрыть карту" : "Показать карту"}
|
||||
</CollapsibleTrigger>
|
||||
<CollapsibleContent>
|
||||
<InternetPathMapCanvas model={model} />
|
||||
</CollapsibleContent>
|
||||
</Collapsible>
|
||||
</OpsPanel>
|
||||
)
|
||||
}
|
||||
@@ -0,0 +1,130 @@
|
||||
"use client"
|
||||
|
||||
import type { ReactNode } from "react"
|
||||
import Link from "next/link"
|
||||
import { Badge } from "@/components/reui/badge"
|
||||
import { IconTile } from "@/components/reui/icon-tile"
|
||||
import { StatusBadge } from "@/components/status-badge"
|
||||
import { Flag } from "@/components/flag"
|
||||
import type { InternetPathViewModel } from "@/lib/dashboard-internet-path"
|
||||
import type { ServerStatus } from "@/lib/data"
|
||||
import { cn } from "@/lib/utils"
|
||||
import { ArrowRightIcon, HomeIcon, RadioIcon, ServerIcon, GlobeIcon } from "lucide-react"
|
||||
|
||||
function pathStateToServerStatus(state: InternetPathViewModel["pathState"]): ServerStatus {
|
||||
if (state === "healthy") return "online"
|
||||
if (state === "failover" || state === "degraded") return "degraded"
|
||||
return "offline"
|
||||
}
|
||||
|
||||
function HopChip({
|
||||
label,
|
||||
name,
|
||||
country,
|
||||
icon,
|
||||
iconClassName,
|
||||
}: {
|
||||
label: string
|
||||
name: string
|
||||
country?: string
|
||||
icon: ReactNode
|
||||
iconClassName?: string
|
||||
}) {
|
||||
return (
|
||||
<div className="flex min-w-0 items-center gap-2 rounded-md border border-border/60 px-2 py-1.5">
|
||||
<IconTile variant="elevated" size="sm" className={cn("shrink-0", iconClassName)} aria-hidden="true">
|
||||
{icon}
|
||||
</IconTile>
|
||||
<div className="min-w-0">
|
||||
<p className="text-muted-foreground text-[10px] font-medium uppercase tracking-wide">{label}</p>
|
||||
<p className="flex items-center gap-1 truncate text-sm font-medium">
|
||||
{country ? <Flag code={country} className="shrink-0" /> : null}
|
||||
<span className="truncate">{name}</span>
|
||||
</p>
|
||||
</div>
|
||||
</div>
|
||||
)
|
||||
}
|
||||
|
||||
/**
|
||||
* Compact live path strip (Frame-friendly). Canvas lives separately.
|
||||
* Preview: https://reui.io/preview/base/stats-12
|
||||
* Docs: https://reui.io/docs/components/base/icon-tile
|
||||
*/
|
||||
export function InternetPathSummary({ model }: { model: InternetPathViewModel | null }) {
|
||||
if (!model) {
|
||||
return (
|
||||
<p className="text-muted-foreground text-sm">
|
||||
Недостаточно данных для пути. Добавьте home-router и проверьте{" "}
|
||||
<Link href="/network-map" className="underline underline-offset-2">
|
||||
карту сети
|
||||
</Link>
|
||||
.
|
||||
</p>
|
||||
)
|
||||
}
|
||||
|
||||
const hop = model.currentHop ?? model.primaryHop
|
||||
const wanName = hop?.wan.name ?? model.activeWanUplink?.name ?? "WAN"
|
||||
const wanIsp = hop?.wan.isp ?? model.activeWanUplink?.isp ?? "—"
|
||||
const ping = hop?.wanJhMetrics.pingMs
|
||||
const dl = hop?.wanJhMetrics.dlMbps
|
||||
|
||||
return (
|
||||
<div className="flex flex-col gap-3">
|
||||
<div className="flex flex-wrap items-center gap-2">
|
||||
<StatusBadge status={pathStateToServerStatus(model.pathState)} />
|
||||
{model.pathState === "failover" ? (
|
||||
<Badge variant="warning-light" size="sm">failover</Badge>
|
||||
) : null}
|
||||
{ping != null ? (
|
||||
<Badge variant="outline" size="sm" className="tabular-nums">
|
||||
{ping} мс
|
||||
</Badge>
|
||||
) : null}
|
||||
{dl != null ? (
|
||||
<Badge variant="outline" size="sm" className="tabular-nums">
|
||||
{Math.round(dl)} ↓ Мбит/с
|
||||
</Badge>
|
||||
) : null}
|
||||
</div>
|
||||
|
||||
<div className="flex flex-col gap-2 @3xl:flex-row @3xl:items-center">
|
||||
<HopChip
|
||||
label="Home"
|
||||
name={model.homeRouter.name}
|
||||
country={model.homeRouter.country}
|
||||
icon={<HomeIcon />}
|
||||
iconClassName="text-success"
|
||||
/>
|
||||
<ArrowRightIcon className="text-muted-foreground hidden size-4 shrink-0 @3xl:block" aria-hidden="true" />
|
||||
<HopChip
|
||||
label="WAN"
|
||||
name={`${wanName} · ${wanIsp}`}
|
||||
icon={<RadioIcon />}
|
||||
iconClassName="text-info"
|
||||
/>
|
||||
<ArrowRightIcon className="text-muted-foreground hidden size-4 shrink-0 @3xl:block" aria-hidden="true" />
|
||||
<HopChip
|
||||
label="JH"
|
||||
name={hop?.jumpHost.name ?? model.fallbackJumpHost?.name ?? "—"}
|
||||
country={hop?.jumpHost.country ?? model.fallbackJumpHost?.country}
|
||||
icon={<ServerIcon />}
|
||||
iconClassName="text-primary"
|
||||
/>
|
||||
<ArrowRightIcon className="text-muted-foreground hidden size-4 shrink-0 @3xl:block" aria-hidden="true" />
|
||||
<HopChip
|
||||
label="Exit"
|
||||
name={hop?.exitNode.name ?? model.fallbackExitNode?.name ?? "—"}
|
||||
country={hop?.exitNode.country ?? model.fallbackExitNode?.country}
|
||||
icon={<GlobeIcon />}
|
||||
iconClassName="text-muted-foreground"
|
||||
/>
|
||||
</div>
|
||||
|
||||
<p className="text-muted-foreground text-xs leading-relaxed">
|
||||
{model.currentPath?.reason ?? model.primaryPath?.reason ?? "Текущий путь не определён"}
|
||||
</p>
|
||||
</div>
|
||||
)
|
||||
}
|
||||
@@ -1,6 +1,6 @@
|
||||
"use client"
|
||||
|
||||
import { useMemo } from "react"
|
||||
import { useMemo, type ReactNode } from "react"
|
||||
import {
|
||||
type ColumnDef,
|
||||
getCoreRowModel,
|
||||
@@ -8,8 +8,9 @@ import {
|
||||
useReactTable,
|
||||
} from "@tanstack/react-table"
|
||||
import type { Backup } from "@/lib/data"
|
||||
import { cn } from "@/lib/utils"
|
||||
import { Button } from "@/components/ui/button"
|
||||
import { Badge } from "@/components/reui/badge"
|
||||
import { IconTile } from "@/components/reui/icon-tile"
|
||||
import { DataGridShell } from "@/components/data-grids/shared/data-grid-shell"
|
||||
import {
|
||||
DATA_GRID_CELL_PAD,
|
||||
@@ -18,13 +19,28 @@ import {
|
||||
} from "@/components/data-grids/shared/data-grid-layout"
|
||||
import { DataGridSortHeader } from "@/components/data-grids/shared/data-grid-sort-header"
|
||||
import { EmptyState } from "@/components/empty-state"
|
||||
import { DownloadIcon, HardDriveIcon, RefreshCwIcon, Trash2Icon } from "lucide-react"
|
||||
import {
|
||||
CloudIcon,
|
||||
DownloadIcon,
|
||||
HardDriveIcon,
|
||||
RefreshCwIcon,
|
||||
Trash2Icon,
|
||||
} from "lucide-react"
|
||||
|
||||
interface BackupsDataGridProps {
|
||||
backups: Backup[]
|
||||
onDownload: (id: string, filename: string) => void
|
||||
onRestore: (backup: Backup) => void
|
||||
onDelete: (id: string) => void
|
||||
onDelete: (backup: Backup) => void
|
||||
emptyAction?: ReactNode
|
||||
emptyTitle?: string
|
||||
emptyDescription?: string
|
||||
}
|
||||
|
||||
function storageBadge(storage: Backup["storage"]) {
|
||||
if (storage === "s3") return { label: "S3", variant: "info-light" as const }
|
||||
if (storage === "both") return { label: "Локально + S3", variant: "success-light" as const }
|
||||
return { label: "Локально", variant: "secondary" as const }
|
||||
}
|
||||
|
||||
function BackupsDataGrid({
|
||||
@@ -32,6 +48,9 @@ function BackupsDataGrid({
|
||||
onDownload,
|
||||
onRestore,
|
||||
onDelete,
|
||||
emptyAction,
|
||||
emptyTitle = "Нет бэкапов",
|
||||
emptyDescription = "Создайте первый бэкап вручную или настройте расписание",
|
||||
}: BackupsDataGridProps) {
|
||||
const columns = useMemo<ColumnDef<Backup>[]>(
|
||||
() => [
|
||||
@@ -39,9 +58,27 @@ function BackupsDataGrid({
|
||||
id: "filename",
|
||||
accessorKey: "filename",
|
||||
header: ({ column }) => <DataGridSortHeader column={column} title="Файл" />,
|
||||
cell: ({ row }) => (
|
||||
<span className="font-mono text-xs font-medium">{row.original.filename}</span>
|
||||
),
|
||||
cell: ({ row }) => {
|
||||
const b = row.original
|
||||
const inS3 = b.storage === "s3" || b.storage === "both"
|
||||
return (
|
||||
<div className="flex items-center gap-3 min-w-0">
|
||||
<IconTile
|
||||
variant="elevated"
|
||||
className={inS3 ? "size-10.5 shrink-0 text-info" : "size-10.5 shrink-0 text-muted-foreground"}
|
||||
aria-hidden="true"
|
||||
>
|
||||
{inS3 ? <CloudIcon /> : <HardDriveIcon />}
|
||||
</IconTile>
|
||||
<div className="min-w-0">
|
||||
<span className="font-mono text-xs font-medium block truncate">{b.filename}</span>
|
||||
{b.uploadError ? (
|
||||
<span className="text-[11px] text-destructive truncate block">{b.uploadError}</span>
|
||||
) : null}
|
||||
</div>
|
||||
</div>
|
||||
)
|
||||
},
|
||||
meta: {
|
||||
headerTitle: "Файл",
|
||||
headerClassName: DATA_GRID_CELL_PAD_FIRST,
|
||||
@@ -78,23 +115,35 @@ function BackupsDataGrid({
|
||||
id: "kind",
|
||||
accessorKey: "kind",
|
||||
header: ({ column }) => <DataGridSortHeader column={column} title="Тип" />,
|
||||
cell: ({ row }) => (
|
||||
<Badge
|
||||
variant={row.original.kind === "manual" ? "info-light" : "secondary"}
|
||||
size="sm"
|
||||
radius="full"
|
||||
>
|
||||
{row.original.kind === "auto" ? "авто" : "вручную"}
|
||||
</Badge>
|
||||
),
|
||||
meta: {
|
||||
headerTitle: "Тип",
|
||||
headerClassName: DATA_GRID_CELL_PAD,
|
||||
cellClassName: DATA_GRID_CELL_PAD,
|
||||
},
|
||||
},
|
||||
{
|
||||
id: "storage",
|
||||
accessorKey: "storage",
|
||||
header: ({ column }) => <DataGridSortHeader column={column} title="Хранилище" />,
|
||||
cell: ({ row }) => {
|
||||
const kind = row.original.kind
|
||||
const badge = storageBadge(row.original.storage)
|
||||
return (
|
||||
<span
|
||||
className={cn(
|
||||
"text-xs px-2 py-0.5 rounded border font-medium",
|
||||
kind === "manual"
|
||||
? "bg-blue-500/10 text-blue-400 border-blue-500/20"
|
||||
: "bg-muted text-muted-foreground border-border",
|
||||
)}
|
||||
>
|
||||
{kind === "auto" ? "авто" : "вручную"}
|
||||
</span>
|
||||
<Badge variant={badge.variant} size="sm" radius="full">
|
||||
{badge.label}
|
||||
</Badge>
|
||||
)
|
||||
},
|
||||
meta: {
|
||||
headerTitle: "Тип",
|
||||
headerTitle: "Хранилище",
|
||||
headerClassName: DATA_GRID_CELL_PAD,
|
||||
cellClassName: DATA_GRID_CELL_PAD,
|
||||
},
|
||||
@@ -135,29 +184,35 @@ function BackupsDataGrid({
|
||||
return (
|
||||
<div className="flex items-center gap-1 justify-end opacity-0 transition-opacity group-hover/row:opacity-100 focus-within:opacity-100">
|
||||
<Button
|
||||
type="button"
|
||||
variant="ghost"
|
||||
size="icon"
|
||||
className="size-7"
|
||||
title="Скачать"
|
||||
aria-label={`Скачать ${b.filename}`}
|
||||
onClick={() => onDownload(b.id, b.filename)}
|
||||
>
|
||||
<DownloadIcon className="size-3.5" />
|
||||
</Button>
|
||||
<Button
|
||||
type="button"
|
||||
variant="ghost"
|
||||
size="icon"
|
||||
className="size-7"
|
||||
title="Восстановить"
|
||||
aria-label={`Восстановить ${b.filename}`}
|
||||
onClick={() => onRestore(b)}
|
||||
>
|
||||
<RefreshCwIcon className="size-3.5" />
|
||||
</Button>
|
||||
<Button
|
||||
type="button"
|
||||
variant="ghost"
|
||||
size="icon"
|
||||
className="size-7 text-destructive hover:text-destructive"
|
||||
title="Удалить"
|
||||
onClick={() => onDelete(b.id)}
|
||||
aria-label={`Удалить ${b.filename}`}
|
||||
onClick={() => onDelete(b)}
|
||||
>
|
||||
<Trash2Icon className="size-3.5" />
|
||||
</Button>
|
||||
@@ -186,9 +241,10 @@ function BackupsDataGrid({
|
||||
if (backups.length === 0) {
|
||||
return (
|
||||
<EmptyState
|
||||
icon={<HardDriveIcon className="size-4" />}
|
||||
title="Нет бэкапов"
|
||||
description="Создайте первый бэкап вручную или настройте расписание"
|
||||
icon={<HardDriveIcon className="size-5" />}
|
||||
title={emptyTitle}
|
||||
description={emptyDescription}
|
||||
action={emptyAction}
|
||||
className="border-0 py-10"
|
||||
/>
|
||||
)
|
||||
|
||||
@@ -0,0 +1,89 @@
|
||||
"use client"
|
||||
|
||||
import { CompactDataGrid, type CompactDataGridColumn } from "@/components/data-grids/compact-data-grid"
|
||||
import { Flag } from "@/components/flag"
|
||||
import { Badge } from "@/components/reui/badge"
|
||||
import { fmtBps, formatBytes } from "@/lib/fmt-rate"
|
||||
import { cn } from "@/lib/utils"
|
||||
import { STATISTICS_UNBOUND_USER_ID, type StatisticsBreakdownRow } from "@mmapp/contracts/statistics"
|
||||
|
||||
export type StatisticsSliceKind = "users" | "servers" | "interfaces" | "countries" | "services" | "asns"
|
||||
|
||||
export function StatisticsBreakdownDataGrid({
|
||||
rows,
|
||||
kind,
|
||||
selectedId,
|
||||
onRowClick,
|
||||
isLoading,
|
||||
density = "full",
|
||||
}: {
|
||||
rows: StatisticsBreakdownRow[]
|
||||
kind: StatisticsSliceKind
|
||||
selectedId?: string
|
||||
onRowClick?: (row: StatisticsBreakdownRow) => void
|
||||
isLoading?: boolean
|
||||
density?: "full" | "mini"
|
||||
}) {
|
||||
const mini = density === "mini"
|
||||
const columns: CompactDataGridColumn<StatisticsBreakdownRow>[] = [
|
||||
{
|
||||
id: "label",
|
||||
header: "Имя",
|
||||
accessorKey: "label",
|
||||
cell: (row) => (
|
||||
<span className={cn("flex items-center gap-2", selectedId === row.id && "font-medium")}>
|
||||
{kind === "countries" && row.id !== "XX" && row.id !== STATISTICS_UNBOUND_USER_ID ? (
|
||||
<Flag code={row.id} size={16} />
|
||||
) : null}
|
||||
<span className="truncate">{row.label || row.id}</span>
|
||||
{selectedId === row.id ? (
|
||||
<Badge variant="outline" size="sm">
|
||||
слайс
|
||||
</Badge>
|
||||
) : null}
|
||||
</span>
|
||||
),
|
||||
},
|
||||
{
|
||||
id: "bytes",
|
||||
header: "Байты",
|
||||
accessorKey: "bytes",
|
||||
cell: (row) => <span className="tabular-nums">{formatBytes(row.bytes)}</span>,
|
||||
},
|
||||
...(!mini
|
||||
? [
|
||||
{
|
||||
id: "packets",
|
||||
header: "Пакеты",
|
||||
accessorKey: "packets" as const,
|
||||
cell: (row: StatisticsBreakdownRow) => (
|
||||
<span className="tabular-nums">{row.packets.toLocaleString("ru-RU")}</span>
|
||||
),
|
||||
},
|
||||
{
|
||||
id: "bps",
|
||||
header: "Средний bitrate",
|
||||
accessorKey: "bps" as const,
|
||||
cell: (row: StatisticsBreakdownRow) => <span className="tabular-nums">{fmtBps(row.bps)}</span>,
|
||||
},
|
||||
]
|
||||
: []),
|
||||
{
|
||||
id: "percent",
|
||||
header: "Доля",
|
||||
accessorKey: "percent",
|
||||
cell: (row) => <span className="tabular-nums">{row.percent.toFixed(1)}%</span>,
|
||||
},
|
||||
]
|
||||
|
||||
return (
|
||||
<CompactDataGrid
|
||||
data={rows}
|
||||
columns={columns}
|
||||
isLoading={isLoading}
|
||||
emptyTitle="Нет трафика"
|
||||
emptyDescription="За выбранный период и слайсы нет данных куба."
|
||||
onRowClick={onRowClick}
|
||||
/>
|
||||
)
|
||||
}
|
||||
@@ -1,13 +1,14 @@
|
||||
"use client"
|
||||
|
||||
import { ReactNode } from "react"
|
||||
import { SearchIcon } from "lucide-react"
|
||||
import { ListFilterIcon, SearchIcon } from "lucide-react"
|
||||
import { cn } from "@/lib/utils"
|
||||
import {
|
||||
InputGroup,
|
||||
InputGroupAddon,
|
||||
InputGroupInput,
|
||||
} from "@/components/ui/input-group"
|
||||
import { Button } from "@/components/ui/button"
|
||||
import {
|
||||
Filters,
|
||||
type Filter,
|
||||
@@ -77,6 +78,12 @@ function DataPageToolbar<T extends string = string>({
|
||||
fields={filterFields}
|
||||
onChange={onFiltersChange}
|
||||
size="sm"
|
||||
trigger={
|
||||
<Button type="button" variant="outline" size="sm">
|
||||
<ListFilterIcon className="size-3.5" />
|
||||
Фильтры
|
||||
</Button>
|
||||
}
|
||||
/>
|
||||
)}
|
||||
{onSearchChange != null && (
|
||||
|
||||
@@ -125,6 +125,117 @@ export function ServiceBrandIcon({ label, size = 22 }: { label: string; size?: n
|
||||
<path d="M13.4 4v9.1a3.3 3.3 0 1 1-2.8-3.3V7.2c1.6.9 3.2 1.4 5 1.5V5.4c-1.4-.1-2.7-.6-3.8-1.4H13.4Z" fill="#FE2C55" transform="translate(1.2 1)" />
|
||||
</BrandSvg>
|
||||
)
|
||||
case "apple":
|
||||
return (
|
||||
<BrandSvg size={size}>
|
||||
<path d="M14.7 6.2c.8-.9 1.3-2.2 1.2-3.5-1.2.1-2.6.8-3.4 1.8-.8.9-1.5 2.2-1.3 3.5 1.3 0 2.6-.8 3.5-1.8Z" fill="#111" />
|
||||
<path d="M16.8 12.2c0-2.2 1.8-3.3 1.9-3.4-1.1-1.6-2.7-1.8-3.3-1.8-1.4-.1-2.7.8-3.4.8s-1.8-.8-3-.8c-1.5 0-3 .9-3.8 2.3-1.6 2.8-.4 7 1.2 9.3.8 1.1 1.7 2.3 2.9 2.3 1.2 0 1.6-.7 3-.7s1.8.7 3 .7 2-.1 2.9-2.2c1.1-1.5 1.5-3 1.5-3.1-.1 0-2.9-1.1-2.9-4.4Z" fill="#111" />
|
||||
</BrandSvg>
|
||||
)
|
||||
case "github":
|
||||
return (
|
||||
<BrandSvg size={size}>
|
||||
<circle cx="12" cy="12" r="10" fill="#181717" />
|
||||
<path d="M12 6.4c-3.1 0-5.6 2.5-5.6 5.6 0 2.5 1.6 4.6 3.8 5.3.3.1.4-.1.4-.3v-1.1c-1.6.3-1.9-.7-1.9-.7-.3-.6-.6-.8-.6-.8-.5-.4 0-.4 0-.4.6 0 .9.6.9.6.5.9 1.4.6 1.7.5.1-.4.2-.6.4-.8-1.2-.1-2.5-.6-2.5-2.8 0-.6.2-1.1.6-1.5-.1-.1-.3-.7 0-1.4 0 0 .5-.2 1.6.6.5-.1 1-.2 1.5-.2s1 .1 1.5.2c1.1-.8 1.6-.6 1.6-.6.3.7.1 1.3 0 1.4.4.4.6.9.6 1.5 0 2.2-1.3 2.6-2.5 2.8.2.2.4.5.4 1.1v1.6c0 .2.1.4.4.3 2.2-.7 3.8-2.8 3.8-5.3 0-3.1-2.5-5.6-5.6-5.6Z" fill="#fff" />
|
||||
</BrandSvg>
|
||||
)
|
||||
case "gitlab":
|
||||
return (
|
||||
<BrandSvg size={size}>
|
||||
<path d="M12 19.2 8.4 8.4h7.2L12 19.2Z" fill="#E24329" />
|
||||
<path d="M12 19.2 8.4 8.4 5.2 16.2 12 19.2Z" fill="#FC6D26" />
|
||||
<path d="M12 19.2 15.6 8.4 18.8 16.2 12 19.2Z" fill="#FC6D26" />
|
||||
<path d="M5.2 16.2 3 8.4h5.4L5.2 16.2Z" fill="#FCA326" />
|
||||
<path d="M18.8 16.2 21 8.4h-5.4l3.2 7.8Z" fill="#FCA326" />
|
||||
</BrandSvg>
|
||||
)
|
||||
case "spotify":
|
||||
return (
|
||||
<BrandSvg size={size}>
|
||||
<circle cx="12" cy="12" r="10" fill="#1DB954" />
|
||||
<path d="M7.2 10.4c3.2-1 6.8-.8 9.6.8M7.6 13c2.6-.8 5.6-.6 8 .6M8 15.4c2-.6 4.4-.4 6.2.4" fill="none" stroke="#fff" strokeWidth="1.5" strokeLinecap="round" />
|
||||
</BrandSvg>
|
||||
)
|
||||
case "x":
|
||||
return (
|
||||
<BrandSvg size={size}>
|
||||
<rect width="24" height="24" rx="5" fill="#111" />
|
||||
<path d="M6.2 5.6h3.2l3 4.2 3.6-4.2H18l-5.2 6.1 5.4 6.7h-3.2l-3.4-4.4-4 4.4H6.4l5.6-6.4Z" fill="#fff" />
|
||||
</BrandSvg>
|
||||
)
|
||||
case "vk":
|
||||
return (
|
||||
<BrandSvg size={size}>
|
||||
<rect width="24" height="24" rx="5" fill="#0077FF" />
|
||||
<path d="M4.8 7.8h2.6c.1 4.2 1.9 6.7 5.4 6.7V7.8h2.4v3.9c1.5-.2 2.9-1.7 3.4-3.9h2.4c-.6 3.3-2.6 5.4-4.4 6.2 1.8.6 4.1 2.4 5 5.2h-2.8c-.7-1.9-2.2-3.4-4-3.6v3.6h-2.4v-3.6c-3.5.1-6.1-2.4-6.6-6.8Z" fill="#fff" />
|
||||
</BrandSvg>
|
||||
)
|
||||
case "zoom":
|
||||
return (
|
||||
<BrandSvg size={size}>
|
||||
<rect width="24" height="24" rx="5" fill="#2D8CFF" />
|
||||
<path d="M5.2 9.2h7.2a2.2 2.2 0 0 1 2.2 2.2v5.2H7.4A2.2 2.2 0 0 1 5.2 14.4Z" fill="#fff" />
|
||||
<path d="M16.2 11.2 20 9.4v7.4l-3.8-1.8Z" fill="#fff" />
|
||||
</BrandSvg>
|
||||
)
|
||||
case "epic":
|
||||
return (
|
||||
<BrandSvg size={size}>
|
||||
<circle cx="12" cy="12" r="10" fill="#111" />
|
||||
<path d="M8.2 7.4h7.6v2H10.6v2h4.6v2h-4.6v3.2H8.2Z" fill="#fff" />
|
||||
</BrandSvg>
|
||||
)
|
||||
case "riot":
|
||||
return (
|
||||
<BrandSvg size={size}>
|
||||
<path d="M5 19.2 12 4.2 19 19.2h-3.2L12 10.6 8.2 19.2Z" fill="#D32936" />
|
||||
<path d="M9.4 19.2h5.2l-2.6-5.2Z" fill="#EB0029" />
|
||||
</BrandSvg>
|
||||
)
|
||||
case "playstation":
|
||||
return (
|
||||
<BrandSvg size={size}>
|
||||
<circle cx="12" cy="12" r="10" fill="#003087" />
|
||||
<path d="M8.2 14.6c-1 .4-1.8.2-2-.4s.4-1.2 1.6-1.6l2-.7v1.6l-1.2.4c-.6.2-.8.4-.7.6.1.2.4.2.9 0l1-.4v1.5Zm3-6.4v8.2c-1.1.4-2.1.5-2.8.2-.9-.4-.9-1.3 0-1.7.5-.2 1.2-.3 2-.2V9.4c0-1.2.5-1.8 1.4-1.5.4.2.7.6.8 1.2Zm5.2 7.6c-1.1.4-2.2.4-3 0-.8-.4-.8-1.2 0-1.6.5-.2 1.2-.3 2-.2v-2.2l-2.4.8V11l4.2-1.5v6.3Z" fill="#fff" />
|
||||
</BrandSvg>
|
||||
)
|
||||
case "roblox":
|
||||
return (
|
||||
<BrandSvg size={size}>
|
||||
<rect width="24" height="24" rx="5" fill="#111" />
|
||||
<path d="M8.4 6.2 17.6 8.8 15.6 17.8 6.4 15.2Z" fill="#fff" />
|
||||
</BrandSvg>
|
||||
)
|
||||
case "digitalocean":
|
||||
return (
|
||||
<BrandSvg size={size}>
|
||||
<circle cx="12" cy="12" r="10" fill="#0080FF" />
|
||||
<path d="M12.4 6.2A5.8 5.8 0 0 0 7.8 16l1.6-1.5A3.6 3.6 0 1 1 16 12h-3.6Z" fill="#fff" />
|
||||
<path d="M12.4 16.2h-1.6v1.6h1.6zm-1.6-2h-1.4v1.4h1.4z" fill="#fff" />
|
||||
</BrandSvg>
|
||||
)
|
||||
case "hetzner":
|
||||
return (
|
||||
<BrandSvg size={size}>
|
||||
<rect width="24" height="24" rx="4" fill="#D50C2D" />
|
||||
<path d="M7.2 6.4h2.6v4.4h4.4V6.4h2.6v11.2h-2.6v-4.4H9.8v4.4H7.2Z" fill="#fff" />
|
||||
</BrandSvg>
|
||||
)
|
||||
case "ovh":
|
||||
return (
|
||||
<BrandSvg size={size}>
|
||||
<rect width="24" height="24" rx="4" fill="#123F6D" />
|
||||
<path d="M4.6 15.6 8.4 8.4h3.2L7.8 15.6Zm6.4 0 3.8-7.2h3.2l-3.8 7.2Zm2.2 0h3.4l1.8-3.4h-3.4Z" fill="#00A2E2" />
|
||||
</BrandSvg>
|
||||
)
|
||||
case "chatgpt":
|
||||
return (
|
||||
<BrandSvg size={size}>
|
||||
<circle cx="12" cy="12" r="10" fill="#10A37F" />
|
||||
<path d="M12.2 6.2c.9-.5 2-.5 2.9 0l2.2 1.3c.9.5 1.4 1.4 1.4 2.4v2.6c0 1-.5 1.9-1.4 2.4l-2.2 1.3c-.9.5-2 .5-2.9 0l-.4-.2c.6-.4 1-1 1.1-1.7l.5.3c.4.2.9.2 1.3 0l2.2-1.3c.4-.2.6-.6.6-1.1V10c0-.4-.2-.8-.6-1.1l-2.2-1.3c-.4-.2-.9-.2-1.3 0L10.2 9c-.4.2-.6.6-.6 1.1v.4h-2V10c0-1 .5-1.9 1.4-2.4Z" fill="#fff" />
|
||||
<path d="M8.8 9.6c.6-.4 1.3-.5 2-.3v2.1c0 .4.2.8.6 1.1l2.2 1.3c.4.2.9.2 1.3 0l.5-.3c.2.7.6 1.3 1.1 1.7l-.4.2c-.9.5-2 .5-2.9 0l-2.2-1.3c-.9-.5-1.4-1.4-1.4-2.4Z" fill="#fff" opacity="0.85" />
|
||||
</BrandSvg>
|
||||
)
|
||||
default:
|
||||
return <GenericCloud size={size} />
|
||||
}
|
||||
|
||||
@@ -0,0 +1,93 @@
|
||||
"use client"
|
||||
|
||||
import type { ReactNode } from "react"
|
||||
import type { LucideIcon } from "lucide-react"
|
||||
import { Badge } from "@/components/reui/badge"
|
||||
import {
|
||||
Frame,
|
||||
FrameHeader,
|
||||
FramePanel,
|
||||
FrameTitle,
|
||||
} from "@/components/reui/frame"
|
||||
import { IconTile } from "@/components/reui/icon-tile"
|
||||
import { cn } from "@/lib/utils"
|
||||
|
||||
/**
|
||||
* Sibling Frame columns for dashboard attention queue.
|
||||
* Preview: https://reui.io/preview/base/dashboard-1 · https://reui.io/preview/base/stats-12
|
||||
* Docs: https://reui.io/docs/components/base/frame · https://reui.io/docs/components/base/icon-tile · https://reui.io/docs/components/base/badge
|
||||
*/
|
||||
export interface AttentionQueueColumn {
|
||||
id: string
|
||||
title: string
|
||||
icon: LucideIcon
|
||||
iconClassName?: string
|
||||
count: number
|
||||
countVariant?: "destructive" | "warning" | "secondary" | "destructive-light" | "warning-light"
|
||||
emptyTitle: string
|
||||
emptyDescription: string
|
||||
emptyAction?: ReactNode
|
||||
children: ReactNode
|
||||
}
|
||||
|
||||
interface AttentionQueueProps {
|
||||
columns: AttentionQueueColumn[]
|
||||
className?: string
|
||||
}
|
||||
|
||||
const DEFAULT_ICON_CLASS = "text-muted-foreground [&_svg]:text-current"
|
||||
|
||||
export function AttentionQueue({ columns, className }: AttentionQueueProps) {
|
||||
return (
|
||||
<div
|
||||
className={cn(
|
||||
"@container grid min-w-0 items-start gap-2 @3xl:grid-cols-3",
|
||||
className,
|
||||
)}
|
||||
>
|
||||
{columns.map((column) => {
|
||||
const Icon = column.icon
|
||||
const isEmpty = column.count === 0
|
||||
return (
|
||||
<Frame key={column.id} dense spacing="sm" className="min-w-0 w-full">
|
||||
<FrameHeader>
|
||||
<div className="flex min-w-0 items-center gap-2">
|
||||
<IconTile
|
||||
variant="elevated"
|
||||
size="sm"
|
||||
className={cn(DEFAULT_ICON_CLASS, column.iconClassName)}
|
||||
aria-hidden="true"
|
||||
>
|
||||
<Icon />
|
||||
</IconTile>
|
||||
<FrameTitle className="min-w-0 truncate">{column.title}</FrameTitle>
|
||||
{column.count > 0 ? (
|
||||
<Badge
|
||||
size="sm"
|
||||
variant={column.countVariant ?? "secondary"}
|
||||
className="tabular-nums"
|
||||
>
|
||||
{column.count}
|
||||
</Badge>
|
||||
) : null}
|
||||
</div>
|
||||
</FrameHeader>
|
||||
<FramePanel className="min-w-0">
|
||||
{isEmpty ? (
|
||||
<div className="flex min-h-24 flex-col items-start justify-center gap-1 py-3">
|
||||
<p className="text-sm font-medium">{column.emptyTitle}</p>
|
||||
<p className="text-muted-foreground text-xs leading-relaxed">
|
||||
{column.emptyDescription}
|
||||
</p>
|
||||
{column.emptyAction ? <div className="pt-1">{column.emptyAction}</div> : null}
|
||||
</div>
|
||||
) : (
|
||||
column.children
|
||||
)}
|
||||
</FramePanel>
|
||||
</Frame>
|
||||
)
|
||||
})}
|
||||
</div>
|
||||
)
|
||||
}
|
||||
@@ -3,3 +3,7 @@ export type { CodeExportFormat, CodeExportSheetProps } from "./code-export-sheet
|
||||
export { KpiStatGrid, KpiStatCardTile, kpiStatItemKey } from "./kpi-stat-grid"
|
||||
export type { KpiStatItem, KpiStatCardData, KpiStatVariant } from "./kpi-stat-grid"
|
||||
export { kpiCols } from "./kpi-cols"
|
||||
export { QuickActionGrid } from "./quick-action-grid"
|
||||
export type { QuickActionItem } from "./quick-action-grid"
|
||||
export { AttentionQueue } from "./attention-queue"
|
||||
export type { AttentionQueueColumn } from "./attention-queue"
|
||||
|
||||
@@ -0,0 +1,161 @@
|
||||
"use client"
|
||||
|
||||
import type { KeyboardEvent, ReactNode } from "react"
|
||||
import Link from "next/link"
|
||||
import {
|
||||
Frame,
|
||||
FrameDescription,
|
||||
FrameHeader,
|
||||
FramePanel,
|
||||
FrameTitle,
|
||||
} from "@/components/reui/frame"
|
||||
import { Badge } from "@/components/reui/badge"
|
||||
import { IconTile } from "@/components/reui/icon-tile"
|
||||
import { cn } from "@/lib/utils"
|
||||
import { kpiCols } from "./kpi-cols"
|
||||
|
||||
type QuickActionBase = {
|
||||
id: string
|
||||
title: string
|
||||
description: string
|
||||
icon?: ReactNode
|
||||
iconClassName?: string
|
||||
badge?: string
|
||||
disabled?: boolean
|
||||
}
|
||||
|
||||
export type QuickActionItem = QuickActionBase &
|
||||
(
|
||||
| { href: string; onClick?: never }
|
||||
| { onClick: () => void; href?: never }
|
||||
)
|
||||
|
||||
interface QuickActionGridProps {
|
||||
actions: QuickActionItem[]
|
||||
title?: string
|
||||
description?: string
|
||||
className?: string
|
||||
}
|
||||
|
||||
const DEFAULT_ICON_CLASS = "text-muted-foreground [&_svg]:text-current"
|
||||
|
||||
function resolveBadge(action: QuickActionItem): string {
|
||||
if (action.badge) return action.badge
|
||||
return action.onClick ? "Выполнить" : "Перейти"
|
||||
}
|
||||
|
||||
function handleActionKeyDown(onActivate: () => void, event: KeyboardEvent<HTMLDivElement>) {
|
||||
if (event.key === "Enter" || event.key === " ") {
|
||||
event.preventDefault()
|
||||
onActivate()
|
||||
}
|
||||
}
|
||||
|
||||
function QuickActionBody({ action }: { action: QuickActionItem }) {
|
||||
return (
|
||||
<div className="relative z-10 flex h-full items-start gap-3">
|
||||
{action.icon ? (
|
||||
<IconTile
|
||||
variant="elevated"
|
||||
aria-hidden="true"
|
||||
className={cn("size-10.5", action.iconClassName ?? DEFAULT_ICON_CLASS)}
|
||||
>
|
||||
{action.icon}
|
||||
</IconTile>
|
||||
) : null}
|
||||
|
||||
<div className="flex min-w-0 flex-1 flex-col gap-0.5">
|
||||
<div className="flex items-start justify-between gap-2">
|
||||
<span className="text-foreground text-sm font-medium">{action.title}</span>
|
||||
<Badge variant="outline" size="sm" className="shrink-0">
|
||||
{resolveBadge(action)}
|
||||
</Badge>
|
||||
</div>
|
||||
<p className="text-muted-foreground line-clamp-2 text-xs leading-relaxed">
|
||||
{action.description}
|
||||
</p>
|
||||
</div>
|
||||
</div>
|
||||
)
|
||||
}
|
||||
|
||||
function panelClassName(disabled?: boolean) {
|
||||
return cn(
|
||||
"relative isolate flex h-full flex-col transition-colors",
|
||||
disabled
|
||||
? "cursor-not-allowed opacity-60"
|
||||
: "hover:bg-muted/40 focus-within:ring-ring cursor-pointer focus-within:ring-2",
|
||||
)
|
||||
}
|
||||
|
||||
/**
|
||||
* KPI-like quick actions strip (horizontal Frame tiles).
|
||||
* Preview: https://reui.io/preview/base/stats-12 · https://reui.io/preview/base/card-12
|
||||
* Docs: https://reui.io/docs/components/base/frame · https://reui.io/docs/components/base/icon-tile
|
||||
*/
|
||||
export function QuickActionGrid({
|
||||
actions,
|
||||
title = "Быстрые действия",
|
||||
description,
|
||||
className,
|
||||
}: QuickActionGridProps) {
|
||||
if (actions.length === 0) return null
|
||||
|
||||
return (
|
||||
<Frame dense spacing="sm" className={cn("@container w-full", className)}>
|
||||
{(title || description) && (
|
||||
<FrameHeader>
|
||||
{title ? <FrameTitle>{title}</FrameTitle> : null}
|
||||
{description ? <FrameDescription>{description}</FrameDescription> : null}
|
||||
</FrameHeader>
|
||||
)}
|
||||
<div className={cn("grid gap-2", kpiCols(actions.length))}>
|
||||
{actions.map((action) => {
|
||||
const label = `${action.title}: ${action.description}`
|
||||
|
||||
if ("href" in action && action.href) {
|
||||
return (
|
||||
<FramePanel key={action.id} className={panelClassName(action.disabled)}>
|
||||
{action.disabled ? (
|
||||
<div aria-disabled aria-label={label}>
|
||||
<QuickActionBody action={action} />
|
||||
</div>
|
||||
) : (
|
||||
<>
|
||||
<QuickActionBody action={action} />
|
||||
<Link
|
||||
href={action.href}
|
||||
className="absolute inset-0 z-20 focus-visible:outline-none"
|
||||
aria-label={label}
|
||||
/>
|
||||
</>
|
||||
)}
|
||||
</FramePanel>
|
||||
)
|
||||
}
|
||||
|
||||
const onClick = action.onClick
|
||||
const onActivate = () => {
|
||||
if (action.disabled || !onClick) return
|
||||
onClick()
|
||||
}
|
||||
|
||||
return (
|
||||
<FramePanel
|
||||
key={action.id}
|
||||
className={panelClassName(action.disabled)}
|
||||
role="button"
|
||||
tabIndex={action.disabled ? -1 : 0}
|
||||
aria-disabled={action.disabled || undefined}
|
||||
aria-label={label}
|
||||
onClick={onActivate}
|
||||
onKeyDown={(e) => handleActionKeyDown(onActivate, e)}
|
||||
>
|
||||
<QuickActionBody action={action} />
|
||||
</FramePanel>
|
||||
)
|
||||
})}
|
||||
</div>
|
||||
</Frame>
|
||||
)
|
||||
}
|
||||
@@ -70,63 +70,72 @@ export function TrafficRxTxChart({
|
||||
rx,
|
||||
tx,
|
||||
range = "1h",
|
||||
embedded = false,
|
||||
}: {
|
||||
rx: number[]
|
||||
tx: number[]
|
||||
range?: string
|
||||
/** Skip outer Frame when already inside OpsPanel / Frame. */
|
||||
embedded?: boolean
|
||||
}) {
|
||||
const rangeMinutes = TRAFFIC_RANGE_MINUTES[range] ?? 60
|
||||
const data = toChartData(rx, tx, rangeMinutes)
|
||||
const tickEvery = Math.max(1, Math.ceil(data.length / 6))
|
||||
|
||||
const chart = (
|
||||
<div className="flex flex-col gap-4">
|
||||
<ChartContainer config={chartConfig} className="-ms-4 aspect-auto h-[220px] w-full">
|
||||
<LineChart data={data} margin={{ top: 5, right: 5, left: 5, bottom: 5 }}>
|
||||
<CartesianGrid
|
||||
strokeDasharray="4 8"
|
||||
vertical={false}
|
||||
stroke="var(--border)"
|
||||
/>
|
||||
<XAxis
|
||||
dataKey="time"
|
||||
axisLine={false}
|
||||
tickLine={false}
|
||||
tick={{ fontSize: 11 }}
|
||||
tickMargin={10}
|
||||
interval={tickEvery - 1}
|
||||
/>
|
||||
<YAxis
|
||||
axisLine={false}
|
||||
tickLine={false}
|
||||
tick={{ fontSize: 11 }}
|
||||
tickFormatter={(v: number) => fmtRate(Number(v))}
|
||||
tickMargin={8}
|
||||
width={72}
|
||||
/>
|
||||
<ChartTooltip content={<CustomTooltip />} />
|
||||
<Line
|
||||
dataKey="rx"
|
||||
type="monotone"
|
||||
stroke="var(--chart-rx)"
|
||||
strokeWidth={2}
|
||||
dot={false}
|
||||
/>
|
||||
<Line
|
||||
dataKey="tx"
|
||||
type="monotone"
|
||||
stroke="var(--chart-tx)"
|
||||
strokeWidth={2}
|
||||
dot={false}
|
||||
/>
|
||||
</LineChart>
|
||||
</ChartContainer>
|
||||
<div className="mb-1 flex items-center justify-center gap-6">
|
||||
<ChartLegendItem label="RX (входящий)" color="var(--chart-rx)" />
|
||||
<ChartLegendItem label="TX (исходящий)" color="var(--chart-tx)" />
|
||||
</div>
|
||||
</div>
|
||||
)
|
||||
|
||||
if (embedded) return chart
|
||||
|
||||
return (
|
||||
<Frame className="w-full">
|
||||
<FramePanel className="flex flex-col gap-6">
|
||||
<ChartContainer config={chartConfig} className="-ms-4 aspect-auto h-[220px] w-full">
|
||||
<LineChart data={data} margin={{ top: 5, right: 5, left: 5, bottom: 5 }}>
|
||||
<CartesianGrid
|
||||
strokeDasharray="4 8"
|
||||
vertical={false}
|
||||
stroke="var(--border)"
|
||||
/>
|
||||
<XAxis
|
||||
dataKey="time"
|
||||
axisLine={false}
|
||||
tickLine={false}
|
||||
tick={{ fontSize: 11 }}
|
||||
tickMargin={10}
|
||||
interval={tickEvery - 1}
|
||||
/>
|
||||
<YAxis
|
||||
axisLine={false}
|
||||
tickLine={false}
|
||||
tick={{ fontSize: 11 }}
|
||||
tickFormatter={(v: number) => fmtRate(Number(v))}
|
||||
tickMargin={8}
|
||||
width={72}
|
||||
/>
|
||||
<ChartTooltip content={<CustomTooltip />} />
|
||||
<Line
|
||||
dataKey="rx"
|
||||
type="monotone"
|
||||
stroke="var(--chart-rx)"
|
||||
strokeWidth={2}
|
||||
dot={false}
|
||||
/>
|
||||
<Line
|
||||
dataKey="tx"
|
||||
type="monotone"
|
||||
stroke="var(--chart-tx)"
|
||||
strokeWidth={2}
|
||||
dot={false}
|
||||
/>
|
||||
</LineChart>
|
||||
</ChartContainer>
|
||||
<div className="mb-1 flex items-center justify-center gap-6">
|
||||
<ChartLegendItem label="RX (входящий)" color="var(--chart-rx)" />
|
||||
<ChartLegendItem label="TX (исходящий)" color="var(--chart-tx)" />
|
||||
</div>
|
||||
</FramePanel>
|
||||
<FramePanel className="flex flex-col gap-6">{chart}</FramePanel>
|
||||
</Frame>
|
||||
)
|
||||
}
|
||||
|
||||
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,57 @@
|
||||
"use client"
|
||||
|
||||
import { Frame, FrameHeader, FramePanel, FrameTitle } from "@/components/reui/frame"
|
||||
import {
|
||||
StatisticsBreakdownDataGrid,
|
||||
type StatisticsSliceKind,
|
||||
} from "@/components/data-grids/statistics-breakdown-data-grid"
|
||||
import { STATISTICS_DIMS } from "@/lib/statistics-dims"
|
||||
import type { StatisticsBreakdownRow, StatisticsDto } from "@mmapp/contracts/statistics"
|
||||
|
||||
const MINI_ROWS = 12
|
||||
|
||||
function rowsForKind(data: StatisticsDto, kind: StatisticsSliceKind): StatisticsBreakdownRow[] {
|
||||
if (kind === "users") return data.users
|
||||
if (kind === "servers") return data.servers
|
||||
if (kind === "interfaces") return data.interfaces
|
||||
if (kind === "countries") return data.countries
|
||||
if (kind === "services") return data.services
|
||||
return data.asns
|
||||
}
|
||||
|
||||
export function BreakdownDashboard({
|
||||
data,
|
||||
hidden,
|
||||
selectedIdFor,
|
||||
onRowClick,
|
||||
isLoading,
|
||||
}: {
|
||||
data: StatisticsDto
|
||||
hidden: Set<StatisticsSliceKind>
|
||||
selectedIdFor: (kind: StatisticsSliceKind) => string | undefined
|
||||
onRowClick: (kind: StatisticsSliceKind, row: StatisticsBreakdownRow) => void
|
||||
isLoading?: boolean
|
||||
}) {
|
||||
const dims = STATISTICS_DIMS.filter((d) => !hidden.has(d.id))
|
||||
return (
|
||||
<div className="grid grid-cols-1 gap-4 p-4 md:grid-cols-2">
|
||||
{dims.map((d) => (
|
||||
<Frame key={d.id} dense>
|
||||
<FrameHeader className="border-b">
|
||||
<FrameTitle>{d.label}</FrameTitle>
|
||||
</FrameHeader>
|
||||
<FramePanel className="max-h-80 overflow-auto p-0">
|
||||
<StatisticsBreakdownDataGrid
|
||||
rows={rowsForKind(data, d.id).slice(0, MINI_ROWS)}
|
||||
kind={d.id}
|
||||
selectedId={selectedIdFor(d.id)}
|
||||
onRowClick={(row) => onRowClick(d.id, row)}
|
||||
isLoading={isLoading}
|
||||
density="mini"
|
||||
/>
|
||||
</FramePanel>
|
||||
</Frame>
|
||||
))}
|
||||
</div>
|
||||
)
|
||||
}
|
||||
@@ -0,0 +1,65 @@
|
||||
import type { DateSelectorI18nConfig } from "@/components/reui/date-selector"
|
||||
|
||||
/** Русские подписи ReUI DateSelector (шапка /statistics). */
|
||||
export const DATE_SELECTOR_RU: DateSelectorI18nConfig = {
|
||||
selectDate: "Выбрать дату",
|
||||
apply: "Применить",
|
||||
cancel: "Отмена",
|
||||
clear: "Сбросить",
|
||||
today: "Сегодня",
|
||||
filterTypes: {
|
||||
is: "равно",
|
||||
before: "до",
|
||||
after: "после",
|
||||
between: "между",
|
||||
},
|
||||
periodTypes: {
|
||||
day: "День",
|
||||
month: "Месяц",
|
||||
quarter: "Квартал",
|
||||
halfYear: "Полугодие",
|
||||
year: "Год",
|
||||
},
|
||||
months: [
|
||||
"Январь",
|
||||
"Февраль",
|
||||
"Март",
|
||||
"Апрель",
|
||||
"Май",
|
||||
"Июнь",
|
||||
"Июль",
|
||||
"Август",
|
||||
"Сентябрь",
|
||||
"Октябрь",
|
||||
"Ноябрь",
|
||||
"Декабрь",
|
||||
],
|
||||
monthsShort: [
|
||||
"янв",
|
||||
"фев",
|
||||
"мар",
|
||||
"апр",
|
||||
"май",
|
||||
"июн",
|
||||
"июл",
|
||||
"авг",
|
||||
"сен",
|
||||
"окт",
|
||||
"ноя",
|
||||
"дек",
|
||||
],
|
||||
quarters: ["I кв.", "II кв.", "III кв.", "IV кв."],
|
||||
halfYears: ["1-е полугодие", "2-е полугодие"],
|
||||
weekdays: [
|
||||
"Воскресенье",
|
||||
"Понедельник",
|
||||
"Вторник",
|
||||
"Среда",
|
||||
"Четверг",
|
||||
"Пятница",
|
||||
"Суббота",
|
||||
],
|
||||
weekdaysShort: ["вс", "пн", "вт", "ср", "чт", "пт", "сб"],
|
||||
placeholder: "Выберите дату…",
|
||||
rangePlaceholder: "Выберите период…",
|
||||
}
|
||||
@@ -0,0 +1,73 @@
|
||||
"use client"
|
||||
|
||||
import {
|
||||
Select,
|
||||
SelectContent,
|
||||
SelectItem,
|
||||
SelectTrigger,
|
||||
SelectValue,
|
||||
} from "@/components/ui/select"
|
||||
import { STATISTICS_DIMS } from "@/lib/statistics-dims"
|
||||
import type { StatisticsPivotDim } from "@mmapp/contracts/statistics"
|
||||
import type { StatisticsSliceKind } from "@/components/data-grids/statistics-breakdown-data-grid"
|
||||
|
||||
export function DimensionSelect({
|
||||
value,
|
||||
onChange,
|
||||
label,
|
||||
}: {
|
||||
value: StatisticsSliceKind
|
||||
onChange: (value: StatisticsSliceKind) => void
|
||||
label?: string
|
||||
}) {
|
||||
const current = STATISTICS_DIMS.find((d) => d.id === value)
|
||||
return (
|
||||
<label className="flex items-center gap-2">
|
||||
{label ? <span className="text-muted-foreground text-xs whitespace-nowrap">{label}</span> : null}
|
||||
<Select value={value} onValueChange={(v) => onChange(String(v ?? value) as StatisticsSliceKind)}>
|
||||
<SelectTrigger size="sm" className="min-w-40">
|
||||
<SelectValue>{current?.label ?? "Измерение"}</SelectValue>
|
||||
</SelectTrigger>
|
||||
<SelectContent align="start">
|
||||
{STATISTICS_DIMS.map((d) => (
|
||||
<SelectItem key={d.id} value={d.id}>
|
||||
{d.label}
|
||||
</SelectItem>
|
||||
))}
|
||||
</SelectContent>
|
||||
</Select>
|
||||
</label>
|
||||
)
|
||||
}
|
||||
|
||||
export function PivotDimSelect({
|
||||
value,
|
||||
onChange,
|
||||
exclude,
|
||||
label,
|
||||
}: {
|
||||
value: StatisticsPivotDim
|
||||
onChange: (value: StatisticsPivotDim) => void
|
||||
exclude?: StatisticsPivotDim
|
||||
label: string
|
||||
}) {
|
||||
const options = STATISTICS_DIMS.filter((d) => d.pivot !== exclude)
|
||||
const current = STATISTICS_DIMS.find((d) => d.pivot === value)
|
||||
return (
|
||||
<label className="flex items-center gap-2">
|
||||
<span className="text-muted-foreground text-xs whitespace-nowrap">{label}</span>
|
||||
<Select value={value} onValueChange={(v) => onChange(String(v ?? value) as StatisticsPivotDim)}>
|
||||
<SelectTrigger size="sm" className="min-w-40">
|
||||
<SelectValue>{current?.label ?? label}</SelectValue>
|
||||
</SelectTrigger>
|
||||
<SelectContent align="start">
|
||||
{options.map((d) => (
|
||||
<SelectItem key={d.pivot} value={d.pivot}>
|
||||
{d.label}
|
||||
</SelectItem>
|
||||
))}
|
||||
</SelectContent>
|
||||
</Select>
|
||||
</label>
|
||||
)
|
||||
}
|
||||
@@ -0,0 +1,188 @@
|
||||
"use client"
|
||||
|
||||
import { useMemo } from "react"
|
||||
import { format } from "date-fns"
|
||||
import { ru } from "date-fns/locale"
|
||||
import { CalendarIcon } from "lucide-react"
|
||||
import {
|
||||
DateSelector,
|
||||
type DateSelectorValue,
|
||||
} from "@/components/reui/date-selector"
|
||||
import { DATE_SELECTOR_RU } from "@/components/statistics/date-selector-i18n"
|
||||
import { Button } from "@/components/ui/button"
|
||||
import { Popover, PopoverContent, PopoverTrigger } from "@/components/ui/popover"
|
||||
|
||||
/**
|
||||
* Период отчёта — DateSelector в Popover (c-date-selector-2).
|
||||
* @see https://reui.io/preview/base/components/c-date-selector-2
|
||||
* @see https://reui.io/docs/components/base/date-selector
|
||||
*/
|
||||
|
||||
export type PeriodPreset = "today" | "24h" | "7d" | "30d" | "month"
|
||||
|
||||
export interface DateRangeYmd {
|
||||
from: string
|
||||
to: string
|
||||
}
|
||||
|
||||
const PRESETS: { id: PeriodPreset; label: string }[] = [
|
||||
{ id: "today", label: "Сегодня" },
|
||||
{ id: "24h", label: "24 ч" },
|
||||
{ id: "7d", label: "7 д" },
|
||||
{ id: "30d", label: "30 д" },
|
||||
{ id: "month", label: "Месяц" },
|
||||
]
|
||||
|
||||
function pad2(n: number): string {
|
||||
return String(n).padStart(2, "0")
|
||||
}
|
||||
|
||||
export function formatYmd(d: Date): string {
|
||||
return `${d.getFullYear()}-${pad2(d.getMonth() + 1)}-${pad2(d.getDate())}`
|
||||
}
|
||||
|
||||
export function parseYmd(s: string): Date | null {
|
||||
const m = /^(\d{4})-(\d{2})-(\d{2})$/.exec(s)
|
||||
if (!m) return null
|
||||
const d = new Date(Number(m[1]), Number(m[2]) - 1, Number(m[3]))
|
||||
return Number.isNaN(d.getTime()) ? null : d
|
||||
}
|
||||
|
||||
export function rangeForPreset(preset: PeriodPreset, now = new Date()): DateRangeYmd {
|
||||
const to = formatYmd(now)
|
||||
if (preset === "today") return { from: to, to }
|
||||
if (preset === "24h") {
|
||||
const from = new Date(now)
|
||||
from.setDate(from.getDate() - 1)
|
||||
return { from: formatYmd(from), to }
|
||||
}
|
||||
if (preset === "7d") {
|
||||
const from = new Date(now)
|
||||
from.setDate(from.getDate() - 6)
|
||||
return { from: formatYmd(from), to }
|
||||
}
|
||||
if (preset === "30d") {
|
||||
const from = new Date(now)
|
||||
from.setDate(from.getDate() - 29)
|
||||
return { from: formatYmd(from), to }
|
||||
}
|
||||
const start = new Date(now.getFullYear(), now.getMonth(), 1)
|
||||
return { from: formatYmd(start), to }
|
||||
}
|
||||
|
||||
export function dateSelectorToRange(value: DateSelectorValue): DateRangeYmd | null {
|
||||
if (value.period === "day") {
|
||||
if (value.operator === "between") {
|
||||
if (!value.startDate || !value.endDate) return null
|
||||
const a = formatYmd(value.startDate)
|
||||
const b = formatYmd(value.endDate)
|
||||
return a <= b ? { from: a, to: b } : { from: b, to: a }
|
||||
}
|
||||
if (value.startDate) {
|
||||
const d = formatYmd(value.startDate)
|
||||
return { from: d, to: d }
|
||||
}
|
||||
}
|
||||
if (value.period === "month" && value.year != null && value.month != null) {
|
||||
const start = new Date(value.year, value.month, 1)
|
||||
const end = new Date(value.year, value.month + 1, 0)
|
||||
return { from: formatYmd(start), to: formatYmd(end) }
|
||||
}
|
||||
if (value.period === "year" && value.year != null) {
|
||||
return { from: `${value.year}-01-01`, to: `${value.year}-12-31` }
|
||||
}
|
||||
if (value.period === "quarter" && value.year != null && value.quarter != null) {
|
||||
const startMonth = value.quarter * 3
|
||||
const start = new Date(value.year, startMonth, 1)
|
||||
const end = new Date(value.year, startMonth + 3, 0)
|
||||
return { from: formatYmd(start), to: formatYmd(end) }
|
||||
}
|
||||
if (value.period === "half-year" && value.year != null && value.halfYear != null) {
|
||||
const startMonth = value.halfYear * 6
|
||||
const start = new Date(value.year, startMonth, 1)
|
||||
const end = new Date(value.year, startMonth + 6, 0)
|
||||
return { from: formatYmd(start), to: formatYmd(end) }
|
||||
}
|
||||
return null
|
||||
}
|
||||
|
||||
export function rangeToSelector(range: DateRangeYmd): DateSelectorValue {
|
||||
return {
|
||||
period: "day",
|
||||
operator: "between",
|
||||
startDate: parseYmd(range.from) ?? undefined,
|
||||
endDate: parseYmd(range.to) ?? undefined,
|
||||
}
|
||||
}
|
||||
|
||||
function formatRangeLabel(range: DateRangeYmd): string {
|
||||
const from = parseYmd(range.from)
|
||||
const to = parseYmd(range.to)
|
||||
if (!from || !to) return "Период"
|
||||
if (range.from === range.to) return format(from, "d MMM yyyy", { locale: ru })
|
||||
return `${format(from, "d MMM", { locale: ru })} – ${format(to, "d MMM yyyy", { locale: ru })}`
|
||||
}
|
||||
|
||||
export function PeriodSelector({
|
||||
range,
|
||||
onChange,
|
||||
}: {
|
||||
range: DateRangeYmd
|
||||
onChange: (next: DateRangeYmd) => void
|
||||
}) {
|
||||
const selectorValue = useMemo(() => rangeToSelector(range), [range])
|
||||
|
||||
function handleSelectorChange(value: DateSelectorValue) {
|
||||
const next = dateSelectorToRange(value)
|
||||
if (!next) return
|
||||
if (next.from === range.from && next.to === range.to) return
|
||||
onChange(next)
|
||||
}
|
||||
|
||||
const activePreset = PRESETS.find((p) => {
|
||||
const r = rangeForPreset(p.id)
|
||||
return r.from === range.from && r.to === range.to
|
||||
})?.id
|
||||
|
||||
return (
|
||||
<div className="flex flex-wrap items-center gap-1.5">
|
||||
{PRESETS.map((p) => (
|
||||
<Button
|
||||
key={p.id}
|
||||
type="button"
|
||||
variant={activePreset === p.id ? "secondary" : "ghost"}
|
||||
size="sm"
|
||||
onClick={() => onChange(rangeForPreset(p.id))}
|
||||
>
|
||||
{p.label}
|
||||
</Button>
|
||||
))}
|
||||
<Popover>
|
||||
<PopoverTrigger
|
||||
render={
|
||||
<Button type="button" variant="outline" size="sm" className="min-w-40 justify-between" />
|
||||
}
|
||||
>
|
||||
<CalendarIcon className="size-3.5" />
|
||||
<span className="tabular-nums">{formatRangeLabel(range)}</span>
|
||||
</PopoverTrigger>
|
||||
<PopoverContent align="end" className="w-auto min-w-[32rem] max-w-[min(100vw-2rem,42rem)] p-3">
|
||||
<DateSelector
|
||||
value={selectorValue}
|
||||
onChange={handleSelectorChange}
|
||||
allowRange
|
||||
defaultPeriodType="day"
|
||||
defaultFilterType="between"
|
||||
periodTypes={["day", "month", "year"]}
|
||||
showTwoMonths
|
||||
weekStartsOn={1}
|
||||
maxYear={2035}
|
||||
dayDateFormat="dd.MM.yyyy"
|
||||
i18n={DATE_SELECTOR_RU}
|
||||
className="sm:w-[470px]"
|
||||
/>
|
||||
</PopoverContent>
|
||||
</Popover>
|
||||
</div>
|
||||
)
|
||||
}
|
||||
@@ -0,0 +1,38 @@
|
||||
"use client"
|
||||
|
||||
import { XIcon } from "lucide-react"
|
||||
import { Badge } from "@/components/reui/badge"
|
||||
import { Button } from "@/components/ui/button"
|
||||
|
||||
export interface SliceChip {
|
||||
key: string
|
||||
label: string
|
||||
}
|
||||
|
||||
export function SliceChips({
|
||||
chips,
|
||||
onRemove,
|
||||
}: {
|
||||
chips: SliceChip[]
|
||||
onRemove: (key: string) => void
|
||||
}) {
|
||||
if (!chips.length) return null
|
||||
return (
|
||||
<div className="flex flex-wrap items-center gap-1.5 border-b px-5 py-2">
|
||||
{chips.map((chip) => (
|
||||
<Badge key={chip.key} variant="outline" size="sm" className="gap-1 pr-0.5">
|
||||
{chip.label}
|
||||
<Button
|
||||
type="button"
|
||||
variant="ghost"
|
||||
size="icon-xs"
|
||||
aria-label={`Снять ${chip.label}`}
|
||||
onClick={() => onRemove(chip.key)}
|
||||
>
|
||||
<XIcon />
|
||||
</Button>
|
||||
</Badge>
|
||||
))}
|
||||
</div>
|
||||
)
|
||||
}
|
||||
@@ -0,0 +1,89 @@
|
||||
"use client"
|
||||
|
||||
import { useMemo } from "react"
|
||||
import { CompactDataGrid, type CompactDataGridColumn } from "@/components/data-grids/compact-data-grid"
|
||||
import { formatBytes } from "@/lib/fmt-rate"
|
||||
import { cn } from "@/lib/utils"
|
||||
import type { StatisticsPivotDto } from "@mmapp/contracts/statistics"
|
||||
|
||||
interface PivotGridRow {
|
||||
id: string
|
||||
label: string
|
||||
total: number
|
||||
[key: string]: string | number
|
||||
}
|
||||
|
||||
export function StatisticsPivotGrid({
|
||||
data,
|
||||
onCellClick,
|
||||
isLoading,
|
||||
}: {
|
||||
data: StatisticsPivotDto
|
||||
onCellClick?: (rowId: string, colId: string) => void
|
||||
isLoading?: boolean
|
||||
}) {
|
||||
const rows: PivotGridRow[] = useMemo(
|
||||
() =>
|
||||
data.rows.map((r) => {
|
||||
const next: PivotGridRow = { id: r.id, label: r.label, total: r.total }
|
||||
for (const col of data.columns) {
|
||||
next[`c:${col.id}`] = r.cells[col.id] ?? 0
|
||||
}
|
||||
return next
|
||||
}),
|
||||
[data],
|
||||
)
|
||||
|
||||
const columns: CompactDataGridColumn<PivotGridRow>[] = [
|
||||
{
|
||||
id: "label",
|
||||
header: "Измерение",
|
||||
accessorKey: "label",
|
||||
cell: (row) => <span className="truncate font-medium">{row.label}</span>,
|
||||
},
|
||||
...data.columns.map((col) => ({
|
||||
id: `c:${col.id}`,
|
||||
header: col.label,
|
||||
accessorKey: `c:${col.id}` as const,
|
||||
cell: (row: PivotGridRow) => {
|
||||
const value = Number(row[`c:${col.id}`] ?? 0)
|
||||
return (
|
||||
<button
|
||||
type="button"
|
||||
className={cn(
|
||||
"tabular-nums text-left hover:underline",
|
||||
col.id === "__other__" || row.id === "__other__" ? "text-muted-foreground" : "",
|
||||
)}
|
||||
onClick={(e) => {
|
||||
e.stopPropagation()
|
||||
if (row.id === "__other__" || col.id === "__other__") return
|
||||
onCellClick?.(row.id, col.id)
|
||||
}}
|
||||
>
|
||||
{data.metric === "packets" ? value.toLocaleString("ru-RU") : formatBytes(value)}
|
||||
</button>
|
||||
)
|
||||
},
|
||||
})),
|
||||
{
|
||||
id: "total",
|
||||
header: "Итого",
|
||||
accessorKey: "total",
|
||||
cell: (row) => (
|
||||
<span className="tabular-nums font-medium">
|
||||
{data.metric === "packets" ? row.total.toLocaleString("ru-RU") : formatBytes(row.total)}
|
||||
</span>
|
||||
),
|
||||
},
|
||||
]
|
||||
|
||||
return (
|
||||
<CompactDataGrid
|
||||
data={rows}
|
||||
columns={columns}
|
||||
isLoading={isLoading}
|
||||
emptyTitle="Нет данных сводной"
|
||||
emptyDescription="Выберите разные измерения строк и колонок."
|
||||
/>
|
||||
)
|
||||
}
|
||||
@@ -0,0 +1,105 @@
|
||||
"use client"
|
||||
|
||||
import { Area, AreaChart, CartesianGrid, XAxis, YAxis } from "recharts"
|
||||
import { Frame, FramePanel } from "@/components/reui/frame"
|
||||
import { ChartContainer, ChartTooltip, type ChartConfig } from "@/components/ui/chart"
|
||||
import { formatBytes } from "@/lib/fmt-rate"
|
||||
import type { StatisticsSeriesPoint } from "@mmapp/contracts/statistics"
|
||||
|
||||
/**
|
||||
* Объём трафика за период — adapt ReUI chart-23 (байты, не live bps).
|
||||
* @see https://reui.io/preview/base/chart-23
|
||||
*/
|
||||
|
||||
const chartConfig = {
|
||||
bytes: { label: "Объём", color: "var(--chart-1)" },
|
||||
} satisfies ChartConfig
|
||||
|
||||
function formatTick(iso: string, grain: "hour" | "day"): string {
|
||||
const d = new Date(iso)
|
||||
if (Number.isNaN(d.getTime())) return iso.slice(0, 10)
|
||||
if (grain === "hour") {
|
||||
return `${String(d.getHours()).padStart(2, "0")}:00`
|
||||
}
|
||||
return d.toLocaleDateString("ru-RU", { day: "2-digit", month: "short" })
|
||||
}
|
||||
|
||||
function CustomTooltip({
|
||||
active,
|
||||
payload,
|
||||
}: {
|
||||
active?: boolean
|
||||
payload?: { payload: { label: string; bytes: number } }[]
|
||||
}) {
|
||||
if (!active || !payload?.length) return null
|
||||
const row = payload[0]?.payload
|
||||
if (!row) return null
|
||||
return (
|
||||
<div className="flex min-w-[120px] flex-col gap-1.5 rounded-lg bg-popover p-3 text-popover-foreground shadow-lg ring-1 ring-foreground/10">
|
||||
<div className="text-[10px] font-medium tracking-wider uppercase opacity-70">{row.label}</div>
|
||||
<div className="text-sm font-semibold tabular-nums">{formatBytes(row.bytes)}</div>
|
||||
</div>
|
||||
)
|
||||
}
|
||||
|
||||
export function StatisticsVolumeChart({
|
||||
series,
|
||||
grain,
|
||||
}: {
|
||||
series: StatisticsSeriesPoint[]
|
||||
grain: "hour" | "day"
|
||||
}) {
|
||||
const data = series.map((p) => ({
|
||||
t: p.t,
|
||||
bytes: p.bytes,
|
||||
label: formatTick(p.t, grain),
|
||||
}))
|
||||
const tickEvery = Math.max(1, Math.ceil(data.length / 8))
|
||||
|
||||
return (
|
||||
<Frame className="w-full">
|
||||
<FramePanel className="flex flex-col gap-4">
|
||||
<div className="flex items-baseline justify-between gap-2">
|
||||
<h2 className="text-sm font-medium">Объём по времени</h2>
|
||||
<span className="text-muted-foreground text-xs">
|
||||
{grain === "hour" ? "по часам" : "по суткам"}
|
||||
</span>
|
||||
</div>
|
||||
{data.length === 0 ? (
|
||||
<p className="text-muted-foreground py-10 text-center text-sm">Нет данных за выбранный период</p>
|
||||
) : (
|
||||
<ChartContainer config={chartConfig} className="-ms-4 aspect-auto h-[220px] w-full">
|
||||
<AreaChart data={data} margin={{ top: 5, right: 5, left: 5, bottom: 5 }}>
|
||||
<CartesianGrid strokeDasharray="4 8" vertical={false} stroke="var(--border)" />
|
||||
<XAxis
|
||||
dataKey="label"
|
||||
axisLine={false}
|
||||
tickLine={false}
|
||||
tick={{ fontSize: 11 }}
|
||||
tickMargin={10}
|
||||
interval={tickEvery - 1}
|
||||
/>
|
||||
<YAxis
|
||||
axisLine={false}
|
||||
tickLine={false}
|
||||
tick={{ fontSize: 11 }}
|
||||
tickFormatter={(v: number) => formatBytes(Number(v))}
|
||||
tickMargin={8}
|
||||
width={72}
|
||||
/>
|
||||
<ChartTooltip content={<CustomTooltip />} />
|
||||
<Area
|
||||
dataKey="bytes"
|
||||
type="monotone"
|
||||
stroke="var(--chart-1)"
|
||||
fill="var(--chart-1)"
|
||||
fillOpacity={0.15}
|
||||
strokeWidth={2}
|
||||
/>
|
||||
</AreaChart>
|
||||
</ChartContainer>
|
||||
)}
|
||||
</FramePanel>
|
||||
</Frame>
|
||||
)
|
||||
}
|
||||
@@ -10,6 +10,7 @@ import { Button } from "@/components/ui/button"
|
||||
import { Input } from "@/components/ui/input"
|
||||
import { CodeExportSheet, type CodeExportFormat } from "@/components/reui-kit/code-export-sheet"
|
||||
import type { TrafficFlowSettingsDto } from "@mmapp/contracts/traffic-flow"
|
||||
import type { GeoipStatusDto } from "@mmapp/contracts/geoip"
|
||||
import {
|
||||
generateTrafficFlowKeys,
|
||||
getTrafficFlowHostFiles,
|
||||
@@ -17,6 +18,7 @@ import {
|
||||
purgeTrafficFlowData,
|
||||
putTrafficFlowSettings,
|
||||
} from "@/shared/api/traffic-flow"
|
||||
import { getGeoipStatus, putGeoipSettings, runGeoipUpdateNow } from "@/shared/api/geoip"
|
||||
import { formatFlowPurgeResult, NetflowPurgeConfirm } from "@/components/traffic/netflow-purge-dialog"
|
||||
import { KeyRoundIcon, DownloadIcon, InfoIcon } from "lucide-react"
|
||||
|
||||
@@ -29,6 +31,127 @@ const HOST_STEPS = [
|
||||
"Проверка: wg show · ss -ulnp | grep 4739 · в этой панели — last datagram.",
|
||||
]
|
||||
|
||||
function fmtDate(iso: string | null | undefined): string {
|
||||
return iso ? new Date(iso).toLocaleString("ru-RU") : "—"
|
||||
}
|
||||
|
||||
function GeoipSettingsSection({ backendUrl }: { backendUrl: string }) {
|
||||
const [status, setStatus] = useState<GeoipStatusDto | null>(null)
|
||||
const [busy, setBusy] = useState(false)
|
||||
const [updating, setUpdating] = useState(false)
|
||||
const [autoOn, setAutoOn] = useState(true)
|
||||
const [intervalHours, setIntervalHours] = useState("168")
|
||||
|
||||
const load = useCallback(async () => {
|
||||
const s = await getGeoipStatus(backendUrl)
|
||||
setStatus(s)
|
||||
setAutoOn(s.settings.enabled)
|
||||
setIntervalHours(String(Math.round(s.settings.updateIntervalSec / 3600)))
|
||||
}, [backendUrl])
|
||||
|
||||
useEffect(() => {
|
||||
void load().catch((e: unknown) => {
|
||||
toast.error(e instanceof Error ? e.message : "Не удалось загрузить GeoIP")
|
||||
})
|
||||
}, [load])
|
||||
|
||||
async function handleSave() {
|
||||
setBusy(true)
|
||||
try {
|
||||
const hours = Math.min(720, Math.max(6, Number.parseInt(intervalHours, 10) || 168))
|
||||
const res = await putGeoipSettings(backendUrl, {
|
||||
enabled: autoOn,
|
||||
updateIntervalSec: hours * 3600,
|
||||
})
|
||||
setStatus(res.status)
|
||||
toast.success("Настройки GeoIP сохранены")
|
||||
} catch (e) {
|
||||
toast.error(e instanceof Error ? e.message : "Не удалось сохранить")
|
||||
} finally {
|
||||
setBusy(false)
|
||||
}
|
||||
}
|
||||
|
||||
async function handleUpdateNow() {
|
||||
setUpdating(true)
|
||||
try {
|
||||
const res = await runGeoipUpdateNow(backendUrl)
|
||||
if (res.ok || res.snapshot.downloaded > 0 || res.snapshot.skippedUnchanged > 0) {
|
||||
toast.success(
|
||||
res.snapshot.downloaded > 0
|
||||
? `Скачано баз: ${res.snapshot.downloaded} (${(res.snapshot.bytes / 1024 / 1024).toFixed(1)} МБ)`
|
||||
: "Базы актуальны, скачивание не требуется",
|
||||
)
|
||||
} else {
|
||||
toast.error(res.snapshot.errors.join("; ") || "Обновление не выполнено")
|
||||
}
|
||||
await load()
|
||||
} catch (e) {
|
||||
toast.error(e instanceof Error ? e.message : "Не удалось обновить базы")
|
||||
} finally {
|
||||
setUpdating(false)
|
||||
}
|
||||
}
|
||||
|
||||
return (
|
||||
<OpsPanel
|
||||
title="GeoIP-базы (GeoLite2)"
|
||||
description="Локальные mmdb MaxMind GeoLite2 с зеркала P3TERX: страна и ASN каждого потока при ingest — мгновенно, включая IPv6, без лимитов RIPEstat."
|
||||
headerRight={
|
||||
<div className="flex items-center gap-2">
|
||||
<Badge variant={status?.countryLoaded ? "success" : "secondary"}>
|
||||
country {status?.countryLoaded ? "ok" : "нет"}
|
||||
</Badge>
|
||||
<Badge variant={status?.asnLoaded ? "success" : "secondary"}>
|
||||
asn {status?.asnLoaded ? "ok" : "нет"}
|
||||
</Badge>
|
||||
</div>
|
||||
}
|
||||
contentClassName="px-5 py-4 flex flex-col gap-4"
|
||||
>
|
||||
<div className="grid grid-cols-1 sm:grid-cols-2 gap-3">
|
||||
<div className="flex items-center gap-3">
|
||||
<FormToggle checked={autoOn} onChange={setAutoOn} />
|
||||
<span className="text-sm">Автообновление</span>
|
||||
</div>
|
||||
<FormField label="Интервал проверки (часов)" hint="Upstream обновляется еженедельно; минимум 6 ч">
|
||||
<Input
|
||||
className="font-mono"
|
||||
value={intervalHours}
|
||||
onChange={(e) => setIntervalHours(e.target.value)}
|
||||
inputMode="numeric"
|
||||
disabled={!autoOn}
|
||||
/>
|
||||
</FormField>
|
||||
</div>
|
||||
|
||||
<p className="text-xs text-muted-foreground">
|
||||
Сборка Country: {fmtDate(status?.settings.countryBuildAt)} · ASN: {fmtDate(status?.settings.asnBuildAt)}
|
||||
{" · "}последняя проверка: {fmtDate(status?.settings.lastCheckAt)}
|
||||
{status?.settings.lastError ? ` · ошибка: ${status.settings.lastError}` : ""}
|
||||
</p>
|
||||
<p className="text-xs text-muted-foreground">
|
||||
Каталог: <span className="font-mono">{status?.dir || "storage/geoip"}</span>. До загрузки баз
|
||||
и при промахе lookup страна/ASN берутся из RIPEstat, как раньше.
|
||||
</p>
|
||||
|
||||
<div className="flex flex-wrap gap-2">
|
||||
<Button size="sm" disabled={busy || updating} onClick={() => { void handleSave() }}>
|
||||
Сохранить GeoIP
|
||||
</Button>
|
||||
<Button size="sm" variant="outline" disabled={busy || updating} onClick={() => { void handleUpdateNow() }}>
|
||||
<DownloadIcon className={updating ? "size-4 animate-spin" : "size-4"} />
|
||||
{updating ? "Обновление…" : "Обновить сейчас"}
|
||||
</Button>
|
||||
</div>
|
||||
|
||||
<p className="text-xs text-muted-foreground">
|
||||
Данные: MaxMind GeoLite2 (CC BY-SA 4.0), зеркало P3TERX/GeoLite.mmdb.
|
||||
</p>
|
||||
</OpsPanel>
|
||||
)
|
||||
}
|
||||
|
||||
function NetflowSettingsPanel({
|
||||
backendUrl,
|
||||
enabled,
|
||||
@@ -276,6 +399,8 @@ function NetflowSettingsPanel({
|
||||
</div>
|
||||
</OpsPanel>
|
||||
|
||||
<GeoipSettingsSection backendUrl={backendUrl} />
|
||||
|
||||
<CodeExportSheet
|
||||
open={exportOpen}
|
||||
onClose={() => setExportOpen(false)}
|
||||
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user