Compare commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
0fdd2fc1e1 | ||
|
|
f15a7348db | ||
|
|
f3c846201c | ||
|
|
ee9804f1bb | ||
|
|
4ce6169d14 | ||
|
|
60a0970d73 | ||
|
|
fc29dcede7 | ||
|
|
5f774ce26e | ||
|
|
25b82997b6 | ||
|
|
b4a3c3a925 | ||
|
|
9c0ee7940e | ||
|
|
5750590b68 | ||
|
|
3c42c114f5 | ||
|
|
5aef419582 | ||
|
|
0c0dfa1df7 | ||
|
|
c162a41bc0 | ||
|
|
97e43b2335 | ||
|
|
db21e1217c | ||
|
|
5bb9066be8 | ||
|
|
b1fd259f10 | ||
|
|
3687bb8fa2 |
@@ -0,0 +1,63 @@
|
||||
# Локальные GeoLite2-базы (Country + ASN) для потоков по странам и ASN
|
||||
|
||||
## Контекст
|
||||
|
||||
Сейчас страна и ASN для netflow-потоков резолвятся через внешний RIPEstat API (`backend/src/services/traffic-flow-ripe.ts`): лимит 30 новых префиксов/мин, очередь на 90, кэш в PG `flow_ip_meta`. Новые IP «дозревают» с задержкой, IPv6 не покрывается (кэш индексируется только по IPv4). Локальные mmdb-базы дают мгновенный синхронный lookup всех IP без внешних вызовов.
|
||||
|
||||
Решения (подтверждены):
|
||||
- Источник — **MaxMind GeoLite2 через P3TERX-зеркало**: `https://github.com/P3TERX/GeoLite.mmdb/raw/download/GeoLite2-Country.mmdb` и `.../GeoLite2-ASN.mmdb`. Без регистрации, ключей и tar-распаковки. Точность по стране у GeoLite2 и IPinfo паритетная (<1% ошибок у обоих, arXiv 2026); выбран P3TERX за надёжность зеркала (5.2k звёзд) и преемственность: текущий RIPE-путь и так читает GeoLite (`maxmind-geo-lite`), история в кэше остаётся консистентной.
|
||||
- **RIPEstat остаётся fallback** (до первой загрузки баз / если lookup не дал результата).
|
||||
- City-базу не качаем (lat/lng фронтенд не использует).
|
||||
|
||||
## Изменения
|
||||
|
||||
### 1. Зависимость
|
||||
- `npm install -w mikrotik-manager-backend maxmind` — sync-чтение mmdb, встроенные TS-типы, без транзитивных зависимостей, Node 22 ок.
|
||||
|
||||
### 2. Новый сервис `backend/src/services/traffic-flow-geoip.ts`
|
||||
(по конвенциям окружения traffic-flow-*: контракт → маршрут → сервис, без БД-логики в маршрутах)
|
||||
- Каталог: `backend/storage/geoip/` (конвенция `storage/backups`), файлы `GeoLite2-Country.mmdb`, `GeoLite2-ASN.mmdb`.
|
||||
- `initGeoip()` — открыть ридеры best-effort при старте (из `index.ts` рядом с `startTrafficFlowListener`), независимо от настроек автообновления: файлы есть — работают.
|
||||
- `lookupGeoip(ip): FlowIpMeta | null` — синхронно: `country.iso_code` (fallback `registered_country.iso_code`) с валидацией `isIsoCountry`, ASN = `autonomous_system_number`, holder = `autonomous_system_organization`; приватные IP → negative-запись как в RIPE (`isNonPublicIp`); в PG не пишем (lookup и так быстрый). IPv6 поддержан ридером.
|
||||
- `resolveFlowIp(ip)` — фасад: `lookupGeoip(ip) ?? lookupRipeCached(ip)`; главный экспорт для потребителей.
|
||||
- `geoipStatus()` — loaded, даты сборки баз (метаданные mmdb). Тест-хук `setGeoipReadersForTests`. Смена ридеров после обновления — атомарная замена ссылок.
|
||||
|
||||
### 3. Коллектор `backend/src/services/geoip-update-collector.ts`
|
||||
`collectGeoipUpdateOnce()` по образцу `certificate-renew-collector.ts`:
|
||||
1. Conditional GET с ETag/If-None-Match из настроек → 304 = skip (фолбэк-сравнение: размер/содержимое).
|
||||
2. Скачивание в `*.tmp` через глобальный `fetch` + AbortController с таймаутом (внешний HTTP из service-слоя — по правилу fastify-backend-drizzle).
|
||||
3. Валидация: открыть ридер из tmp-файла, пробой 8.8.8.8 (страна US, ASN 15169).
|
||||
4. `fs.rename` атомарная подмена, старый файл → `*.prev` (откат, если новый ридер не открылся).
|
||||
5. Перезагрузка ридеров, статус в настройках; snapshot для `scheduler_runs` (checked/downloaded/skipped/bytes/error).
|
||||
|
||||
### 4. Планировщик (`backend/src/services/scheduler.ts`)
|
||||
- `JOB_KEYS` += `geoip_update`; case в `runSchedulerJobBody`; блок в `refreshScheduler()` по образцу `certificates_renew`: интервал `Math.max(6ч, updateIntervalSec*1000)`, по умолчанию 7 дней (upstream обновляется еженедельно) + немедленный первый запуск при включённой настройке.
|
||||
|
||||
### 5. Схема и миграция
|
||||
- `backend/src/db/schema.ts`: singleton `geoip_settings` — `enabled` (default true), `updateIntervalSec` (default 604800), `lastCheckAt`, `lastSuccessAt`, `lastError`, `countryBuildAt`, `asnBuildAt`, `etagsJson` (jsonb), `createdAt/updatedAt`.
|
||||
- Миграция: `npm run db:generate` → файл в `backend/drizzle/`.
|
||||
|
||||
### 6. API + контракты
|
||||
- `packages/contracts/src/geoip.ts`: zod-схемы настроек/статуса (все входы — Zod, по правилам проекта).
|
||||
- Новый `backend/src/routes/geoip.ts`, регистрация в `index.ts` с prefix `/api`:
|
||||
- `GET /api/geoip` — настройки + статус (ready, даты сборки, последняя проверка/ошибка);
|
||||
- `PUT /api/geoip` — сохранить настройки, затем `refreshScheduler()`;
|
||||
- `POST /api/geoip/update` — запустить загрузку сейчас (409, если уже идёт; флаг-гард как в коллекторах).
|
||||
|
||||
### 7. Интеграция в пайплайн (geoip-first, RIPE-fallback)
|
||||
- `traffic-flow-engine.ts` (`queueParsedFlows`, ~строка 336): `lookupRipeCached` → `resolveFlowIp`. Логика misses не меняется: при готовом mmdb публичные IP (v4+v6) резолвятся сразу, очередь RIPE пустеет; до скачивания баз — прежнее поведение.
|
||||
- Остальные вызовы `lookupRipeCached` → `resolveFlowIp` (grep: как минимум `traffic-flow-analytics.ts` ~258–271).
|
||||
- `classifyFlowDst`/бренды не трогаем: holder из mmdb (org name) встаёт в существующие `HOLDER_BRANDS`-регулярки как есть.
|
||||
|
||||
### 8. Frontend (по next-shadcn-production / ui-guardian: только переиспользование)
|
||||
- Секция «GeoIP-базы (GeoLite2)» внутри существующей `components/traffic/netflow-settings-panel.tsx`: статус (готово/не скачано, даты сборки Country/ASN, последняя проверка, ошибка), тумблер автообновления, интервал, кнопка «Обновить сейчас» с индикатором. Только уже используемые в панели примитивы (Switch/Button/поля) — никаких новых визуальных паттернов и Card-shell. API-клиент через существующие http-хелперы.
|
||||
|
||||
### 9. Хаускипинг, тесты, проверки
|
||||
- `backend/.gitignore`: `storage/geoip/`.
|
||||
- Тесты `backend/src/services/traffic-flow-geoip.test.ts` + скрипт `test:geoip` (по образцу `test:traffic-flow`): приоритет фасада (geoip hit → RIPE не зовётся; miss → fallback), negative на приватных IP, фильтрация EU/ZZ через `isIsoCountry`, коллектор с мокнутым fetch (304-skip, битый файл → подмены нет, `.prev` сохранён), dims по стране/ASN с засеянным ридером.
|
||||
- Проверки после реализации (обязательно по правилам): типы/сборка бэка (`npm run build -w mikrotik-manager-backend`), типы фронта при правке UI (`npx tsc --noEmit`), `npm run test:geoip` и `test:traffic-flow`; предупреждения не игнорировать.
|
||||
- Коммит: `feat(netflow): <subject по-русски>` — новая пользовательская фича (мгновенные страна/ASN в потоках), по commit-messages-ru.
|
||||
- README: короткий раздел о GeoIP; примечание, что в Docker `storage/geoip` ephemeral без тома — базы перекачаются после пересоздания контейнера (~17 МБ); при желании смонтировать volume.
|
||||
|
||||
## Что это даёт
|
||||
Страна и ASN появляются у потока мгновенно при ingest (включая IPv6), без ограничения скорости RIPE; dims `country`/`asn` в `flow_daily_dims`, аналитика (карта, топы, monthly) становятся полными сразу. Внешняя зависимость от stat.ripe.net остаётся только как fallback до первой загрузки баз.
|
||||
+130
-42
@@ -1,6 +1,6 @@
|
||||
"use client"
|
||||
|
||||
import { Fragment, useState, useMemo, useEffect } from "react"
|
||||
import { useState, useMemo, useEffect } from "react"
|
||||
import { PageHeader } from "@/components/page-header"
|
||||
import { DataPageToolbar } from "@/components/data-page-toolbar"
|
||||
import { BgpSessionsDataGrid } from "@/components/data-grids/bgp-sessions-data-grid"
|
||||
@@ -25,6 +25,9 @@ import {
|
||||
} from "lucide-react"
|
||||
import { useDataSource } from "@/lib/data-source"
|
||||
import { requestJson } from "@/shared/api/http-client"
|
||||
import { servers as mockServers, type Server } from "@/lib/data"
|
||||
import { ServerRailLayout, ServerRailMobileButton } from "@/components/server-rail-layout"
|
||||
import { ALL_SERVERS_ID, type ServerTileItem } from "@/components/server-tile-rail"
|
||||
|
||||
// ─── types ────────────────────────────────────────────────────────────────────
|
||||
|
||||
@@ -233,6 +236,37 @@ interface BackendBgpSession {
|
||||
capabilities: string[]; lastError: string | null
|
||||
}
|
||||
|
||||
interface BackendServer {
|
||||
id: number
|
||||
name: string
|
||||
host: string
|
||||
type?: Server["type"]
|
||||
site?: string
|
||||
country: string
|
||||
asn?: string
|
||||
enabled: boolean
|
||||
status?: Server["status"]
|
||||
latency?: number | null
|
||||
}
|
||||
|
||||
function mapBackendServer(s: BackendServer): Server {
|
||||
return {
|
||||
id: String(s.id),
|
||||
name: s.name || s.host,
|
||||
host: s.host,
|
||||
model: "—",
|
||||
os: "—",
|
||||
site: s.site ?? "",
|
||||
country: s.country || "UN",
|
||||
asn: s.asn ?? "",
|
||||
type: s.type ?? "exit-node",
|
||||
enabled: s.enabled,
|
||||
status: s.status ?? "online",
|
||||
latency: s.latency ?? null,
|
||||
sessions: 0,
|
||||
}
|
||||
}
|
||||
|
||||
function backendToFrontend(b: BackendBgpSession): BgpSession {
|
||||
return {
|
||||
id: `${b.serverId}-${b.id}`,
|
||||
@@ -623,27 +657,40 @@ const TABS: Array<{ id: BgpTab; label: string; icon: React.ReactNode }> = [
|
||||
|
||||
export default function BgpPage() {
|
||||
const [activeTab, setActiveTab] = useState<BgpTab>("sessions")
|
||||
const [selectedServerId, setSelectedServerId] = useState(ALL_SERVERS_ID)
|
||||
|
||||
const { mode, backendUrl } = useDataSource()
|
||||
const isLive = mode === "live"
|
||||
|
||||
const [liveSessions, setLiveSessions] = useState<BgpSession[]>([])
|
||||
const [liveServers, setLiveServers] = useState<Server[]>([])
|
||||
const [loading, setLoading] = useState(false)
|
||||
const [fetchedAt, setFetchedAt] = useState<Date | null>(null)
|
||||
const [liveError, setLiveError] = useState<string | null>(null)
|
||||
const [fetchTick, setFetchTick] = useState(0)
|
||||
|
||||
useEffect(() => {
|
||||
if (!isLive) return
|
||||
if (!isLive) {
|
||||
queueMicrotask(() => {
|
||||
setLiveSessions([])
|
||||
setLiveServers([])
|
||||
setLiveError(null)
|
||||
})
|
||||
return
|
||||
}
|
||||
let cancelled = false
|
||||
queueMicrotask(() => {
|
||||
if (cancelled) return
|
||||
setLoading(true)
|
||||
setLiveError(null)
|
||||
void requestJson<BackendBgpSession[]>(backendUrl, "/api/bgp/sessions")
|
||||
.then(data => {
|
||||
void Promise.all([
|
||||
requestJson<BackendBgpSession[]>(backendUrl, "/api/bgp/sessions"),
|
||||
requestJson<BackendServer[]>(backendUrl, "/api/servers"),
|
||||
])
|
||||
.then(([data, servers]) => {
|
||||
if (cancelled) return
|
||||
setLiveSessions(data.map(backendToFrontend))
|
||||
setLiveServers(servers.filter((s) => s.enabled).map(mapBackendServer))
|
||||
setFetchedAt(new Date())
|
||||
setLoading(false)
|
||||
})
|
||||
@@ -656,50 +703,87 @@ export default function BgpPage() {
|
||||
return () => { cancelled = true }
|
||||
}, [isLive, backendUrl, fetchTick])
|
||||
|
||||
// Use live or mock data for all tabs and KPI
|
||||
const sessions = isLive ? liveSessions : SESSIONS
|
||||
const allSessions = isLive ? liveSessions : SESSIONS
|
||||
const displayServers = isLive ? liveServers : mockServers.filter((s) => s.enabled)
|
||||
|
||||
const effectiveServerId =
|
||||
selectedServerId === ALL_SERVERS_ID || displayServers.some((s) => s.id === selectedServerId)
|
||||
? selectedServerId
|
||||
: ALL_SERVERS_ID
|
||||
|
||||
const sessions = useMemo(() => {
|
||||
if (effectiveServerId === ALL_SERVERS_ID) return allSessions
|
||||
return allSessions.filter((s) => s.serverId === effectiveServerId)
|
||||
}, [allSessions, effectiveServerId])
|
||||
|
||||
const railItems = useMemo<ServerTileItem[]>(() => {
|
||||
const counts = new Map<string, number>()
|
||||
for (const s of allSessions) {
|
||||
counts.set(s.serverId, (counts.get(s.serverId) ?? 0) + 1)
|
||||
}
|
||||
return displayServers.map((s) => ({
|
||||
id: s.id,
|
||||
name: s.name,
|
||||
host: s.host,
|
||||
site: s.site,
|
||||
country: s.country,
|
||||
status: s.status,
|
||||
type: s.type,
|
||||
count: counts.get(s.id) ?? 0,
|
||||
enabled: s.enabled,
|
||||
title: [s.name, s.host, s.asn].filter(Boolean).join(" · "),
|
||||
}))
|
||||
}, [displayServers, allSessions])
|
||||
|
||||
const established = sessions.filter(s => s.state === "Established").length
|
||||
const notEstab = sessions.length - established
|
||||
const totalRx = sessions.reduce((a, s) => a + s.prefixesRx, 0)
|
||||
const serverCount = useMemo(
|
||||
() => new Set(liveSessions.map(s => s.serverId)).size,
|
||||
[liveSessions],
|
||||
() => new Set(sessions.map(s => s.serverId)).size,
|
||||
[sessions],
|
||||
)
|
||||
|
||||
return (
|
||||
<div className="flex flex-col h-full">
|
||||
<PageHeader
|
||||
crumbs={[{ label: "Управление" }, { label: "BGP" }]}
|
||||
actions={
|
||||
<>
|
||||
<Button variant="outline" size="sm" onClick={() => setFetchTick(t => t + 1)}>
|
||||
<RefreshCwIcon className={cn("size-4", loading && "animate-spin")} />
|
||||
Обновить
|
||||
</Button>
|
||||
<Button variant="outline" size="sm"><DownloadIcon className="size-4" />Экспорт</Button>
|
||||
</>
|
||||
}
|
||||
/>
|
||||
|
||||
{/* tab bar */}
|
||||
<div className="border-b bg-background shrink-0">
|
||||
<div className="flex items-center px-6">
|
||||
{TABS.map(t => (
|
||||
<button key={t.id} onClick={() => setActiveTab(t.id)}
|
||||
className={cn(
|
||||
"flex items-center gap-2 px-4 py-3 text-sm font-medium border-b-2 transition-colors -mb-px",
|
||||
activeTab === t.id
|
||||
? "border-primary text-foreground"
|
||||
: "border-transparent text-muted-foreground hover:text-foreground hover:border-border",
|
||||
)}>
|
||||
{t.icon}{t.label}
|
||||
</button>
|
||||
))}
|
||||
<ServerRailLayout
|
||||
items={railItems}
|
||||
selectedId={effectiveServerId}
|
||||
onSelect={setSelectedServerId}
|
||||
showAll
|
||||
allCount={displayServers.length}
|
||||
loading={isLive && loading && displayServers.length === 0}
|
||||
header={
|
||||
<PageHeader
|
||||
crumbs={[{ label: "Управление" }, { label: "BGP" }]}
|
||||
actions={
|
||||
<>
|
||||
<ServerRailMobileButton />
|
||||
<Button variant="outline" size="sm" onClick={() => setFetchTick(t => t + 1)}>
|
||||
<RefreshCwIcon className={cn("size-4", loading && "animate-spin")} />
|
||||
Обновить
|
||||
</Button>
|
||||
<Button variant="outline" size="sm"><DownloadIcon className="size-4" />Экспорт</Button>
|
||||
</>
|
||||
}
|
||||
/>
|
||||
}
|
||||
banner={
|
||||
<div className="border-b bg-background shrink-0">
|
||||
<div className="flex items-center px-6">
|
||||
{TABS.map(t => (
|
||||
<button key={t.id} onClick={() => setActiveTab(t.id)}
|
||||
className={cn(
|
||||
"flex items-center gap-2 px-4 py-3 text-sm font-medium border-b-2 transition-colors -mb-px",
|
||||
activeTab === t.id
|
||||
? "border-primary text-foreground"
|
||||
: "border-transparent text-muted-foreground hover:text-foreground hover:border-border",
|
||||
)}>
|
||||
{t.icon}{t.label}
|
||||
</button>
|
||||
))}
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<div className="flex-1 overflow-y-auto p-6">
|
||||
}
|
||||
>
|
||||
<div className="flex flex-col gap-5">
|
||||
|
||||
{/* data source banner */}
|
||||
@@ -729,11 +813,16 @@ export default function BgpPage() {
|
||||
<AlertDescription className="text-xs">Ошибка загрузки: {liveError}</AlertDescription>
|
||||
</Alert>
|
||||
)}
|
||||
{isLive && !loading && liveSessions.length === 0 && !liveError && fetchedAt && (
|
||||
{isLive && !loading && allSessions.length === 0 && !liveError && fetchedAt && (
|
||||
<div className="rounded-md border border-border bg-muted/30 px-4 py-6 text-center text-sm text-muted-foreground">
|
||||
BGP не настроен ни на одном сервере
|
||||
</div>
|
||||
)}
|
||||
{isLive && !loading && allSessions.length > 0 && sessions.length === 0 && !liveError && (
|
||||
<div className="rounded-md border border-border bg-muted/30 px-4 py-6 text-center text-sm text-muted-foreground">
|
||||
На выбранном сервере нет BGP-сессий
|
||||
</div>
|
||||
)}
|
||||
{mode === "mock" && (
|
||||
<span className="inline-flex w-fit items-center gap-1.5 rounded-full border border-border bg-muted/40 px-2.5 py-0.5 text-[11px] font-medium text-muted-foreground">
|
||||
Моковые данные
|
||||
@@ -794,7 +883,6 @@ export default function BgpPage() {
|
||||
{activeTab === "analytics" && <AnalyticsTab sessions={sessions} />}
|
||||
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</ServerRailLayout>
|
||||
)
|
||||
}
|
||||
|
||||
+253
-70
@@ -1,15 +1,17 @@
|
||||
"use client"
|
||||
|
||||
import { useMemo, useState } from "react"
|
||||
import { useCallback, useEffect, useMemo, useState } from "react"
|
||||
import { PageHeader } from "@/components/page-header"
|
||||
import { routerContainers, servers } from "@/lib/data"
|
||||
import type { RouterContainer } from "@/lib/data"
|
||||
import { routerContainers as mockContainers, servers as mockServers } from "@/lib/data"
|
||||
import type { RouterContainer, Server } from "@/lib/data"
|
||||
import { Flag } from "@/components/flag"
|
||||
import { Frame, FramePanel } from "@/components/reui/frame"
|
||||
import { KpiStatGrid } from "@/components/reui-kit/kpi-stat-grid"
|
||||
import { OpsPanel } from "@/components/ops-panel"
|
||||
import { Button } from "@/components/ui/button"
|
||||
import { Alert, AlertDescription } from "@/components/ui/alert"
|
||||
import { cn } from "@/lib/utils"
|
||||
import { toast } from "sonner"
|
||||
import {
|
||||
DropdownMenu, DropdownMenuTrigger, DropdownMenuContent,
|
||||
DropdownMenuItem, DropdownMenuSeparator,
|
||||
@@ -18,18 +20,47 @@ import {
|
||||
BoxIcon, PlayIcon, StopCircleIcon, SearchIcon,
|
||||
MoreHorizontalIcon, Trash2Icon, PencilIcon, PowerIcon,
|
||||
CodeXmlIcon, ActivityIcon, ServerIcon,
|
||||
TerminalIcon, AlertCircleIcon,
|
||||
TerminalIcon, AlertCircleIcon, RefreshCwIcon,
|
||||
} from "lucide-react"
|
||||
import {
|
||||
Sheet, SheetContent, SheetHeader, SheetTitle,
|
||||
SheetDescription, SheetFooter, SheetClose,
|
||||
} from "@/components/ui/sheet"
|
||||
import { CodeExportSheet } from "@/components/reui-kit/code-export-sheet"
|
||||
import { useDataSource } from "@/lib/data-source"
|
||||
import { requestJson } from "@/shared/api/http-client"
|
||||
import { ServerRailLayout, ServerRailMobileButton } from "@/components/server-rail-layout"
|
||||
import { ALL_SERVERS_ID, type ServerTileItem } from "@/components/server-tile-rail"
|
||||
|
||||
// ─── helpers ──────────────────────────────────────────────────────────────────
|
||||
interface BackendServer {
|
||||
id: number
|
||||
name: string
|
||||
host: string
|
||||
type?: Server["type"]
|
||||
site?: string
|
||||
country: string
|
||||
asn?: string
|
||||
enabled: boolean
|
||||
status?: Server["status"]
|
||||
latency?: number | null
|
||||
}
|
||||
|
||||
function serverFor(id: string) {
|
||||
return servers.find((s) => s.id === id)
|
||||
interface ContainersApiResponse {
|
||||
containers: RouterContainer[]
|
||||
}
|
||||
|
||||
function mapBackendServer(s: BackendServer): Server {
|
||||
return {
|
||||
id: String(s.id),
|
||||
name: s.name || s.host,
|
||||
host: s.host,
|
||||
model: "—",
|
||||
os: "—",
|
||||
site: s.site ?? "",
|
||||
country: s.country || "UN",
|
||||
asn: s.asn ?? "",
|
||||
type: s.type ?? "exit-node",
|
||||
enabled: s.enabled,
|
||||
status: s.status ?? "online",
|
||||
latency: s.latency ?? null,
|
||||
sessions: 0,
|
||||
}
|
||||
}
|
||||
|
||||
function statusConfig(status: RouterContainer["status"]) {
|
||||
@@ -52,10 +83,8 @@ function statusConfig(status: RouterContainer["status"]) {
|
||||
}[status]
|
||||
}
|
||||
|
||||
// ─── RSC generator ────────────────────────────────────────────────────────────
|
||||
|
||||
function generateContainerRsc(c: RouterContainer): string {
|
||||
const srv = serverFor(c.serverId)
|
||||
function generateContainerRsc(c: RouterContainer, serverById: Record<string, Server>): string {
|
||||
const srv = serverById[c.serverId]
|
||||
const lines: string[] = []
|
||||
lines.push(`# RouterOS Container — ${c.name}`)
|
||||
if (srv) lines.push(`# Сервер: ${srv.name} (${srv.host})`)
|
||||
@@ -63,13 +92,11 @@ function generateContainerRsc(c: RouterContainer): string {
|
||||
lines.push(`# RouterOS 7.4+ · /container`)
|
||||
lines.push(``)
|
||||
|
||||
// interface
|
||||
for (const iface of c.interfaces) {
|
||||
lines.push(`/interface/veth/add name=${iface} address=172.17.0.2/24 gateway=172.17.0.1`)
|
||||
}
|
||||
lines.push(``)
|
||||
|
||||
// envs
|
||||
if (c.envs.length > 0) {
|
||||
lines.push(`/container/envs/add name=${c.name}-envs \\`)
|
||||
for (const { key, value } of c.envs) {
|
||||
@@ -78,7 +105,6 @@ function generateContainerRsc(c: RouterContainer): string {
|
||||
lines.push(``)
|
||||
}
|
||||
|
||||
// mounts
|
||||
for (const m of c.mounts) {
|
||||
lines.push(`/container/mounts/add name=${c.name}-mount-${m.dst.replace(/\//g, "-").slice(1)} \\`)
|
||||
if (m.src) lines.push(` src=${m.src} \\`)
|
||||
@@ -86,7 +112,6 @@ function generateContainerRsc(c: RouterContainer): string {
|
||||
lines.push(``)
|
||||
}
|
||||
|
||||
// container
|
||||
lines.push(`/container/add \\`)
|
||||
lines.push(` remote-image=${c.image}:${c.tag} \\`)
|
||||
lines.push(` interface=${c.interfaces[0] ?? "veth-container"} \\`)
|
||||
@@ -100,12 +125,18 @@ function generateContainerRsc(c: RouterContainer): string {
|
||||
return lines.join("\n")
|
||||
}
|
||||
|
||||
// ─── Export Sheet ─────────────────────────────────────────────────────────────
|
||||
|
||||
function ExportSheet({ open, container, onClose }: {
|
||||
open: boolean; container: RouterContainer | null; onClose: () => void
|
||||
function ExportSheet({
|
||||
open, container, onClose, serverById,
|
||||
}: {
|
||||
open: boolean
|
||||
container: RouterContainer | null
|
||||
onClose: () => void
|
||||
serverById: Record<string, Server>
|
||||
}) {
|
||||
const code = useMemo(() => container ? generateContainerRsc(container) : "", [container])
|
||||
const code = useMemo(
|
||||
() => (container ? generateContainerRsc(container, serverById) : ""),
|
||||
[container, serverById],
|
||||
)
|
||||
|
||||
return (
|
||||
<CodeExportSheet
|
||||
@@ -125,17 +156,29 @@ function ExportSheet({ open, container, onClose }: {
|
||||
)
|
||||
}
|
||||
|
||||
// ─── Container card ───────────────────────────────────────────────────────────
|
||||
|
||||
function ContainerCard({
|
||||
container,
|
||||
server,
|
||||
live,
|
||||
busy,
|
||||
onExport,
|
||||
onStart,
|
||||
onStop,
|
||||
onRestart,
|
||||
onRemove,
|
||||
}: {
|
||||
container: RouterContainer
|
||||
server?: Server
|
||||
live: boolean
|
||||
busy: boolean
|
||||
onExport: () => void
|
||||
onStart: () => void
|
||||
onStop: () => void
|
||||
onRestart: () => void
|
||||
onRemove: () => void
|
||||
}) {
|
||||
const srv = serverFor(container.serverId)
|
||||
const cfg = statusConfig(container.status)
|
||||
const canMutate = live && Boolean(container.rosId)
|
||||
|
||||
return (
|
||||
<Frame dense className="w-full overflow-hidden">
|
||||
@@ -150,29 +193,36 @@ function ContainerCard({
|
||||
</div>
|
||||
<DropdownMenu>
|
||||
<DropdownMenuTrigger render={
|
||||
<Button variant="ghost" size="icon" className="size-7 shrink-0">
|
||||
<Button variant="ghost" size="icon" className="size-7 shrink-0" disabled={busy}>
|
||||
<MoreHorizontalIcon className="size-4" />
|
||||
</Button>
|
||||
} />
|
||||
<DropdownMenuContent side="bottom" align="end">
|
||||
{container.status === "running" ? (
|
||||
<DropdownMenuItem><StopCircleIcon className="size-4 text-amber-500" />Остановить</DropdownMenuItem>
|
||||
<DropdownMenuItem disabled={!canMutate} onClick={onStop}>
|
||||
<StopCircleIcon className="size-4 text-amber-500" />Остановить
|
||||
</DropdownMenuItem>
|
||||
) : (
|
||||
<DropdownMenuItem><PlayIcon className="size-4 text-emerald-500" />Запустить</DropdownMenuItem>
|
||||
<DropdownMenuItem disabled={!canMutate} onClick={onStart}>
|
||||
<PlayIcon className="size-4 text-emerald-500" />Запустить
|
||||
</DropdownMenuItem>
|
||||
)}
|
||||
<DropdownMenuItem><TerminalIcon className="size-4" />Логи</DropdownMenuItem>
|
||||
<DropdownMenuItem><PencilIcon className="size-4" />Редактировать</DropdownMenuItem>
|
||||
<DropdownMenuItem disabled><TerminalIcon className="size-4" />Логи</DropdownMenuItem>
|
||||
<DropdownMenuItem disabled><PencilIcon className="size-4" />Редактировать</DropdownMenuItem>
|
||||
<DropdownMenuItem onClick={onExport}><CodeXmlIcon className="size-4" />Экспорт .rsc</DropdownMenuItem>
|
||||
<DropdownMenuSeparator />
|
||||
<DropdownMenuItem><PowerIcon className="size-4" />Перезапустить</DropdownMenuItem>
|
||||
<DropdownMenuItem disabled={!canMutate} onClick={onRestart}>
|
||||
<PowerIcon className="size-4" />Перезапустить
|
||||
</DropdownMenuItem>
|
||||
<DropdownMenuSeparator />
|
||||
<DropdownMenuItem variant="destructive"><Trash2Icon className="size-4" />Удалить</DropdownMenuItem>
|
||||
<DropdownMenuItem variant="destructive" disabled={!canMutate} onClick={onRemove}>
|
||||
<Trash2Icon className="size-4" />Удалить
|
||||
</DropdownMenuItem>
|
||||
</DropdownMenuContent>
|
||||
</DropdownMenu>
|
||||
</div>
|
||||
|
||||
<div className="px-4 py-3 flex flex-col gap-3">
|
||||
{/* image */}
|
||||
<div className="flex items-center gap-2">
|
||||
<BoxIcon className="size-3.5 text-muted-foreground shrink-0" />
|
||||
<span className="font-mono text-xs text-foreground/80">
|
||||
@@ -180,17 +230,15 @@ function ContainerCard({
|
||||
</span>
|
||||
</div>
|
||||
|
||||
{/* server */}
|
||||
{srv && (
|
||||
{server && (
|
||||
<div className="flex items-center gap-2 text-xs text-muted-foreground">
|
||||
<ServerIcon className="size-3.5 shrink-0" />
|
||||
<Flag code={srv.country} size={12} />
|
||||
<span className="font-mono">{srv.name}</span>
|
||||
<Flag code={server.country} size={12} />
|
||||
<span className="font-mono">{server.name}</span>
|
||||
</div>
|
||||
)}
|
||||
|
||||
{/* uptime + stats */}
|
||||
{container.status === "running" && (
|
||||
{container.status === "running" && (container.uptime || container.cpu !== undefined || container.memMb !== undefined) && (
|
||||
<div className="flex items-center gap-4 text-xs text-muted-foreground border-t pt-2.5">
|
||||
{container.uptime && (
|
||||
<div className="flex items-center gap-1">
|
||||
@@ -216,7 +264,6 @@ function ContainerCard({
|
||||
</div>
|
||||
)}
|
||||
|
||||
{/* interfaces */}
|
||||
{container.interfaces.length > 0 && (
|
||||
<div className="flex flex-wrap gap-1">
|
||||
{container.interfaces.map((i) => (
|
||||
@@ -227,7 +274,6 @@ function ContainerCard({
|
||||
</div>
|
||||
)}
|
||||
|
||||
{/* mounts */}
|
||||
{container.mounts.length > 0 && (
|
||||
<div className="flex flex-col gap-1">
|
||||
{container.mounts.map((m, idx) => (
|
||||
@@ -249,50 +295,183 @@ function ContainerCard({
|
||||
)
|
||||
}
|
||||
|
||||
// ════════════════════════════════════════════════════════════════════════════
|
||||
export default function ContainersPage() {
|
||||
const [search, setSearch] = useState("")
|
||||
const [statusFilter, setStatusFilter] = useState<RouterContainer["status"] | "all">("all")
|
||||
const { mode, backendUrl } = useDataSource()
|
||||
const isLive = mode === "live"
|
||||
|
||||
const [search, setSearch] = useState("")
|
||||
const [statusFilter, setStatusFilter] = useState<RouterContainer["status"] | "all">("all")
|
||||
const [exportContainer, setExportContainer] = useState<RouterContainer | null>(null)
|
||||
const [selectedServerId, setSelectedServerId] = useState(ALL_SERVERS_ID)
|
||||
|
||||
const [liveContainers, setLiveContainers] = useState<RouterContainer[]>([])
|
||||
const [liveServers, setLiveServers] = useState<Server[]>([])
|
||||
const [loading, setLoading] = useState(false)
|
||||
const [busyId, setBusyId] = useState<string | null>(null)
|
||||
const [liveError, setLiveError] = useState<string | null>(null)
|
||||
|
||||
const loadLive = useCallback(async () => {
|
||||
if (!isLive) return
|
||||
setLoading(true)
|
||||
setLiveError(null)
|
||||
try {
|
||||
const [cRes, sRes] = await Promise.all([
|
||||
requestJson<ContainersApiResponse>(backendUrl, "/api/containers"),
|
||||
requestJson<BackendServer[]>(backendUrl, "/api/servers"),
|
||||
])
|
||||
setLiveContainers(cRes.containers ?? [])
|
||||
setLiveServers(sRes.filter((s) => s.enabled).map(mapBackendServer))
|
||||
} catch (e) {
|
||||
setLiveError(e instanceof Error ? e.message : "Ошибка загрузки")
|
||||
setLiveContainers([])
|
||||
} finally {
|
||||
setLoading(false)
|
||||
}
|
||||
}, [isLive, backendUrl])
|
||||
|
||||
useEffect(() => {
|
||||
if (!isLive) {
|
||||
queueMicrotask(() => {
|
||||
setLiveContainers([])
|
||||
setLiveServers([])
|
||||
setLiveError(null)
|
||||
})
|
||||
return
|
||||
}
|
||||
queueMicrotask(() => {
|
||||
void loadLive()
|
||||
})
|
||||
}, [isLive, loadLive])
|
||||
|
||||
const displayContainers = isLive ? liveContainers : mockContainers
|
||||
const displayServers = isLive ? liveServers : mockServers.filter((s) => s.enabled)
|
||||
|
||||
const effectiveServerId =
|
||||
selectedServerId === ALL_SERVERS_ID || displayServers.some((s) => s.id === selectedServerId)
|
||||
? selectedServerId
|
||||
: ALL_SERVERS_ID
|
||||
|
||||
const scoped = useMemo(() => {
|
||||
if (effectiveServerId === ALL_SERVERS_ID) return displayContainers
|
||||
return displayContainers.filter((c) => c.serverId === effectiveServerId)
|
||||
}, [displayContainers, effectiveServerId])
|
||||
|
||||
const serverById = useMemo(
|
||||
() => Object.fromEntries(displayServers.map((s) => [s.id, s])),
|
||||
[displayServers],
|
||||
)
|
||||
|
||||
const railItems = useMemo<ServerTileItem[]>(() => (
|
||||
displayServers.map((s) => ({
|
||||
id: s.id,
|
||||
name: s.name,
|
||||
host: s.host,
|
||||
site: s.site,
|
||||
country: s.country,
|
||||
status: s.status,
|
||||
type: s.type,
|
||||
enabled: s.enabled,
|
||||
meta: String(displayContainers.filter((c) => c.serverId === s.id).length),
|
||||
}))
|
||||
), [displayServers, displayContainers])
|
||||
|
||||
const filtered = useMemo(() => {
|
||||
return routerContainers.filter((c) => {
|
||||
return scoped.filter((c) => {
|
||||
if (statusFilter !== "all" && c.status !== statusFilter) return false
|
||||
if (!search) return true
|
||||
const q = search.toLowerCase()
|
||||
return (
|
||||
c.name.toLowerCase().includes(q) ||
|
||||
c.image.toLowerCase().includes(q) ||
|
||||
(serverFor(c.serverId)?.name.toLowerCase().includes(q) ?? false)
|
||||
(serverById[c.serverId]?.name.toLowerCase().includes(q) ?? false)
|
||||
)
|
||||
})
|
||||
}, [search, statusFilter])
|
||||
}, [search, statusFilter, scoped, serverById])
|
||||
|
||||
const running = routerContainers.filter((c) => c.status === "running").length
|
||||
const stopped = routerContainers.filter((c) => c.status === "stopped").length
|
||||
const errors = routerContainers.filter((c) => c.status === "error").length
|
||||
const running = scoped.filter((c) => c.status === "running").length
|
||||
const stopped = scoped.filter((c) => c.status === "stopped").length
|
||||
const errors = scoped.filter((c) => c.status === "error").length
|
||||
|
||||
async function mutate(c: RouterContainer, action: "start" | "stop" | "restart" | "remove") {
|
||||
if (!isLive || !c.rosId) {
|
||||
toast.info("Действие доступно только в live-режиме")
|
||||
return
|
||||
}
|
||||
if (action === "remove" && !window.confirm(`Удалить контейнер ${c.name}?`)) return
|
||||
setBusyId(c.id)
|
||||
try {
|
||||
await requestJson(backendUrl, `/api/servers/${c.serverId}/containers/${action}`, {
|
||||
method: "POST",
|
||||
body: JSON.stringify({ rosId: c.rosId }),
|
||||
})
|
||||
const labels = { start: "запущен", stop: "остановлен", restart: "перезапущен", remove: "удалён" }
|
||||
toast.success(`${c.name}: ${labels[action]}`)
|
||||
await loadLive()
|
||||
} catch (e) {
|
||||
toast.error(e instanceof Error ? e.message : "Ошибка RouterOS")
|
||||
} finally {
|
||||
setBusyId(null)
|
||||
}
|
||||
}
|
||||
|
||||
return (
|
||||
<div className="flex flex-col h-full">
|
||||
<PageHeader
|
||||
crumbs={[{ label: "Управление" }, { label: "Контейнеры" }]}
|
||||
actions={
|
||||
<Button size="sm">
|
||||
<BoxIcon className="size-4" />Новый контейнер
|
||||
</Button>
|
||||
}
|
||||
/>
|
||||
|
||||
<div className="flex-1 overflow-y-auto p-6">
|
||||
<>
|
||||
<ServerRailLayout
|
||||
items={railItems}
|
||||
selectedId={effectiveServerId}
|
||||
onSelect={setSelectedServerId}
|
||||
showAll
|
||||
allCount={displayServers.length}
|
||||
loading={isLive && loading && displayServers.length === 0}
|
||||
header={
|
||||
<PageHeader
|
||||
crumbs={[{ label: "Управление" }, { label: "Контейнеры" }]}
|
||||
actions={
|
||||
<>
|
||||
<ServerRailMobileButton />
|
||||
<Button
|
||||
variant="outline"
|
||||
size="sm"
|
||||
onClick={() => { void loadLive() }}
|
||||
disabled={!isLive || loading}
|
||||
>
|
||||
<RefreshCwIcon className={cn("size-4", loading && "animate-spin")} />
|
||||
Обновить
|
||||
</Button>
|
||||
<Button size="sm">
|
||||
<BoxIcon className="size-4" />Новый контейнер
|
||||
</Button>
|
||||
</>
|
||||
}
|
||||
/>
|
||||
}
|
||||
>
|
||||
<div className="flex flex-col gap-5">
|
||||
|
||||
{isLive && liveError && (
|
||||
<Alert variant="warning" className="py-2">
|
||||
<AlertCircleIcon />
|
||||
<AlertDescription className="text-xs">Ошибка загрузки: {liveError}</AlertDescription>
|
||||
</Alert>
|
||||
)}
|
||||
{isLive && !loading && displayContainers.length === 0 && !liveError && (
|
||||
<div className="rounded-md border border-border bg-muted/30 px-4 py-6 text-center text-sm text-muted-foreground">
|
||||
Контейнеры не найдены. Нужен пакет container (RouterOS 7.4+).
|
||||
</div>
|
||||
)}
|
||||
{mode === "mock" && (
|
||||
<span className="inline-flex w-fit items-center gap-1.5 rounded-full border border-border bg-muted/40 px-2.5 py-0.5 text-[11px] font-medium text-muted-foreground">
|
||||
Моковые данные
|
||||
</span>
|
||||
)}
|
||||
|
||||
<KpiStatGrid
|
||||
aria-label="Сводка контейнеров"
|
||||
items={[
|
||||
{
|
||||
id: "all",
|
||||
label: "Всего",
|
||||
value: routerContainers.length,
|
||||
value: scoped.length,
|
||||
icon: <BoxIcon className="size-4" />,
|
||||
iconClassName: "text-muted-foreground",
|
||||
},
|
||||
@@ -321,7 +500,6 @@ export default function ContainersPage() {
|
||||
]}
|
||||
/>
|
||||
|
||||
{/* Info banner */}
|
||||
<div className="flex items-start gap-3 rounded-lg bg-violet-500/5 border border-violet-500/20 px-4 py-3 text-sm">
|
||||
<BoxIcon className="size-5 text-violet-500 shrink-0 mt-0.5" />
|
||||
<div>
|
||||
@@ -333,7 +511,6 @@ export default function ContainersPage() {
|
||||
</div>
|
||||
</div>
|
||||
|
||||
{/* Toolbar */}
|
||||
<div className="flex items-center gap-3 flex-wrap">
|
||||
<div className="flex items-center gap-2 h-8 px-3 border border-input rounded-md bg-background min-w-[240px]">
|
||||
<SearchIcon className="size-3.5 text-muted-foreground shrink-0" />
|
||||
@@ -363,7 +540,6 @@ export default function ContainersPage() {
|
||||
<span className="text-sm text-muted-foreground ml-auto">{filtered.length} контейнеров</span>
|
||||
</div>
|
||||
|
||||
{/* Grid */}
|
||||
{filtered.length === 0 ? (
|
||||
<div className="flex flex-col items-center justify-center py-16 text-center text-muted-foreground">
|
||||
<BoxIcon className="size-10 mb-3 opacity-20" />
|
||||
@@ -376,13 +552,19 @@ export default function ContainersPage() {
|
||||
<ContainerCard
|
||||
key={c.id}
|
||||
container={c}
|
||||
server={serverById[c.serverId]}
|
||||
live={isLive}
|
||||
busy={busyId === c.id}
|
||||
onExport={() => setExportContainer(c)}
|
||||
onStart={() => { void mutate(c, "start") }}
|
||||
onStop={() => { void mutate(c, "stop") }}
|
||||
onRestart={() => { void mutate(c, "restart") }}
|
||||
onRemove={() => { void mutate(c, "remove") }}
|
||||
/>
|
||||
))}
|
||||
</div>
|
||||
)}
|
||||
|
||||
{/* RouterOS reference */}
|
||||
<OpsPanel title="RouterOS 7.4+ · /container — быстрые команды" contentClassName="px-5 py-4">
|
||||
<div className="grid grid-cols-1 sm:grid-cols-3 gap-4 text-xs font-mono">
|
||||
{[
|
||||
@@ -442,13 +624,14 @@ export default function ContainersPage() {
|
||||
</OpsPanel>
|
||||
|
||||
</div>
|
||||
</div>
|
||||
</ServerRailLayout>
|
||||
|
||||
<ExportSheet
|
||||
open={!!exportContainer}
|
||||
container={exportContainer}
|
||||
onClose={() => setExportContainer(null)}
|
||||
serverById={serverById}
|
||||
/>
|
||||
</div>
|
||||
</>
|
||||
)
|
||||
}
|
||||
|
||||
+172
-138
@@ -27,7 +27,7 @@ import {
|
||||
StarIcon, ArrowUpDownIcon, ArrowUpIcon, ArrowDownIcon,
|
||||
FileCodeIcon, CopyIcon, NetworkIcon, TagIcon,
|
||||
ArrowRightIcon, AlertTriangleIcon, LoaderCircleIcon, RouteIcon,
|
||||
CheckCircle2Icon, XCircleIcon, CircleDashedIcon, RefreshCwIcon,
|
||||
RefreshCwIcon, HistoryIcon,
|
||||
} from "lucide-react"
|
||||
import {
|
||||
Sheet, SheetContent, SheetHeader, SheetTitle, SheetDescription, SheetFooter,
|
||||
@@ -36,13 +36,13 @@ import { Tabs, TabsContent, TabsList, TabsTrigger } from "@/components/ui/tabs"
|
||||
import { Tooltip, TooltipContent, TooltipTrigger } from "@/components/ui/tooltip"
|
||||
import { toast } from "sonner"
|
||||
import { CodeExportSheet } from "@/components/reui-kit/code-export-sheet"
|
||||
import { ConfigHistorySheet } from "@/components/config-history-sheet"
|
||||
import type { ConfigRevisionDto } from "@/lib/config-revisions"
|
||||
import { ServerRailLayout, ServerRailMobileButton } from "@/components/server-rail-layout"
|
||||
import { type ServerTileItem } from "@/components/server-tile-rail"
|
||||
|
||||
// ── helpers ────────────────────────────────────────────────────────────────────
|
||||
|
||||
type FilterRouterSyncStatus = "synced" | "drift" | "missing"
|
||||
|
||||
function newId() { return `r${Date.now()}-${Math.random().toString(36).slice(2, 6)}` }
|
||||
function innerIpToGateway(ip: string) { return ip.split("/")[0] }
|
||||
|
||||
@@ -1239,6 +1239,9 @@ interface BackendServer {
|
||||
interface LiveFiltersResponse {
|
||||
rulesets: ServerFilterRuleset[]
|
||||
greTunnels: GreTunnel[]
|
||||
live?: boolean
|
||||
stale?: boolean
|
||||
error?: string
|
||||
}
|
||||
|
||||
function buildRulesets(serverList: Server[], sourceRulesets: ServerFilterRuleset[]): ServerFilterRuleset[] {
|
||||
@@ -1325,18 +1328,14 @@ export default function FiltersPage() {
|
||||
const [sheetMode, setSheetMode] = useState<"create" | "edit">("create")
|
||||
const [sheetInitial, setSheetInitial]= useState<RuleForm>(emptyForm())
|
||||
const [editingId, setEditingId] = useState<string | null>(null)
|
||||
const [previewOpen, setPreviewOpen] = useState(false)
|
||||
const [copyOpen, setCopyOpen] = useState(false)
|
||||
const [syncBusy, setSyncBusy] = useState<"from" | "to" | null>(null)
|
||||
const [routerCompare, setRouterCompare] = useState<{
|
||||
serverId: string
|
||||
byCommunity: Record<string, FilterRouterSyncStatus>
|
||||
} | null>(null)
|
||||
const [routerCompareLoading, setRouterCompareLoading] = useState(false)
|
||||
|
||||
useEffect(() => {
|
||||
setRouterCompare(null)
|
||||
}, [selectedServerId])
|
||||
const [previewOpen, setPreviewOpen] = useState(false)
|
||||
const [copyOpen, setCopyOpen] = useState(false)
|
||||
const [applyBusy, setApplyBusy] = useState(false)
|
||||
const [liveStale, setLiveStale] = useState(false)
|
||||
const [historyOpen, setHistoryOpen] = useState(false)
|
||||
const [historyLoading, setHistoryLoading] = useState(false)
|
||||
const [historyRestoring, setHistoryRestoring] = useState(false)
|
||||
const [revisions, setRevisions] = useState<ConfigRevisionDto[]>([])
|
||||
|
||||
useEffect(() => {
|
||||
if (!isLive) {
|
||||
@@ -1347,6 +1346,7 @@ export default function FiltersPage() {
|
||||
setRulesets(buildRulesets(servers, serverFilterRulesets))
|
||||
setSelectedServerId(servers[0]?.id ?? "")
|
||||
setLiveLoadState("idle")
|
||||
setLiveStale(false)
|
||||
})
|
||||
return
|
||||
}
|
||||
@@ -1390,18 +1390,43 @@ export default function FiltersPage() {
|
||||
})
|
||||
}, [isLive, apiFetch])
|
||||
|
||||
const loadLiveRules = useCallback(async (serverId: string) => {
|
||||
if (!isLive || !serverId) return
|
||||
try {
|
||||
const fresh = await apiFetch<LiveFiltersResponse>(
|
||||
`/api/filters/rules?serverId=${encodeURIComponent(serverId)}`,
|
||||
)
|
||||
const liveRules = fresh.rulesets.find((r) => r.serverId === serverId)?.rules ?? fresh.rulesets[0]?.rules ?? []
|
||||
setRulesets((prev) => {
|
||||
const has = prev.some((rs) => rs.serverId === serverId)
|
||||
if (!has) return [...prev, { serverId, rules: liveRules }]
|
||||
return prev.map((rs) => rs.serverId === serverId ? { ...rs, rules: liveRules } : rs)
|
||||
})
|
||||
setLiveStale(Boolean(fresh.stale))
|
||||
if (fresh.greTunnels?.length) {
|
||||
setGreByServer((prev) => ({ ...prev, [serverId]: fresh.greTunnels }))
|
||||
}
|
||||
} catch (err) {
|
||||
setLiveStale(true)
|
||||
toast.error("Не удалось прочитать правила с роутера", { description: String(err) })
|
||||
}
|
||||
}, [isLive, apiFetch])
|
||||
|
||||
useEffect(() => {
|
||||
if (!isLive || !selectedServerId) return
|
||||
if (!isLive || !selectedServerId || liveLoadState !== "idle") return
|
||||
if (!liveServers.some((s) => s.id === selectedServerId)) return
|
||||
void Promise.all([
|
||||
loadLiveRules(selectedServerId),
|
||||
ensureGreTunnels(selectedServerId),
|
||||
ensureRecursiveRoutes(selectedServerId),
|
||||
])
|
||||
}, [isLive, selectedServerId, ensureGreTunnels, ensureRecursiveRoutes])
|
||||
}, [isLive, selectedServerId, liveLoadState, liveServers, ensureGreTunnels, ensureRecursiveRoutes, loadLiveRules])
|
||||
|
||||
const allServers = isLive ? liveServers : servers
|
||||
const allTunnels = isLive ? (greByServer[selectedServerId] ?? []) : greTunnels
|
||||
const allServers = !isLive || liveLoadState === "error" ? servers : liveServers
|
||||
const allTunnels = !isLive || liveLoadState === "error" ? greTunnels : (greByServer[selectedServerId] ?? [])
|
||||
const selectedServer = allServers.find(s => s.id === selectedServerId) ?? allServers[0]
|
||||
const totalRules = rulesets.reduce((s, r) => s + r.rules.length, 0)
|
||||
const mutationsLocked = isLive && (applyBusy || liveStale || liveLoadState === "error")
|
||||
|
||||
const filterRailItems = useMemo<ServerTileItem[]>(() => (
|
||||
allServers.map((s) => ({
|
||||
@@ -1427,21 +1452,6 @@ export default function FiltersPage() {
|
||||
[rulesets, selectedServerId],
|
||||
)
|
||||
|
||||
const fetchRouterCompare = useCallback(async () => {
|
||||
if (!isLive || !selectedServerId) return
|
||||
setRouterCompareLoading(true)
|
||||
try {
|
||||
const d = await apiFetch<{ byCommunity: Record<string, FilterRouterSyncStatus> }>(
|
||||
`/api/filters/router-compare?serverId=${encodeURIComponent(selectedServerId)}`,
|
||||
)
|
||||
setRouterCompare({ serverId: selectedServerId, byCommunity: d.byCommunity })
|
||||
} catch {
|
||||
setRouterCompare(null)
|
||||
} finally {
|
||||
setRouterCompareLoading(false)
|
||||
}
|
||||
}, [isLive, selectedServerId, apiFetch])
|
||||
|
||||
const filteredRules = useMemo(() => {
|
||||
const q = search.toLowerCase()
|
||||
if (!q) return currentRules
|
||||
@@ -1453,68 +1463,94 @@ export default function FiltersPage() {
|
||||
)
|
||||
}, [currentRules, search, communityNameMap])
|
||||
|
||||
const updateRules = useCallback((serverId: string, updater: (rules: FilterRule[]) => FilterRule[]) => {
|
||||
setRouterCompare(rc => (rc && rc.serverId === serverId ? null : rc))
|
||||
setRulesets(prev => {
|
||||
const next = prev.map(rs =>
|
||||
rs.serverId === serverId ? { ...rs, rules: updater(rs.rules) } : rs
|
||||
)
|
||||
if (isLive) {
|
||||
void apiFetch<{ ok: boolean }>("/api/filters/rules", {
|
||||
method: "PUT",
|
||||
body: JSON.stringify({ rulesets: next }),
|
||||
}).catch(() => {})
|
||||
}
|
||||
return next
|
||||
})
|
||||
}, [isLive, apiFetch])
|
||||
|
||||
const syncFromRouter = useCallback(async () => {
|
||||
if (!isLive || syncBusy) return
|
||||
setSyncBusy("from")
|
||||
const applyRules = useCallback(async (
|
||||
serverId: string,
|
||||
nextRules: FilterRule[],
|
||||
source: "apply" | "copy" = "apply",
|
||||
) => {
|
||||
const prev = rulesets
|
||||
setRulesets((p) => p.map((rs) => rs.serverId === serverId ? { ...rs, rules: nextRules } : rs))
|
||||
if (!isLive) return
|
||||
if (liveStale) {
|
||||
setRulesets(prev)
|
||||
toast.error("Роутер недоступен — изменения заблокированы")
|
||||
return
|
||||
}
|
||||
setApplyBusy(true)
|
||||
try {
|
||||
await apiFetch<{ ok: boolean }>("/api/filters/sync/from-router", {
|
||||
method: "POST",
|
||||
body: JSON.stringify({ serverId: selectedServerId }),
|
||||
const res = await apiFetch<{ ok: boolean; rules?: FilterRule[] }>("/api/filters/rules", {
|
||||
method: "PUT",
|
||||
body: JSON.stringify({ serverId, rules: nextRules, source }),
|
||||
})
|
||||
const fresh = await apiFetch<LiveFiltersResponse>("/api/filters/rules")
|
||||
setRulesets(buildRulesets(allServers, fresh.rulesets))
|
||||
if (res.rules) {
|
||||
setRulesets((p) => p.map((rs) => rs.serverId === serverId ? { ...rs, rules: res.rules ?? nextRules } : rs))
|
||||
}
|
||||
toast.success("Правила применены на роутер")
|
||||
} catch (err) {
|
||||
setRulesets(prev)
|
||||
toast.error("Не удалось применить правила на роутер", { description: String(err) })
|
||||
} finally {
|
||||
setApplyBusy(false)
|
||||
}
|
||||
}, [isLive, apiFetch, rulesets, liveStale])
|
||||
|
||||
const refreshFromRouter = useCallback(async () => {
|
||||
if (!isLive || !selectedServerId || applyBusy) return
|
||||
setApplyBusy(true)
|
||||
try {
|
||||
await loadLiveRules(selectedServerId)
|
||||
await Promise.all([
|
||||
ensureGreTunnels(selectedServerId),
|
||||
ensureRecursiveRoutes(selectedServerId),
|
||||
])
|
||||
await fetchRouterCompare()
|
||||
} finally {
|
||||
setSyncBusy(null)
|
||||
setApplyBusy(false)
|
||||
}
|
||||
}, [isLive, syncBusy, apiFetch, allServers, selectedServerId, ensureGreTunnels, ensureRecursiveRoutes, fetchRouterCompare])
|
||||
}, [isLive, selectedServerId, applyBusy, loadLiveRules, ensureGreTunnels, ensureRecursiveRoutes])
|
||||
|
||||
const syncToRouter = useCallback(async () => {
|
||||
if (!isLive || syncBusy || !selectedServerId) return
|
||||
setSyncBusy("to")
|
||||
const loadRevisions = useCallback(async () => {
|
||||
if (!isLive || !selectedServerId) return
|
||||
setHistoryLoading(true)
|
||||
try {
|
||||
const res = await apiFetch<{
|
||||
ok: boolean
|
||||
updatedServers: number
|
||||
pushedRules: number
|
||||
errors?: Array<{ serverId: number; error: string }>
|
||||
}>("/api/filters/sync/to-router", {
|
||||
method: "POST",
|
||||
body: JSON.stringify({ serverId: selectedServerId }),
|
||||
})
|
||||
if (res.ok) {
|
||||
toast.success(`Загружено правил на роутер: ${res.pushedRules}`)
|
||||
} else {
|
||||
const detail = res.errors?.[0]?.error ?? "неизвестная ошибка"
|
||||
toast.error("Не удалось загрузить правила на роутер", { description: detail })
|
||||
}
|
||||
await fetchRouterCompare()
|
||||
const res = await apiFetch<{ revisions: ConfigRevisionDto[] }>(
|
||||
`/api/filters/revisions?serverId=${encodeURIComponent(selectedServerId)}`,
|
||||
)
|
||||
setRevisions(res.revisions)
|
||||
} catch (err) {
|
||||
toast.error("Не удалось загрузить правила на роутер", { description: String(err) })
|
||||
toast.error("Не удалось загрузить историю", { description: String(err) })
|
||||
setRevisions([])
|
||||
} finally {
|
||||
setSyncBusy(null)
|
||||
setHistoryLoading(false)
|
||||
}
|
||||
}, [isLive, syncBusy, selectedServerId, apiFetch, fetchRouterCompare])
|
||||
}, [isLive, selectedServerId, apiFetch])
|
||||
|
||||
const restoreRevision = useCallback(async (id: string) => {
|
||||
if (!isLive || !selectedServerId) return
|
||||
setHistoryRestoring(true)
|
||||
try {
|
||||
const res = await apiFetch<{ ok: boolean; rules?: FilterRule[] }>(
|
||||
`/api/filters/revisions/${encodeURIComponent(id)}/restore`,
|
||||
{ method: "POST", body: JSON.stringify({ serverId: selectedServerId }) },
|
||||
)
|
||||
if (res.rules) {
|
||||
setRulesets((p) => p.map((rs) => rs.serverId === selectedServerId ? { ...rs, rules: res.rules ?? [] } : rs))
|
||||
} else {
|
||||
await loadLiveRules(selectedServerId)
|
||||
}
|
||||
setLiveStale(false)
|
||||
toast.success("Версия применена на роутер")
|
||||
await loadRevisions()
|
||||
} catch (err) {
|
||||
toast.error("Не удалось откатить", { description: String(err) })
|
||||
} finally {
|
||||
setHistoryRestoring(false)
|
||||
}
|
||||
}, [isLive, selectedServerId, apiFetch, loadLiveRules, loadRevisions])
|
||||
|
||||
const updateRules = useCallback((serverId: string, updater: (rules: FilterRule[]) => FilterRule[]) => {
|
||||
const current = rulesets.find((rs) => rs.serverId === serverId)?.rules ?? []
|
||||
void applyRules(serverId, updater(current))
|
||||
}, [rulesets, applyRules])
|
||||
|
||||
const openCreate = () => {
|
||||
setSheetInitial(emptyForm()); setSheetMode("create"); setEditingId(null); setSheetOpen(true)
|
||||
@@ -1532,6 +1568,7 @@ export default function FiltersPage() {
|
||||
}
|
||||
|
||||
const handleSave = (form: RuleForm) => {
|
||||
if (mutationsLocked) return
|
||||
const { gatewayKind: _gk, ...payload } = form
|
||||
if (sheetMode === "create") {
|
||||
updateRules(selectedServerId, rules => [
|
||||
@@ -1549,34 +1586,30 @@ export default function FiltersPage() {
|
||||
setSheetOpen(false)
|
||||
}
|
||||
|
||||
const handleDelete = (id: string) => updateRules(selectedServerId, rules => rules.filter(r => r.id !== id))
|
||||
const handleDelete = (id: string) => {
|
||||
if (mutationsLocked) return
|
||||
updateRules(selectedServerId, rules => rules.filter(r => r.id !== id))
|
||||
}
|
||||
|
||||
const handleCopyRules = useCallback((targetServerId: string, rules: FilterRule[], mode: CopyMode) => {
|
||||
updateRules(targetServerId, existing =>
|
||||
mode === "replace" ? rules : [...existing, ...rules]
|
||||
)
|
||||
}, [updateRules])
|
||||
const existing = rulesets.find((rs) => rs.serverId === targetServerId)?.rules ?? []
|
||||
const next = mode === "replace" ? rules : [...existing, ...rules]
|
||||
void applyRules(targetServerId, next, "copy")
|
||||
}, [rulesets, applyRules])
|
||||
const handleMoveUp = (index: number) => {
|
||||
if (index === 0) return
|
||||
if (mutationsLocked || index === 0) return
|
||||
updateRules(selectedServerId, rules => {
|
||||
const n = [...rules]; [n[index - 1], n[index]] = [n[index], n[index - 1]]; return n
|
||||
})
|
||||
}
|
||||
const handleMoveDown = (index: number) => {
|
||||
if (mutationsLocked) return
|
||||
updateRules(selectedServerId, rules => {
|
||||
if (index >= rules.length - 1) return rules
|
||||
const n = [...rules]; [n[index], n[index + 1]] = [n[index + 1], n[index]]; return n
|
||||
})
|
||||
}
|
||||
|
||||
if (!selectedServer) {
|
||||
return (
|
||||
<div className="flex h-full items-center justify-center text-sm text-muted-foreground">
|
||||
Нет доступных серверов
|
||||
</div>
|
||||
)
|
||||
}
|
||||
|
||||
if (isLive && liveLoadState === "loading") {
|
||||
return (
|
||||
<div className="flex h-full flex-col items-center justify-center gap-3 text-sm text-muted-foreground">
|
||||
@@ -1586,6 +1619,14 @@ export default function FiltersPage() {
|
||||
)
|
||||
}
|
||||
|
||||
if (!selectedServer) {
|
||||
return (
|
||||
<div className="flex h-full items-center justify-center text-sm text-muted-foreground">
|
||||
Нет доступных серверов
|
||||
</div>
|
||||
)
|
||||
}
|
||||
|
||||
return (
|
||||
<>
|
||||
<ServerRailLayout
|
||||
@@ -1604,35 +1645,25 @@ export default function FiltersPage() {
|
||||
<Button
|
||||
variant="outline"
|
||||
size="sm"
|
||||
onClick={syncFromRouter}
|
||||
disabled={syncBusy !== null}
|
||||
title="Синхронизация Router → БД"
|
||||
onClick={() => void refreshFromRouter()}
|
||||
disabled={applyBusy}
|
||||
title="Прочитать актуальные правила с роутера"
|
||||
>
|
||||
{syncBusy === "from" ? "Синк Router → DB…" : "Router → DB"}
|
||||
<RefreshCwIcon className={cn("size-4", applyBusy && "animate-spin")} />
|
||||
Обновить
|
||||
</Button>
|
||||
<Button
|
||||
variant="outline"
|
||||
size="sm"
|
||||
onClick={syncToRouter}
|
||||
disabled={syncBusy !== null}
|
||||
title="Синхронизация БД → Router"
|
||||
onClick={() => {
|
||||
setHistoryOpen(true)
|
||||
void loadRevisions()
|
||||
}}
|
||||
disabled={applyBusy}
|
||||
title="История версий и откат на CHR"
|
||||
>
|
||||
{syncBusy === "to" ? "Синк DB → Router…" : "DB → Router"}
|
||||
</Button>
|
||||
<Button
|
||||
variant="outline"
|
||||
size="sm"
|
||||
onClick={() => void fetchRouterCompare()}
|
||||
disabled={syncBusy !== null || routerCompareLoading}
|
||||
title="Сравнить правила в БД с цепочкой bgp-in на MikroTik"
|
||||
className="gap-1.5"
|
||||
>
|
||||
{routerCompareLoading ? (
|
||||
<LoaderCircleIcon className="size-4 animate-spin" />
|
||||
) : (
|
||||
<RefreshCwIcon className="size-4" />
|
||||
)}
|
||||
Сверить
|
||||
<HistoryIcon className="size-4" />
|
||||
История
|
||||
</Button>
|
||||
</>
|
||||
)}
|
||||
@@ -1642,12 +1673,12 @@ export default function FiltersPage() {
|
||||
<Button
|
||||
variant="outline" size="sm"
|
||||
onClick={() => setCopyOpen(true)}
|
||||
disabled={currentRules.length === 0}
|
||||
disabled={currentRules.length === 0 || mutationsLocked}
|
||||
title="Копировать правила на другой сервер"
|
||||
>
|
||||
<CopyIcon className="size-4" />Копировать
|
||||
</Button>
|
||||
<Button size="sm" onClick={openCreate}>
|
||||
<Button size="sm" onClick={openCreate} disabled={mutationsLocked}>
|
||||
<PlusIcon className="size-4" />Новое правило
|
||||
</Button>
|
||||
</>
|
||||
@@ -1662,6 +1693,12 @@ export default function FiltersPage() {
|
||||
Бекенд недоступен — показаны демо-данные из lib/data. Проверьте URL бекенда в настройках.
|
||||
</div>
|
||||
)}
|
||||
{isLive && liveStale && liveLoadState !== "error" && (
|
||||
<div className="shrink-0 border-b border-amber-500/30 bg-amber-500/10 px-6 py-2.5 text-xs text-amber-700 dark:text-amber-400 flex items-center gap-2">
|
||||
<AlertTriangleIcon className="size-3.5 shrink-0" />
|
||||
Роутер недоступен — показан кэш. Изменения заблокированы, пока не удастся прочитать CHR.
|
||||
</div>
|
||||
)}
|
||||
<div className="border-b px-4 py-3 flex items-center gap-3 flex-wrap shrink-0 md:px-6">
|
||||
<div className="relative min-w-[200px] max-w-xs flex-1">
|
||||
<SearchIcon className="absolute left-2.5 top-1/2 -translate-y-1/2 size-3.5 text-muted-foreground pointer-events-none" />
|
||||
@@ -1745,15 +1782,7 @@ export default function FiltersPage() {
|
||||
)}>{selectedServer.latency}мс</span>
|
||||
)}
|
||||
<div className="ml-auto flex items-center gap-2 text-xs text-muted-foreground flex-wrap justify-end">
|
||||
{isLive && routerCompare?.serverId === selectedServerId && currentRules.length > 0 && (
|
||||
<span className="font-mono tabular-nums">
|
||||
роутер:{" "}
|
||||
<span className="text-emerald-600 dark:text-emerald-500">
|
||||
{Object.values(routerCompare.byCommunity).filter(s => s === "synced").length}
|
||||
</span>
|
||||
/{currentRules.length} совпало
|
||||
</span>
|
||||
)}
|
||||
{applyBusy && <span>Применение на роутер…</span>}
|
||||
<span>{currentRules.length} правил</span>
|
||||
</div>
|
||||
</div>
|
||||
@@ -1790,12 +1819,6 @@ export default function FiltersPage() {
|
||||
serversList={allServers}
|
||||
communityNameMap={communityNameMap}
|
||||
recursiveRoutes={recRoutesByServer[selectedServerId] ?? []}
|
||||
routerSyncByCommunity={
|
||||
!isLive || !routerCompare || routerCompare.serverId !== selectedServerId
|
||||
? null
|
||||
: routerCompare.byCommunity
|
||||
}
|
||||
isLive={isLive}
|
||||
enableSorting={!!search}
|
||||
onEdit={openEdit}
|
||||
onDelete={handleDelete}
|
||||
@@ -1805,7 +1828,7 @@ export default function FiltersPage() {
|
||||
)}
|
||||
|
||||
{/* add rule shortcut */}
|
||||
<button onClick={openCreate}
|
||||
<button onClick={openCreate} disabled={mutationsLocked}
|
||||
className="w-full flex items-center gap-2 px-5 py-2 text-xs text-muted-foreground hover:text-foreground hover:bg-muted/20 transition-colors border-t">
|
||||
<PlusIcon className="size-3.5" />
|
||||
Добавить правило для {selectedServer.name}
|
||||
@@ -1854,6 +1877,17 @@ export default function FiltersPage() {
|
||||
recRoutesByServer={recRoutesByServer}
|
||||
ensureRecursiveFor={ensureRecursiveRoutes}
|
||||
/>
|
||||
|
||||
<ConfigHistorySheet
|
||||
open={historyOpen}
|
||||
onOpenChange={setHistoryOpen}
|
||||
title="История фильтров"
|
||||
itemLabel="правил"
|
||||
revisions={revisions}
|
||||
loading={historyLoading}
|
||||
restoring={historyRestoring}
|
||||
onRestore={restoreRevision}
|
||||
/>
|
||||
</>
|
||||
)
|
||||
}
|
||||
|
||||
@@ -49,12 +49,14 @@ import {
|
||||
PowerIcon, CheckCircleIcon,
|
||||
PlayIcon, SquareIcon, RotateCcwIcon, ZapIcon,
|
||||
CheckCircle2Icon, XCircleIcon, MinusCircleIcon, SkipForwardIcon,
|
||||
SlidersHorizontalIcon, RefreshCwIcon,
|
||||
SlidersHorizontalIcon, RefreshCwIcon, HistoryIcon,
|
||||
} from "lucide-react"
|
||||
import { CodeExportSheet } from "@/components/reui-kit/code-export-sheet"
|
||||
import { ServerRailLayout, ServerRailMobileButton } from "@/components/server-rail-layout"
|
||||
import { ALL_SERVERS_ID, type ServerTileItem } from "@/components/server-tile-rail"
|
||||
import { toast } from "sonner"
|
||||
import { ConfigHistorySheet } from "@/components/config-history-sheet"
|
||||
import type { ConfigRevisionDto } from "@/lib/config-revisions"
|
||||
|
||||
// ─── Types ────────────────────────────────────────────────────────────────────
|
||||
|
||||
@@ -1947,6 +1949,10 @@ function FirewallPageInner() {
|
||||
const [exportOpen, setExportOpen] = useState(false)
|
||||
const [editingAddr, setEditingAddr] = useState<Partial<AddressListEntry> | null>(null)
|
||||
const [addrSheetOpen, setAddrSheetOpen] = useState(false)
|
||||
const [historyOpen, setHistoryOpen] = useState(false)
|
||||
const [revisions, setRevisions] = useState<ConfigRevisionDto[]>([])
|
||||
const [historyLoading, setHistoryLoading] = useState(false)
|
||||
const [historyRestoring, setHistoryRestoring] = useState(false)
|
||||
|
||||
const loadLive = useCallback(async () => {
|
||||
if (!isLive) return
|
||||
@@ -1970,6 +1976,42 @@ function FirewallPageInner() {
|
||||
}
|
||||
}, [isLive, apiFetch])
|
||||
|
||||
const historyServerId = selectedServerId === ALL_SERVERS_ID ? null : selectedServerId
|
||||
|
||||
const loadRevisions = useCallback(async () => {
|
||||
if (!isLive || !historyServerId) return
|
||||
setHistoryLoading(true)
|
||||
try {
|
||||
const res = await apiFetch<{ revisions: ConfigRevisionDto[] }>(
|
||||
`/api/firewall/revisions?serverId=${encodeURIComponent(historyServerId)}`,
|
||||
)
|
||||
setRevisions(res.revisions)
|
||||
} catch (err) {
|
||||
toast.error("Не удалось загрузить историю", { description: String(err) })
|
||||
setRevisions([])
|
||||
} finally {
|
||||
setHistoryLoading(false)
|
||||
}
|
||||
}, [isLive, historyServerId, apiFetch])
|
||||
|
||||
const restoreRevision = useCallback(async (id: string) => {
|
||||
if (!isLive || !historyServerId) return
|
||||
setHistoryRestoring(true)
|
||||
try {
|
||||
await apiFetch(
|
||||
`/api/firewall/revisions/${encodeURIComponent(id)}/restore`,
|
||||
{ method: "POST", body: JSON.stringify({ serverId: historyServerId }) },
|
||||
)
|
||||
toast.success("Версия применена на роутер")
|
||||
await loadLive()
|
||||
await loadRevisions()
|
||||
} catch (err) {
|
||||
toast.error("Не удалось откатить", { description: String(err) })
|
||||
} finally {
|
||||
setHistoryRestoring(false)
|
||||
}
|
||||
}, [isLive, historyServerId, apiFetch, loadLive, loadRevisions])
|
||||
|
||||
useEffect(() => {
|
||||
if (!isLive) {
|
||||
queueMicrotask(() => {
|
||||
@@ -2373,6 +2415,19 @@ function FirewallPageInner() {
|
||||
<RefreshCwIcon className={cn("size-4", dataLoading && "animate-spin")} />
|
||||
Обновить
|
||||
</Button>
|
||||
<Button
|
||||
variant="outline"
|
||||
size="sm"
|
||||
onClick={() => {
|
||||
setHistoryOpen(true)
|
||||
void loadRevisions()
|
||||
}}
|
||||
disabled={!isLive || !historyServerId || dataLoading}
|
||||
title={!historyServerId ? "Выберите сервер, чтобы смотреть историю" : "История версий и откат на CHR"}
|
||||
>
|
||||
<HistoryIcon className="size-4" />
|
||||
История
|
||||
</Button>
|
||||
<Button variant="outline" size="sm" onClick={() => setExportOpen(true)}>
|
||||
<CodeXmlIcon className="size-4" />Экспорт .rsc
|
||||
</Button>
|
||||
@@ -2586,6 +2641,17 @@ function FirewallPageInner() {
|
||||
onClose={() => setExportOpen(false)}
|
||||
rules={familyRules}
|
||||
/>
|
||||
|
||||
<ConfigHistorySheet
|
||||
open={historyOpen}
|
||||
onOpenChange={setHistoryOpen}
|
||||
title="История Firewall"
|
||||
itemLabel="объектов"
|
||||
revisions={revisions}
|
||||
loading={historyLoading}
|
||||
restoring={historyRestoring}
|
||||
onRestore={restoreRevision}
|
||||
/>
|
||||
</>
|
||||
)
|
||||
}
|
||||
|
||||
+289
-58
@@ -13,11 +13,23 @@ import { useDataSource } from "@/lib/data-source"
|
||||
import { requestJson } from "@/shared/api/http-client"
|
||||
import { cn } from "@/lib/utils"
|
||||
import { toast } from "sonner"
|
||||
import { Frame, FramePanel } from "@/components/reui/frame"
|
||||
import { KpiStatGrid } from "@/components/reui-kit/kpi-stat-grid"
|
||||
import { OpsPanel } from "@/components/ops-panel"
|
||||
import { Button } from "@/components/ui/button"
|
||||
import { Input } from "@/components/ui/input"
|
||||
import { ConfigHistorySheet } from "@/components/config-history-sheet"
|
||||
import type { ConfigRevisionDto } from "@/lib/config-revisions"
|
||||
import {
|
||||
AlertDialog,
|
||||
AlertDialogAction,
|
||||
AlertDialogCancel,
|
||||
AlertDialogContent,
|
||||
AlertDialogDescription,
|
||||
AlertDialogFooter,
|
||||
AlertDialogHeader,
|
||||
AlertDialogMedia,
|
||||
AlertDialogTitle,
|
||||
} from "@/components/ui/alert-dialog"
|
||||
import {
|
||||
Sheet, SheetContent, SheetHeader, SheetTitle,
|
||||
SheetDescription, SheetFooter, SheetClose,
|
||||
@@ -31,7 +43,7 @@ import {
|
||||
LockIcon, LockOpenIcon, ShieldCheckIcon, NetworkIcon,
|
||||
EyeIcon, EyeOffIcon, ChevronDownIcon, ChevronRightIcon,
|
||||
CodeXmlIcon, PencilIcon, PowerIcon, Trash2Icon,
|
||||
DatabaseIcon,
|
||||
DatabaseIcon, HistoryIcon, TriangleAlertIcon,
|
||||
} from "lucide-react"
|
||||
import { CodeExportSheet } from "@/components/reui-kit/code-export-sheet"
|
||||
import { ServerRailLayout, ServerRailMobileButton } from "@/components/server-rail-layout"
|
||||
@@ -56,6 +68,10 @@ const STATUS_MAP: Record<GreStatus, { label: string; dot: string }> = {
|
||||
down: { label: "Down", dot: "bg-red-500" },
|
||||
}
|
||||
|
||||
function greStatusMeta(status: GreStatus | undefined) {
|
||||
return STATUS_MAP[status ?? "degraded"] ?? STATUS_MAP.degraded
|
||||
}
|
||||
|
||||
// ─── RouterOS code generator ─────────────────────────────────────────────────
|
||||
|
||||
function generateRosCommands(t: GreTunnel, serverById: Record<string, Server>): string {
|
||||
@@ -91,10 +107,10 @@ function generateRosCommands(t: GreTunnel, serverById: Record<string, Server>):
|
||||
lines.push(` address=${t.localInnerIp} \\`)
|
||||
lines.push(` interface=${t.name}`)
|
||||
|
||||
// IPsec manual equivalent
|
||||
if (t.ipsec) {
|
||||
const ikeMode = t.ipsec.ikeVersion === "ikev2" ? "ike2" : "ike1"
|
||||
const pfsGroup = t.ipsec.pfs ? t.ipsec.dhGroup : "none"
|
||||
// IPsec: live CHR имеет только ipsec-secret; proposal — у моков/формы
|
||||
if (t.ipsec?.encAlg && t.ipsec.authAlg) {
|
||||
const ikeMode = t.ipsec.ikeVersion === "ikev1" ? "ike1" : "ike2"
|
||||
const pfsGroup = t.ipsec.pfs ? (t.ipsec.dhGroup ?? "none") : "none"
|
||||
|
||||
lines.push("")
|
||||
lines.push("# ── IPsec (авто через ipsec-secret; ручной эквивалент) ───────")
|
||||
@@ -111,13 +127,16 @@ function generateRosCommands(t: GreTunnel, serverById: Record<string, Server>):
|
||||
lines.push(` enc-algorithms=${ENC_ROS[t.ipsec.encAlg]} \\`)
|
||||
lines.push(` auth-algorithms=${AUTH_ROS[t.ipsec.authAlg]} \\`)
|
||||
lines.push(` pfs-group=${pfsGroup} \\`)
|
||||
lines.push(` lifetime=${t.ipsec.lifetime}`)
|
||||
lines.push(` lifetime=${t.ipsec.lifetime ?? "1d"}`)
|
||||
lines.push("")
|
||||
lines.push(`/ip ipsec policy add \\`)
|
||||
lines.push(` src-address=${t.localAddress !== "0.0.0.0" ? t.localAddress + "/32" : "0.0.0.0/0"} \\`)
|
||||
lines.push(` dst-address=${t.remoteAddress}/32 \\`)
|
||||
lines.push(` proposal=${t.name} \\`)
|
||||
lines.push(` tunnel=yes`)
|
||||
} else if (t.ipsec) {
|
||||
lines.push("")
|
||||
lines.push("# IPsec: peer/policy создаёт RouterOS по ipsec-secret")
|
||||
}
|
||||
|
||||
return lines.join("\n")
|
||||
@@ -126,7 +145,7 @@ function generateRosCommands(t: GreTunnel, serverById: Record<string, Server>):
|
||||
// ─── small ui helpers ────────────────────────────────────────────────────────
|
||||
|
||||
function TunnelStatus({ status }: { status: GreStatus }) {
|
||||
const s = STATUS_MAP[status]
|
||||
const s = greStatusMeta(status)
|
||||
return (
|
||||
<span className="inline-flex items-center gap-1.5 text-sm">
|
||||
<span className={`size-1.5 rounded-full ${s.dot}`} />
|
||||
@@ -164,6 +183,7 @@ interface BackendServer {
|
||||
|
||||
interface GreTunnelsApiResponse {
|
||||
tunnels: GreTunnel[]
|
||||
failures?: Array<{ serverId: string; serverName?: string; error: string }>
|
||||
}
|
||||
|
||||
function makeApiFetch(backendUrl: string) {
|
||||
@@ -237,7 +257,6 @@ export default function GrePage() {
|
||||
const [liveTunnels, setLiveTunnels] = useState<GreTunnel[]>([])
|
||||
const [dataLoading, setDataLoading] = useState(false)
|
||||
const [dataError, setDataError] = useState<string | null>(null)
|
||||
const [syncJhBusy, setSyncJhBusy] = useState(false)
|
||||
|
||||
const [pageTab, setPageTab] = useState<PageTab>("tunnels")
|
||||
const [tabFilter, setTabFilter] = useState<TabFilter>("all")
|
||||
@@ -245,6 +264,15 @@ export default function GrePage() {
|
||||
const [selectedServerId, setSelectedServerId] = useState(ALL_SERVERS_ID)
|
||||
|
||||
const [tunnelOpen, setTunnelOpen] = useState(false)
|
||||
const [tunnelMode, setTunnelMode] = useState<"create" | "edit">("create")
|
||||
const [editingTunnel, setEditingTunnel] = useState<GreTunnel | null>(null)
|
||||
const [pendingDelete, setPendingDelete] = useState<GreTunnel | null>(null)
|
||||
const [mutateBusy, setMutateBusy] = useState(false)
|
||||
const [liveStale, setLiveStale] = useState(false)
|
||||
const [historyOpen, setHistoryOpen] = useState(false)
|
||||
const [revisions, setRevisions] = useState<ConfigRevisionDto[]>([])
|
||||
const [historyLoading, setHistoryLoading] = useState(false)
|
||||
const [historyRestoring, setHistoryRestoring] = useState(false)
|
||||
const [poolOpen, setPoolOpen] = useState(false)
|
||||
const [codePreviewTunnel, setCodePreviewTunnel] = useState<GreTunnel | null>(null)
|
||||
|
||||
@@ -261,14 +289,19 @@ export default function GrePage() {
|
||||
try {
|
||||
const [backendServers, greRes] = await Promise.all([
|
||||
apiFetch<BackendServer[]>("/api/servers"),
|
||||
apiFetch<GreTunnelsApiResponse>("/api/filters/gre-tunnels"),
|
||||
apiFetch<GreTunnelsApiResponse>("/api/gre/tunnels"),
|
||||
])
|
||||
setLiveServers(backendServers.map(mapBackendToServer))
|
||||
setLiveTunnels(greRes.tunnels)
|
||||
setLiveStale(false)
|
||||
if (greRes.failures?.length) {
|
||||
toast.warning(
|
||||
`Не удалось опросить: ${greRes.failures.map((f) => f.serverName ?? f.serverId).join(", ")}`,
|
||||
)
|
||||
}
|
||||
} catch (e) {
|
||||
setDataError(e instanceof Error ? e.message : "Ошибка загрузки")
|
||||
setLiveServers([])
|
||||
setLiveTunnels([])
|
||||
setLiveStale(true)
|
||||
} finally {
|
||||
setDataLoading(false)
|
||||
}
|
||||
@@ -280,6 +313,7 @@ export default function GrePage() {
|
||||
setLiveServers([])
|
||||
setLiveTunnels([])
|
||||
setDataError(null)
|
||||
setLiveStale(false)
|
||||
})
|
||||
return
|
||||
}
|
||||
@@ -323,39 +357,186 @@ export default function GrePage() {
|
||||
[displayPools],
|
||||
)
|
||||
|
||||
const syncJhToDb = useCallback(async () => {
|
||||
if (!isLive || syncJhBusy) return
|
||||
const jh = displayServers.filter((s) => s.type === "jump-host" && s.enabled)
|
||||
if (jh.length === 0) {
|
||||
toast.info("Нет включённых Jump Host в списке серверов")
|
||||
const historyServerId = selectedServerId === ALL_SERVERS_ID ? null : selectedServerId
|
||||
const mutationsLocked = isLive && (mutateBusy || liveStale || historyRestoring)
|
||||
|
||||
const loadRevisions = useCallback(async () => {
|
||||
if (!isLive || !historyServerId) return
|
||||
setHistoryLoading(true)
|
||||
try {
|
||||
const res = await apiFetch<{ revisions: ConfigRevisionDto[] }>(
|
||||
`/api/gre/revisions?serverId=${encodeURIComponent(historyServerId)}`,
|
||||
)
|
||||
setRevisions(res.revisions)
|
||||
} catch (err) {
|
||||
toast.error("Не удалось загрузить историю", { description: String(err) })
|
||||
setRevisions([])
|
||||
} finally {
|
||||
setHistoryLoading(false)
|
||||
}
|
||||
}, [isLive, historyServerId, apiFetch])
|
||||
|
||||
const restoreRevision = useCallback(async (id: string) => {
|
||||
if (!isLive || !historyServerId) return
|
||||
setHistoryRestoring(true)
|
||||
try {
|
||||
await apiFetch(
|
||||
`/api/gre/revisions/${encodeURIComponent(id)}/restore`,
|
||||
{ method: "POST", body: JSON.stringify({ serverId: historyServerId }) },
|
||||
)
|
||||
toast.success("Версия применена на роутер")
|
||||
await loadLive()
|
||||
await loadRevisions()
|
||||
} catch (err) {
|
||||
toast.error("Не удалось откатить", { description: String(err) })
|
||||
} finally {
|
||||
setHistoryRestoring(false)
|
||||
}
|
||||
}, [isLive, historyServerId, apiFetch, loadLive, loadRevisions])
|
||||
|
||||
function tunnelWriteBody(form: typeof defaultTunnelForm) {
|
||||
return {
|
||||
serverId: form.serverId,
|
||||
name: form.name.trim(),
|
||||
localAddress: form.localAddress.trim() || undefined,
|
||||
remoteAddress: form.remoteAddress.trim(),
|
||||
localInnerIp: form.localInnerIp.trim() || undefined,
|
||||
remoteInnerIp: form.remoteInnerIp.trim() || undefined,
|
||||
comment: form.comment || undefined,
|
||||
enabled: form.enabled,
|
||||
mtu: form.mtu,
|
||||
keepaliveInterval: form.keepaliveInterval,
|
||||
keepaliveRetries: form.keepaliveRetries,
|
||||
dscp: form.dscp,
|
||||
clampTcpMss: form.clampTcpMss,
|
||||
allowFastPath: form.allowFastPath,
|
||||
ipsecSecret: form.ipsecEnabled ? form.ipsecSecret : undefined,
|
||||
}
|
||||
}
|
||||
|
||||
async function submitTunnel() {
|
||||
if (!isLive) {
|
||||
toast.info("Создание на роутер доступно только в live-режиме")
|
||||
return
|
||||
}
|
||||
setSyncJhBusy(true)
|
||||
const errors: string[] = []
|
||||
try {
|
||||
for (const s of jh) {
|
||||
try {
|
||||
await apiFetch<{ ok: boolean }>("/api/filters/sync/from-router", {
|
||||
method: "POST",
|
||||
body: JSON.stringify({ serverId: s.id }),
|
||||
})
|
||||
} catch (e) {
|
||||
errors.push(`${s.name}: ${e instanceof Error ? e.message : "ошибка"}`)
|
||||
}
|
||||
}
|
||||
const fresh = await apiFetch<GreTunnelsApiResponse>("/api/filters/gre-tunnels")
|
||||
setLiveTunnels(fresh.tunnels)
|
||||
if (errors.length) {
|
||||
toast.warning(`Синхронизировано JH: ${jh.length - errors.length}/${jh.length}. Ошибки: ${errors.join("; ")}`)
|
||||
} else {
|
||||
toast.success(`Правила с ${jh.length} JH записаны в БД, список GRE обновлён.`)
|
||||
}
|
||||
} catch (e) {
|
||||
toast.error(e instanceof Error ? e.message : "Ошибка после синхронизации")
|
||||
} finally {
|
||||
setSyncJhBusy(false)
|
||||
if (liveStale) {
|
||||
toast.error("Роутер недоступен — изменения заблокированы")
|
||||
return
|
||||
}
|
||||
}, [isLive, syncJhBusy, apiFetch, displayServers])
|
||||
if (!tForm.name.trim() || !tForm.serverId || !tForm.remoteAddress.trim()) {
|
||||
toast.error("Заполните имя, сервер и удалённый адрес")
|
||||
return
|
||||
}
|
||||
if (tForm.ipsecEnabled && tForm.ipsecSecret.trim().length < 8) {
|
||||
toast.error("Для IPsec нужен PSK не короче 8 символов")
|
||||
return
|
||||
}
|
||||
setMutateBusy(true)
|
||||
try {
|
||||
if (tunnelMode === "edit" && editingTunnel) {
|
||||
await apiFetch("/api/gre/tunnels", {
|
||||
method: "PATCH",
|
||||
body: JSON.stringify({
|
||||
...tunnelWriteBody(tForm),
|
||||
rosId: editingTunnel.id,
|
||||
name: editingTunnel.name,
|
||||
}),
|
||||
})
|
||||
toast.success(`Туннель ${tForm.name} обновлён`)
|
||||
} else {
|
||||
await apiFetch("/api/gre/tunnels", {
|
||||
method: "POST",
|
||||
body: JSON.stringify(tunnelWriteBody(tForm)),
|
||||
})
|
||||
toast.success(`Туннель ${tForm.name} создан`)
|
||||
}
|
||||
setTunnelOpen(false)
|
||||
setEditingTunnel(null)
|
||||
await loadLive()
|
||||
} catch (err) {
|
||||
toast.error("Не удалось сохранить туннель", { description: String(err) })
|
||||
} finally {
|
||||
setMutateBusy(false)
|
||||
}
|
||||
}
|
||||
|
||||
async function toggleTunnel(t: GreTunnel) {
|
||||
if (!isLive || mutationsLocked) return
|
||||
setMutateBusy(true)
|
||||
try {
|
||||
await apiFetch("/api/gre/tunnels", {
|
||||
method: "PATCH",
|
||||
body: JSON.stringify({
|
||||
serverId: t.serverId,
|
||||
rosId: t.id,
|
||||
name: t.name,
|
||||
enabled: !t.enabled,
|
||||
remoteAddress: t.remoteAddress,
|
||||
}),
|
||||
})
|
||||
toast.success(t.enabled ? `Выключен ${t.name}` : `Включён ${t.name}`)
|
||||
await loadLive()
|
||||
} catch (err) {
|
||||
toast.error("Не удалось изменить туннель", { description: String(err) })
|
||||
} finally {
|
||||
setMutateBusy(false)
|
||||
}
|
||||
}
|
||||
|
||||
async function confirmDeleteTunnel() {
|
||||
const t = pendingDelete
|
||||
if (!t || !isLive) return
|
||||
setMutateBusy(true)
|
||||
try {
|
||||
await apiFetch("/api/gre/tunnels", {
|
||||
method: "DELETE",
|
||||
body: JSON.stringify({ serverId: t.serverId, rosId: t.id, name: t.name }),
|
||||
})
|
||||
toast.success(`Удалён ${t.name}`)
|
||||
setPendingDelete(null)
|
||||
await loadLive()
|
||||
} catch (err) {
|
||||
toast.error("Не удалось удалить туннель", { description: String(err) })
|
||||
} finally {
|
||||
setMutateBusy(false)
|
||||
}
|
||||
}
|
||||
|
||||
function openCreateTunnel() {
|
||||
setTunnelMode("create")
|
||||
setEditingTunnel(null)
|
||||
setTForm({
|
||||
...defaultTunnelForm,
|
||||
serverId: selectedServerId === ALL_SERVERS_ID ? "" : selectedServerId,
|
||||
})
|
||||
setTunnelOpen(true)
|
||||
}
|
||||
|
||||
function openEditTunnel(t: GreTunnel) {
|
||||
setTunnelMode("edit")
|
||||
setEditingTunnel(t)
|
||||
setTForm({
|
||||
...defaultTunnelForm,
|
||||
name: t.name,
|
||||
serverId: t.serverId,
|
||||
localAddress: t.localAddress === "0.0.0.0" ? "" : t.localAddress,
|
||||
remoteAddress: t.remoteAddress,
|
||||
poolId: t.poolId === "live" ? "" : t.poolId,
|
||||
localInnerIp: t.localInnerIp,
|
||||
remoteInnerIp: t.remoteInnerIp,
|
||||
comment: t.comment,
|
||||
enabled: t.enabled,
|
||||
ipsecEnabled: !!t.ipsec,
|
||||
ipsecSecret: t.ipsec?.secret ?? "",
|
||||
mtu: t.mtu,
|
||||
keepaliveInterval: t.keepaliveInterval,
|
||||
keepaliveRetries: t.keepaliveRetries,
|
||||
dscp: String(t.dscp),
|
||||
clampTcpMss: t.clampTcpMss,
|
||||
allowFastPath: t.allowFastPath,
|
||||
})
|
||||
setTunnelOpen(true)
|
||||
}
|
||||
|
||||
useEffect(() => {
|
||||
if (dataError) toast.error(dataError)
|
||||
@@ -403,6 +584,14 @@ export default function GrePage() {
|
||||
showAll
|
||||
allCount={displayServers.length}
|
||||
loading={isLive && dataLoading && displayServers.length === 0}
|
||||
banner={
|
||||
isLive && liveStale ? (
|
||||
<div className="shrink-0 border-b border-amber-500/30 bg-amber-500/10 px-6 py-2.5 text-xs text-amber-700 dark:text-amber-400 flex items-center gap-2">
|
||||
<TriangleAlertIcon className="size-3.5 shrink-0" />
|
||||
Роутер недоступен — показан кэш. Изменения заблокированы, пока не удастся прочитать CHR.
|
||||
</div>
|
||||
) : null
|
||||
}
|
||||
header={
|
||||
<PageHeader
|
||||
crumbs={[{ label: "Управление" }, { label: "GRE-туннели" }]}
|
||||
@@ -422,14 +611,17 @@ export default function GrePage() {
|
||||
<Button
|
||||
variant="outline"
|
||||
size="sm"
|
||||
onClick={() => { void syncJhToDb() }}
|
||||
disabled={!isLive || syncJhBusy || dataLoading}
|
||||
title="Загрузить правила фильтрации с каждого Jump Host в БД и обновить опрос GRE"
|
||||
onClick={() => {
|
||||
setHistoryOpen(true)
|
||||
void loadRevisions()
|
||||
}}
|
||||
disabled={!isLive || !historyServerId || dataLoading}
|
||||
title={!historyServerId ? "Выберите сервер, чтобы смотреть историю" : "История версий и откат на CHR"}
|
||||
>
|
||||
<DatabaseIcon className={cn("size-4", syncJhBusy && "animate-pulse")} />
|
||||
JH → БД
|
||||
<HistoryIcon className="size-4" />
|
||||
История
|
||||
</Button>
|
||||
<Button size="sm" onClick={() => { setTForm(defaultTunnelForm); setTunnelOpen(true) }}>
|
||||
<Button size="sm" onClick={openCreateTunnel} disabled={mutateBusy}>
|
||||
<PlusIcon className="size-4" />Добавить туннель
|
||||
</Button>
|
||||
</>
|
||||
@@ -521,6 +713,10 @@ export default function GrePage() {
|
||||
servers={displayServers}
|
||||
pools={displayPools}
|
||||
onCodePreview={setCodePreviewTunnel}
|
||||
onEdit={openEditTunnel}
|
||||
onToggle={(t) => { void toggleTunnel(t) }}
|
||||
onDelete={setPendingDelete}
|
||||
mutationsLocked={mutationsLocked}
|
||||
/>
|
||||
</DataPageCard>
|
||||
)}
|
||||
@@ -548,7 +744,7 @@ export default function GrePage() {
|
||||
<div className="flex flex-wrap gap-2">
|
||||
{poolTunnels.map((t, tunnelIndex) => (
|
||||
<div key={`${t.id}:${t.serverId}:${t.name}:${tunnelIndex}`} className="flex items-center gap-2 border border-border rounded-md px-3 py-1.5 bg-muted/30 text-xs">
|
||||
<span className={`size-1.5 rounded-full ${STATUS_MAP[t.status].dot}`} />
|
||||
<span className={`size-1.5 rounded-full ${greStatusMeta(t.status).dot}`} />
|
||||
<span className="font-mono font-medium">{t.name}</span>
|
||||
<span className="text-muted-foreground">{t.localInnerIp} ↔ {t.remoteInnerIp}</span>
|
||||
{t.ipsec && <LockIcon className="size-3 text-emerald-400" />}
|
||||
@@ -609,8 +805,8 @@ export default function GrePage() {
|
||||
codePreviewTunnel ? (
|
||||
<div className="flex flex-wrap gap-3 text-xs shrink-0">
|
||||
<span className="flex items-center gap-1.5">
|
||||
<span className={`size-1.5 rounded-full ${STATUS_MAP[codePreviewTunnel.status].dot}`} />
|
||||
{STATUS_MAP[codePreviewTunnel.status].label}
|
||||
<span className={`size-1.5 rounded-full ${greStatusMeta(codePreviewTunnel.status).dot}`} />
|
||||
{greStatusMeta(codePreviewTunnel.status).label}
|
||||
</span>
|
||||
<span className="text-muted-foreground">·</span>
|
||||
<span>{serverById[codePreviewTunnel.serverId]?.name}</span>
|
||||
@@ -625,7 +821,7 @@ export default function GrePage() {
|
||||
<span className="text-muted-foreground">·</span>
|
||||
<span className="flex items-center gap-1 text-success">
|
||||
<LockIcon className="size-3" />
|
||||
IPsec {IKE_LABELS[codePreviewTunnel.ipsec.ikeVersion]}
|
||||
IPsec {codePreviewTunnel.ipsec.ikeVersion ? IKE_LABELS[codePreviewTunnel.ipsec.ikeVersion] : "PSK"}
|
||||
</span>
|
||||
</>
|
||||
) : null}
|
||||
@@ -638,18 +834,18 @@ export default function GrePage() {
|
||||
<Sheet open={tunnelOpen} onOpenChange={setTunnelOpen}>
|
||||
<SheetContent side="right" className="w-full sm:max-w-lg flex flex-col gap-0 p-0">
|
||||
<SheetHeader className="px-6 pt-6 pb-4 border-b shrink-0">
|
||||
<SheetTitle>Новый GRE-туннель</SheetTitle>
|
||||
<SheetDescription>RouterOS 7.20+ · /interface gre add</SheetDescription>
|
||||
<SheetTitle>{tunnelMode === "edit" ? "Редактировать GRE-туннель" : "Новый GRE-туннель"}</SheetTitle>
|
||||
<SheetDescription>RouterOS 7.20+ · /interface gre {tunnelMode === "edit" ? "set" : "add"}</SheetDescription>
|
||||
</SheetHeader>
|
||||
|
||||
<div className="flex-1 overflow-y-auto px-6 py-5 flex flex-col gap-5">
|
||||
<div className="flex flex-col gap-4">
|
||||
<SectionTitle>Основные</SectionTitle>
|
||||
<FormField label="Имя интерфейса" required hint="Только латиница, цифры и дефис, например gre-msk-spb">
|
||||
<Input className="font-mono" placeholder="gre-msk-spb" value={tForm.name} onChange={(e) => setT("name", e.target.value)} />
|
||||
<Input className="font-mono" placeholder="gre-msk-spb" value={tForm.name} disabled={tunnelMode === "edit"} onChange={(e) => setT("name", e.target.value)} />
|
||||
</FormField>
|
||||
<FormField label="Сервер (MikroTik)" required>
|
||||
<select value={tForm.serverId} onChange={(e) => setT("serverId", e.target.value)}
|
||||
<select value={tForm.serverId} onChange={(e) => setT("serverId", e.target.value)} disabled={tunnelMode === "edit"}
|
||||
className="h-8 w-full rounded-lg border border-input bg-background px-2.5 text-sm text-foreground outline-none focus-visible:border-ring focus-visible:ring-3 focus-visible:ring-ring/50">
|
||||
<option value="" disabled>Выбрать сервер…</option>
|
||||
{displayServers.map((s) => <option key={s.id} value={s.id}>{s.name} ({s.site})</option>)}
|
||||
@@ -676,7 +872,7 @@ export default function GrePage() {
|
||||
|
||||
<div className="flex flex-col gap-4">
|
||||
<SectionTitle>Внутренний IP</SectionTitle>
|
||||
<FormField label="IP-пул" required hint="Из какого пула выделяется /30-блок">
|
||||
<FormField label="IP-пул" hint="Необязательно — внутренний IP можно указать вручную">
|
||||
<select value={tForm.poolId} onChange={(e) => setT("poolId", e.target.value)}
|
||||
className="h-8 w-full rounded-lg border border-input bg-background px-2.5 text-sm text-foreground outline-none focus-visible:border-ring focus-visible:ring-3 focus-visible:ring-ring/50">
|
||||
<option value="" disabled>Выбрать пул…</option>
|
||||
@@ -798,7 +994,9 @@ export default function GrePage() {
|
||||
|
||||
<SheetFooter className="px-6 py-4 border-t shrink-0 flex-row gap-2">
|
||||
<SheetClose render={<Button variant="outline" className="flex-1" />}>Отмена</SheetClose>
|
||||
<Button className="flex-1" onClick={() => setTunnelOpen(false)}>Создать туннель</Button>
|
||||
<Button className="flex-1" onClick={() => void submitTunnel()} disabled={mutateBusy}>
|
||||
{tunnelMode === "edit" ? "Сохранить" : "Создать туннель"}
|
||||
</Button>
|
||||
</SheetFooter>
|
||||
</SheetContent>
|
||||
</Sheet>
|
||||
@@ -845,6 +1043,39 @@ export default function GrePage() {
|
||||
</SheetFooter>
|
||||
</SheetContent>
|
||||
</Sheet>
|
||||
|
||||
<ConfigHistorySheet
|
||||
open={historyOpen}
|
||||
onOpenChange={setHistoryOpen}
|
||||
title="История GRE"
|
||||
itemLabel="туннелей"
|
||||
revisions={revisions}
|
||||
loading={historyLoading}
|
||||
restoring={historyRestoring}
|
||||
onRestore={restoreRevision}
|
||||
/>
|
||||
|
||||
<AlertDialog open={!!pendingDelete} onOpenChange={(v) => { if (!v) setPendingDelete(null) }}>
|
||||
<AlertDialogContent size="default">
|
||||
<AlertDialogHeader>
|
||||
<AlertDialogMedia className="bg-destructive/10 text-destructive">
|
||||
<Trash2Icon />
|
||||
</AlertDialogMedia>
|
||||
<AlertDialogTitle>Удалить GRE-туннель?</AlertDialogTitle>
|
||||
<AlertDialogDescription>
|
||||
{pendingDelete
|
||||
? `${pendingDelete.name} на сервере ${serverById[pendingDelete.serverId]?.name ?? pendingDelete.serverId}. Будут удалены интерфейс и связанный /ip/address.`
|
||||
: null}
|
||||
</AlertDialogDescription>
|
||||
</AlertDialogHeader>
|
||||
<AlertDialogFooter>
|
||||
<AlertDialogCancel onClick={() => setPendingDelete(null)}>Отмена</AlertDialogCancel>
|
||||
<AlertDialogAction variant="destructive" onClick={() => void confirmDeleteTunnel()}>
|
||||
Удалить
|
||||
</AlertDialogAction>
|
||||
</AlertDialogFooter>
|
||||
</AlertDialogContent>
|
||||
</AlertDialog>
|
||||
</>
|
||||
)
|
||||
}
|
||||
|
||||
+530
-51
@@ -27,7 +27,9 @@ import {
|
||||
findServerByGreRemote,
|
||||
greSourceWanIndexOnMap,
|
||||
greTunnelProbe,
|
||||
placeCountryServiceNodes,
|
||||
placeServiceNodes,
|
||||
SERVICE_COL_W,
|
||||
type GreMapEdge,
|
||||
type WanJhEdge,
|
||||
} from "@/lib/network-map-layout"
|
||||
@@ -55,8 +57,9 @@ import {
|
||||
matchNetflowForWan,
|
||||
type MatchedNetflowHop,
|
||||
} from "@/lib/map-netflow-hops"
|
||||
import type { FlowMapHop, FlowMapHopsDto, FlowMapService, FlowMapServiceEdge, FlowMapServicePath } from "@mmapp/contracts/traffic-flow"
|
||||
import type { FlowMapCountryServiceGroup, FlowMapHop, FlowMapHopsDto, FlowMapService, FlowMapServiceEdge, FlowMapServicePath } from "@mmapp/contracts/traffic-flow"
|
||||
import { ServiceBrandIcon } from "@/components/network-map/service-brand-icon"
|
||||
import { CountryFlagSvg } from "@/components/network-map/country-flag-svg"
|
||||
import { Button } from "@/components/ui/button"
|
||||
import { StatusBadge } from "@/components/status-badge"
|
||||
import { StatusDot } from "@/components/status-dot"
|
||||
@@ -67,8 +70,9 @@ import {
|
||||
CableIcon, CopyIcon, ActivityIcon, ExternalLinkIcon,
|
||||
} from "lucide-react"
|
||||
import { cn } from "@/lib/utils"
|
||||
import { formatServicePathLabel, formatServicePathTitle } from "@/lib/format-service-path-label"
|
||||
import Link from "next/link"
|
||||
import { Flag } from "@/components/flag"
|
||||
import { Flag, countryName } from "@/components/flag"
|
||||
|
||||
// ─── Resource metrics (для мини-блока справа; числа детерминированы по id узла) ─
|
||||
|
||||
@@ -284,6 +288,12 @@ const MOCK_MAP_SERVICES: FlowMapService[] = [
|
||||
{ id: "svc:aws", label: "AWS", category: "CDN", bytes: 9_000_000, bps: 3_600_000, share: 0.09 },
|
||||
]
|
||||
|
||||
const MOCK_MAP_COUNTRIES: FlowMapService[] = [
|
||||
{ id: "cc:us", label: "US", category: "Страна", bytes: 22_000_000, bps: 8_800_000, share: 0.38 },
|
||||
{ id: "cc:nl", label: "NL", category: "Страна", bytes: 14_000_000, bps: 5_600_000, share: 0.31 },
|
||||
{ id: "cc:de", label: "DE", category: "Страна", bytes: 9_000_000, bps: 3_600_000, share: 0.21 },
|
||||
]
|
||||
|
||||
const MOCK_MAP_SERVICE_EDGES: FlowMapServiceEdge[] = [
|
||||
{ fromId: "srv2", toId: "svc:google", bytes: 14_000_000, bps: 5_600_000, bpsFwd: 4_200_000, bpsRev: 1_400_000, clientName: "Alice", clients: [{ id: "u1", name: "Alice" }] },
|
||||
{ fromId: "srv3", toId: "svc:google", bytes: 8_000_000, bps: 3_200_000, bpsFwd: 2_400_000, bpsRev: 800_000, clientName: "Bob", clients: [{ id: "u2", name: "Bob" }] },
|
||||
@@ -292,6 +302,14 @@ const MOCK_MAP_SERVICE_EDGES: FlowMapServiceEdge[] = [
|
||||
{ fromId: "srv3", toId: "svc:aws", bytes: 9_000_000, bps: 3_600_000, bpsFwd: 2_700_000, bpsRev: 900_000, clientName: "Bob", clients: [{ id: "u2", name: "Bob" }] },
|
||||
]
|
||||
|
||||
const MOCK_MAP_COUNTRY_EDGES: FlowMapServiceEdge[] = [
|
||||
{ fromId: "srv2", toId: "cc:us", bytes: 14_000_000, bps: 5_600_000, bpsFwd: 4_200_000, bpsRev: 1_400_000, clientName: "Alice", clients: [{ id: "u1", name: "Alice" }] },
|
||||
{ fromId: "srv3", toId: "cc:us", bytes: 8_000_000, bps: 3_200_000, bpsFwd: 2_400_000, bpsRev: 800_000, clientName: "Bob", clients: [{ id: "u2", name: "Bob" }] },
|
||||
{ fromId: "srv2", toId: "cc:nl", bytes: 9_000_000, bps: 3_600_000, bpsFwd: 2_800_000, bpsRev: 800_000, clientName: "Alice", clients: [{ id: "u1", name: "Alice" }] },
|
||||
{ fromId: "srv3", toId: "cc:nl", bytes: 5_000_000, bps: 2_000_000, bpsFwd: 1_500_000, bpsRev: 500_000, clientName: "Bob", clients: [{ id: "u2", name: "Bob" }] },
|
||||
{ fromId: "srv3", toId: "cc:de", bytes: 9_000_000, bps: 3_600_000, bpsFwd: 2_700_000, bpsRev: 900_000, clientName: "Bob", clients: [{ id: "u2", name: "Bob" }] },
|
||||
]
|
||||
|
||||
const MOCK_MAP_SERVICE_PATHS: FlowMapServicePath[] = [
|
||||
{ clientId: "u1", clientName: "Alice", viaId: "srv1", viaName: "mt-msk-core-01", enId: "srv2", enName: "mt-spb-edge-01", serviceId: "svc:google", bytes: 14_000_000, bps: 5_600_000 },
|
||||
{ clientId: "u2", clientName: "Bob", viaId: "srv1", viaName: "mt-msk-core-01", enId: "srv3", enName: "mt-fra-edge-01", serviceId: "svc:google", bytes: 8_000_000, bps: 3_200_000 },
|
||||
@@ -300,6 +318,56 @@ const MOCK_MAP_SERVICE_PATHS: FlowMapServicePath[] = [
|
||||
{ clientId: "u2", clientName: "Bob", viaId: "srv1", viaName: "mt-msk-core-01", enId: "srv3", enName: "mt-fra-edge-01", serviceId: "svc:aws", bytes: 9_000_000, bps: 3_600_000 },
|
||||
]
|
||||
|
||||
const MOCK_MAP_COUNTRY_PATHS: FlowMapServicePath[] = [
|
||||
{ clientId: "u1", clientName: "Alice", viaId: "srv1", viaName: "mt-msk-core-01", enId: "srv2", enName: "mt-spb-edge-01", serviceId: "cc:us", bytes: 14_000_000, bps: 5_600_000 },
|
||||
{ clientId: "u2", clientName: "Bob", viaId: "srv1", viaName: "mt-msk-core-01", enId: "srv3", enName: "mt-fra-edge-01", serviceId: "cc:us", bytes: 8_000_000, bps: 3_200_000 },
|
||||
{ clientId: "u1", clientName: "Alice", viaId: "srv1", viaName: "mt-msk-core-01", enId: "srv2", enName: "mt-spb-edge-01", serviceId: "cc:nl", bytes: 9_000_000, bps: 3_600_000 },
|
||||
{ clientId: "u2", clientName: "Bob", viaId: "srv1", viaName: "mt-msk-core-01", enId: "srv3", enName: "mt-fra-edge-01", serviceId: "cc:nl", bytes: 5_000_000, bps: 2_000_000 },
|
||||
{ clientId: "u2", clientName: "Bob", viaId: "srv1", viaName: "mt-msk-core-01", enId: "srv3", enName: "mt-fra-edge-01", serviceId: "cc:de", bytes: 9_000_000, bps: 3_600_000 },
|
||||
]
|
||||
|
||||
/** Доли — от байтов страны cc:us (22M из моков выше). */
|
||||
const MOCK_COUNTRY_SERVICE_GROUPS: FlowMapCountryServiceGroup[] = [
|
||||
{
|
||||
countryId: "cc:us",
|
||||
services: [
|
||||
{ id: "cc:us|svc:google", label: "Google", category: "Веб", bytes: 12_000_000, bps: 4_800_000, share: 12 / 22 },
|
||||
{ id: "cc:us|svc:cloudflare", label: "Cloudflare", category: "CDN", bytes: 7_000_000, bps: 2_800_000, share: 7 / 22 },
|
||||
{ id: "cc:us|svc:aws", label: "AWS", category: "CDN", bytes: 3_000_000, bps: 1_200_000, share: 3 / 22 },
|
||||
],
|
||||
edges: [
|
||||
{ fromId: "cc:us", toId: "cc:us|svc:google", bytes: 12_000_000, bps: 4_800_000, bpsFwd: 9_000_000, bpsRev: 3_000_000, clientName: "Alice", clients: [{ id: "u1", name: "Alice" }] },
|
||||
{ fromId: "cc:us", toId: "cc:us|svc:cloudflare", bytes: 7_000_000, bps: 2_800_000, bpsFwd: 5_250_000, bpsRev: 1_750_000, clientName: "Alice", clients: [{ id: "u1", name: "Alice" }] },
|
||||
{ fromId: "cc:us", toId: "cc:us|svc:aws", bytes: 3_000_000, bps: 1_200_000, bpsFwd: 2_250_000, bpsRev: 750_000, clientName: "Bob", clients: [{ id: "u2", name: "Bob" }] },
|
||||
],
|
||||
paths: [
|
||||
{ clientId: "u1", clientName: "Alice", viaId: "srv1", viaName: "mt-msk-core-01", enId: "srv2", enName: "mt-spb-edge-01", serviceId: "cc:us|svc:google", bytes: 8_000_000, bps: 3_200_000 },
|
||||
{ clientId: "u2", clientName: "Bob", viaId: "srv1", viaName: "mt-msk-core-01", enId: "srv3", enName: "mt-fra-edge-01", serviceId: "cc:us|svc:google", bytes: 4_000_000, bps: 1_600_000 },
|
||||
{ clientId: "u1", clientName: "Alice", viaId: "srv1", viaName: "mt-msk-core-01", enId: "srv2", enName: "mt-spb-edge-01", serviceId: "cc:us|svc:cloudflare", bytes: 7_000_000, bps: 2_800_000 },
|
||||
{ clientId: "u2", clientName: "Bob", viaId: "srv1", viaName: "mt-msk-core-01", enId: "srv3", enName: "mt-fra-edge-01", serviceId: "cc:us|svc:aws", bytes: 3_000_000, bps: 1_200_000 },
|
||||
],
|
||||
},
|
||||
]
|
||||
|
||||
const DEST_MODE_KEY = "mm-network-map-dest-mode"
|
||||
type DestMode = "services" | "countries"
|
||||
|
||||
function readDestMode(): DestMode {
|
||||
if (typeof window === "undefined") return "services"
|
||||
try {
|
||||
return sessionStorage.getItem(DEST_MODE_KEY) === "countries" ? "countries" : "services"
|
||||
} catch {
|
||||
return "services"
|
||||
}
|
||||
}
|
||||
|
||||
function destDisplayLabel(node: FlowMapService | undefined, mode: DestMode, fallback = ""): string {
|
||||
if (!node) return fallback
|
||||
if (mode !== "countries") return node.label
|
||||
if (node.id === "cc:other" || node.label === "Прочее") return "Прочее"
|
||||
return countryName(node.label)
|
||||
}
|
||||
|
||||
function servicePathKey(p: Pick<FlowMapServicePath, "clientId" | "viaId" | "enId" | "serviceId">): string {
|
||||
return `${p.clientId}|${p.viaId}|${p.enId}|${p.serviceId}`
|
||||
}
|
||||
@@ -737,6 +805,10 @@ function ServiceNode({
|
||||
isSel,
|
||||
isVis,
|
||||
isDragged,
|
||||
destMode,
|
||||
iso,
|
||||
dim,
|
||||
shareLabel,
|
||||
onClick,
|
||||
onMouseDown,
|
||||
}: {
|
||||
@@ -747,20 +819,27 @@ function ServiceNode({
|
||||
isSel: boolean
|
||||
isVis: boolean
|
||||
isDragged: boolean
|
||||
destMode: DestMode
|
||||
iso?: string
|
||||
/** Приглушение узла при раскрытии другой страны (остаётся на холсте). */
|
||||
dim?: boolean
|
||||
/** Подпись доли в tooltip: у вложенных сервисов — доля страны, не окна. */
|
||||
shareLabel?: string
|
||||
onClick: () => void
|
||||
onMouseDown: (e: React.MouseEvent) => void
|
||||
}) {
|
||||
const bw = MAP_SERVICE_NODE_W
|
||||
const bh = MAP_SERVICE_NODE_H
|
||||
const flagIso = destMode === "countries" && iso && iso !== "Прочее" ? iso : ""
|
||||
return (
|
||||
<g
|
||||
transform={`translate(${x},${y})`}
|
||||
style={{ cursor: isDragged ? "grabbing" : "grab", transition: isDragged ? "none" : "opacity 0.25s" }}
|
||||
opacity={isVis ? 1 : 0.08}
|
||||
opacity={isVis ? (dim ? 0.35 : 1) : 0.08}
|
||||
onMouseDown={(e) => { e.stopPropagation(); onMouseDown(e) }}
|
||||
onClick={(e) => { e.stopPropagation(); onClick() }}
|
||||
>
|
||||
<title>{`${label} · ${serviceSharePct(share)} трафика окна`}</title>
|
||||
<title>{`${label} · ${serviceSharePct(share)} ${shareLabel ?? "payload окна"}`}</title>
|
||||
{isSel && (
|
||||
<rect
|
||||
x={-bw / 2 - 6}
|
||||
@@ -785,7 +864,9 @@ function ServiceNode({
|
||||
strokeWidth={isSel ? 2.2 : 1.4}
|
||||
/>
|
||||
<g transform="translate(-11,-24)" pointerEvents="none">
|
||||
<ServiceBrandIcon label={label} size={22} />
|
||||
{flagIso
|
||||
? <CountryFlagSvg iso={flagIso} size={22} />
|
||||
: <ServiceBrandIcon label={destMode === "countries" ? "Прочее" : label} size={22} />}
|
||||
</g>
|
||||
<text textAnchor="middle" y="14" fontSize="8.5" fontWeight="700" fill="#e0f2fe" fontFamily="ui-monospace,monospace">
|
||||
{label}
|
||||
@@ -803,6 +884,7 @@ function ServicePathList({
|
||||
services,
|
||||
highlight,
|
||||
viaMode,
|
||||
destMode,
|
||||
onToggle,
|
||||
}: {
|
||||
paths: FlowMapServicePath[]
|
||||
@@ -810,6 +892,7 @@ function ServicePathList({
|
||||
services: FlowMapService[]
|
||||
highlight: { viaId: string; enId: string; serviceId: string } | null
|
||||
viaMode: "via" | "service"
|
||||
destMode: DestMode
|
||||
onToggle: (p: FlowMapServicePath) => void
|
||||
}) {
|
||||
if (paths.length === 0) {
|
||||
@@ -820,9 +903,16 @@ function ServicePathList({
|
||||
{paths.map((p) => {
|
||||
const rowKey = servicePathKey(p)
|
||||
const via = servers.find((s) => s.id === p.viaId)
|
||||
const viaLabel = via?.site || p.viaName
|
||||
const en = servers.find((s) => s.id === p.enId)
|
||||
const svc = services.find((s) => s.id === p.serviceId)
|
||||
const mid = viaMode === "via" ? viaLabel : (svc?.label ?? p.serviceId)
|
||||
const destLabel = destDisplayLabel(svc, destMode, p.serviceId)
|
||||
const label = formatServicePathLabel(p, viaMode, {
|
||||
viaName: via?.name,
|
||||
viaSite: via?.site,
|
||||
enName: en?.name,
|
||||
serviceLabel: destLabel,
|
||||
})
|
||||
const title = formatServicePathTitle(label, destLabel)
|
||||
const active = Boolean(
|
||||
highlight
|
||||
&& highlight.viaId === p.viaId
|
||||
@@ -833,13 +923,14 @@ function ServicePathList({
|
||||
<button
|
||||
key={rowKey}
|
||||
type="button"
|
||||
title={title}
|
||||
onClick={() => onToggle(p)}
|
||||
className={cn(
|
||||
"flex items-center justify-between gap-2 rounded-md px-2 py-1.5 text-left text-xs transition-colors",
|
||||
active ? "bg-cyan-500/15 ring-1 ring-cyan-500/40" : "hover:bg-muted/50",
|
||||
)}
|
||||
>
|
||||
<span className="font-mono truncate min-w-0">{p.clientName} · {mid}</span>
|
||||
<span className="font-mono truncate min-w-0">{label}</span>
|
||||
<span className="font-mono text-emerald-400 tabular-nums shrink-0">
|
||||
{formatNetflowRate({ bytes: p.bytes, bps: p.bps, bpsFwd: p.bps, bpsRev: 0 })}
|
||||
</span>
|
||||
@@ -1091,7 +1182,16 @@ export default function NetworkMapPage() {
|
||||
const [mapServices, setMapServices] = useState<FlowMapService[]>([])
|
||||
const [mapServiceEdges, setMapServiceEdges] = useState<FlowMapServiceEdge[]>([])
|
||||
const [mapServicePaths, setMapServicePaths] = useState<FlowMapServicePath[]>([])
|
||||
const [mapCountries, setMapCountries] = useState<FlowMapService[]>([])
|
||||
const [mapCountryEdges, setMapCountryEdges] = useState<FlowMapServiceEdge[]>([])
|
||||
const [mapCountryPaths, setMapCountryPaths] = useState<FlowMapServicePath[]>([])
|
||||
const [mapCountryServiceGroups, setMapCountryServiceGroups] = useState<FlowMapCountryServiceGroup[]>([])
|
||||
const [mapSharePct, setMapSharePct] = useState(5)
|
||||
const [mapNamedBytes, setMapNamedBytes] = useState(0)
|
||||
const [mapTotalBytes, setMapTotalBytes] = useState(0)
|
||||
const [mapWindowSec, setMapWindowSec] = useState(300)
|
||||
const [mapAsnLoaded, setMapAsnLoaded] = useState(true)
|
||||
const [mapCountryLoaded, setMapCountryLoaded] = useState(true)
|
||||
/** FQDN из GRE outer → IPv4 (ответ POST /api/network/resolve-hosts), для матчинга с WAN. */
|
||||
const [greResolvedIpv4ByHost, setGreResolvedIpv4ByHost] = useState<Record<string, string>>({})
|
||||
const [dataError, setDataError] = useState<string | null>(null)
|
||||
@@ -1187,7 +1287,14 @@ export default function NetworkMapPage() {
|
||||
setMapServices(MOCK_MAP_SERVICES)
|
||||
setMapServiceEdges(MOCK_MAP_SERVICE_EDGES)
|
||||
setMapServicePaths(MOCK_MAP_SERVICE_PATHS)
|
||||
setMapCountries(MOCK_MAP_COUNTRIES)
|
||||
setMapCountryEdges(MOCK_MAP_COUNTRY_EDGES)
|
||||
setMapCountryPaths(MOCK_MAP_COUNTRY_PATHS)
|
||||
setMapCountryServiceGroups(MOCK_COUNTRY_SERVICE_GROUPS)
|
||||
setMapSharePct(5)
|
||||
setMapNamedBytes(0)
|
||||
setMapTotalBytes(0)
|
||||
setMapCountryLoaded(true)
|
||||
setDataError(null)
|
||||
})
|
||||
return
|
||||
@@ -1216,6 +1323,41 @@ export default function NetworkMapPage() {
|
||||
// ── Interaction ─────────────────────────────────────────────────────────────
|
||||
const [selected, setSelected] = useState<Server | null>(null)
|
||||
const [selectedService, setSelectedService] = useState<FlowMapService | null>(null)
|
||||
/** Раскрытая страна (режим «Страны»): справа столбец её сервисов. Не персистится. */
|
||||
const [expandedCountryId, setExpandedCountryId] = useState<string | null>(null)
|
||||
const [destMode, setDestModeState] = useState<DestMode>("services")
|
||||
useEffect(() => {
|
||||
queueMicrotask(() => setDestModeState(readDestMode()))
|
||||
}, [])
|
||||
function setDestMode(mode: DestMode) {
|
||||
setDestModeState(mode)
|
||||
setSelectedService(null)
|
||||
setExpandedCountryId(null)
|
||||
setHighlightedPath(null)
|
||||
try { sessionStorage.setItem(DEST_MODE_KEY, mode) } catch { /* private mode */ }
|
||||
}
|
||||
const expandedCountryGroup = useMemo(
|
||||
() => destMode === "countries" && expandedCountryId
|
||||
? mapCountryServiceGroups.find((g) => g.countryId === expandedCountryId) ?? null
|
||||
: null,
|
||||
[destMode, expandedCountryId, mapCountryServiceGroups],
|
||||
)
|
||||
const nestedServices = useMemo(() => expandedCountryGroup?.services ?? [], [expandedCountryGroup])
|
||||
const liveSelectedService = selectedService
|
||||
? (
|
||||
(destMode === "countries" ? mapCountries : mapServices)
|
||||
.find((s) => s.id === selectedService.id)
|
||||
?? nestedServices.find((s) => s.id === selectedService.id)
|
||||
?? selectedService
|
||||
)
|
||||
: null
|
||||
const selectedNestedService = liveSelectedService
|
||||
&& nestedServices.some((s) => s.id === liveSelectedService.id)
|
||||
? liveSelectedService
|
||||
: null
|
||||
const expandedCountry = expandedCountryId
|
||||
? mapCountries.find((c) => c.id === expandedCountryId) ?? null
|
||||
: null
|
||||
const [highlightedPath, setHighlightedPath] = useState<{ viaId: string; enId: string; serviceId: string } | null>(null)
|
||||
const [selWanIdx, setSelWanIdx] = useState<number | null>(null)
|
||||
const [hoveredId, setHoveredId] = useState<string | null>(null)
|
||||
@@ -1266,7 +1408,12 @@ export default function NetworkMapPage() {
|
||||
setMapServices(MOCK_MAP_SERVICES)
|
||||
setMapServiceEdges(MOCK_MAP_SERVICE_EDGES)
|
||||
setMapServicePaths(MOCK_MAP_SERVICE_PATHS)
|
||||
setMapCountries(MOCK_MAP_COUNTRIES)
|
||||
setMapCountryEdges(MOCK_MAP_COUNTRY_EDGES)
|
||||
setMapCountryPaths(MOCK_MAP_COUNTRY_PATHS)
|
||||
setMapCountryServiceGroups(MOCK_COUNTRY_SERVICE_GROUPS)
|
||||
setMapSharePct(5)
|
||||
setMapCountryLoaded(true)
|
||||
})
|
||||
return
|
||||
}
|
||||
@@ -1276,6 +1423,10 @@ export default function NetworkMapPage() {
|
||||
setMapServices([])
|
||||
setMapServiceEdges([])
|
||||
setMapServicePaths([])
|
||||
setMapCountries([])
|
||||
setMapCountryEdges([])
|
||||
setMapCountryPaths([])
|
||||
setMapCountryServiceGroups([])
|
||||
})
|
||||
return
|
||||
}
|
||||
@@ -1291,7 +1442,16 @@ export default function NetworkMapPage() {
|
||||
setMapServices(res.services ?? [])
|
||||
setMapServiceEdges(res.serviceEdges ?? [])
|
||||
setMapServicePaths(res.servicePaths ?? [])
|
||||
setMapCountries(res.countries ?? [])
|
||||
setMapCountryEdges(res.countryEdges ?? [])
|
||||
setMapCountryPaths(res.countryPaths ?? [])
|
||||
setMapCountryServiceGroups(res.countryServiceGroups ?? [])
|
||||
if (res.mapServiceMinSharePct != null) setMapSharePct(res.mapServiceMinSharePct)
|
||||
setMapNamedBytes(res.namedBytes ?? 0)
|
||||
setMapTotalBytes(res.totalBytes ?? 0)
|
||||
if (res.asnLoaded != null) setMapAsnLoaded(res.asnLoaded)
|
||||
if (res.countryLoaded != null) setMapCountryLoaded(res.countryLoaded)
|
||||
if (res.windowSec) setMapWindowSec(res.windowSec)
|
||||
})
|
||||
.catch((err: unknown) => {
|
||||
if (cancelled) return
|
||||
@@ -1494,9 +1654,13 @@ export default function NetworkMapPage() {
|
||||
return m
|
||||
}, [homeRouters, wanJhEdges, mapHops, showNetflow])
|
||||
|
||||
const visibleMapServices = showServices ? mapServices : []
|
||||
const destNodes = destMode === "countries" ? mapCountries : mapServices
|
||||
const destEdges = destMode === "countries" ? mapCountryEdges : mapServiceEdges
|
||||
const destPaths = destMode === "countries" ? mapCountryPaths : mapServicePaths
|
||||
|
||||
const visibleMapServices = showServices ? destNodes : []
|
||||
const visibleServiceEdges = showServices
|
||||
? drawableServiceEdges(visibleMapServices, mapServiceEdges, mapServers, greEdges, nodePosById)
|
||||
? drawableServiceEdges(visibleMapServices, destEdges, mapServers, greEdges, nodePosById)
|
||||
: []
|
||||
|
||||
const nodes = mapServers
|
||||
@@ -1518,8 +1682,24 @@ export default function NetworkMapPage() {
|
||||
.map((s) => nodePosById[s.id])
|
||||
.filter((p): p is { x: number; y: number } => Boolean(p)),
|
||||
)
|
||||
// Раскрытая страна: колонка стран уходит влево, правый x занимает столбец её сервисов.
|
||||
const countryColShift = expandedCountryGroup ? SERVICE_COL_W : 0
|
||||
const autoDestPos = countryColShift
|
||||
? Object.fromEntries(
|
||||
Object.entries(autoServicePos).map(([id, p]) => [id, { x: p.x - countryColShift, y: p.y }]),
|
||||
)
|
||||
: autoServicePos
|
||||
const servicePosById = Object.fromEntries(
|
||||
visibleMapServices.map((s) => [s.id, servicePositions[s.id] ?? autoServicePos[s.id]!]),
|
||||
visibleMapServices.map((s) => [s.id, servicePositions[s.id] ?? autoDestPos[s.id]!]),
|
||||
)
|
||||
const autoNestedPos = placeCountryServiceNodes(
|
||||
nestedServices.map((s) => s.id),
|
||||
expandedCountryId ? servicePosById[expandedCountryId] ?? autoDestPos[expandedCountryId] : undefined,
|
||||
)
|
||||
const nestedPosById = Object.fromEntries(
|
||||
nestedServices
|
||||
.map((s) => [s.id, servicePositions[s.id] ?? autoNestedPos[s.id]] as const)
|
||||
.filter((entry): entry is readonly [string, { x: number; y: number }] => Boolean(entry[1])),
|
||||
)
|
||||
|
||||
// ── Refs ─────────────────────────────────────────────────────────────────────
|
||||
@@ -1604,6 +1784,7 @@ export default function NetworkMapPage() {
|
||||
setSelectedGreEdge(null)
|
||||
setSelectedService(null)
|
||||
setHighlightedPath(null)
|
||||
setExpandedCountryId(null)
|
||||
}
|
||||
if (e.key === "=" || e.key === "+") applyZoomCenter(1.25)
|
||||
if (e.key === "-") applyZoomCenter(1 / 1.25)
|
||||
@@ -1701,6 +1882,7 @@ export default function NetworkMapPage() {
|
||||
setSelectedGreEdge(null)
|
||||
setSelectedService(null)
|
||||
setHighlightedPath(null)
|
||||
setExpandedCountryId(null)
|
||||
}
|
||||
}
|
||||
|
||||
@@ -1735,6 +1917,34 @@ export default function NetworkMapPage() {
|
||||
}
|
||||
|
||||
// ── Side panel ────────────────────────────────────────────────────────────
|
||||
useEffect(() => {
|
||||
if (
|
||||
expandedCountryId
|
||||
&& (!mapCountries.some((s) => s.id === expandedCountryId)
|
||||
|| !mapCountryServiceGroups.some((g) => g.countryId === expandedCountryId))
|
||||
) {
|
||||
queueMicrotask(() => setExpandedCountryId(null))
|
||||
}
|
||||
}, [mapCountries, mapCountryServiceGroups, expandedCountryId])
|
||||
useEffect(() => {
|
||||
if (!selectedService) return
|
||||
const stillVisible =
|
||||
destNodes.some((s) => s.id === selectedService.id)
|
||||
|| nestedServices.some((s) => s.id === selectedService.id)
|
||||
if (!stillVisible) {
|
||||
queueMicrotask(() => {
|
||||
setSelectedService(null)
|
||||
setHighlightedPath(null)
|
||||
})
|
||||
}
|
||||
}, [destMode, destNodes, nestedServices, selectedService])
|
||||
useEffect(() => {
|
||||
if (!showServices) queueMicrotask(() => setExpandedCountryId(null))
|
||||
}, [showServices])
|
||||
// Сдвиг колонки стран меняет систему координат: сбрасываем drag-овчины сервисов.
|
||||
useEffect(() => {
|
||||
queueMicrotask(() => setServicePositions({}))
|
||||
}, [expandedCountryId])
|
||||
function selectServer(s: Server) {
|
||||
setSelectedGreEdge(null)
|
||||
setSelectedService(null)
|
||||
@@ -1749,6 +1959,10 @@ export default function NetworkMapPage() {
|
||||
setSelWanIdx(null)
|
||||
setHighlightedPath(null)
|
||||
setSelectedService((prev: FlowMapService | null) => prev?.id === svc.id ? null : svc)
|
||||
// Клик по стране в режиме «Страны» раскрывает столбец её сервисов; повторный — сворачивает.
|
||||
if (destMode === "countries" && svc.id.startsWith("cc:") && !svc.id.includes("|")) {
|
||||
setExpandedCountryId((prev) => (prev === svc.id ? null : svc.id))
|
||||
}
|
||||
}
|
||||
function selectWan(s: Server, wanIdx: number) {
|
||||
setSelectedGreEdge(null)
|
||||
@@ -1898,6 +2112,27 @@ export default function NetworkMapPage() {
|
||||
)}
|
||||
</div>
|
||||
|
||||
{/* Dest overlay: services vs countries */}
|
||||
<div className="flex items-center gap-0.5 rounded-md border border-border bg-muted/40 p-0.5">
|
||||
{([
|
||||
{ value: "services" as const, label: "Сервисы" },
|
||||
{ value: "countries" as const, label: "Страны" },
|
||||
]).map((b) => (
|
||||
<button
|
||||
key={b.value}
|
||||
onClick={() => setDestMode(b.value)}
|
||||
className={cn(
|
||||
"px-2.5 py-1 text-xs rounded transition-colors whitespace-nowrap",
|
||||
destMode === b.value
|
||||
? "bg-background text-foreground shadow-sm"
|
||||
: "text-muted-foreground hover:text-foreground",
|
||||
)}
|
||||
>
|
||||
{b.label}
|
||||
</button>
|
||||
))}
|
||||
</div>
|
||||
|
||||
{/* Layers dropdown */}
|
||||
<div className="relative">
|
||||
<button
|
||||
@@ -1916,7 +2151,7 @@ export default function NetworkMapPage() {
|
||||
{([
|
||||
{ key: "showPingBadges", label: "Ping-значки", val: showPingBadges, set: setShowPingBadges, hint: "P" },
|
||||
{ key: "showNetflow", label: "NetFlow", val: showNetflow, set: setShowNetflow, hint: "" },
|
||||
{ key: "showServices", label: "Сервисы", val: showServices, set: setShowServices, hint: "" },
|
||||
{ key: "showServices", label: "Назначения", val: showServices, set: setShowServices, hint: "" },
|
||||
{ key: "showAnimDots", label: "Анимация трафика", val: showAnimDots, set: setShowAnimDots, hint: "" },
|
||||
{ key: "showMinimap", label: "Минимап", val: showMinimap, set: setShowMinimap, hint: "M" },
|
||||
{ key: "showHints", label: "Горячие клавиши", val: showHints, set: setShowHints, hint: "" },
|
||||
@@ -1943,9 +2178,14 @@ export default function NetworkMapPage() {
|
||||
))}
|
||||
<p className="px-3 pt-1.5 pb-1 text-[10px] text-muted-foreground leading-snug">
|
||||
{mapSharePct > 0
|
||||
? `Порог доли сервиса ≥ ${mapSharePct}% · Настройки → NetFlow`
|
||||
: "Порог доли выключен (все бренды, макс. 20) · Настройки → NetFlow"}
|
||||
? `Порог доли ≥ ${mapSharePct}% · Настройки → NetFlow`
|
||||
: "Порог доли выключен (все узлы, макс. 20) · Настройки → NetFlow"}
|
||||
</p>
|
||||
{destMode === "countries" && !mapCountryLoaded && (
|
||||
<p className="px-3 pb-1 text-[10px] text-amber-500 leading-snug">
|
||||
GeoIP Country не загружен, страны из RIPE-кэша
|
||||
</p>
|
||||
)}
|
||||
{(Object.keys(nodePositions).length > 0 || Object.keys(satPositions).length > 0 || Object.keys(servicePositions).length > 0) && (
|
||||
<div className="border-t border-border/50 mt-1 pt-1">
|
||||
<button
|
||||
@@ -2262,6 +2502,32 @@ export default function NetworkMapPage() {
|
||||
{visibleMapServices.map((svc) => {
|
||||
const pos = servicePosById[svc.id]
|
||||
if (!pos) return null
|
||||
return (
|
||||
<ServiceNode
|
||||
key={svc.id}
|
||||
label={destDisplayLabel(svc, destMode)}
|
||||
share={svc.share}
|
||||
x={pos.x}
|
||||
y={pos.y}
|
||||
isSel={selectedService?.id === svc.id}
|
||||
isVis
|
||||
dim={Boolean(expandedCountryGroup) && svc.id !== expandedCountryId}
|
||||
isDragged={draggedSvcId === svc.id}
|
||||
destMode={destMode}
|
||||
iso={svc.label}
|
||||
onMouseDown={(e) => onServiceMouseDown(e, svc.id, pos.x, pos.y)}
|
||||
onClick={() => {
|
||||
if (suppressClickRef.current) { suppressClickRef.current = false; return }
|
||||
selectService(svc)
|
||||
}}
|
||||
/>
|
||||
)
|
||||
})}
|
||||
|
||||
{/* ── Сервисы раскрытой страны (второй столбец справа) ── */}
|
||||
{expandedCountryGroup && nestedServices.map((svc) => {
|
||||
const pos = nestedPosById[svc.id]
|
||||
if (!pos) return null
|
||||
return (
|
||||
<ServiceNode
|
||||
key={svc.id}
|
||||
@@ -2272,6 +2538,8 @@ export default function NetworkMapPage() {
|
||||
isSel={selectedService?.id === svc.id}
|
||||
isVis
|
||||
isDragged={draggedSvcId === svc.id}
|
||||
destMode="services"
|
||||
shareLabel="трафика страны"
|
||||
onMouseDown={(e) => onServiceMouseDown(e, svc.id, pos.x, pos.y)}
|
||||
onClick={() => {
|
||||
if (suppressClickRef.current) { suppressClickRef.current = false; return }
|
||||
@@ -2311,15 +2579,16 @@ export default function NetworkMapPage() {
|
||||
&& highlightedPath.serviceId === edge.toId,
|
||||
)
|
||||
const pathDim = Boolean(highlightedPath) && !pathHit
|
||||
const hl = pathHit || (!highlightedPath && (selectedService?.id === edge.toId || selected?.id === edge.fromId))
|
||||
const hl = pathHit || (!highlightedPath && (selectedService?.id === edge.toId || selected?.id === edge.fromId || expandedCountryId === edge.toId))
|
||||
const countryDim = !hl && Boolean(expandedCountryGroup) && edge.toId !== expandedCountryId
|
||||
const svc = visibleMapServices.find((s) => s.id === edge.toId)
|
||||
const enName = mapServers.find((s) => s.id === edge.fromId)?.name ?? edge.fromId
|
||||
const clientLabel = (edge.clients?.map((c) => c.name).filter(Boolean).join(", ") || edge.clientName || "—")
|
||||
const pathTitle = `${clientLabel} → ${enName} → ${svc?.label ?? edge.toId}`
|
||||
const pathTitle = `${clientLabel} → ${enName} → ${destDisplayLabel(svc, destMode, edge.toId)}`
|
||||
return (
|
||||
<g
|
||||
key={`${edge.fromId}|${edge.toId}`}
|
||||
opacity={pathDim ? 0.12 : hl ? 1 : 0.72}
|
||||
opacity={pathDim ? 0.12 : hl ? 1 : countryDim ? 0.35 : 0.72}
|
||||
style={{ transition: "opacity 0.3s" }}
|
||||
>
|
||||
<title>{pathTitle}</title>
|
||||
@@ -2365,6 +2634,81 @@ export default function NetworkMapPage() {
|
||||
)
|
||||
})}
|
||||
|
||||
{/* ── Раскрытая страна → её сервисы ── */}
|
||||
{expandedCountryGroup?.edges.map((edge) => {
|
||||
const from = servicePosById[edge.fromId]
|
||||
const to = nestedPosById[edge.toId]
|
||||
if (!from || !to) return null
|
||||
const hop: MatchedNetflowHop = {
|
||||
bytes: edge.bytes,
|
||||
bps: edge.bps,
|
||||
bpsFwd: edge.bpsFwd,
|
||||
bpsRev: edge.bpsRev,
|
||||
}
|
||||
const clipped = clipSegmentCircleToRect(
|
||||
from.x,
|
||||
from.y,
|
||||
MAP_SERVICE_NODE_W / 2,
|
||||
to.x,
|
||||
to.y,
|
||||
MAP_SERVICE_NODE_W / 2,
|
||||
MAP_SERVICE_NODE_H / 2,
|
||||
)
|
||||
const { mx, my } = edgeBadgePosition(clipped.x1, clipped.y1, clipped.x2, clipped.y2, 0.55, 16)
|
||||
const hl = selectedService?.id === edge.toId
|
||||
const svc = nestedServices.find((s) => s.id === edge.toId)
|
||||
const country = mapCountries.find((s) => s.id === edge.fromId)
|
||||
const clientLabel = (edge.clients?.map((c) => c.name).filter(Boolean).join(", ") || edge.clientName || "—")
|
||||
const pathTitle = `${clientLabel} → ${destDisplayLabel(country, "countries", edge.fromId)} → ${edge.toId.split("|")[1] ?? edge.toId}`
|
||||
return (
|
||||
<g
|
||||
key={`${edge.fromId}|${edge.toId}`}
|
||||
opacity={hl ? 1 : 0.72}
|
||||
style={{ transition: "opacity 0.3s" }}
|
||||
>
|
||||
<title>{pathTitle}</title>
|
||||
<line
|
||||
x1={clipped.x1} y1={clipped.y1} x2={clipped.x2} y2={clipped.y2}
|
||||
stroke="#22d3ee"
|
||||
strokeWidth={hopHasRate(hop) ? 2 : 1.3}
|
||||
strokeDasharray="4 4"
|
||||
opacity="0.9"
|
||||
pointerEvents="none"
|
||||
/>
|
||||
<line
|
||||
x1={clipped.x1} y1={clipped.y1} x2={clipped.x2} y2={clipped.y2}
|
||||
stroke="#00000000"
|
||||
strokeWidth={14}
|
||||
strokeLinecap="round"
|
||||
style={{ cursor: "pointer" }}
|
||||
onPointerDown={(ev) => { ev.stopPropagation() }}
|
||||
onClick={(ev) => {
|
||||
ev.stopPropagation()
|
||||
if (svc) selectService(svc)
|
||||
}}
|
||||
/>
|
||||
{showAnimDots && hopHasRate(hop) && (
|
||||
<circle r="3" fill="#67e8f9" opacity="0.85" pointerEvents="none">
|
||||
<animateMotion dur="2.6s" repeatCount="indefinite"
|
||||
path={`M ${clipped.x1} ${clipped.y1} L ${clipped.x2} ${clipped.y2}`} />
|
||||
</circle>
|
||||
)}
|
||||
{hopHasRate(hop) && (
|
||||
<NetflowRateBadge
|
||||
mx={mx}
|
||||
my={my}
|
||||
hop={hop}
|
||||
onOpen={(ev) => {
|
||||
ev.stopPropagation()
|
||||
const hit = nestedServices.find((s) => s.id === edge.toId)
|
||||
if (hit) selectService(hit)
|
||||
}}
|
||||
/>
|
||||
)}
|
||||
</g>
|
||||
)
|
||||
})}
|
||||
|
||||
{/* ── Hover tooltip ── */}
|
||||
{hoveredNode && !isDragging && (
|
||||
<SvgTooltip n={hoveredNode} />
|
||||
@@ -2403,7 +2747,9 @@ export default function NetworkMapPage() {
|
||||
|
||||
<g transform="translate(10, 164)">
|
||||
<rect width="14" height="14" rx="4" fill="#08202c" stroke="#22d3ee" strokeWidth="1.2" />
|
||||
<text x="22" y="11" fontSize="8.5" fill="#cbd5e1" fontFamily="system-ui">Сервис</text>
|
||||
<text x="22" y="11" fontSize="8.5" fill="#cbd5e1" fontFamily="system-ui">
|
||||
{destMode === "countries" ? "Страна" : "Сервис"}
|
||||
</text>
|
||||
</g>
|
||||
|
||||
<line x1="10" y1="186" x2="130" y2="186" stroke="rgba(255,255,255,0.07)" strokeWidth="1" />
|
||||
@@ -2467,7 +2813,7 @@ export default function NetworkMapPage() {
|
||||
satPos={effectiveSatPos}
|
||||
wanJhEdges={visibleWanJhEdges}
|
||||
homeRouters={homeRouters}
|
||||
servicePos={servicePosById}
|
||||
servicePos={{ ...servicePosById, ...nestedPosById }}
|
||||
onClose={() => setShowMinimap(false)}
|
||||
onPan={(x, y) => setPan({ x, y })}
|
||||
/>
|
||||
@@ -2702,16 +3048,87 @@ export default function NetworkMapPage() {
|
||||
})()}
|
||||
</div>
|
||||
</>
|
||||
) : selectedService ? (
|
||||
) : selectedNestedService ? (
|
||||
<>
|
||||
<div className="flex items-start gap-2 px-4 py-3 border-b">
|
||||
<div className="mt-0.5">
|
||||
<ServiceBrandIcon label={selectedService.label} size={22} />
|
||||
<ServiceBrandIcon label={selectedNestedService.label} size={22} />
|
||||
</div>
|
||||
<div className="flex-1 min-w-0">
|
||||
<p className="font-mono font-semibold text-sm truncate">{selectedService.label}</p>
|
||||
<p className="font-mono font-semibold text-sm truncate">
|
||||
{selectedNestedService.label}
|
||||
</p>
|
||||
<p className="text-xs text-muted-foreground mt-0.5">
|
||||
Конечный сервис · {selectedService.category}
|
||||
{`Сервис в ${destDisplayLabel(expandedCountry ?? undefined, "countries")} · ${selectedNestedService.category}`}
|
||||
</p>
|
||||
</div>
|
||||
<button
|
||||
type="button"
|
||||
onClick={() => setSelectedService(null)}
|
||||
className="text-muted-foreground hover:text-foreground transition-colors"
|
||||
>
|
||||
<XIcon className="size-4" />
|
||||
</button>
|
||||
</div>
|
||||
<div className="flex-1 overflow-y-auto px-4 py-4 flex flex-col gap-4">
|
||||
<div className="flex flex-col gap-0">
|
||||
<div className="flex items-center justify-between py-2 border-b border-border/50">
|
||||
<span className="text-xs text-muted-foreground">Доля страны</span>
|
||||
<span className="text-xs font-mono font-medium text-cyan-400">{serviceSharePct(selectedNestedService.share)}</span>
|
||||
</div>
|
||||
{mapTotalBytes > 0 && (
|
||||
<div className="flex items-center justify-between py-2 border-b border-border/50">
|
||||
<span className="text-xs text-muted-foreground">Доля окна</span>
|
||||
<span className="text-xs font-mono font-medium text-cyan-400">
|
||||
{serviceSharePct(selectedNestedService.bytes / mapTotalBytes)}
|
||||
</span>
|
||||
</div>
|
||||
)}
|
||||
<div className="flex items-center justify-between py-2 border-b border-border/50">
|
||||
<span className="text-xs text-muted-foreground">Скорость</span>
|
||||
<span className="text-xs font-mono font-medium">
|
||||
{formatNetflowRate({
|
||||
bytes: selectedNestedService.bytes,
|
||||
bps: selectedNestedService.bps,
|
||||
bpsFwd: selectedNestedService.bps,
|
||||
bpsRev: 0,
|
||||
})}
|
||||
</span>
|
||||
</div>
|
||||
</div>
|
||||
<div>
|
||||
<p className="text-xs font-semibold text-muted-foreground uppercase tracking-wider mb-2">Выход</p>
|
||||
<ServicePathList
|
||||
paths={(expandedCountryGroup?.paths ?? [])
|
||||
.filter((p) => p.serviceId === selectedNestedService.id)
|
||||
.slice()
|
||||
.sort((a, b) => b.bps - a.bps)}
|
||||
servers={mapServers}
|
||||
services={nestedServices}
|
||||
highlight={highlightedPath}
|
||||
viaMode="via"
|
||||
destMode="services"
|
||||
onToggle={togglePathHighlight}
|
||||
/>
|
||||
</div>
|
||||
</div>
|
||||
</>
|
||||
) : liveSelectedService ? (
|
||||
<>
|
||||
<div className="flex items-start gap-2 px-4 py-3 border-b">
|
||||
<div className="mt-0.5">
|
||||
{destMode === "countries" && liveSelectedService.label !== "Прочее" && liveSelectedService.id !== "cc:other"
|
||||
? <Flag code={liveSelectedService.label} size={22} />
|
||||
: <ServiceBrandIcon label={destMode === "countries" ? "Прочее" : liveSelectedService.label} size={22} />}
|
||||
</div>
|
||||
<div className="flex-1 min-w-0">
|
||||
<p className="font-mono font-semibold text-sm truncate">
|
||||
{destDisplayLabel(liveSelectedService, destMode)}
|
||||
</p>
|
||||
<p className="text-xs text-muted-foreground mt-0.5">
|
||||
{destMode === "countries"
|
||||
? `Конечная страна${liveSelectedService.label !== "Прочее" ? ` · ${liveSelectedService.label}` : ""}`
|
||||
: `Конечный сервис · ${liveSelectedService.category}`}
|
||||
</p>
|
||||
</div>
|
||||
<button
|
||||
@@ -2726,50 +3143,107 @@ export default function NetworkMapPage() {
|
||||
<div className="flex flex-col gap-0">
|
||||
<div className="flex items-center justify-between py-2 border-b border-border/50">
|
||||
<span className="text-xs text-muted-foreground">Доля окна</span>
|
||||
<span className="text-xs font-mono font-medium text-cyan-400">{serviceSharePct(selectedService.share)}</span>
|
||||
<span className="text-xs font-mono font-medium text-cyan-400">{serviceSharePct(liveSelectedService.share)}</span>
|
||||
</div>
|
||||
{mapTotalBytes > 0 && (
|
||||
<div className="flex items-center justify-between py-2 border-b border-border/50">
|
||||
<span className="text-xs text-muted-foreground">Классифицировано</span>
|
||||
<span className="text-xs font-mono font-medium text-cyan-400">
|
||||
{formatNetflowRate({
|
||||
bytes: mapNamedBytes,
|
||||
bps: (mapNamedBytes * 8) / Math.max(1, mapWindowSec),
|
||||
bpsFwd: 0,
|
||||
bpsRev: 0,
|
||||
})}
|
||||
{" из "}
|
||||
{formatNetflowRate({
|
||||
bytes: mapTotalBytes,
|
||||
bps: (mapTotalBytes * 8) / Math.max(1, mapWindowSec),
|
||||
bpsFwd: 0,
|
||||
bpsRev: 0,
|
||||
})}
|
||||
</span>
|
||||
</div>
|
||||
)}
|
||||
{!mapAsnLoaded && destMode === "services" && (
|
||||
<div className="flex items-center justify-between py-2 border-b border-border/50">
|
||||
<span className="text-xs text-muted-foreground">GeoLite2 ASN</span>
|
||||
<span className="text-xs font-mono font-medium text-amber-500">не загружена</span>
|
||||
</div>
|
||||
)}
|
||||
{destMode === "countries" && !mapCountryLoaded && (
|
||||
<div className="flex items-center justify-between py-2 border-b border-border/50">
|
||||
<span className="text-xs text-muted-foreground">GeoIP Country</span>
|
||||
<span className="text-xs font-mono font-medium text-amber-500">RIPE-кэш</span>
|
||||
</div>
|
||||
)}
|
||||
<div className="flex items-center justify-between py-2 border-b border-border/50">
|
||||
<span className="text-xs text-muted-foreground">Скорость</span>
|
||||
<span className="text-xs font-mono font-medium">
|
||||
{formatNetflowRate({
|
||||
bytes: selectedService.bytes,
|
||||
bps: selectedService.bps,
|
||||
bpsFwd: selectedService.bps,
|
||||
bytes: liveSelectedService.bytes,
|
||||
bps: liveSelectedService.bps,
|
||||
bpsFwd: liveSelectedService.bps,
|
||||
bpsRev: 0,
|
||||
})}
|
||||
</span>
|
||||
</div>
|
||||
</div>
|
||||
{expandedCountryGroup && liveSelectedService.id === expandedCountryGroup.countryId && (
|
||||
<div>
|
||||
<p className="text-xs font-semibold text-muted-foreground uppercase tracking-wider mb-2">Сервисы в стране</p>
|
||||
<div className="flex flex-col gap-1">
|
||||
{expandedCountryGroup.services.map((svc) => (
|
||||
<button
|
||||
key={svc.id}
|
||||
type="button"
|
||||
onClick={() => selectService(svc)}
|
||||
className={cn(
|
||||
"flex items-center justify-between gap-2 rounded-md px-2 py-1.5 text-left text-xs transition-colors",
|
||||
selectedService?.id === svc.id ? "bg-cyan-500/15 ring-1 ring-cyan-500/40" : "hover:bg-muted/50",
|
||||
)}
|
||||
>
|
||||
<span className="flex items-center gap-1.5 min-w-0">
|
||||
<ServiceBrandIcon label={svc.label} size={14} />
|
||||
<span className="font-mono truncate">{svc.label}</span>
|
||||
</span>
|
||||
<span className="font-mono text-cyan-400 tabular-nums shrink-0">{serviceSharePct(svc.share)}</span>
|
||||
</button>
|
||||
))}
|
||||
</div>
|
||||
</div>
|
||||
)}
|
||||
<div>
|
||||
<p className="text-xs font-semibold text-muted-foreground uppercase tracking-wider mb-2">Выход</p>
|
||||
<div className="flex flex-col gap-1.5">
|
||||
{visibleServiceEdges.filter((e) => e.toId === selectedService.id).map((e) => {
|
||||
<div className="flex flex-col gap-3">
|
||||
{visibleServiceEdges.filter((e) => e.toId === liveSelectedService.id).map((e) => {
|
||||
const src = mapServers.find((s) => s.id === e.fromId)
|
||||
const enPaths = destPaths
|
||||
.filter((p) => p.serviceId === liveSelectedService.id && p.enId === e.fromId)
|
||||
.slice()
|
||||
.sort((a, b) => b.bps - a.bps)
|
||||
return (
|
||||
<div key={`${e.fromId}|${e.toId}`} className="flex items-center justify-between text-xs">
|
||||
<span className="font-mono truncate">{src?.name ?? e.fromId}</span>
|
||||
<span className="font-mono text-emerald-400 tabular-nums">
|
||||
{formatNetflowRate({ bytes: e.bytes, bps: e.bps, bpsFwd: e.bpsFwd, bpsRev: e.bpsRev })}
|
||||
</span>
|
||||
<div key={`${e.fromId}|${e.toId}`} className="flex flex-col gap-1.5">
|
||||
<div className="flex items-center justify-between text-xs">
|
||||
<span className="font-mono truncate">{src?.name ?? e.fromId}</span>
|
||||
<span className="font-mono text-emerald-400 tabular-nums">
|
||||
{formatNetflowRate({ bytes: e.bytes, bps: e.bps, bpsFwd: e.bpsFwd, bpsRev: e.bpsRev })}
|
||||
</span>
|
||||
</div>
|
||||
<ServicePathList
|
||||
paths={enPaths}
|
||||
servers={mapServers}
|
||||
services={destNodes}
|
||||
highlight={highlightedPath}
|
||||
viaMode="via"
|
||||
destMode={destMode}
|
||||
onToggle={togglePathHighlight}
|
||||
/>
|
||||
</div>
|
||||
)
|
||||
})}
|
||||
</div>
|
||||
</div>
|
||||
<div>
|
||||
<p className="text-xs font-semibold text-muted-foreground uppercase tracking-wider mb-2">Пути</p>
|
||||
<ServicePathList
|
||||
paths={mapServicePaths
|
||||
.filter((p) => p.serviceId === selectedService.id)
|
||||
.slice()
|
||||
.sort((a, b) => b.bps - a.bps)}
|
||||
servers={mapServers}
|
||||
services={mapServices}
|
||||
highlight={highlightedPath}
|
||||
viaMode="via"
|
||||
onToggle={togglePathHighlight}
|
||||
/>
|
||||
</div>
|
||||
</div>
|
||||
</>
|
||||
) : selected ? (
|
||||
@@ -3001,14 +3475,19 @@ export default function NetworkMapPage() {
|
||||
<div>
|
||||
<p className="text-xs font-semibold text-muted-foreground uppercase tracking-wider mb-2">Пути</p>
|
||||
<ServicePathList
|
||||
paths={mapServicePaths
|
||||
.filter((p) => p.viaId === selected.id || p.enId === selected.id)
|
||||
paths={destPaths
|
||||
.filter((p) => (
|
||||
selected.type === "exit-node"
|
||||
? p.enId === selected.id
|
||||
: p.viaId === selected.id
|
||||
))
|
||||
.slice()
|
||||
.sort((a, b) => b.bps - a.bps)}
|
||||
servers={mapServers}
|
||||
services={mapServices}
|
||||
services={destNodes}
|
||||
highlight={highlightedPath}
|
||||
viaMode="service"
|
||||
destMode={destMode}
|
||||
onToggle={togglePathHighlight}
|
||||
/>
|
||||
</div>
|
||||
|
||||
@@ -122,11 +122,17 @@ interface BackendBfdSession {
|
||||
packetsRx: number; packetsTx: number; stateChanges: number
|
||||
}
|
||||
|
||||
interface BackendOspfRoute {
|
||||
id: string; serverId: number; serverName: string; serverSite: string
|
||||
destination: string; type: OspfRoute["type"]; cost: number; nextHop: string; via: string; area: string
|
||||
}
|
||||
|
||||
interface BackendOspfAll {
|
||||
neighbors: BackendNeighbor[]
|
||||
interfaces: BackendInterface[]
|
||||
instances: BackendInstance[]
|
||||
bfdSessions: BackendBfdSession[]
|
||||
routes?: BackendOspfRoute[]
|
||||
}
|
||||
|
||||
function isRefInterfaceName(name: string): boolean {
|
||||
@@ -213,6 +219,22 @@ function backendToBfdSession(b: BackendBfdSession): BfdSession {
|
||||
}
|
||||
}
|
||||
|
||||
function backendToRoute(b: BackendOspfRoute): OspfRoute {
|
||||
const allowed: OspfRoute["type"][] = ["O", "O IA", "O E1", "O E2"]
|
||||
const type = allowed.includes(b.type) ? b.type : "O"
|
||||
return {
|
||||
id: `${b.serverId}-${b.id}`,
|
||||
destination: b.destination,
|
||||
type,
|
||||
cost: b.cost,
|
||||
nextHop: b.nextHop,
|
||||
via: b.via,
|
||||
serverId: String(b.serverId),
|
||||
serverLabel: b.serverName,
|
||||
area: b.area || "—",
|
||||
}
|
||||
}
|
||||
|
||||
// ─── mock data ────────────────────────────────────────────────────────────────
|
||||
|
||||
const COST_STEP = 10
|
||||
@@ -1172,7 +1194,7 @@ export default function OspfPage() {
|
||||
}, [isLive, backendUrl, fetchTick])
|
||||
|
||||
// Derive frontend types from backend data or use mocks
|
||||
const { items, neighbors, graphNodes, graphEdges, routerIds, bfdSessions } = useMemo(() => {
|
||||
const { items, neighbors, graphNodes, graphEdges, routerIds, bfdSessions, routes } = useMemo(() => {
|
||||
if (isLive && liveData) {
|
||||
// Build interface→cost map for neighbor cost lookup
|
||||
const ifaceMap = new Map<string, number>()
|
||||
@@ -1185,6 +1207,7 @@ export default function OspfPage() {
|
||||
.filter((item) => !isRefInterfaceName(item.interfaceName))
|
||||
const neighbors = liveData.neighbors.map(b => backendToNeighbor(b, ifaceMap))
|
||||
const bfdSessions = (liveData.bfdSessions ?? []).map(backendToBfdSession)
|
||||
const routes = (liveData.routes ?? []).map(backendToRoute)
|
||||
|
||||
// Build routerIds from instances
|
||||
const routerIds: Record<string, string> = {}
|
||||
@@ -1195,7 +1218,7 @@ export default function OspfPage() {
|
||||
}
|
||||
|
||||
const { nodes: graphNodes, edges: graphEdges } = buildLiveGraph(neighbors)
|
||||
return { items, neighbors, graphNodes, graphEdges, routerIds, bfdSessions }
|
||||
return { items, neighbors, graphNodes, graphEdges, routerIds, bfdSessions, routes }
|
||||
}
|
||||
if (isLive) {
|
||||
return {
|
||||
@@ -1205,6 +1228,7 @@ export default function OspfPage() {
|
||||
graphEdges: [],
|
||||
routerIds: {},
|
||||
bfdSessions: [],
|
||||
routes: [],
|
||||
}
|
||||
}
|
||||
return {
|
||||
@@ -1214,6 +1238,7 @@ export default function OspfPage() {
|
||||
graphEdges: MOCK_GRAPH_EDGES,
|
||||
routerIds: MOCK_ROUTER_IDS,
|
||||
bfdSessions: MOCK_BFD,
|
||||
routes: MOCK_ROUTES,
|
||||
}
|
||||
}, [isLive, liveData])
|
||||
|
||||
@@ -1257,6 +1282,7 @@ export default function OspfPage() {
|
||||
const displayItems = filterServerId === ALL_SERVERS_ID ? items : items.filter(i => i.routerKey === filterServerId)
|
||||
const displayNeighbors = filterServerId === ALL_SERVERS_ID ? neighbors : neighbors.filter(n => n.localRouter === filterServerId)
|
||||
const displayBfdSessions = filterServerId === ALL_SERVERS_ID ? bfdSessions : bfdSessions.filter(b => b.serverId === filterServerId)
|
||||
const displayRoutes = filterServerId === ALL_SERVERS_ID ? routes : routes.filter(r => r.serverId === filterServerId)
|
||||
|
||||
const ospfRailItems = useMemo<ServerTileItem[]>(() => (
|
||||
ospfServers.map((s) => {
|
||||
@@ -1398,7 +1424,7 @@ export default function OspfPage() {
|
||||
routerIds={routerIds}
|
||||
/>
|
||||
)}
|
||||
{activeTab === "routes" && <RoutesTab routes={isLive ? [] : MOCK_ROUTES} />}
|
||||
{activeTab === "routes" && <RoutesTab routes={displayRoutes} />}
|
||||
{activeTab === "bfd" && <BfdTab sessions={displayBfdSessions} />}
|
||||
|
||||
</div>
|
||||
|
||||
@@ -18,9 +18,12 @@ import { Flag } from "@/components/flag"
|
||||
import { useDataSource } from "@/lib/data-source"
|
||||
import { cn } from "@/lib/utils"
|
||||
import { servers as mockServers, type Server } from "@/lib/data"
|
||||
import { PlusIcon, SaveIcon, TrashIcon, SearchIcon, XIcon, PencilIcon, CheckIcon, AlertCircleIcon } from "lucide-react"
|
||||
import { PlusIcon, TrashIcon, SearchIcon, XIcon, PencilIcon, CheckIcon, AlertCircleIcon, RefreshCwIcon, HistoryIcon, AlertTriangleIcon } from "lucide-react"
|
||||
import { toast } from "sonner"
|
||||
import { requestJson } from "@/shared/api/http-client"
|
||||
import { ServerRailLayout, ServerRailMobileButton } from "@/components/server-rail-layout"
|
||||
import { ConfigHistorySheet } from "@/components/config-history-sheet"
|
||||
import type { ConfigRevisionDto } from "@/lib/config-revisions"
|
||||
import { type ServerTileItem } from "@/components/server-tile-rail"
|
||||
|
||||
interface BackendServer {
|
||||
@@ -364,7 +367,7 @@ export default function RecursiveRoutesPage() {
|
||||
const [servers, setServers] = useState<Server[]>([])
|
||||
const [selectedServerId, setSelectedServerId] = useState<string>("")
|
||||
const [rows, setRows] = useState<RecursiveRouteRow[]>([])
|
||||
const [busy, setBusy] = useState<"load" | "save" | "from" | "to" | null>(null)
|
||||
const [busy, setBusy] = useState<"load" | "apply" | null>(null)
|
||||
const [search, setSearch] = useState("")
|
||||
const [sheetOpen, setSheetOpen] = useState(false)
|
||||
const [sheetMode, setSheetMode] = useState<"create" | "edit">("create")
|
||||
@@ -373,6 +376,11 @@ export default function RecursiveRoutesPage() {
|
||||
const [gatewayOptions, setGatewayOptions] = useState<GatewayOption[]>([])
|
||||
const [expandedGroupKey, setExpandedGroupKey] = useState<string | null>(null)
|
||||
const [opError, setOpError] = useState<string | null>(null)
|
||||
const [liveStale, setLiveStale] = useState(false)
|
||||
const [historyOpen, setHistoryOpen] = useState(false)
|
||||
const [historyLoading, setHistoryLoading] = useState(false)
|
||||
const [historyRestoring, setHistoryRestoring] = useState(false)
|
||||
const [revisions, setRevisions] = useState<ConfigRevisionDto[]>([])
|
||||
/** В live не дергаем API с id мока (srv1…) пока не подтянули /api/servers */
|
||||
const [liveServerListReady, setLiveServerListReady] = useState(false)
|
||||
|
||||
@@ -407,10 +415,13 @@ export default function RecursiveRoutesPage() {
|
||||
setOpError(null)
|
||||
setBusy("load")
|
||||
try {
|
||||
const res = await apiFetch<{ routes: RecursiveRouteRow[] }>(`/api/recursive-routes?serverId=${selectedServerId}`)
|
||||
const res = await apiFetch<{ routes: RecursiveRouteRow[]; stale?: boolean }>(
|
||||
`/api/recursive-routes?serverId=${selectedServerId}`,
|
||||
)
|
||||
setRows(res.routes)
|
||||
setLiveStale(Boolean(res.stale))
|
||||
} catch (e) {
|
||||
setRows([])
|
||||
setLiveStale(true)
|
||||
setOpError(e instanceof Error ? e.message : "Не удалось загрузить маршруты")
|
||||
} finally {
|
||||
setBusy(null)
|
||||
@@ -437,56 +448,65 @@ export default function RecursiveRoutesPage() {
|
||||
void loadGateways()
|
||||
}, [loadGateways])
|
||||
|
||||
const saveToDb = useCallback(async () => {
|
||||
if (!isLive || !selectedServerId) return
|
||||
const applyRoutes = useCallback(async (next: RecursiveRouteRow[]) => {
|
||||
if (!isLive || !selectedServerId || liveStale) return
|
||||
const prev = rows
|
||||
setRows(next)
|
||||
setOpError(null)
|
||||
setBusy("save")
|
||||
setBusy("apply")
|
||||
try {
|
||||
await apiFetch<{ ok: boolean }>("/api/recursive-routes", {
|
||||
const res = await apiFetch<{ ok: boolean; routes?: RecursiveRouteRow[] }>("/api/recursive-routes", {
|
||||
method: "PUT",
|
||||
body: JSON.stringify({ serverId: selectedServerId, routes: rows }),
|
||||
body: JSON.stringify({ serverId: selectedServerId, routes: next }),
|
||||
})
|
||||
await loadRoutes()
|
||||
setRows(res.routes ?? next)
|
||||
setLiveStale(false)
|
||||
toast.success("Маршруты применены на роутер")
|
||||
} catch (e) {
|
||||
setOpError(e instanceof Error ? e.message : "Не удалось сохранить маршруты в БД")
|
||||
setRows(prev)
|
||||
const msg = e instanceof Error ? e.message : "Не удалось применить маршруты на роутер"
|
||||
setOpError(msg)
|
||||
toast.error(msg)
|
||||
} finally {
|
||||
setBusy(null)
|
||||
}
|
||||
}, [isLive, selectedServerId, rows, apiFetch, loadRoutes])
|
||||
}, [isLive, selectedServerId, liveStale, rows, apiFetch])
|
||||
|
||||
const syncFromRouter = useCallback(async () => {
|
||||
const loadRevisions = useCallback(async () => {
|
||||
if (!isLive || !selectedServerId) return
|
||||
setOpError(null)
|
||||
setBusy("from")
|
||||
setHistoryLoading(true)
|
||||
try {
|
||||
await apiFetch<{ ok: boolean }>("/api/recursive-routes/sync/from-router", {
|
||||
method: "POST",
|
||||
body: JSON.stringify({ serverId: selectedServerId }),
|
||||
})
|
||||
await loadRoutes()
|
||||
const res = await apiFetch<{ revisions: ConfigRevisionDto[] }>(
|
||||
`/api/recursive-routes/revisions?serverId=${encodeURIComponent(selectedServerId)}`,
|
||||
)
|
||||
setRevisions(res.revisions)
|
||||
} catch (e) {
|
||||
setOpError(e instanceof Error ? e.message : "Не удалось синхронизировать маршруты с роутера")
|
||||
toast.error(e instanceof Error ? e.message : "Не удалось загрузить историю")
|
||||
setRevisions([])
|
||||
} finally {
|
||||
setBusy(null)
|
||||
}
|
||||
}, [isLive, selectedServerId, apiFetch, loadRoutes])
|
||||
|
||||
const syncToRouter = useCallback(async () => {
|
||||
if (!isLive || !selectedServerId) return
|
||||
setOpError(null)
|
||||
setBusy("to")
|
||||
try {
|
||||
await apiFetch<{ ok: boolean }>("/api/recursive-routes/sync/to-router", {
|
||||
method: "POST",
|
||||
body: JSON.stringify({ serverId: selectedServerId }),
|
||||
})
|
||||
} catch (e) {
|
||||
setOpError(e instanceof Error ? e.message : "Не удалось применить маршруты на роутер")
|
||||
} finally {
|
||||
setBusy(null)
|
||||
setHistoryLoading(false)
|
||||
}
|
||||
}, [isLive, selectedServerId, apiFetch])
|
||||
|
||||
const restoreRevision = useCallback(async (id: string) => {
|
||||
if (!isLive || !selectedServerId) return
|
||||
setHistoryRestoring(true)
|
||||
try {
|
||||
const res = await apiFetch<{ ok: boolean; routes?: RecursiveRouteRow[] }>(
|
||||
`/api/recursive-routes/revisions/${encodeURIComponent(id)}/restore`,
|
||||
{ method: "POST", body: JSON.stringify({ serverId: selectedServerId }) },
|
||||
)
|
||||
setRows(res.routes ?? [])
|
||||
setLiveStale(false)
|
||||
toast.success("Версия применена на роутер")
|
||||
await loadRevisions()
|
||||
} catch (e) {
|
||||
toast.error(e instanceof Error ? e.message : "Не удалось откатить")
|
||||
} finally {
|
||||
setHistoryRestoring(false)
|
||||
}
|
||||
}, [isLive, selectedServerId, apiFetch, loadRevisions])
|
||||
|
||||
function groupKeyOf(row: RecursiveRouteRow): string {
|
||||
return row.dstAddress.trim().toLowerCase()
|
||||
}
|
||||
@@ -529,22 +549,26 @@ export default function RecursiveRoutesPage() {
|
||||
disabled: false,
|
||||
country: ep.country || inferCountry(ep.gateway) || "",
|
||||
})
|
||||
let next: RecursiveRouteRow[]
|
||||
if (sheetMode === "create") {
|
||||
const base = `new-${Date.now()}-${Math.random().toString(36).slice(2, 6)}`
|
||||
const expanded = v.endpoints.map((ep, i) => toRow(ep, `${base}-${i}`))
|
||||
setRows(prev => [...prev, ...expanded])
|
||||
next = [...rows, ...expanded]
|
||||
} else if (editingGroupKey) {
|
||||
setRows(prev => {
|
||||
const kept = prev.filter(r => groupKeyOf(r) !== editingGroupKey)
|
||||
const base = `edit-${Date.now()}-${Math.random().toString(36).slice(2, 6)}`
|
||||
const expanded = v.endpoints.map((ep, i) => toRow(ep, `${base}-${i}`))
|
||||
return [...kept, ...expanded]
|
||||
})
|
||||
const kept = rows.filter(r => groupKeyOf(r) !== editingGroupKey)
|
||||
const base = `edit-${Date.now()}-${Math.random().toString(36).slice(2, 6)}`
|
||||
const expanded = v.endpoints.map((ep, i) => toRow(ep, `${base}-${i}`))
|
||||
next = [...kept, ...expanded]
|
||||
} else {
|
||||
setSheetOpen(false)
|
||||
return
|
||||
}
|
||||
setSheetOpen(false)
|
||||
void applyRoutes(next)
|
||||
}
|
||||
|
||||
const currentServer = servers.find(s => s.id === selectedServerId)
|
||||
const mutationsLocked = !isLive || busy !== null || liveStale
|
||||
const rrRailItems = useMemo<ServerTileItem[]>(() => (
|
||||
servers.map((s) => ({
|
||||
id: s.id,
|
||||
@@ -602,16 +626,33 @@ export default function RecursiveRoutesPage() {
|
||||
actions={
|
||||
<>
|
||||
<ServerRailMobileButton />
|
||||
<Button variant="outline" size="sm" onClick={syncFromRouter} disabled={!isLive || busy !== null}>
|
||||
{busy === "from" ? "Синхронизация..." : "Router => DB"}
|
||||
</Button>
|
||||
<Button variant="outline" size="sm" onClick={syncToRouter} disabled={!isLive || busy !== null}>
|
||||
{busy === "to" ? "Применение..." : "DB => Router"}
|
||||
</Button>
|
||||
<Button variant="outline" size="sm" onClick={saveToDb} disabled={!isLive || busy !== null}>
|
||||
<SaveIcon className="size-4" />Сохранить в БД
|
||||
</Button>
|
||||
<Button size="sm" onClick={openCreate} disabled={!isLive || busy !== null}>
|
||||
{isLive && (
|
||||
<>
|
||||
<Button
|
||||
variant="outline"
|
||||
size="sm"
|
||||
onClick={() => void loadRoutes()}
|
||||
disabled={busy !== null}
|
||||
title="Прочитать маршруты с роутера"
|
||||
>
|
||||
<RefreshCwIcon className={cn("size-4", busy === "load" && "animate-spin")} />
|
||||
Обновить
|
||||
</Button>
|
||||
<Button
|
||||
variant="outline"
|
||||
size="sm"
|
||||
onClick={() => {
|
||||
setHistoryOpen(true)
|
||||
void loadRevisions()
|
||||
}}
|
||||
disabled={busy !== null}
|
||||
>
|
||||
<HistoryIcon className="size-4" />
|
||||
История
|
||||
</Button>
|
||||
</>
|
||||
)}
|
||||
<Button size="sm" onClick={openCreate} disabled={mutationsLocked}>
|
||||
<PlusIcon className="size-4" />Добавить
|
||||
</Button>
|
||||
</>
|
||||
@@ -643,6 +684,12 @@ export default function RecursiveRoutesPage() {
|
||||
{opError}
|
||||
</div>
|
||||
)}
|
||||
{liveStale && (
|
||||
<div className="w-full text-xs text-amber-700 dark:text-amber-400 bg-amber-500/10 border border-amber-500/20 rounded-md px-3 py-2 flex items-center gap-2">
|
||||
<AlertTriangleIcon className="size-3.5 shrink-0" />
|
||||
Роутер недоступен — показан кэш. Изменения заблокированы.
|
||||
</div>
|
||||
)}
|
||||
</div>
|
||||
}
|
||||
>
|
||||
@@ -679,10 +726,13 @@ export default function RecursiveRoutesPage() {
|
||||
expandedKey={expandedGroupKey}
|
||||
onExpandedChange={setExpandedGroupKey}
|
||||
onEdit={openEdit}
|
||||
onDelete={(g) => setRows((prev) => prev.filter((r) => groupKeyOf(r) !== g.key))}
|
||||
onDelete={(g) => {
|
||||
if (mutationsLocked) return
|
||||
void applyRoutes(rows.filter((r) => groupKeyOf(r) !== g.key))
|
||||
}}
|
||||
/>
|
||||
<button onClick={openCreate}
|
||||
className="w-full flex items-center gap-2 px-5 py-2 text-xs text-muted-foreground hover:text-foreground hover:bg-muted/20 transition-colors border-t">
|
||||
<button onClick={openCreate} disabled={mutationsLocked}
|
||||
className="w-full flex items-center gap-2 px-5 py-2 text-xs text-muted-foreground hover:text-foreground hover:bg-muted/20 transition-colors border-t disabled:opacity-50">
|
||||
<PlusIcon className="size-3.5" />
|
||||
Добавить маршрут
|
||||
</button>
|
||||
@@ -698,6 +748,17 @@ export default function RecursiveRoutesPage() {
|
||||
onClose={() => setSheetOpen(false)}
|
||||
gateways={gatewayOptions}
|
||||
/>
|
||||
|
||||
<ConfigHistorySheet
|
||||
open={historyOpen}
|
||||
onOpenChange={setHistoryOpen}
|
||||
title="История маршрутов"
|
||||
itemLabel="маршрутов"
|
||||
revisions={revisions}
|
||||
loading={historyLoading}
|
||||
restoring={historyRestoring}
|
||||
onRestore={restoreRevision}
|
||||
/>
|
||||
</>
|
||||
)
|
||||
}
|
||||
|
||||
@@ -0,0 +1,558 @@
|
||||
"use client"
|
||||
|
||||
import { Suspense, useCallback, useEffect, useMemo, useState } from "react"
|
||||
import { useSearchParams } from "next/navigation"
|
||||
import {
|
||||
ActivityIcon,
|
||||
DatabaseIcon,
|
||||
GaugeIcon,
|
||||
ServerIcon,
|
||||
UsersIcon,
|
||||
} from "lucide-react"
|
||||
import { PageHeader } from "@/components/page-header"
|
||||
import { KpiStatGrid } from "@/components/reui-kit/kpi-stat-grid"
|
||||
import { DataPageCard } from "@/components/data-page-card"
|
||||
import { DataPageToolbar } from "@/components/data-page-toolbar"
|
||||
import { SegmentedControl } from "@/components/form-kit"
|
||||
import { EmptyState } from "@/components/empty-state"
|
||||
import { PeriodSelector, rangeForPreset, type DateRangeYmd } from "@/components/statistics/period-selector"
|
||||
import { StatisticsVolumeChart } from "@/components/statistics/statistics-volume-chart"
|
||||
import { DimensionSelect, PivotDimSelect } from "@/components/statistics/dimension-select"
|
||||
import { SliceChips } from "@/components/statistics/slice-chips"
|
||||
import { BreakdownDashboard } from "@/components/statistics/breakdown-dashboard"
|
||||
import { StatisticsPivotGrid } from "@/components/statistics/statistics-pivot-grid"
|
||||
import {
|
||||
StatisticsBreakdownDataGrid,
|
||||
type StatisticsSliceKind,
|
||||
} from "@/components/data-grids/statistics-breakdown-data-grid"
|
||||
import { Alert, AlertDescription, AlertTitle } from "@/components/reui/alert"
|
||||
import type { Filter } from "@/components/reui/filters"
|
||||
import { STATISTICS_FILTER_FIELDS } from "@/lib/data-filters/statistics-filter-fields"
|
||||
import {
|
||||
isStatisticsPivotDim,
|
||||
isStatisticsSliceKind,
|
||||
STATISTICS_DIMS,
|
||||
} from "@/lib/statistics-dims"
|
||||
import { useDataSource } from "@/lib/data-source"
|
||||
import { fmtBps, formatBytes } from "@/lib/fmt-rate"
|
||||
import {
|
||||
getStatistics,
|
||||
getStatisticsPivot,
|
||||
STATISTICS_UNBOUND_USER_ID,
|
||||
type StatisticsDto,
|
||||
type StatisticsPivotDto,
|
||||
type StatisticsQuery,
|
||||
} from "@/shared/api/statistics"
|
||||
import type { StatisticsBreakdownRow, StatisticsPivotDim } from "@mmapp/contracts/statistics"
|
||||
|
||||
/**
|
||||
* BI-куб трафика: критерий → остальные разрезы + pivot.
|
||||
* Preview: https://reui.io/preview/base/dashboard-1 · https://reui.io/preview/base/stats-12
|
||||
* · https://reui.io/preview/base/data-grid-filtering-2 · https://reui.io/preview/base/solution-analytics-8
|
||||
* · https://reui.io/docs/components/base/frame · https://reui.io/docs/components/base/data-grid
|
||||
*/
|
||||
|
||||
const EMPTY: StatisticsDto = {
|
||||
from: "",
|
||||
to: "",
|
||||
grain: "day",
|
||||
kpis: {
|
||||
bytes: 0,
|
||||
packets: 0,
|
||||
avgBps: 0,
|
||||
users: 0,
|
||||
servers: 0,
|
||||
ifaces: 0,
|
||||
topCountry: "",
|
||||
topService: "",
|
||||
},
|
||||
series: [],
|
||||
users: [],
|
||||
servers: [],
|
||||
interfaces: [],
|
||||
countries: [],
|
||||
services: [],
|
||||
asns: [],
|
||||
}
|
||||
|
||||
const EMPTY_PIVOT: StatisticsPivotDto = {
|
||||
rowDim: "country",
|
||||
colDim: "service",
|
||||
metric: "bytes",
|
||||
columns: [],
|
||||
rows: [],
|
||||
otherBytes: 0,
|
||||
}
|
||||
|
||||
interface CubeSlices {
|
||||
country?: string
|
||||
service?: string
|
||||
asn?: string
|
||||
serverId?: string
|
||||
userId?: string
|
||||
iface?: string
|
||||
}
|
||||
|
||||
const SLICE_KEYS = ["country", "service", "asn", "serverId", "userId", "iface"] as const
|
||||
|
||||
function readRange(sp: URLSearchParams): DateRangeYmd {
|
||||
const from = sp.get("from")
|
||||
const to = sp.get("to")
|
||||
if (from && to && from <= to) return { from, to }
|
||||
return rangeForPreset("7d")
|
||||
}
|
||||
|
||||
function readDim(sp: URLSearchParams): StatisticsSliceKind {
|
||||
const t = sp.get("dim") ?? sp.get("tab")
|
||||
return t && isStatisticsSliceKind(t) ? t : "users"
|
||||
}
|
||||
|
||||
function readView(sp: URLSearchParams): "explore" | "pivot" {
|
||||
return sp.get("view") === "pivot" ? "pivot" : "explore"
|
||||
}
|
||||
|
||||
function readPlanes(sp: URLSearchParams): "unique" | "all" {
|
||||
return sp.get("planes") === "all" ? "all" : "unique"
|
||||
}
|
||||
|
||||
function readPivotDim(sp: URLSearchParams, key: string, fallback: StatisticsPivotDim): StatisticsPivotDim {
|
||||
const v = sp.get(key)
|
||||
return v && isStatisticsPivotDim(v) ? v : fallback
|
||||
}
|
||||
|
||||
function readSlices(sp: URLSearchParams): CubeSlices {
|
||||
const next: CubeSlices = {}
|
||||
for (const key of SLICE_KEYS) {
|
||||
const v = sp.get(key)?.trim()
|
||||
if (v) next[key] = v
|
||||
}
|
||||
return next
|
||||
}
|
||||
|
||||
function slicesToFilters(slices: CubeSlices): Filter[] {
|
||||
return SLICE_KEYS.flatMap((key) => {
|
||||
const val = slices[key]
|
||||
if (!val) return []
|
||||
return [{ id: key, field: key, operator: "is", values: [val] }]
|
||||
})
|
||||
}
|
||||
|
||||
function filtersToSlices(filters: Filter[]): CubeSlices {
|
||||
const next: CubeSlices = {}
|
||||
for (const f of filters) {
|
||||
const raw = String(f.values[0] ?? "").trim()
|
||||
if (!raw) continue
|
||||
if (f.field === "country") next.country = raw.toUpperCase().slice(0, 2)
|
||||
else if (f.field === "service") next.service = raw
|
||||
else if (f.field === "asn") next.asn = raw.replace(/[^\d]/g, "")
|
||||
else if (f.field === "serverId") next.serverId = raw
|
||||
else if (f.field === "userId") next.userId = raw
|
||||
else if (f.field === "iface") next.iface = raw
|
||||
}
|
||||
return next
|
||||
}
|
||||
|
||||
function toQuery(range: DateRangeYmd, slices: CubeSlices, planes: "unique" | "all"): StatisticsQuery {
|
||||
const serverId = slices.serverId ? Number(slices.serverId) : undefined
|
||||
const asn = slices.asn != null && slices.asn !== "" ? Number(slices.asn) : undefined
|
||||
return {
|
||||
from: range.from,
|
||||
to: range.to,
|
||||
serverId: Number.isFinite(serverId) && (serverId ?? 0) > 0 ? serverId : undefined,
|
||||
userId: slices.userId,
|
||||
iface: slices.iface,
|
||||
country: slices.country && slices.country.length === 2 ? slices.country : undefined,
|
||||
service: slices.service,
|
||||
asn: Number.isFinite(asn) ? asn : undefined,
|
||||
planes,
|
||||
}
|
||||
}
|
||||
|
||||
function selectedIdForKind(kind: StatisticsSliceKind, slices: CubeSlices): string | undefined {
|
||||
if (kind === "users") return slices.userId
|
||||
if (kind === "servers") return slices.serverId
|
||||
if (kind === "countries") return slices.country
|
||||
if (kind === "services") return slices.service
|
||||
if (kind === "asns") return slices.asn
|
||||
if (kind === "interfaces" && slices.serverId && slices.iface) {
|
||||
return `${slices.serverId}:${slices.iface}`
|
||||
}
|
||||
if (kind === "interfaces") return slices.iface
|
||||
return undefined
|
||||
}
|
||||
|
||||
function rowsForKind(data: StatisticsDto, kind: StatisticsSliceKind) {
|
||||
if (kind === "users") return data.users
|
||||
if (kind === "servers") return data.servers
|
||||
if (kind === "interfaces") return data.interfaces
|
||||
if (kind === "countries") return data.countries
|
||||
if (kind === "services") return data.services
|
||||
return data.asns
|
||||
}
|
||||
|
||||
function hasAnySlice(slices: CubeSlices): boolean {
|
||||
return SLICE_KEYS.some((k) => Boolean(slices[k]))
|
||||
}
|
||||
|
||||
function hiddenKinds(slices: CubeSlices): Set<StatisticsSliceKind> {
|
||||
const hidden = new Set<StatisticsSliceKind>()
|
||||
if (slices.userId) hidden.add("users")
|
||||
if (slices.serverId) hidden.add("servers")
|
||||
if (slices.iface) hidden.add("interfaces")
|
||||
if (slices.country) hidden.add("countries")
|
||||
if (slices.service) hidden.add("services")
|
||||
if (slices.asn) hidden.add("asns")
|
||||
return hidden
|
||||
}
|
||||
|
||||
function applyDimValue(slices: CubeSlices, kind: StatisticsSliceKind, rowId: string): CubeSlices {
|
||||
const next: CubeSlices = { ...slices }
|
||||
if (kind === "users") {
|
||||
if (rowId === STATISTICS_UNBOUND_USER_ID) return next
|
||||
if (next.userId === rowId) delete next.userId
|
||||
else next.userId = rowId
|
||||
} else if (kind === "servers") {
|
||||
if (next.serverId === rowId) delete next.serverId
|
||||
else next.serverId = rowId
|
||||
} else if (kind === "countries") {
|
||||
if (next.country === rowId) delete next.country
|
||||
else next.country = rowId
|
||||
} else if (kind === "services") {
|
||||
if (next.service === rowId) delete next.service
|
||||
else next.service = rowId
|
||||
} else if (kind === "asns") {
|
||||
if (next.asn === rowId) delete next.asn
|
||||
else next.asn = rowId
|
||||
} else {
|
||||
const colon = rowId.indexOf(":")
|
||||
const sid = colon >= 0 ? rowId.slice(0, colon) : undefined
|
||||
const iface = colon >= 0 ? rowId.slice(colon + 1) : rowId
|
||||
if (next.iface === iface && next.serverId === sid) {
|
||||
delete next.iface
|
||||
delete next.serverId
|
||||
} else {
|
||||
next.iface = iface
|
||||
if (sid) next.serverId = sid
|
||||
}
|
||||
}
|
||||
return next
|
||||
}
|
||||
|
||||
function applyPivotDim(slices: CubeSlices, dim: StatisticsPivotDim, id: string): CubeSlices {
|
||||
const kind = STATISTICS_DIMS.find((d) => d.pivot === dim)?.id ?? "users"
|
||||
return applyDimValue(slices, kind, id)
|
||||
}
|
||||
|
||||
function chipList(slices: CubeSlices): { key: string; label: string }[] {
|
||||
const chips: { key: string; label: string }[] = []
|
||||
if (slices.country) chips.push({ key: "country", label: `страна ${slices.country}` })
|
||||
if (slices.service) chips.push({ key: "service", label: `сервис ${slices.service}` })
|
||||
if (slices.asn) chips.push({ key: "asn", label: `ASN ${slices.asn}` })
|
||||
if (slices.serverId) chips.push({ key: "serverId", label: `сервер ${slices.serverId}` })
|
||||
if (slices.userId) chips.push({ key: "userId", label: `пользователь ${slices.userId}` })
|
||||
if (slices.iface) chips.push({ key: "iface", label: `iface ${slices.iface}` })
|
||||
return chips
|
||||
}
|
||||
|
||||
function StatisticsPageInner() {
|
||||
const searchParams = useSearchParams()
|
||||
const { mode, backendUrl, prefsHydrated } = useDataSource()
|
||||
const isLive = mode === "live"
|
||||
|
||||
const range = useMemo(() => readRange(searchParams), [searchParams])
|
||||
const slices = useMemo(() => readSlices(searchParams), [searchParams])
|
||||
const filters = useMemo(() => slicesToFilters(slices), [slices])
|
||||
const dim = useMemo(() => readDim(searchParams), [searchParams])
|
||||
const view = useMemo(() => readView(searchParams), [searchParams])
|
||||
const planes = useMemo(() => readPlanes(searchParams), [searchParams])
|
||||
const pivotRow = useMemo(() => readPivotDim(searchParams, "pivotRow", "country"), [searchParams])
|
||||
const pivotCol = useMemo(() => readPivotDim(searchParams, "pivotCol", "service"), [searchParams])
|
||||
|
||||
const [data, setData] = useState<StatisticsDto>(EMPTY)
|
||||
const [pivot, setPivot] = useState<StatisticsPivotDto>(EMPTY_PIVOT)
|
||||
const [loading, setLoading] = useState(false)
|
||||
const [error, setError] = useState<string | null>(null)
|
||||
|
||||
const replaceParams = useCallback(
|
||||
(patch: Record<string, string | undefined>) => {
|
||||
const sp = new URLSearchParams(searchParams.toString())
|
||||
for (const [k, v] of Object.entries(patch)) {
|
||||
if (v) sp.set(k, v)
|
||||
else sp.delete(k)
|
||||
}
|
||||
const qs = sp.toString()
|
||||
if (qs === searchParams.toString()) return
|
||||
window.history.replaceState(null, "", qs ? `/statistics?${qs}` : "/statistics")
|
||||
},
|
||||
[searchParams],
|
||||
)
|
||||
|
||||
const setRange = useCallback(
|
||||
(next: DateRangeYmd) => {
|
||||
replaceParams({ from: next.from, to: next.to })
|
||||
},
|
||||
[replaceParams],
|
||||
)
|
||||
|
||||
const setSlices = useCallback(
|
||||
(next: CubeSlices) => {
|
||||
replaceParams({
|
||||
country: next.country,
|
||||
service: next.service,
|
||||
asn: next.asn,
|
||||
serverId: next.serverId,
|
||||
userId: next.userId,
|
||||
iface: next.iface,
|
||||
})
|
||||
},
|
||||
[replaceParams],
|
||||
)
|
||||
|
||||
useEffect(() => {
|
||||
if (!prefsHydrated || !isLive) return
|
||||
let cancelled = false
|
||||
void (async () => {
|
||||
setLoading(true)
|
||||
setError(null)
|
||||
try {
|
||||
const query = toQuery(range, slices, planes)
|
||||
const dto = await getStatistics(backendUrl, query)
|
||||
if (!cancelled) setData(dto)
|
||||
if (view === "pivot" && pivotRow !== pivotCol) {
|
||||
const matrix = await getStatisticsPivot(backendUrl, {
|
||||
...query,
|
||||
row: pivotRow,
|
||||
col: pivotCol,
|
||||
metric: "bytes",
|
||||
})
|
||||
if (!cancelled) setPivot(matrix)
|
||||
}
|
||||
} catch (e: unknown) {
|
||||
if (!cancelled) {
|
||||
setData(EMPTY)
|
||||
setPivot(EMPTY_PIVOT)
|
||||
setError(e instanceof Error ? e.message : "Не удалось загрузить статистику")
|
||||
}
|
||||
} finally {
|
||||
if (!cancelled) setLoading(false)
|
||||
}
|
||||
})()
|
||||
return () => {
|
||||
cancelled = true
|
||||
}
|
||||
}, [backendUrl, isLive, prefsHydrated, range, slices, view, pivotRow, pivotCol, planes])
|
||||
|
||||
const viewData = isLive ? data : EMPTY
|
||||
const sliced = hasAnySlice(slices)
|
||||
const emptyCube = !isLive || (!loading && viewData.kpis.bytes === 0 && viewData.interfaces.length === 0)
|
||||
|
||||
function handleRowClick(kind: StatisticsSliceKind, row: StatisticsBreakdownRow) {
|
||||
if (kind === "users" && row.id === STATISTICS_UNBOUND_USER_ID) return
|
||||
if (kind === "interfaces" && row.label.includes("· дубль")) return
|
||||
setSlices(applyDimValue(slices, kind, row.id))
|
||||
}
|
||||
|
||||
function handlePivotCell(rowId: string, colId: string) {
|
||||
if (rowId === "__other__" || colId === "__other__") return
|
||||
let next = applyPivotDim(slices, pivotRow, rowId)
|
||||
next = applyPivotDim(next, pivotCol, colId)
|
||||
replaceParams({
|
||||
country: next.country,
|
||||
service: next.service,
|
||||
asn: next.asn,
|
||||
serverId: next.serverId,
|
||||
userId: next.userId,
|
||||
iface: next.iface,
|
||||
view: "explore",
|
||||
})
|
||||
}
|
||||
|
||||
const kpis = viewData.kpis
|
||||
const chips = chipList(slices)
|
||||
const countLabel =
|
||||
view === "pivot"
|
||||
? `${pivot.rows.length} × ${pivot.columns.length}`
|
||||
: sliced
|
||||
? `${STATISTICS_DIMS.filter((d) => !hiddenKinds(slices).has(d.id)).length} разрезов`
|
||||
: `${rowsForKind(viewData, dim).length} строк`
|
||||
|
||||
return (
|
||||
<div className="flex h-full flex-col">
|
||||
<PageHeader
|
||||
crumbs={[{ label: "Обзор", href: "/dashboard" }, { label: "Статистика" }]}
|
||||
actions={<PeriodSelector range={range} onChange={setRange} />}
|
||||
/>
|
||||
|
||||
<div className="flex flex-1 flex-col gap-4 overflow-y-auto px-4 py-4 md:gap-6 md:px-6 md:py-5">
|
||||
{!isLive ? (
|
||||
<Alert>
|
||||
<AlertTitle>Живые данные выключены</AlertTitle>
|
||||
<AlertDescription>
|
||||
Куб статистики строится из IPFIX. Переключитесь на живой источник, чтобы увидеть отчёт.
|
||||
</AlertDescription>
|
||||
</Alert>
|
||||
) : null}
|
||||
|
||||
{error ? (
|
||||
<Alert variant="destructive">
|
||||
<AlertTitle>Ошибка загрузки</AlertTitle>
|
||||
<AlertDescription>{error}</AlertDescription>
|
||||
</Alert>
|
||||
) : null}
|
||||
|
||||
{!slices.serverId && isLive && !emptyCube ? (
|
||||
<Alert>
|
||||
<AlertTitle>Уникальный объём</AlertTitle>
|
||||
<AlertDescription>
|
||||
Объём — трафик клиентов на GRE/WG, без повторного учёта JH↔EN и WAN.
|
||||
</AlertDescription>
|
||||
</Alert>
|
||||
) : null}
|
||||
|
||||
<KpiStatGrid
|
||||
aria-label="Сводка трафика"
|
||||
isLoading={loading}
|
||||
skeletonCount={5}
|
||||
items={[
|
||||
{
|
||||
id: "bytes",
|
||||
label: "Объём",
|
||||
value: formatBytes(kpis.bytes),
|
||||
hint: "GRE/WG клиентов, без hops",
|
||||
icon: <DatabaseIcon />,
|
||||
iconClassName: "text-muted-foreground",
|
||||
},
|
||||
{
|
||||
id: "packets",
|
||||
label: "Пакеты",
|
||||
value: kpis.packets.toLocaleString("ru-RU"),
|
||||
hint: kpis.topService ? `топ: ${kpis.topService}` : undefined,
|
||||
icon: <ActivityIcon />,
|
||||
iconClassName: "text-muted-foreground",
|
||||
},
|
||||
{
|
||||
id: "bps",
|
||||
label: "Средний bitrate",
|
||||
value: fmtBps(kpis.avgBps),
|
||||
icon: <GaugeIcon />,
|
||||
iconClassName: "text-muted-foreground",
|
||||
},
|
||||
{
|
||||
id: "users",
|
||||
label: "Пользователи",
|
||||
value: String(kpis.users),
|
||||
icon: <UsersIcon />,
|
||||
iconClassName: "text-muted-foreground",
|
||||
},
|
||||
{
|
||||
id: "servers",
|
||||
label: "Серверы",
|
||||
value: String(kpis.servers),
|
||||
hint: slices.serverId
|
||||
? (kpis.ifaces ? `${kpis.ifaces} iface` : undefined)
|
||||
: planes === "all"
|
||||
? "WAN и дубли в списке"
|
||||
: "без WAN и overlay",
|
||||
icon: <ServerIcon />,
|
||||
iconClassName: "text-muted-foreground",
|
||||
},
|
||||
]}
|
||||
/>
|
||||
|
||||
<StatisticsVolumeChart series={viewData.series} grain={viewData.grain} />
|
||||
|
||||
<DataPageCard>
|
||||
<DataPageToolbar
|
||||
leading={
|
||||
<div className="flex flex-wrap items-center gap-3">
|
||||
<SegmentedControl
|
||||
value={view}
|
||||
onChange={(next) => replaceParams({ view: next === "pivot" ? "pivot" : "explore" })}
|
||||
options={[
|
||||
{ value: "explore", label: "Разрез" },
|
||||
{ value: "pivot", label: "Сводка" },
|
||||
]}
|
||||
/>
|
||||
<SegmentedControl
|
||||
value={planes}
|
||||
onChange={(next) => replaceParams({ planes: next === "all" ? "all" : undefined })}
|
||||
options={[
|
||||
{ value: "unique", label: "Уникальный" },
|
||||
{ value: "all", label: "Все плоскости" },
|
||||
]}
|
||||
/>
|
||||
{view === "explore" && !sliced ? (
|
||||
<DimensionSelect
|
||||
label="Критерий"
|
||||
value={dim}
|
||||
onChange={(next) => replaceParams({ dim: next })}
|
||||
/>
|
||||
) : null}
|
||||
{view === "pivot" ? (
|
||||
<>
|
||||
<PivotDimSelect
|
||||
label="Строки"
|
||||
value={pivotRow}
|
||||
exclude={pivotCol}
|
||||
onChange={(next) => replaceParams({ pivotRow: next })}
|
||||
/>
|
||||
<PivotDimSelect
|
||||
label="Колонки"
|
||||
value={pivotCol}
|
||||
exclude={pivotRow}
|
||||
onChange={(next) => replaceParams({ pivotCol: next })}
|
||||
/>
|
||||
</>
|
||||
) : null}
|
||||
</div>
|
||||
}
|
||||
filters={filters}
|
||||
onFiltersChange={(next) => setSlices(filtersToSlices(next))}
|
||||
filterFields={STATISTICS_FILTER_FIELDS}
|
||||
countLabel={countLabel}
|
||||
/>
|
||||
<SliceChips
|
||||
chips={chips}
|
||||
onRemove={(key) => {
|
||||
const next = { ...slices }
|
||||
delete next[key as keyof CubeSlices]
|
||||
setSlices(next)
|
||||
}}
|
||||
/>
|
||||
{emptyCube ? (
|
||||
<EmptyState
|
||||
title="Нет данных куба"
|
||||
description="За выбранный период нет IPFIX-фактов. Куб заполняется с момента деплоя, без бэкфилла за год."
|
||||
/>
|
||||
) : view === "pivot" ? (
|
||||
<StatisticsPivotGrid data={isLive ? pivot : EMPTY_PIVOT} onCellClick={handlePivotCell} isLoading={loading} />
|
||||
) : sliced ? (
|
||||
<BreakdownDashboard
|
||||
data={viewData}
|
||||
hidden={hiddenKinds(slices)}
|
||||
selectedIdFor={(kind) => selectedIdForKind(kind, slices)}
|
||||
onRowClick={handleRowClick}
|
||||
isLoading={loading}
|
||||
/>
|
||||
) : (
|
||||
<StatisticsBreakdownDataGrid
|
||||
rows={rowsForKind(viewData, dim)}
|
||||
kind={dim}
|
||||
selectedId={selectedIdForKind(dim, slices)}
|
||||
onRowClick={(row) => handleRowClick(dim, row)}
|
||||
isLoading={loading}
|
||||
/>
|
||||
)}
|
||||
</DataPageCard>
|
||||
</div>
|
||||
</div>
|
||||
)
|
||||
}
|
||||
|
||||
export default function StatisticsPage() {
|
||||
return (
|
||||
<Suspense fallback={null}>
|
||||
<StatisticsPageInner />
|
||||
</Suspense>
|
||||
)
|
||||
}
|
||||
+190
-45
@@ -1,30 +1,63 @@
|
||||
"use client"
|
||||
|
||||
import { useMemo, useState } from "react"
|
||||
import { useCallback, useEffect, useMemo, useState } from "react"
|
||||
import { PageHeader } from "@/components/page-header"
|
||||
import { vxlanTunnels, servers } from "@/lib/data"
|
||||
import type { VxlanTunnel } from "@/lib/data"
|
||||
import { vxlanTunnels as mockVxlanTunnels, servers as mockServers } from "@/lib/data"
|
||||
import type { Server, VxlanTunnel } from "@/lib/data"
|
||||
import { KpiStatGrid } from "@/components/reui-kit/kpi-stat-grid"
|
||||
import { OpsPanel } from "@/components/ops-panel"
|
||||
import { DataPageCard } from "@/components/data-page-card"
|
||||
import { DataPageToolbar } from "@/components/data-page-toolbar"
|
||||
import { VxlanDataGrid } from "@/components/data-grids/vxlan-data-grid"
|
||||
import { Button } from "@/components/ui/button"
|
||||
import { Alert, AlertDescription } from "@/components/ui/alert"
|
||||
import {
|
||||
NetworkIcon, PlusIcon, CodeXmlIcon, LayersIcon,
|
||||
NetworkIcon, PlusIcon, LayersIcon, RefreshCwIcon, AlertCircleIcon,
|
||||
} from "lucide-react"
|
||||
import { CodeExportSheet } from "@/components/reui-kit/code-export-sheet"
|
||||
import { useDataSource } from "@/lib/data-source"
|
||||
import { requestJson } from "@/shared/api/http-client"
|
||||
import { cn } from "@/lib/utils"
|
||||
import { ServerRailLayout, ServerRailMobileButton } from "@/components/server-rail-layout"
|
||||
import { ALL_SERVERS_ID, type ServerTileItem } from "@/components/server-tile-rail"
|
||||
|
||||
// ─── helpers ──────────────────────────────────────────────────────────────────
|
||||
|
||||
function serverFor(id: string) {
|
||||
return servers.find((s) => s.id === id)
|
||||
interface BackendServer {
|
||||
id: number
|
||||
name: string
|
||||
host: string
|
||||
type?: Server["type"]
|
||||
site?: string
|
||||
country: string
|
||||
asn?: string
|
||||
enabled: boolean
|
||||
status?: Server["status"]
|
||||
latency?: number | null
|
||||
}
|
||||
|
||||
// ─── RSC generator ───────────────────────────────────────────────────────────
|
||||
interface VxlanApiResponse {
|
||||
tunnels: VxlanTunnel[]
|
||||
}
|
||||
|
||||
function generateVxlanRsc(t: VxlanTunnel): string {
|
||||
const srv = serverFor(t.serverId)
|
||||
function mapBackendServer(s: BackendServer): Server {
|
||||
return {
|
||||
id: String(s.id),
|
||||
name: s.name || s.host,
|
||||
host: s.host,
|
||||
model: "—",
|
||||
os: "—",
|
||||
site: s.site ?? "",
|
||||
country: s.country || "UN",
|
||||
asn: s.asn ?? "",
|
||||
type: s.type ?? "exit-node",
|
||||
enabled: s.enabled,
|
||||
status: s.status ?? "online",
|
||||
latency: s.latency ?? null,
|
||||
sessions: 0,
|
||||
}
|
||||
}
|
||||
|
||||
function generateVxlanRsc(t: VxlanTunnel, serverById: Record<string, Server>): string {
|
||||
const srv = serverById[t.serverId]
|
||||
const lines: string[] = []
|
||||
lines.push(`# VXLAN — ${t.name} · VNI ${t.vni}`)
|
||||
if (srv) lines.push(`# Сервер: ${srv.name} (${srv.host})`)
|
||||
@@ -42,7 +75,6 @@ function generateVxlanRsc(t: VxlanTunnel): string {
|
||||
if (!t.enabled) lines.push(` disabled=yes \\`)
|
||||
lines.push(``)
|
||||
|
||||
// FDB entries for remote VTEPs
|
||||
for (const vtep of t.remoteVteps) {
|
||||
lines.push(`/interface/vxlan/vteps/add \\`)
|
||||
lines.push(` interface=${t.name} \\`)
|
||||
@@ -50,7 +82,6 @@ function generateVxlanRsc(t: VxlanTunnel): string {
|
||||
lines.push(``)
|
||||
}
|
||||
|
||||
// Bridge
|
||||
lines.push(`# Добавить в bridge:`)
|
||||
lines.push(`/interface/bridge/port/add \\`)
|
||||
lines.push(` bridge=bridge-overlay \\`)
|
||||
@@ -59,12 +90,18 @@ function generateVxlanRsc(t: VxlanTunnel): string {
|
||||
return lines.join("\n")
|
||||
}
|
||||
|
||||
// ─── Export Sheet ─────────────────────────────────────────────────────────────
|
||||
|
||||
function ExportSheet({ open, tunnel, onClose }: {
|
||||
open: boolean; tunnel: VxlanTunnel | null; onClose: () => void
|
||||
function ExportSheet({
|
||||
open, tunnel, onClose, serverById,
|
||||
}: {
|
||||
open: boolean
|
||||
tunnel: VxlanTunnel | null
|
||||
onClose: () => void
|
||||
serverById: Record<string, Server>
|
||||
}) {
|
||||
const code = useMemo(() => tunnel ? generateVxlanRsc(tunnel) : "", [tunnel])
|
||||
const code = useMemo(
|
||||
() => (tunnel ? generateVxlanRsc(tunnel, serverById) : ""),
|
||||
[tunnel, serverById],
|
||||
)
|
||||
|
||||
return (
|
||||
<CodeExportSheet
|
||||
@@ -84,46 +121,156 @@ function ExportSheet({ open, tunnel, onClose }: {
|
||||
)
|
||||
}
|
||||
|
||||
// ─── Export Sheet ─────────────────────────────────────────────────────────────
|
||||
export default function VxlanPage() {
|
||||
const [search, setSearch] = useState("")
|
||||
const { mode, backendUrl } = useDataSource()
|
||||
const isLive = mode === "live"
|
||||
|
||||
const [search, setSearch] = useState("")
|
||||
const [exportTunnel, setExportTunnel] = useState<VxlanTunnel | null>(null)
|
||||
const [selectedServerId, setSelectedServerId] = useState(ALL_SERVERS_ID)
|
||||
|
||||
const [liveTunnels, setLiveTunnels] = useState<VxlanTunnel[]>([])
|
||||
const [liveServers, setLiveServers] = useState<Server[]>([])
|
||||
const [loading, setLoading] = useState(false)
|
||||
const [liveError, setLiveError] = useState<string | null>(null)
|
||||
|
||||
const loadLive = useCallback(async () => {
|
||||
if (!isLive) return
|
||||
setLoading(true)
|
||||
setLiveError(null)
|
||||
try {
|
||||
const [tunnelsRes, serversRes] = await Promise.all([
|
||||
requestJson<VxlanApiResponse>(backendUrl, "/api/vxlan"),
|
||||
requestJson<BackendServer[]>(backendUrl, "/api/servers"),
|
||||
])
|
||||
setLiveTunnels(tunnelsRes.tunnels ?? [])
|
||||
setLiveServers(serversRes.filter((s) => s.enabled).map(mapBackendServer))
|
||||
} catch (e) {
|
||||
setLiveError(e instanceof Error ? e.message : "Ошибка загрузки")
|
||||
setLiveTunnels([])
|
||||
} finally {
|
||||
setLoading(false)
|
||||
}
|
||||
}, [isLive, backendUrl])
|
||||
|
||||
useEffect(() => {
|
||||
if (!isLive) {
|
||||
queueMicrotask(() => {
|
||||
setLiveTunnels([])
|
||||
setLiveServers([])
|
||||
setLiveError(null)
|
||||
})
|
||||
return
|
||||
}
|
||||
queueMicrotask(() => {
|
||||
void loadLive()
|
||||
})
|
||||
}, [isLive, loadLive])
|
||||
|
||||
const displayTunnels = isLive ? liveTunnels : mockVxlanTunnels
|
||||
const displayServers = isLive ? liveServers : mockServers.filter((s) => s.enabled)
|
||||
|
||||
const effectiveServerId =
|
||||
selectedServerId === ALL_SERVERS_ID || displayServers.some((s) => s.id === selectedServerId)
|
||||
? selectedServerId
|
||||
: ALL_SERVERS_ID
|
||||
|
||||
const scopedTunnels = useMemo(() => {
|
||||
if (effectiveServerId === ALL_SERVERS_ID) return displayTunnels
|
||||
return displayTunnels.filter((t) => t.serverId === effectiveServerId)
|
||||
}, [displayTunnels, effectiveServerId])
|
||||
|
||||
const serverById = useMemo(
|
||||
() => Object.fromEntries(displayServers.map((s) => [s.id, s])),
|
||||
[displayServers],
|
||||
)
|
||||
|
||||
const railItems = useMemo<ServerTileItem[]>(() => (
|
||||
displayServers.map((s) => ({
|
||||
id: s.id,
|
||||
name: s.name,
|
||||
host: s.host,
|
||||
site: s.site,
|
||||
country: s.country,
|
||||
status: s.status,
|
||||
type: s.type,
|
||||
enabled: s.enabled,
|
||||
meta: String(displayTunnels.filter((t) => t.serverId === s.id).length),
|
||||
}))
|
||||
), [displayServers, displayTunnels])
|
||||
|
||||
const filtered = useMemo(() => {
|
||||
if (!search) return vxlanTunnels
|
||||
if (!search) return scopedTunnels
|
||||
const q = search.toLowerCase()
|
||||
return vxlanTunnels.filter((t) =>
|
||||
return scopedTunnels.filter((t) =>
|
||||
t.name.includes(q) ||
|
||||
String(t.vni).includes(q) ||
|
||||
t.vtepIp.includes(q) ||
|
||||
(serverFor(t.serverId)?.name.toLowerCase().includes(q) ?? false)
|
||||
(serverById[t.serverId]?.name.toLowerCase().includes(q) ?? false),
|
||||
)
|
||||
}, [search])
|
||||
}, [search, scopedTunnels, serverById])
|
||||
|
||||
const upCount = vxlanTunnels.filter((t) => t.status === "up").length
|
||||
const vnis = new Set(vxlanTunnels.map((t) => t.vni)).size
|
||||
const upCount = scopedTunnels.filter((t) => t.status === "up").length
|
||||
const vnis = new Set(scopedTunnels.map((t) => t.vni)).size
|
||||
|
||||
return (
|
||||
<div className="flex flex-col h-full">
|
||||
<PageHeader
|
||||
crumbs={[{ label: "Управление" }, { label: "VXLAN" }]}
|
||||
actions={
|
||||
<Button size="sm">
|
||||
<PlusIcon className="size-4" />Новый VXLAN
|
||||
</Button>
|
||||
}
|
||||
/>
|
||||
|
||||
<div className="flex-1 overflow-y-auto p-6">
|
||||
<>
|
||||
<ServerRailLayout
|
||||
items={railItems}
|
||||
selectedId={effectiveServerId}
|
||||
onSelect={setSelectedServerId}
|
||||
showAll
|
||||
allCount={displayServers.length}
|
||||
loading={isLive && loading && displayServers.length === 0}
|
||||
header={
|
||||
<PageHeader
|
||||
crumbs={[{ label: "Управление" }, { label: "VXLAN" }]}
|
||||
actions={
|
||||
<>
|
||||
<ServerRailMobileButton />
|
||||
<Button
|
||||
variant="outline"
|
||||
size="sm"
|
||||
onClick={() => { void loadLive() }}
|
||||
disabled={!isLive || loading}
|
||||
>
|
||||
<RefreshCwIcon className={cn("size-4", loading && "animate-spin")} />
|
||||
Обновить
|
||||
</Button>
|
||||
<Button size="sm">
|
||||
<PlusIcon className="size-4" />Новый VXLAN
|
||||
</Button>
|
||||
</>
|
||||
}
|
||||
/>
|
||||
}
|
||||
>
|
||||
<div className="flex flex-col gap-5">
|
||||
|
||||
{isLive && liveError && (
|
||||
<Alert variant="warning" className="py-2">
|
||||
<AlertCircleIcon />
|
||||
<AlertDescription className="text-xs">Ошибка загрузки: {liveError}</AlertDescription>
|
||||
</Alert>
|
||||
)}
|
||||
{isLive && !loading && displayTunnels.length === 0 && !liveError && (
|
||||
<div className="rounded-md border border-border bg-muted/30 px-4 py-6 text-center text-sm text-muted-foreground">
|
||||
На опрошенных серверах нет VXLAN-интерфейсов
|
||||
</div>
|
||||
)}
|
||||
{mode === "mock" && (
|
||||
<span className="inline-flex w-fit items-center gap-1.5 rounded-full border border-border bg-muted/40 px-2.5 py-0.5 text-[11px] font-medium text-muted-foreground">
|
||||
Моковые данные
|
||||
</span>
|
||||
)}
|
||||
|
||||
<KpiStatGrid
|
||||
aria-label="Сводка VXLAN"
|
||||
items={[
|
||||
{
|
||||
id: "tunnels",
|
||||
label: "Туннелей",
|
||||
value: vxlanTunnels.length,
|
||||
value: scopedTunnels.length,
|
||||
icon: <NetworkIcon className="size-4" />,
|
||||
iconClassName: "text-muted-foreground",
|
||||
},
|
||||
@@ -144,14 +291,13 @@ export default function VxlanPage() {
|
||||
{
|
||||
id: "servers",
|
||||
label: "Серверов",
|
||||
value: new Set(vxlanTunnels.map((t) => t.serverId)).size,
|
||||
value: new Set(scopedTunnels.map((t) => t.serverId)).size,
|
||||
icon: <NetworkIcon className="size-4" />,
|
||||
iconClassName: "text-primary",
|
||||
},
|
||||
]}
|
||||
/>
|
||||
|
||||
{/* Info banner */}
|
||||
<div className="flex items-start gap-3 rounded-lg bg-sky-500/5 border border-sky-500/20 px-4 py-3 text-sm">
|
||||
<NetworkIcon className="size-5 text-sky-500 shrink-0 mt-0.5" />
|
||||
<div>
|
||||
@@ -163,7 +309,6 @@ export default function VxlanPage() {
|
||||
</div>
|
||||
</div>
|
||||
|
||||
{/* Table */}
|
||||
<DataPageCard>
|
||||
<DataPageToolbar
|
||||
search={search}
|
||||
@@ -173,12 +318,11 @@ export default function VxlanPage() {
|
||||
/>
|
||||
<VxlanDataGrid
|
||||
tunnels={filtered}
|
||||
servers={servers}
|
||||
servers={displayServers}
|
||||
onExport={setExportTunnel}
|
||||
/>
|
||||
</DataPageCard>
|
||||
|
||||
{/* Reference */}
|
||||
<OpsPanel title="RouterOS 7 · /interface/vxlan — быстрые команды" contentClassName="px-5 py-4">
|
||||
<div className="grid grid-cols-1 sm:grid-cols-3 gap-4 text-xs font-mono">
|
||||
{[
|
||||
@@ -232,13 +376,14 @@ export default function VxlanPage() {
|
||||
</OpsPanel>
|
||||
|
||||
</div>
|
||||
</div>
|
||||
</ServerRailLayout>
|
||||
|
||||
<ExportSheet
|
||||
open={!!exportTunnel}
|
||||
tunnel={exportTunnel}
|
||||
onClose={() => setExportTunnel(null)}
|
||||
serverById={serverById}
|
||||
/>
|
||||
</div>
|
||||
</>
|
||||
)
|
||||
}
|
||||
|
||||
@@ -57,10 +57,12 @@ import {
|
||||
type ServerTileItem,
|
||||
} from "@/components/server-tile-rail"
|
||||
import { toast } from "sonner"
|
||||
import { ConfigHistorySheet } from "@/components/config-history-sheet"
|
||||
import type { ConfigRevisionDto } from "@/lib/config-revisions"
|
||||
import {
|
||||
ShieldCheckIcon, PlusIcon, KeyRoundIcon,
|
||||
UsersIcon, ActivityIcon, RefreshCwIcon, UploadIcon, InfoIcon,
|
||||
Trash2Icon, CodeXmlIcon, AlertCircleIcon,
|
||||
Trash2Icon, CodeXmlIcon, AlertCircleIcon, HistoryIcon,
|
||||
} from "lucide-react"
|
||||
|
||||
type WgWorkspaceTab = "interfaces" | "peers" | "cli"
|
||||
@@ -188,6 +190,10 @@ export default function WireGuardPage() {
|
||||
const [exportPeerId, setExportPeerId] = useState<string | null>(null)
|
||||
const [peerIface, setPeerIface] = useState<WgIfaceWithServer | null>(null)
|
||||
const [pendingDelete, setPendingDelete] = useState<PendingDelete | null>(null)
|
||||
const [historyOpen, setHistoryOpen] = useState(false)
|
||||
const [revisions, setRevisions] = useState<ConfigRevisionDto[]>([])
|
||||
const [historyLoading, setHistoryLoading] = useState(false)
|
||||
const [historyRestoring, setHistoryRestoring] = useState(false)
|
||||
const [liveExport, setLiveExport] = useState<{
|
||||
rsc?: string
|
||||
conf?: string
|
||||
@@ -242,6 +248,44 @@ export default function WireGuardPage() {
|
||||
? selectedServerId
|
||||
: ALL_SERVERS_ID
|
||||
|
||||
const historyServerId = effectiveServerId === ALL_SERVERS_ID ? null : effectiveServerId
|
||||
|
||||
const loadRevisions = useCallback(async () => {
|
||||
if (!isLive || !historyServerId) return
|
||||
setHistoryLoading(true)
|
||||
try {
|
||||
const res = await requestJson<{ revisions: ConfigRevisionDto[] }>(
|
||||
backendUrl,
|
||||
`/api/wireguard/revisions?serverId=${encodeURIComponent(historyServerId)}`,
|
||||
)
|
||||
setRevisions(res.revisions)
|
||||
} catch (err) {
|
||||
toast.error("Не удалось загрузить историю", { description: String(err) })
|
||||
setRevisions([])
|
||||
} finally {
|
||||
setHistoryLoading(false)
|
||||
}
|
||||
}, [isLive, historyServerId, backendUrl])
|
||||
|
||||
const restoreRevision = useCallback(async (id: string) => {
|
||||
if (!isLive || !historyServerId) return
|
||||
setHistoryRestoring(true)
|
||||
try {
|
||||
await requestJson(
|
||||
backendUrl,
|
||||
`/api/wireguard/revisions/${encodeURIComponent(id)}/restore`,
|
||||
{ method: "POST", body: JSON.stringify({ serverId: historyServerId }) },
|
||||
)
|
||||
toast.success("Версия применена на роутер")
|
||||
await loadLive()
|
||||
await loadRevisions()
|
||||
} catch (err) {
|
||||
toast.error("Не удалось откатить", { description: String(err) })
|
||||
} finally {
|
||||
setHistoryRestoring(false)
|
||||
}
|
||||
}, [isLive, historyServerId, backendUrl, loadLive, loadRevisions])
|
||||
|
||||
const scopedIfaces = useMemo(() => {
|
||||
if (effectiveServerId === ALL_SERVERS_ID) return displayIfaces
|
||||
return displayIfaces.filter((i) => i.serverId === effectiveServerId)
|
||||
@@ -544,6 +588,7 @@ export default function WireGuardPage() {
|
||||
<>
|
||||
<ServerRailMobileButton />
|
||||
{isLive && (
|
||||
<>
|
||||
<Button
|
||||
size="sm"
|
||||
variant="outline"
|
||||
@@ -553,6 +598,20 @@ export default function WireGuardPage() {
|
||||
<RefreshCwIcon className={`size-4 ${loading ? "animate-spin" : ""}`} />
|
||||
Обновить
|
||||
</Button>
|
||||
<Button
|
||||
size="sm"
|
||||
variant="outline"
|
||||
disabled={loading || !historyServerId}
|
||||
title={!historyServerId ? "Выберите сервер, чтобы смотреть историю" : "История версий и откат на CHR"}
|
||||
onClick={() => {
|
||||
setHistoryOpen(true)
|
||||
void loadRevisions()
|
||||
}}
|
||||
>
|
||||
<HistoryIcon className="size-4" />
|
||||
История
|
||||
</Button>
|
||||
</>
|
||||
)}
|
||||
<Button size="sm" variant="outline" onClick={() => setImportOpen(true)}>
|
||||
<UploadIcon className="size-4" />
|
||||
@@ -822,6 +881,17 @@ export default function WireGuardPage() {
|
||||
</AlertDialogFooter>
|
||||
</AlertDialogContent>
|
||||
</AlertDialog>
|
||||
|
||||
<ConfigHistorySheet
|
||||
open={historyOpen}
|
||||
onOpenChange={setHistoryOpen}
|
||||
title="История WireGuard"
|
||||
itemLabel="интерфейсов"
|
||||
revisions={revisions}
|
||||
loading={historyLoading}
|
||||
restoring={historyRestoring}
|
||||
onRestore={restoreRevision}
|
||||
/>
|
||||
</>
|
||||
)
|
||||
}
|
||||
|
||||
@@ -0,0 +1,28 @@
|
||||
-- Statistics cube: hour + daily facts (server × iface × country × service × ASN).
|
||||
-- Compact types. FILLFACTOR/autovacuum нельзя на partitioned parent (PG 42809) —
|
||||
-- задаются на листовых партициях в ensurePartitionFor.
|
||||
-- Retention: DROP partitions only (see PARTITION_SPECS).
|
||||
|
||||
CREATE TABLE IF NOT EXISTS flow_hour_facts (
|
||||
server_id BIGINT NOT NULL REFERENCES servers(id) ON DELETE CASCADE,
|
||||
bucket_at TIMESTAMPTZ NOT NULL,
|
||||
iface TEXT NOT NULL,
|
||||
country CHAR(2) NOT NULL,
|
||||
service TEXT NOT NULL,
|
||||
asn INTEGER NOT NULL,
|
||||
bytes BIGINT NOT NULL DEFAULT 0,
|
||||
packets BIGINT NOT NULL DEFAULT 0,
|
||||
PRIMARY KEY (server_id, bucket_at, iface, country, service, asn)
|
||||
) PARTITION BY RANGE (bucket_at);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS flow_daily_facts (
|
||||
server_id BIGINT NOT NULL REFERENCES servers(id) ON DELETE CASCADE,
|
||||
day DATE NOT NULL,
|
||||
iface TEXT NOT NULL,
|
||||
country CHAR(2) NOT NULL,
|
||||
service TEXT NOT NULL,
|
||||
asn INTEGER NOT NULL,
|
||||
bytes BIGINT NOT NULL DEFAULT 0,
|
||||
packets BIGINT NOT NULL DEFAULT 0,
|
||||
PRIMARY KEY (server_id, day, iface, country, service, asn)
|
||||
) PARTITION BY RANGE (day);
|
||||
@@ -0,0 +1,16 @@
|
||||
-- История desired/actual снапшотов managed-секций (фильтры, рекурсивные маршруты).
|
||||
-- Retention — prune в сервисе (последние 50 на пару server+section).
|
||||
|
||||
CREATE TABLE IF NOT EXISTS config_revisions (
|
||||
id TEXT PRIMARY KEY,
|
||||
server_id BIGINT NOT NULL REFERENCES servers(id) ON DELETE CASCADE,
|
||||
section TEXT NOT NULL,
|
||||
source TEXT NOT NULL,
|
||||
fingerprint TEXT NOT NULL,
|
||||
payload JSONB NOT NULL DEFAULT '[]'::jsonb,
|
||||
note TEXT,
|
||||
created_at TIMESTAMPTZ NOT NULL DEFAULT now()
|
||||
);
|
||||
|
||||
CREATE INDEX IF NOT EXISTS idx_config_revisions_server_section_created
|
||||
ON config_revisions (server_id, section, created_at DESC);
|
||||
@@ -0,0 +1,7 @@
|
||||
-- IPFIX postNAT (IANA 225/226) + postNAPT ports (IANA 227/228) from MikroTik Traffic Flow.
|
||||
-- Needed to rebuild facts with the same internet dest as the network map.
|
||||
|
||||
ALTER TABLE flow_buckets ADD COLUMN IF NOT EXISTS nat_src INET;
|
||||
ALTER TABLE flow_buckets ADD COLUMN IF NOT EXISTS nat_dst INET;
|
||||
ALTER TABLE flow_buckets ADD COLUMN IF NOT EXISTS nat_src_port INTEGER NOT NULL DEFAULT 0;
|
||||
ALTER TABLE flow_buckets ADD COLUMN IF NOT EXISTS nat_dst_port INTEGER NOT NULL DEFAULT 0;
|
||||
@@ -12,14 +12,17 @@
|
||||
"db:migrate": "drizzle-kit migrate",
|
||||
"db:studio": "drizzle-kit studio",
|
||||
"db:migrate-from-sqlite": "tsx src/scripts/migrate-sqlite-to-pg.ts",
|
||||
"facts:rebuild": "tsx src/scripts/rebuild-flow-facts.ts",
|
||||
"test:auth": "tsx src/lib/permissions.test.ts && tsx src/plugins/auth.smoke.test.ts",
|
||||
"test:wireguard": "npx tsx src/services/wireguard-config.test.ts",
|
||||
"test:traffic-rate": "tsx src/services/traffic-rate.test.ts",
|
||||
"test:traffic-flow": "tsx src/services/traffic-flow-parse.test.ts && tsx src/services/traffic-flow-map-exporter.test.ts && tsx src/services/traffic-flow-ifaces.test.ts && tsx src/services/traffic-flow-dedup.test.ts && tsx src/services/traffic-flow-planes.test.ts && tsx src/services/traffic-flow-ip.test.ts && tsx src/services/traffic-flow-classify.test.ts && tsx src/services/traffic-flow-ripe.test.ts && tsx src/services/traffic-flow-brands.test.ts && tsx src/services/traffic-flow-ingest.test.ts && tsx src/services/traffic-flow-analytics.test.ts && tsx src/services/traffic-flow-map-hops.test.ts && tsx src/services/traffic-flow-purge.test.ts && tsx src/services/traffic-flow-geoip.test.ts",
|
||||
"test:traffic-flow": "tsx src/services/traffic-flow-parse.test.ts && tsx src/services/traffic-flow-map-exporter.test.ts && tsx src/services/traffic-flow-ifaces.test.ts && tsx src/services/traffic-flow-ifindex.test.ts && tsx src/services/traffic-flow-dedup.test.ts && tsx src/services/traffic-flow-planes.test.ts && tsx src/services/traffic-flow-ip.test.ts && tsx src/services/traffic-flow-dest.test.ts && tsx src/services/traffic-flow-classify.test.ts && tsx src/services/traffic-flow-ripe.test.ts && tsx src/services/traffic-flow-brands.test.ts && tsx src/services/traffic-flow-ingest.test.ts && tsx src/services/traffic-flow-analytics.test.ts && tsx src/services/traffic-flow-map-hops.test.ts && tsx src/services/traffic-flow-purge.test.ts && tsx src/services/traffic-flow-geoip.test.ts && tsx src/services/traffic-flow-facts.test.ts && tsx src/services/traffic-flow-facts-filter.test.ts && tsx src/services/traffic-flow-facts-rebuild.test.ts && tsx src/services/statistics-aggregate.test.ts",
|
||||
"test:users": "tsx src/modules/users/iface-type.test.ts && tsx src/modules/users/bindings.test.ts",
|
||||
"test:pg": "tsx src/db/sql-bind.test.ts && tsx src/db/sqlite-json.test.ts && tsx src/db/traffic-flags.test.ts && tsx src/db/pg-schema.test.ts",
|
||||
"test:pg": "tsx src/db/sql-bind.test.ts && tsx src/db/sqlite-json.test.ts && tsx src/db/traffic-flags.test.ts && tsx src/db/pg-schema.test.ts && tsx src/services/config-revisions.test.ts",
|
||||
"test:config-sync": "tsx src/services/config-apply-plan.test.ts && tsx src/services/entity-snapshots.test.ts",
|
||||
"test:backups": "tsx src/services/s3-backup-client.test.ts",
|
||||
"test": "npm run test:alert-engine && npm run test:auth && npm run test:wireguard && npm run test:traffic-rate && npm run test:traffic-flow && npm run test:users && npm run test:pg && npm run test:backups",
|
||||
"test:live-maps": "tsx src/services/ospf-route-parse.test.ts && tsx src/services/vxlan-live.test.ts && tsx src/services/containers-live.test.ts",
|
||||
"test": "npm run test:alert-engine && npm run test:auth && npm run test:wireguard && npm run test:traffic-rate && npm run test:traffic-flow && npm run test:users && npm run test:pg && npm run test:backups && npm run test:live-maps && npm run test:config-sync",
|
||||
"test:geoip": "tsx src/services/traffic-flow-geoip.test.ts"
|
||||
},
|
||||
"dependencies": {
|
||||
|
||||
@@ -8,11 +8,19 @@ export interface PartitionSpec {
|
||||
keepDays: number
|
||||
}
|
||||
|
||||
/** Leaf-only: PG forbids storage params on partitioned parents (SQLSTATE 42809). */
|
||||
const FACT_LEAF_STORAGE =
|
||||
"fillfactor = 70, autovacuum_vacuum_scale_factor = 0.05, autovacuum_vacuum_cost_limit = 2000"
|
||||
|
||||
const FACT_PARENTS = new Set(["flow_hour_facts", "flow_daily_facts"])
|
||||
|
||||
export const PARTITION_SPECS: PartitionSpec[] = [
|
||||
{ parent: "flow_buckets", kind: "day", keepDays: 4 },
|
||||
{ parent: "flow_minute_stats", kind: "day", keepDays: 4 },
|
||||
{ parent: "flow_minute_dims", kind: "day", keepDays: 4 },
|
||||
{ parent: "flow_daily_dims", kind: "month", keepDays: 420 },
|
||||
{ parent: "flow_hour_facts", kind: "day", keepDays: 3 },
|
||||
{ parent: "flow_daily_facts", kind: "month", keepDays: 420 },
|
||||
{ parent: "traffic_samples", kind: "week", keepDays: 21 },
|
||||
{ parent: "servers_rest_ping_samples", kind: "week", keepDays: 35 },
|
||||
{ parent: "uptime_probe_samples", kind: "week", keepDays: 21 },
|
||||
@@ -110,6 +118,9 @@ export async function ensurePartitionFor(
|
||||
await pool.query(
|
||||
`CREATE TABLE IF NOT EXISTS ${name} PARTITION OF ${parent} FOR VALUES FROM ('${from}') TO ('${to}')`,
|
||||
)
|
||||
if (FACT_PARENTS.has(parent)) {
|
||||
await pool.query(`ALTER TABLE ${name} SET (${FACT_LEAF_STORAGE})`)
|
||||
}
|
||||
return name
|
||||
}
|
||||
|
||||
|
||||
@@ -9,6 +9,8 @@ if (!(await withPgOrSkip())) {
|
||||
process.exit(0)
|
||||
}
|
||||
|
||||
await applySqlMigrations(pool)
|
||||
|
||||
{
|
||||
const { rows } = await dbQuery<{ n: string }>(`SELECT COUNT(*)::text AS n FROM servers`)
|
||||
assert.ok(rows[0])
|
||||
@@ -114,13 +116,17 @@ if (!(await withPgOrSkip())) {
|
||||
SELECT column_name, udt_name
|
||||
FROM information_schema.columns
|
||||
WHERE table_schema = 'public' AND table_name = 'flow_buckets'
|
||||
AND column_name IN ('src', 'dst', 'next_hop', 'proto')
|
||||
AND column_name IN ('src', 'dst', 'next_hop', 'proto', 'nat_src', 'nat_dst', 'nat_src_port', 'nat_dst_port')
|
||||
`)
|
||||
const by = Object.fromEntries(rows.map((r) => [r.column_name, r.udt_name]))
|
||||
assert.equal(by.src, "inet")
|
||||
assert.equal(by.dst, "inet")
|
||||
assert.equal(by.next_hop, "inet")
|
||||
assert.equal(by.proto, "int2")
|
||||
assert.equal(by.nat_src, "inet")
|
||||
assert.equal(by.nat_dst, "inet")
|
||||
assert.equal(by.nat_src_port, "int4")
|
||||
assert.equal(by.nat_dst_port, "int4")
|
||||
}
|
||||
|
||||
{
|
||||
@@ -163,6 +169,29 @@ if (!(await withPgOrSkip())) {
|
||||
await dbQuery(`DELETE FROM servers WHERE name = 'pg-wipe-idempotent'`)
|
||||
}
|
||||
|
||||
{
|
||||
const mig = await dbQuery<{ id: string }>(
|
||||
`SELECT id FROM schema_migrations WHERE id = '0006_config_revisions'`,
|
||||
)
|
||||
assert.equal(mig.rows.length, 1, "0006 применена")
|
||||
|
||||
const { rows } = await dbQuery<{ column_name: string; udt_name: string }>(`
|
||||
SELECT column_name, udt_name FROM information_schema.columns
|
||||
WHERE table_schema = 'public' AND table_name = 'config_revisions'
|
||||
`)
|
||||
const by = Object.fromEntries(rows.map((r) => [r.column_name, r.udt_name]))
|
||||
assert.equal(by.payload, "jsonb")
|
||||
assert.equal(by.fingerprint, "text")
|
||||
assert.equal(by.section, "text")
|
||||
}
|
||||
|
||||
{
|
||||
const mig = await dbQuery<{ id: string }>(
|
||||
`SELECT id FROM schema_migrations WHERE id = '0007_flow_buckets_nat'`,
|
||||
)
|
||||
assert.equal(mig.rows.length, 1, "0007 применена")
|
||||
}
|
||||
|
||||
{
|
||||
const marker = await dbQuery<{ sqlite_imported_at: string | null }>(
|
||||
`SELECT sqlite_imported_at FROM data_migration WHERE id = 1`,
|
||||
|
||||
@@ -93,6 +93,19 @@ export const filterRules = pgTable("filter_rules", {
|
||||
index("idx_filter_rules_server_sort").on(t.serverId, t.sortOrder),
|
||||
])
|
||||
|
||||
export const configRevisions = pgTable("config_revisions", {
|
||||
id: text("id").primaryKey(),
|
||||
serverId: intPkRef().references(() => servers.id, { onDelete: "cascade" }),
|
||||
section: text("section", { enum: ["filters", "recursive-routes", "firewall", "wireguard", "gre"] }).notNull(),
|
||||
source: text("source", { enum: ["apply", "rollback", "observed", "copy"] }).notNull(),
|
||||
fingerprint: text("fingerprint").notNull(),
|
||||
payload: jsonb("payload").$type<unknown>().notNull().default(sql`'[]'::jsonb`),
|
||||
note: text("note"),
|
||||
createdAt: ts("created_at").notNull().defaultNow(),
|
||||
}, (t) => [
|
||||
index("idx_config_revisions_server_section_created").on(t.serverId, t.section, t.createdAt),
|
||||
])
|
||||
|
||||
export const recursiveRoutes = pgTable("recursive_routes", {
|
||||
id: idIdentity().primaryKey(),
|
||||
serverId: intPkRef().references(() => servers.id, { onDelete: "cascade" }),
|
||||
@@ -208,6 +221,36 @@ export const flowDailyDims = pgTable("flow_daily_dims", {
|
||||
index("idx_flow_daily_dims_day").on(t.day, t.dim),
|
||||
])
|
||||
|
||||
/** Hour-grain traffic cube for statistics (≤48h). No secondary indexes. */
|
||||
export const flowHourFacts = pgTable("flow_hour_facts", {
|
||||
serverId: bigint("server_id", { mode: "number" }).notNull()
|
||||
.references(() => servers.id, { onDelete: "cascade" }),
|
||||
bucketAt: ts("bucket_at").notNull(),
|
||||
iface: text("iface").notNull(),
|
||||
country: text("country").notNull(),
|
||||
service: text("service").notNull(),
|
||||
asn: integer("asn").notNull(),
|
||||
bytes: bigint("bytes", { mode: "number" }).notNull().default(0),
|
||||
packets: bigint("packets", { mode: "number" }).notNull().default(0),
|
||||
}, (t) => [
|
||||
primaryKey({ columns: [t.serverId, t.bucketAt, t.iface, t.country, t.service, t.asn] }),
|
||||
])
|
||||
|
||||
/** Daily-grain traffic cube for statistics (long window). No secondary indexes. */
|
||||
export const flowDailyFacts = pgTable("flow_daily_facts", {
|
||||
serverId: bigint("server_id", { mode: "number" }).notNull()
|
||||
.references(() => servers.id, { onDelete: "cascade" }),
|
||||
day: date("day", { mode: "string" }).notNull(),
|
||||
iface: text("iface").notNull(),
|
||||
country: text("country").notNull(),
|
||||
service: text("service").notNull(),
|
||||
asn: integer("asn").notNull(),
|
||||
bytes: bigint("bytes", { mode: "number" }).notNull().default(0),
|
||||
packets: bigint("packets", { mode: "number" }).notNull().default(0),
|
||||
}, (t) => [
|
||||
primaryKey({ columns: [t.serverId, t.day, t.iface, t.country, t.service, t.asn] }),
|
||||
])
|
||||
|
||||
export const flowBuckets = pgTable("flow_buckets", {
|
||||
serverId: intPkRef().references(() => servers.id, { onDelete: "cascade" }),
|
||||
bucketAt: ts("bucket_at").notNull(),
|
||||
@@ -223,6 +266,10 @@ export const flowBuckets = pgTable("flow_buckets", {
|
||||
nextHop: inet("next_hop"),
|
||||
flowStartMs: bigint("flow_start_ms", { mode: "number" }).notNull().default(0),
|
||||
flowEndMs: bigint("flow_end_ms", { mode: "number" }).notNull().default(0),
|
||||
natSrc: inet("nat_src"),
|
||||
natDst: inet("nat_dst"),
|
||||
natSrcPort: integer("nat_src_port").notNull().default(0),
|
||||
natDstPort: integer("nat_dst_port").notNull().default(0),
|
||||
}, (t) => [
|
||||
primaryKey({
|
||||
name: "flow_buckets_pkey",
|
||||
@@ -703,6 +750,7 @@ export type ServerInsert = typeof servers.$inferInsert
|
||||
export type Snapshot = typeof serverSnapshots.$inferSelect
|
||||
export type SnapshotInsert = typeof serverSnapshots.$inferInsert
|
||||
export type FilterRuleRow = typeof filterRules.$inferSelect
|
||||
export type ConfigRevisionRow = typeof configRevisions.$inferSelect
|
||||
export type RecursiveRouteRow = typeof recursiveRoutes.$inferSelect
|
||||
export type TrafficSettingsRow = typeof trafficSettings.$inferSelect
|
||||
export type TrafficFlowSettingsRow = typeof trafficFlowSettings.$inferSelect
|
||||
|
||||
@@ -125,6 +125,7 @@ const TABLES: TableCopy[] = [
|
||||
["src_port", "int"], ["dst_port", "int"], ["bytes", "int"], ["packets", "int"],
|
||||
["in_iface", "text"], ["out_iface", "text"], ["next_hop", "inet"],
|
||||
["flow_start_ms", "int"], ["flow_end_ms", "int"],
|
||||
["nat_src", "inet"], ["nat_dst", "inet"], ["nat_src_port", "int"], ["nat_dst_port", "int"],
|
||||
]},
|
||||
{ table: "flow_minute_stats", timeCol: "bucket_at", retentionDays: 3, columns: [
|
||||
["server_id", "int"], ["bucket_at", "ts"], ["bytes", "int"], ["packets", "int"],
|
||||
|
||||
@@ -29,8 +29,12 @@ import certificatesRoutes from "./routes/certificates.js"
|
||||
import systemDatabaseRoutes from "./routes/system-database.js"
|
||||
import eventsRoutes from "./routes/events.js"
|
||||
import wireguardRoutes from "./routes/wireguard.js"
|
||||
import vxlanRoutes from "./routes/vxlan.js"
|
||||
import containersRoutes from "./routes/containers.js"
|
||||
import firewallRoutes from "./routes/firewall.js"
|
||||
import greRoutes from "./routes/gre.js"
|
||||
import usersRoutes from "./routes/users.js"
|
||||
import statisticsRoutes from "./routes/statistics.js"
|
||||
import { refreshScheduler, stopScheduler } from "./services/scheduler.js"
|
||||
import { getFlowWorkerHealth, startTrafficFlowListener, stopTrafficFlowListener } from "./services/traffic-flow-ingest.js"
|
||||
import { initGeoip } from "./services/traffic-flow-geoip.js"
|
||||
@@ -133,8 +137,12 @@ export async function buildApp(opts?: {
|
||||
await app.register(systemDatabaseRoutes, { prefix: "/api" })
|
||||
await app.register(eventsRoutes, { prefix: "/api" })
|
||||
await app.register(wireguardRoutes, { prefix: "/api" })
|
||||
await app.register(vxlanRoutes, { prefix: "/api" })
|
||||
await app.register(containersRoutes, { prefix: "/api" })
|
||||
await app.register(firewallRoutes, { prefix: "/api" })
|
||||
await app.register(greRoutes, { prefix: "/api" })
|
||||
await app.register(usersRoutes, { prefix: "/api" })
|
||||
await app.register(statisticsRoutes, { prefix: "/api" })
|
||||
|
||||
if (opts?.startScheduler !== false) {
|
||||
await refreshScheduler()
|
||||
|
||||
@@ -18,7 +18,7 @@ assert.equal(
|
||||
"mm:settings:admin",
|
||||
)
|
||||
assert.equal(
|
||||
permissionForRequest("GET", "/api/traffic/servers/1/live"),
|
||||
permissionForRequest("GET", "/api/statistics"),
|
||||
"mm:traffic:read",
|
||||
)
|
||||
assert.equal(
|
||||
@@ -41,6 +41,14 @@ assert.equal(
|
||||
permissionForRequest("GET", "/api/firewall/all"),
|
||||
"mm:network:read",
|
||||
)
|
||||
assert.equal(
|
||||
permissionForRequest("GET", "/api/gre/tunnels"),
|
||||
"mm:network:read",
|
||||
)
|
||||
assert.equal(
|
||||
permissionForRequest("POST", "/api/gre/tunnels"),
|
||||
"mm:network:write",
|
||||
)
|
||||
assert.equal(
|
||||
permissionForRequest("GET", "/api/users"),
|
||||
"mm:users:read",
|
||||
|
||||
@@ -107,7 +107,7 @@ const RULES: Rule[] = [
|
||||
},
|
||||
{
|
||||
methods: ["GET"],
|
||||
match: (p) => p.startsWith("/api/traffic"),
|
||||
match: (p) => p.startsWith("/api/traffic") || p.startsWith("/api/statistics"),
|
||||
permission: "mm:traffic:read",
|
||||
},
|
||||
{
|
||||
@@ -155,7 +155,8 @@ const RULES: Rule[] = [
|
||||
p.startsWith("/api/internet-path") ||
|
||||
p.startsWith("/api/exec") ||
|
||||
p.startsWith("/api/wireguard") ||
|
||||
p.startsWith("/api/firewall"),
|
||||
p.startsWith("/api/firewall") ||
|
||||
p.startsWith("/api/gre"),
|
||||
permission: "mm:network:read",
|
||||
},
|
||||
{
|
||||
@@ -168,7 +169,8 @@ const RULES: Rule[] = [
|
||||
p.startsWith("/api/internet-path") ||
|
||||
p.startsWith("/api/exec") ||
|
||||
p.startsWith("/api/wireguard") ||
|
||||
p.startsWith("/api/firewall"),
|
||||
p.startsWith("/api/firewall") ||
|
||||
p.startsWith("/api/gre"),
|
||||
permission: "mm:network:write",
|
||||
},
|
||||
]
|
||||
|
||||
@@ -19,5 +19,31 @@ export function managedComment(label: string): string {
|
||||
}
|
||||
|
||||
export function managedRecursiveComment(comment?: string | null): string {
|
||||
return comment ? `${PRODUCT_NAME}:recursive ${comment}` : `${PRODUCT_NAME}:recursive`
|
||||
const stripped = stripManagedRecursiveComment(comment ?? "")
|
||||
return stripped ? `${PRODUCT_NAME}:recursive ${stripped}` : `${PRODUCT_NAME}:recursive`
|
||||
}
|
||||
|
||||
const LEGACY_RECURSIVE_PREFIX = /^recursive:\s*/i
|
||||
|
||||
export function stripManagedRecursiveComment(comment: string): string {
|
||||
const value = comment.trim()
|
||||
if (!value) return ""
|
||||
const managedPrefixes = [
|
||||
`${PRODUCT_NAME}:recursive`,
|
||||
`${LEGACY_PRODUCT_NAME}:recursive`,
|
||||
]
|
||||
for (const prefix of managedPrefixes) {
|
||||
if (value.startsWith(prefix)) return value.slice(prefix.length).trim()
|
||||
}
|
||||
if (LEGACY_RECURSIVE_PREFIX.test(value)) {
|
||||
return value.replace(LEGACY_RECURSIVE_PREFIX, "").trim()
|
||||
}
|
||||
return value
|
||||
}
|
||||
|
||||
/** Owned recursive route: MM/legacy prefix or old `recursive:` mask. */
|
||||
export function isOwnedRecursiveComment(comment: string | undefined): boolean {
|
||||
if (!comment) return false
|
||||
const value = comment.trim()
|
||||
return hasManagedRecursiveComment(value) || LEGACY_RECURSIVE_PREFIX.test(value)
|
||||
}
|
||||
|
||||
@@ -0,0 +1,58 @@
|
||||
import { z } from "zod"
|
||||
import type { FastifyPluginAsyncZod } from "@fastify/type-provider-zod"
|
||||
import {
|
||||
getEnabledServerById,
|
||||
listContainers,
|
||||
listContainersForServer,
|
||||
removeContainer,
|
||||
restartContainer,
|
||||
startContainer,
|
||||
stopContainer,
|
||||
} from "../services/containers-live.js"
|
||||
import { ServerIdParamSchema, type ServerIdParams } from "../types/server.js"
|
||||
|
||||
const RosIdBodySchema = z.object({
|
||||
rosId: z.string().min(1),
|
||||
})
|
||||
|
||||
type RosIdBody = z.infer<typeof RosIdBodySchema>
|
||||
type MutateFn = typeof startContainer
|
||||
|
||||
const containersRoutes: FastifyPluginAsyncZod = async (app) => {
|
||||
app.get("/containers", async (_req, reply) => {
|
||||
const containers = await listContainers()
|
||||
return reply.send({ containers })
|
||||
})
|
||||
|
||||
app.get("/servers/:id/containers", { schema: { params: ServerIdParamSchema } }, async (req, reply) => {
|
||||
const params = req.params as ServerIdParams
|
||||
const server = await getEnabledServerById(params.id)
|
||||
if (!server) return reply.status(404).send({ error: "Server not found" })
|
||||
const containers = await listContainersForServer(server)
|
||||
return reply.send({ containers })
|
||||
})
|
||||
|
||||
function registerMutate(path: string, fn: MutateFn) {
|
||||
app.post(path, { schema: { params: ServerIdParamSchema, body: RosIdBodySchema } }, async (req, reply) => {
|
||||
const params = req.params as ServerIdParams
|
||||
const body = req.body as RosIdBody
|
||||
const server = await getEnabledServerById(params.id)
|
||||
if (!server) return reply.status(404).send({ error: "Server not found" })
|
||||
try {
|
||||
await fn(server, body.rosId)
|
||||
return reply.send({ ok: true })
|
||||
} catch (err) {
|
||||
return reply.status(502).send({
|
||||
error: err instanceof Error ? err.message : "Ошибка RouterOS",
|
||||
})
|
||||
}
|
||||
})
|
||||
}
|
||||
|
||||
registerMutate("/servers/:id/containers/start", startContainer)
|
||||
registerMutate("/servers/:id/containers/stop", stopContainer)
|
||||
registerMutate("/servers/:id/containers/restart", restartContainer)
|
||||
registerMutate("/servers/:id/containers/remove", removeContainer)
|
||||
}
|
||||
|
||||
export default containersRoutes
|
||||
+204
-259
@@ -1,14 +1,20 @@
|
||||
import { and, asc, eq, inArray } from "drizzle-orm"
|
||||
import { and, asc, eq } from "drizzle-orm"
|
||||
import { z } from "zod"
|
||||
import type { FastifyPluginAsyncZod } from "@fastify/type-provider-zod"
|
||||
import { db } from "../db/index.js"
|
||||
import { filterRules, recursiveRoutes, servers } from "../db/schema.js"
|
||||
import { MikrotikClient } from "../services/mikrotik.js"
|
||||
import { parseDbServerId } from "../utils/server-id.js"
|
||||
import { appendEvent } from "../modules/events/service/events-service.js"
|
||||
import { managedComment } from "../managed-markers.js"
|
||||
import { planBgpInApply } from "../services/config-apply-plan.js"
|
||||
import {
|
||||
hasManagedCommentPrefix,
|
||||
managedComment,
|
||||
} from "../managed-markers.js"
|
||||
appendRevisionIfChanged,
|
||||
canonicalFilterRules,
|
||||
getRevisionById,
|
||||
listRevisions,
|
||||
type ConfigRevisionSource,
|
||||
} from "../services/config-revisions.js"
|
||||
|
||||
type ServerRow = typeof servers.$inferSelect
|
||||
|
||||
@@ -296,47 +302,6 @@ async function resolveRouteTargets(serverId: number, rule: ApiFilterRule): Promi
|
||||
return { gateway: rule.gateway, outIface: tid }
|
||||
}
|
||||
|
||||
function normalizeCommunity(c: string): string {
|
||||
return (c ?? "").trim()
|
||||
}
|
||||
|
||||
/** Одинаковый эффект на роутере при одинаковой community (blackhole vs gateway + out-interface) */
|
||||
async function ruleEffectSignature(serverId: number, r: ApiFilterRule): Promise<string> {
|
||||
if (r.action === "blackhole") return `bh:${normalizeCommunity(r.community)}`
|
||||
const { gateway, outIface } = await resolveRouteTargets(serverId, r)
|
||||
return `rt:${normalizeCommunity(r.community)}:${gateway}:${outIface}`
|
||||
}
|
||||
|
||||
export type FilterRouterCompareStatus = "synced" | "drift" | "missing"
|
||||
|
||||
async function compareDbRulesWithRouter(
|
||||
serverId: number,
|
||||
dbRules: ApiFilterRule[],
|
||||
remoteRules: ApiFilterRule[],
|
||||
): Promise<Record<string, FilterRouterCompareStatus>> {
|
||||
const remoteSigByComm = new Map<string, string>()
|
||||
for (const rr of remoteRules) {
|
||||
const c = normalizeCommunity(rr.community)
|
||||
if (!remoteSigByComm.has(c)) {
|
||||
remoteSigByComm.set(c, await ruleEffectSignature(serverId, rr))
|
||||
}
|
||||
}
|
||||
const out: Record<string, FilterRouterCompareStatus> = {}
|
||||
for (const dr of dbRules) {
|
||||
const c = normalizeCommunity(dr.community)
|
||||
const sigD = await ruleEffectSignature(serverId, dr)
|
||||
const sigR = remoteSigByComm.get(c)
|
||||
if (sigR === undefined) {
|
||||
out[c] = "missing"
|
||||
} else if (sigR !== sigD) {
|
||||
out[c] = "drift"
|
||||
} else {
|
||||
out[c] = "synced"
|
||||
}
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
async function toRouterRuleBody(serverId: number, rules: ApiFilterRule[]): Promise<string> {
|
||||
if (rules.length === 0) return ""
|
||||
// Группируем по эффекту (action + gateway + out-interface). Communities с одним и тем же
|
||||
@@ -421,44 +386,80 @@ async function replaceDbRules(serverId: number, rules: ApiFilterRule[]) {
|
||||
)
|
||||
}
|
||||
|
||||
const filtersRoutes: FastifyPluginAsyncZod = async (app) => {
|
||||
/** Сравнение правил в БД с живым bgp-in на MikroTik (один запрос API к роутеру) */
|
||||
app.get("/filters/router-compare", async (req, reply) => {
|
||||
const q = req.query as { serverId?: string | number }
|
||||
const serverId = parseDbServerId(q.serverId)
|
||||
if (serverId === null) {
|
||||
return reply.status(400).send({ error: "serverId is required" })
|
||||
}
|
||||
async function cacheRulesetsForServer(serverId: number): Promise<ApiFilterRule[]> {
|
||||
const rows = await db
|
||||
.select()
|
||||
.from(filterRules)
|
||||
.where(eq(filterRules.serverId, serverId))
|
||||
.orderBy(asc(filterRules.sortOrder))
|
||||
return rows.map((r) => ({
|
||||
id: String(r.id),
|
||||
community: r.community,
|
||||
communityName: r.communityName ?? undefined,
|
||||
action: r.action,
|
||||
gateway: r.gateway,
|
||||
gatewayTunnelId: r.gatewayTunnelId,
|
||||
description: r.description,
|
||||
}))
|
||||
}
|
||||
|
||||
const server = (await db.select().from(servers).where(eq(servers.id, serverId)).limit(1))[0]
|
||||
if (!server) return reply.status(404).send({ error: "Server not found" })
|
||||
async function applyFiltersToServer(
|
||||
server: ServerRow,
|
||||
rules: ApiFilterRule[],
|
||||
source: ConfigRevisionSource,
|
||||
): Promise<{ pushed: number; action: string; conflictsRemoved: number }> {
|
||||
const client = MikrotikClient.fromServer(server)
|
||||
const existing = await client.get<RosFilterRule[]>("/routing/filter/rule")
|
||||
const plan = planBgpInApply(existing, rules.length)
|
||||
const managedCommentValue = managedComment(server.name || server.host)
|
||||
|
||||
try {
|
||||
const remote = await fetchServerFilters(server)
|
||||
const rows = await db
|
||||
.select()
|
||||
.from(filterRules)
|
||||
.where(eq(filterRules.serverId, serverId))
|
||||
.orderBy(asc(filterRules.sortOrder))
|
||||
if (plan.action === "patch" && plan.managedId) {
|
||||
const ruleBody = await toRouterRuleBody(server.id, rules)
|
||||
await client.patch(
|
||||
`/routing/filter/rule/${encodeURIComponent(plan.managedId)}`,
|
||||
{
|
||||
chain: "bgp-in",
|
||||
comment: managedCommentValue,
|
||||
rule: ruleBody,
|
||||
disabled: "no",
|
||||
},
|
||||
)
|
||||
} else if (plan.action === "create") {
|
||||
const ruleBody = await toRouterRuleBody(server.id, rules)
|
||||
await client.post("/routing/filter/rule/add", {
|
||||
chain: "bgp-in",
|
||||
comment: managedCommentValue,
|
||||
rule: ruleBody,
|
||||
})
|
||||
} else if (plan.action === "delete" && plan.managedId) {
|
||||
await client.delete(
|
||||
`/routing/filter/rule/${encodeURIComponent(plan.managedId)}`,
|
||||
)
|
||||
}
|
||||
|
||||
const dbRules: ApiFilterRule[] = rows.map(r => ({
|
||||
id: String(r.id),
|
||||
community: r.community,
|
||||
communityName: r.communityName ?? undefined,
|
||||
action: r.action,
|
||||
gateway: r.gateway,
|
||||
gatewayTunnelId: r.gatewayTunnelId,
|
||||
description: r.description,
|
||||
}))
|
||||
for (const id of plan.conflictIds) {
|
||||
await client.delete(`/routing/filter/rule/${encodeURIComponent(id)}`)
|
||||
}
|
||||
|
||||
const byCommunity = await compareDbRulesWithRouter(serverId, dbRules, remote.rules)
|
||||
return reply.send({ byCommunity })
|
||||
} catch (err) {
|
||||
app.log.error({ serverId, err: String(err) }, "filters router-compare failed")
|
||||
return reply.status(500).send({ error: String(err) })
|
||||
}
|
||||
await replaceDbRules(server.id, rules)
|
||||
await appendRevisionIfChanged({
|
||||
serverId: server.id,
|
||||
section: "filters",
|
||||
source,
|
||||
payload: canonicalFilterRules(rules),
|
||||
})
|
||||
|
||||
return {
|
||||
pushed: rules.length,
|
||||
action: plan.action,
|
||||
conflictsRemoved: plan.conflictIds.length,
|
||||
}
|
||||
}
|
||||
|
||||
const RevisionIdParamSchema = z.object({ id: z.string().min(1) })
|
||||
|
||||
const filtersRoutes: FastifyPluginAsyncZod = async (app) => {
|
||||
|
||||
/** GRE с роутеров: один сервер (?serverId) или все включённые (без query) — для /gre, карты сети */
|
||||
app.get("/filters/gre-tunnels", async (req, reply) => {
|
||||
const q = req.query as { serverId?: string | number }
|
||||
@@ -487,74 +488,103 @@ const filtersRoutes: FastifyPluginAsyncZod = async (app) => {
|
||||
return reply.send({ tunnels: results.flat() })
|
||||
})
|
||||
|
||||
/** Только правила фильтров из БД (без опроса MikroTik за GRE) */
|
||||
app.get("/filters/rules", async (_req, reply) => {
|
||||
/** Без serverId — cache для дашборда. С serverId — live с CHR, cache fallback. */
|
||||
app.get("/filters/rules", async (req, reply) => {
|
||||
const q = req.query as { serverId?: string | number }
|
||||
const serverId = parseDbServerId(q.serverId)
|
||||
const allServers = await db.select().from(servers).where(eq(servers.enabled, true))
|
||||
const dbRulesets = await toApiRulesets(allServers)
|
||||
|
||||
return reply.send({
|
||||
rulesets: dbRulesets,
|
||||
greTunnels: [] as LiveGreTunnel[],
|
||||
})
|
||||
})
|
||||
|
||||
app.put("/filters/rules", async (req, reply) => {
|
||||
const body = req.body as { rulesets?: Array<{ serverId: string; rules: ApiFilterRule[] }> }
|
||||
const payload = body.rulesets ?? []
|
||||
const serverIds = payload.map(r => Number.parseInt(r.serverId, 10)).filter(Number.isFinite)
|
||||
if (serverIds.length > 0) {
|
||||
await db.delete(filterRules).where(inArray(filterRules.serverId, serverIds))
|
||||
}
|
||||
for (const rs of payload) {
|
||||
const sid = Number.parseInt(rs.serverId, 10)
|
||||
if (!Number.isFinite(sid)) continue
|
||||
await replaceDbRules(sid, rs.rules ?? [])
|
||||
}
|
||||
return reply.send({ ok: true })
|
||||
})
|
||||
|
||||
app.post("/filters/sync/from-router", async (_req, reply) => {
|
||||
const body = _req.body as { serverId?: string | number } | undefined
|
||||
const rawServerId = body?.serverId
|
||||
const serverId = parseDbServerId(rawServerId)
|
||||
if (serverId === null) {
|
||||
return reply.status(400).send({ error: "serverId is required" })
|
||||
const dbRulesets = await toApiRulesets(allServers)
|
||||
return reply.send({
|
||||
rulesets: dbRulesets,
|
||||
greTunnels: [] as LiveGreTunnel[],
|
||||
live: false,
|
||||
stale: false,
|
||||
})
|
||||
}
|
||||
|
||||
const server = (await db.select().from(servers).where(eq(servers.id, serverId)).limit(1))[0]
|
||||
const server = allServers.find((s) => s.id === serverId)
|
||||
?? (await db.select().from(servers).where(eq(servers.id, serverId)).limit(1))[0]
|
||||
if (!server) return reply.status(404).send({ error: "Server not found" })
|
||||
|
||||
try {
|
||||
app.log.info({ serverId, host: server.host }, "Filters sync from router started")
|
||||
await appendEvent({
|
||||
level: "info",
|
||||
eventType: "filters.sync.from_router.started",
|
||||
sourceModule: "filters",
|
||||
title: "Синхронизация фильтров запущена",
|
||||
message: `${server.name || server.host} → БД`,
|
||||
entityType: "server",
|
||||
entityId: String(serverId),
|
||||
})
|
||||
const remote = await fetchServerFilters(server)
|
||||
await replaceDbRules(server.id, remote.rules)
|
||||
app.log.info({ serverId, totalRules: remote.rules.length }, "Filters sync from router completed")
|
||||
await appendRevisionIfChanged({
|
||||
serverId: server.id,
|
||||
section: "filters",
|
||||
source: "observed",
|
||||
payload: canonicalFilterRules(remote.rules),
|
||||
})
|
||||
const cached = await cacheRulesetsForServer(server.id)
|
||||
return reply.send({
|
||||
rulesets: [{ serverId: String(server.id), rules: cached }],
|
||||
greTunnels: remote.tunnels,
|
||||
live: true,
|
||||
stale: false,
|
||||
})
|
||||
} catch (err) {
|
||||
app.log.warn({ serverId, err: String(err) }, "filters live GET failed, serving cache")
|
||||
const cached = await cacheRulesetsForServer(server.id)
|
||||
return reply.send({
|
||||
rulesets: [{ serverId: String(server.id), rules: cached }],
|
||||
greTunnels: [] as LiveGreTunnel[],
|
||||
live: false,
|
||||
stale: true,
|
||||
error: String(err),
|
||||
})
|
||||
}
|
||||
})
|
||||
|
||||
app.put("/filters/rules", async (req, reply) => {
|
||||
const body = req.body as {
|
||||
serverId?: string | number
|
||||
rules?: ApiFilterRule[]
|
||||
source?: ConfigRevisionSource
|
||||
}
|
||||
const serverId = parseDbServerId(body.serverId)
|
||||
if (serverId === null) return reply.status(400).send({ error: "serverId is required" })
|
||||
const server = (await db.select().from(servers).where(eq(servers.id, serverId)).limit(1))[0]
|
||||
if (!server) return reply.status(404).send({ error: "Server not found" })
|
||||
|
||||
const rules = body.rules ?? []
|
||||
const source: ConfigRevisionSource = body.source === "copy" ? "copy" : "apply"
|
||||
try {
|
||||
await appendEvent({
|
||||
level: "info",
|
||||
eventType: "filters.sync.from_router.done",
|
||||
eventType: "filters.apply.started",
|
||||
sourceModule: "filters",
|
||||
title: "Синхронизация фильтров завершена",
|
||||
message: `${server.name || server.host}: ${remote.rules.length} правил`,
|
||||
title: "Применение фильтров на роутер",
|
||||
message: `${server.name || server.host}: ${rules.length} правил`,
|
||||
entityType: "server",
|
||||
entityId: String(serverId),
|
||||
})
|
||||
return reply.send({ ok: true, updatedServers: 1, totalRules: remote.rules.length, serverId })
|
||||
const result = await applyFiltersToServer(server, rules, source)
|
||||
const cached = await cacheRulesetsForServer(server.id)
|
||||
await appendEvent({
|
||||
level: "info",
|
||||
eventType: "filters.apply.done",
|
||||
sourceModule: "filters",
|
||||
title: "Фильтры применены",
|
||||
message: `${server.name || server.host}: ${result.pushed} правил (${result.action})`,
|
||||
entityType: "server",
|
||||
entityId: String(serverId),
|
||||
})
|
||||
return reply.send({
|
||||
ok: true,
|
||||
serverId,
|
||||
pushedRules: result.pushed,
|
||||
action: result.action,
|
||||
rules: cached,
|
||||
})
|
||||
} catch (err) {
|
||||
app.log.error({ serverId, err: String(err) }, "Filters sync from router failed")
|
||||
app.log.error({ serverId, err: String(err) }, "filters apply failed")
|
||||
await appendEvent({
|
||||
level: "critical",
|
||||
eventType: "filters.sync.from_router.failed",
|
||||
eventType: "filters.apply.failed",
|
||||
sourceModule: "filters",
|
||||
title: "Ошибка синхронизации фильтров",
|
||||
title: "Ошибка применения фильтров",
|
||||
message: `${server.name || server.host}: ${String(err)}`,
|
||||
entityType: "server",
|
||||
entityId: String(serverId),
|
||||
@@ -563,145 +593,60 @@ const filtersRoutes: FastifyPluginAsyncZod = async (app) => {
|
||||
}
|
||||
})
|
||||
|
||||
app.post("/filters/sync/to-router", async (req, reply) => {
|
||||
app.get("/filters/revisions", async (req, reply) => {
|
||||
const q = req.query as { serverId?: string | number }
|
||||
const serverId = parseDbServerId(q.serverId)
|
||||
if (serverId === null) return reply.status(400).send({ error: "serverId is required" })
|
||||
const revisions = await listRevisions(serverId, "filters")
|
||||
return reply.send({ revisions })
|
||||
})
|
||||
|
||||
app.post("/filters/revisions/:id/restore", {
|
||||
schema: { params: RevisionIdParamSchema },
|
||||
}, async (req, reply) => {
|
||||
const { id } = req.params
|
||||
const body = req.body as { serverId?: string | number } | undefined
|
||||
const requestedServerId = parseDbServerId(body?.serverId)
|
||||
|
||||
const allServers = await db.select().from(servers).where(eq(servers.enabled, true))
|
||||
const targetServers = requestedServerId !== null
|
||||
? allServers.filter(s => s.id === requestedServerId)
|
||||
: allServers
|
||||
|
||||
if (requestedServerId !== null && targetServers.length === 0) {
|
||||
return reply.status(404).send({ error: "Server not found" })
|
||||
const rev = await getRevisionById(id)
|
||||
if (!rev) return reply.status(404).send({ error: "Revision not found" })
|
||||
if (rev.section !== "filters") return reply.status(400).send({ error: "Revision section mismatch" })
|
||||
const requested = parseDbServerId(body?.serverId)
|
||||
if (requested !== null && requested !== rev.serverId) {
|
||||
return reply.status(400).send({ error: "Revision belongs to another server" })
|
||||
}
|
||||
const server = (await db.select().from(servers).where(eq(servers.id, rev.serverId)).limit(1))[0]
|
||||
if (!server) return reply.status(404).send({ error: "Server not found" })
|
||||
|
||||
let updatedServers = 0
|
||||
let pushedRules = 0
|
||||
const errors: Array<{ serverId: number; error: string }> = []
|
||||
await appendEvent({
|
||||
level: "info",
|
||||
eventType: "filters.sync.to_router.started",
|
||||
sourceModule: "filters",
|
||||
title: "Отправка фильтров на роутеры запущена",
|
||||
message: `Целевых серверов: ${targetServers.length}`,
|
||||
payload: { requestedServerId },
|
||||
})
|
||||
|
||||
for (const server of targetServers) {
|
||||
try {
|
||||
app.log.info({ serverId: server.id, host: server.host }, "Filters sync to router started")
|
||||
const client = MikrotikClient.fromServer(server)
|
||||
const existing = await client.get<RosFilterRule[]>("/routing/filter/rule")
|
||||
|
||||
const isInBgpIn = (r: RosFilterRule) =>
|
||||
(r.chain ?? "").trim().toLowerCase() === "bgp-in"
|
||||
const managedCommentValue = managedComment(server.name || server.host)
|
||||
|
||||
// Уже созданное нами правило — будем PATCH'ить, чтобы сохранить ID/позицию в цепочке.
|
||||
const managedRule = existing.find(
|
||||
r => isInBgpIn(r) && hasManagedCommentPrefix(r.comment ?? ""),
|
||||
)
|
||||
|
||||
// Конфликтующие легаси-правила в bgp-in (без нашего comment, но с bgp-communities) —
|
||||
// удаляем после успешного upsert: иначе старое правило с `else { reject; }`
|
||||
// отрабатывает первым и перебивает наш upsert.
|
||||
const conflictIds = existing
|
||||
.filter(r =>
|
||||
isInBgpIn(r) &&
|
||||
!hasManagedCommentPrefix(r.comment ?? "") &&
|
||||
/bgp-communities/i.test(r.rule ?? ""),
|
||||
)
|
||||
.map(r => r[".id"])
|
||||
.filter((id): id is string => Boolean(id))
|
||||
|
||||
const rows = await db.select().from(filterRules)
|
||||
.where(and(eq(filterRules.serverId, server.id)))
|
||||
.orderBy(asc(filterRules.sortOrder))
|
||||
|
||||
const rules: ApiFilterRule[] = rows.map(r => ({
|
||||
id: String(r.id),
|
||||
community: r.community,
|
||||
communityName: r.communityName ?? undefined,
|
||||
action: r.action,
|
||||
gateway: r.gateway,
|
||||
gatewayTunnelId: r.gatewayTunnelId,
|
||||
description: r.description,
|
||||
}))
|
||||
|
||||
// Upsert: PATCH существующего managed-правила или POST /add нового.
|
||||
// Если ошибка — конфликтные правила НЕ удаляем (роутер не остаётся с пустым bgp-in).
|
||||
// Путь `/routing/filter/rule/add` обязателен: голый POST на коллекцию RouterOS REST
|
||||
// трактует как «вызов команды» и отдаёт 400 «no such command».
|
||||
// См. https://help.mikrotik.com/docs/spaces/ROS/pages/47579162/REST+API
|
||||
if (rules.length > 0) {
|
||||
const ruleBody = await toRouterRuleBody(server.id, rules)
|
||||
if (managedRule && managedRule[".id"]) {
|
||||
await client.patch(
|
||||
`/routing/filter/rule/${encodeURIComponent(managedRule[".id"])}`,
|
||||
{
|
||||
chain: "bgp-in",
|
||||
comment: managedCommentValue,
|
||||
rule: ruleBody,
|
||||
disabled: "no",
|
||||
},
|
||||
)
|
||||
app.log.info({ serverId: server.id, id: managedRule[".id"] }, "bgp-in rule updated")
|
||||
} else {
|
||||
await client.post("/routing/filter/rule/add", {
|
||||
chain: "bgp-in",
|
||||
comment: managedCommentValue,
|
||||
rule: ruleBody,
|
||||
})
|
||||
app.log.info({ serverId: server.id }, "bgp-in rule created")
|
||||
}
|
||||
pushedRules += rules.length
|
||||
} else if (managedRule && managedRule[".id"]) {
|
||||
// В БД нет правил → удаляем наш managed-rule на роутере.
|
||||
await client.delete(
|
||||
`/routing/filter/rule/${encodeURIComponent(managedRule[".id"])}`,
|
||||
)
|
||||
app.log.info({ serverId: server.id }, "bgp-in rule removed (no rules in DB)")
|
||||
}
|
||||
|
||||
for (const id of conflictIds) {
|
||||
await client.delete(`/routing/filter/rule/${encodeURIComponent(id)}`)
|
||||
}
|
||||
|
||||
updatedServers += 1
|
||||
app.log.info(
|
||||
{
|
||||
serverId: server.id,
|
||||
mode: managedRule ? "patch" : "create",
|
||||
conflictsRemoved: conflictIds.length,
|
||||
pushed: rules.length,
|
||||
},
|
||||
"Filters sync to router completed",
|
||||
)
|
||||
} catch (err) {
|
||||
app.log.error({ serverId: server.id, err: String(err) }, "filters sync to-router failed")
|
||||
errors.push({ serverId: server.id, error: String(err) })
|
||||
const raw = Array.isArray(rev.payload) ? rev.payload : []
|
||||
const rules: ApiFilterRule[] = raw.map((item, idx) => {
|
||||
const r = item as Partial<ApiFilterRule>
|
||||
return {
|
||||
id: `rev-${idx}`,
|
||||
community: r.community ?? "",
|
||||
communityName: r.communityName,
|
||||
action: r.action === "blackhole" ? "blackhole" : "route",
|
||||
gateway: r.gateway ?? "",
|
||||
gatewayTunnelId: r.gatewayTunnelId ?? "",
|
||||
description: r.description ?? "",
|
||||
}
|
||||
}
|
||||
})
|
||||
|
||||
await appendEvent({
|
||||
level: errors.length === 0 ? "info" : "warning",
|
||||
eventType: errors.length === 0 ? "filters.sync.to_router.done" : "filters.sync.to_router.partial",
|
||||
sourceModule: "filters",
|
||||
title: errors.length === 0 ? "Отправка фильтров завершена" : "Отправка фильтров завершена с ошибками",
|
||||
message: `Успешно: ${updatedServers}, ошибок: ${errors.length}, правил: ${pushedRules}`,
|
||||
payload: {
|
||||
updatedServers,
|
||||
pushedRules,
|
||||
errors,
|
||||
},
|
||||
})
|
||||
return reply.send({
|
||||
ok: errors.length === 0,
|
||||
updatedServers,
|
||||
pushedRules,
|
||||
errors,
|
||||
})
|
||||
try {
|
||||
const result = await applyFiltersToServer(server, rules, "rollback")
|
||||
const cached = await cacheRulesetsForServer(server.id)
|
||||
await appendEvent({
|
||||
level: "info",
|
||||
eventType: "filters.rollback.done",
|
||||
sourceModule: "filters",
|
||||
title: "Откат фильтров",
|
||||
message: `${server.name || server.host}: ${result.pushed} правил`,
|
||||
entityType: "server",
|
||||
entityId: String(server.id),
|
||||
})
|
||||
return reply.send({ ok: true, rules: cached, pushedRules: result.pushed })
|
||||
} catch (err) {
|
||||
app.log.error({ serverId: server.id, err: String(err) }, "filters restore failed")
|
||||
return reply.status(500).send({ error: String(err) })
|
||||
}
|
||||
})
|
||||
}
|
||||
|
||||
|
||||
@@ -2,7 +2,20 @@ import type { FastifyPluginAsyncZod } from "@fastify/type-provider-zod"
|
||||
import { z } from "zod"
|
||||
import { MikrotikClient, MikrotikError, encodeRosId, firewallRestPath } from "../services/mikrotik.js"
|
||||
import { getEnabledServerById } from "../services/wireguard-live.js"
|
||||
import { listFirewallAll } from "../services/firewall-live.js"
|
||||
import {
|
||||
captureFirewallSnapshot,
|
||||
fetchFirewallState,
|
||||
listFirewallAll,
|
||||
} from "../services/firewall-live.js"
|
||||
import {
|
||||
captureAndAppendRevision,
|
||||
listRevisions,
|
||||
loadRevisionForRestore,
|
||||
type ConfigRevisionSource,
|
||||
} from "../services/config-revisions.js"
|
||||
import { parseFirewallSnapshot, planFirewallRestore } from "../services/entity-snapshots.js"
|
||||
import { executeRosOps } from "../services/ros-ops.js"
|
||||
import { parseDbServerId } from "../utils/server-id.js"
|
||||
import type { FirewallFamily, FirewallTable } from "../types/server.js"
|
||||
|
||||
const FamilySchema = z.enum(["ip", "ip6"])
|
||||
@@ -120,6 +133,20 @@ async function requireServer(serverId: string) {
|
||||
return await getEnabledServerById(serverId)
|
||||
}
|
||||
|
||||
async function recordFirewall(
|
||||
server: NonNullable<Awaited<ReturnType<typeof requireServer>>>,
|
||||
source: ConfigRevisionSource,
|
||||
) {
|
||||
await captureAndAppendRevision({
|
||||
serverId: server.id,
|
||||
section: "firewall",
|
||||
source,
|
||||
capture: () => captureFirewallSnapshot(server),
|
||||
})
|
||||
}
|
||||
|
||||
const RevisionIdParamSchema = z.object({ id: z.string().min(1) })
|
||||
|
||||
const firewallRoutes: FastifyPluginAsyncZod = async (app) => {
|
||||
app.get("/firewall/all", async (_req, reply) => {
|
||||
const data = await listFirewallAll()
|
||||
@@ -138,6 +165,7 @@ const firewallRoutes: FastifyPluginAsyncZod = async (app) => {
|
||||
const path = firewallRestPath(body.family as FirewallFamily, body.table as FirewallTable)
|
||||
try {
|
||||
await client.put(path, ruleToRos(body))
|
||||
await recordFirewall(server, "apply")
|
||||
return reply.status(201).send({ ok: true })
|
||||
} catch (e) {
|
||||
return reply.status(502).send({ error: `RouterOS: ${rosErr(e)}` })
|
||||
@@ -156,6 +184,7 @@ const firewallRoutes: FastifyPluginAsyncZod = async (app) => {
|
||||
const path = `${firewallRestPath(body.family as FirewallFamily, body.table as FirewallTable)}/${encodeRosId(body.rosId)}`
|
||||
try {
|
||||
await client.patch(path, ruleToRos(body))
|
||||
await recordFirewall(server, "apply")
|
||||
return reply.send({ ok: true })
|
||||
} catch (e) {
|
||||
return reply.status(502).send({ error: `RouterOS: ${rosErr(e)}` })
|
||||
@@ -174,6 +203,7 @@ const firewallRoutes: FastifyPluginAsyncZod = async (app) => {
|
||||
const path = `${firewallRestPath(body.family, body.table)}/${encodeRosId(body.rosId)}`
|
||||
try {
|
||||
await client.patch(path, { disabled: body.disabled ? "yes" : "no" })
|
||||
await recordFirewall(server, "apply")
|
||||
return reply.send({ ok: true })
|
||||
} catch (e) {
|
||||
return reply.status(502).send({ error: `RouterOS: ${rosErr(e)}` })
|
||||
@@ -192,6 +222,7 @@ const firewallRoutes: FastifyPluginAsyncZod = async (app) => {
|
||||
const path = `${firewallRestPath(body.family, body.table)}/${encodeRosId(body.rosId)}`
|
||||
try {
|
||||
await client.delete(path)
|
||||
await recordFirewall(server, "apply")
|
||||
return reply.send({ ok: true })
|
||||
} catch (e) {
|
||||
return reply.status(502).send({ error: `RouterOS: ${rosErr(e)}` })
|
||||
@@ -213,6 +244,7 @@ const firewallRoutes: FastifyPluginAsyncZod = async (app) => {
|
||||
numbers: body.rosId,
|
||||
...(body.destinationRosId ? { destination: body.destinationRosId } : {}),
|
||||
})
|
||||
await recordFirewall(server, "apply")
|
||||
return reply.send({ ok: true })
|
||||
} catch (e) {
|
||||
return reply.status(502).send({ error: `RouterOS: ${rosErr(e)}` })
|
||||
@@ -230,6 +262,7 @@ const firewallRoutes: FastifyPluginAsyncZod = async (app) => {
|
||||
const client = MikrotikClient.fromServer(server)
|
||||
try {
|
||||
await client.put(firewallRestPath(body.family, "address-list"), addressToRos(body))
|
||||
await recordFirewall(server, "apply")
|
||||
return reply.status(201).send({ ok: true })
|
||||
} catch (e) {
|
||||
return reply.status(502).send({ error: `RouterOS: ${rosErr(e)}` })
|
||||
@@ -248,6 +281,7 @@ const firewallRoutes: FastifyPluginAsyncZod = async (app) => {
|
||||
const path = `${firewallRestPath(body.family, "address-list")}/${encodeRosId(body.rosId)}`
|
||||
try {
|
||||
await client.patch(path, addressToRos(body))
|
||||
await recordFirewall(server, "apply")
|
||||
return reply.send({ ok: true })
|
||||
} catch (e) {
|
||||
return reply.status(502).send({ error: `RouterOS: ${rosErr(e)}` })
|
||||
@@ -266,6 +300,7 @@ const firewallRoutes: FastifyPluginAsyncZod = async (app) => {
|
||||
const path = `${firewallRestPath(body.family, "address-list")}/${encodeRosId(body.rosId)}`
|
||||
try {
|
||||
await client.patch(path, { disabled: body.disabled ? "yes" : "no" })
|
||||
await recordFirewall(server, "apply")
|
||||
return reply.send({ ok: true })
|
||||
} catch (e) {
|
||||
return reply.status(502).send({ error: `RouterOS: ${rosErr(e)}` })
|
||||
@@ -284,11 +319,48 @@ const firewallRoutes: FastifyPluginAsyncZod = async (app) => {
|
||||
const path = `${firewallRestPath(body.family, "address-list")}/${encodeRosId(body.rosId)}`
|
||||
try {
|
||||
await client.delete(path)
|
||||
await recordFirewall(server, "apply")
|
||||
return reply.send({ ok: true })
|
||||
} catch (e) {
|
||||
return reply.status(502).send({ error: `RouterOS: ${rosErr(e)}` })
|
||||
}
|
||||
})
|
||||
|
||||
app.get("/firewall/revisions", async (req, reply) => {
|
||||
const q = req.query as { serverId?: string | number }
|
||||
const serverId = parseDbServerId(q.serverId)
|
||||
if (serverId === null) return reply.status(400).send({ error: "serverId is required" })
|
||||
const revisions = await listRevisions(serverId, "firewall")
|
||||
return reply.send({ revisions })
|
||||
})
|
||||
|
||||
app.post("/firewall/revisions/:id/restore", {
|
||||
schema: { params: RevisionIdParamSchema },
|
||||
}, async (req, reply) => {
|
||||
const { id } = req.params
|
||||
const body = req.body as { serverId?: string | number } | undefined
|
||||
const loaded = await loadRevisionForRestore({
|
||||
id,
|
||||
section: "firewall",
|
||||
requestedServerId: parseDbServerId(body?.serverId),
|
||||
})
|
||||
if (!loaded.ok) return reply.status(loaded.status).send({ error: loaded.error })
|
||||
const client = MikrotikClient.fromServer(loaded.server)
|
||||
try {
|
||||
const desired = parseFirewallSnapshot(loaded.row.payload)
|
||||
const state = await fetchFirewallState(loaded.server)
|
||||
const ops = planFirewallRestore(desired, {
|
||||
rules: state.liveRules,
|
||||
addressLists: state.liveLists,
|
||||
})
|
||||
await executeRosOps(client, ops)
|
||||
await recordFirewall(loaded.server, "rollback")
|
||||
const next = await fetchFirewallState(loaded.server)
|
||||
return reply.send({ ok: true, rules: next.rules, addressLists: next.addressLists })
|
||||
} catch (e) {
|
||||
return reply.status(502).send({ error: `RouterOS: ${rosErr(e)}` })
|
||||
}
|
||||
})
|
||||
}
|
||||
|
||||
export default firewallRoutes
|
||||
|
||||
@@ -0,0 +1,229 @@
|
||||
import type { FastifyPluginAsyncZod } from "@fastify/type-provider-zod"
|
||||
import { z } from "zod"
|
||||
import { MikrotikClient, MikrotikError } from "../services/mikrotik.js"
|
||||
import { getEnabledServerById } from "../services/wireguard-live.js"
|
||||
import {
|
||||
captureGreSnapshot,
|
||||
fetchGreState,
|
||||
formatKeepalive,
|
||||
listGreTunnels,
|
||||
parseKeepalive,
|
||||
} from "../services/gre-live.js"
|
||||
import {
|
||||
canonicalGreSnapshot,
|
||||
parseGreSnapshot,
|
||||
planGreCreate,
|
||||
planGreDelete,
|
||||
planGreRestore,
|
||||
} from "../services/entity-snapshots.js"
|
||||
import { executeRosOps } from "../services/ros-ops.js"
|
||||
import {
|
||||
captureAndAppendRevision,
|
||||
listRevisions,
|
||||
loadRevisionForRestore,
|
||||
type ConfigRevisionSource,
|
||||
} from "../services/config-revisions.js"
|
||||
import { parseDbServerId } from "../utils/server-id.js"
|
||||
|
||||
const TunnelWriteSchema = z.object({
|
||||
serverId: z.string().min(1),
|
||||
name: z.string().min(1),
|
||||
rosId: z.string().optional(),
|
||||
localAddress: z.string().optional(),
|
||||
remoteAddress: z.string().min(1),
|
||||
localInnerIp: z.string().optional(),
|
||||
remoteInnerIp: z.string().optional(),
|
||||
comment: z.string().optional(),
|
||||
enabled: z.boolean().optional(),
|
||||
mtu: z.number().optional(),
|
||||
keepaliveInterval: z.number().optional(),
|
||||
keepaliveRetries: z.number().optional(),
|
||||
dscp: z.union([z.literal("inherit"), z.number(), z.string()]).optional(),
|
||||
clampTcpMss: z.boolean().optional(),
|
||||
allowFastPath: z.boolean().optional(),
|
||||
ipsecSecret: z.string().optional(),
|
||||
})
|
||||
|
||||
const TunnelKeySchema = z.object({
|
||||
serverId: z.string().min(1),
|
||||
rosId: z.string().optional(),
|
||||
name: z.string().optional(),
|
||||
disabled: z.boolean().optional(),
|
||||
})
|
||||
|
||||
const RevisionIdParamSchema = z.object({ id: z.string().min(1) })
|
||||
|
||||
function rosErr(e: unknown): string {
|
||||
if (e instanceof MikrotikError) return e.message
|
||||
if (e instanceof Error) return e.message
|
||||
return String(e)
|
||||
}
|
||||
|
||||
async function recordGre(
|
||||
server: NonNullable<Awaited<ReturnType<typeof getEnabledServerById>>>,
|
||||
source: ConfigRevisionSource,
|
||||
) {
|
||||
await captureAndAppendRevision({
|
||||
serverId: server.id,
|
||||
section: "gre",
|
||||
source,
|
||||
capture: () => captureGreSnapshot(server),
|
||||
})
|
||||
}
|
||||
|
||||
function tunnelFromBody(body: z.infer<typeof TunnelWriteSchema> & { keepaliveInterval?: number; keepaliveRetries?: number }) {
|
||||
const dscp = body.dscp == null
|
||||
? "inherit"
|
||||
: typeof body.dscp === "number"
|
||||
? String(body.dscp)
|
||||
: body.dscp
|
||||
const keepalive = body.keepaliveInterval === undefined && body.keepaliveRetries === undefined
|
||||
? undefined
|
||||
: formatKeepalive(body.keepaliveInterval ?? 0, body.keepaliveRetries ?? 10)
|
||||
return canonicalGreSnapshot({
|
||||
tunnels: [{
|
||||
name: body.name,
|
||||
localAddress: body.localAddress ?? "",
|
||||
remoteAddress: body.remoteAddress,
|
||||
localInnerIp: body.localInnerIp ?? "",
|
||||
remoteInnerIp: body.remoteInnerIp ?? "",
|
||||
comment: body.comment ?? "",
|
||||
disabled: body.enabled === false,
|
||||
mtu: body.mtu ?? 1476,
|
||||
keepalive: keepalive ?? "0",
|
||||
dscp,
|
||||
clampTcpMss: body.clampTcpMss,
|
||||
allowFastPath: body.allowFastPath,
|
||||
ipsecSecret: body.ipsecSecret ?? "",
|
||||
}],
|
||||
}).tunnels[0]!
|
||||
}
|
||||
|
||||
const greRoutes: FastifyPluginAsyncZod = async (app) => {
|
||||
app.get("/gre/tunnels", async (req, reply) => {
|
||||
const q = req.query as { serverId?: string | number }
|
||||
const sid = parseDbServerId(q.serverId)
|
||||
const result = await listGreTunnels({ serverId: sid !== null ? String(sid) : undefined })
|
||||
return reply.send(result)
|
||||
})
|
||||
|
||||
app.post("/gre/tunnels", async (req, reply) => {
|
||||
const parsed = TunnelWriteSchema.safeParse(req.body ?? {})
|
||||
if (!parsed.success) {
|
||||
return reply.status(400).send({ error: "Некорректное тело запроса", details: parsed.error.flatten() })
|
||||
}
|
||||
const body = parsed.data
|
||||
const server = await getEnabledServerById(body.serverId)
|
||||
if (!server) return reply.status(404).send({ error: "Сервер не найден" })
|
||||
const client = MikrotikClient.fromServer(server)
|
||||
try {
|
||||
const tunnel = tunnelFromBody(body)
|
||||
await executeRosOps(client, planGreCreate(tunnel))
|
||||
await recordGre(server, "apply")
|
||||
const state = await fetchGreState(server)
|
||||
const created = state.tunnels.find((t) => t.name === tunnel.name)
|
||||
return reply.status(201).send(created ?? { ok: true, name: tunnel.name })
|
||||
} catch (e) {
|
||||
return reply.status(502).send({ error: `RouterOS: ${rosErr(e)}` })
|
||||
}
|
||||
})
|
||||
|
||||
app.patch("/gre/tunnels", async (req, reply) => {
|
||||
const parsed = TunnelWriteSchema.partial().required({ serverId: true }).safeParse(req.body ?? {})
|
||||
if (!parsed.success) {
|
||||
return reply.status(400).send({ error: "Некорректное тело запроса", details: parsed.error.flatten() })
|
||||
}
|
||||
const body = parsed.data
|
||||
const server = await getEnabledServerById(body.serverId)
|
||||
if (!server) return reply.status(404).send({ error: "Сервер не найден" })
|
||||
try {
|
||||
const state = await fetchGreState(server)
|
||||
const live = state.gre.find((g) =>
|
||||
(body.rosId && g.rosId === body.rosId) || (body.name && g.name === body.name),
|
||||
)
|
||||
if (!live) return reply.status(404).send({ error: "Туннель не найден" })
|
||||
const merged = tunnelFromBody({
|
||||
serverId: body.serverId,
|
||||
name: body.name || live.name,
|
||||
localAddress: body.localAddress ?? live.localAddress,
|
||||
remoteAddress: body.remoteAddress || live.remoteAddress,
|
||||
localInnerIp: body.localInnerIp ?? state.addrs.find((a) => a.interfaceName === live.name)?.address ?? "",
|
||||
remoteInnerIp: body.remoteInnerIp,
|
||||
comment: body.comment ?? live.comment,
|
||||
enabled: body.enabled ?? !live.disabled,
|
||||
mtu: body.mtu ?? live.mtu,
|
||||
keepaliveInterval: body.keepaliveInterval ?? parseKeepalive(live.keepalive).interval,
|
||||
keepaliveRetries: body.keepaliveRetries ?? parseKeepalive(live.keepalive).retries,
|
||||
dscp: body.dscp ?? live.dscp,
|
||||
clampTcpMss: body.clampTcpMss ?? live.clampTcpMss,
|
||||
allowFastPath: body.allowFastPath ?? live.allowFastPath,
|
||||
ipsecSecret: body.ipsecSecret ?? live.ipsecSecret,
|
||||
})
|
||||
const ops = planGreRestore(
|
||||
{ tunnels: state.snapshot.tunnels.map((t) => t.name === live.name ? merged : t) },
|
||||
{ gre: state.gre, addrs: state.addrs },
|
||||
)
|
||||
await executeRosOps(state.client, ops)
|
||||
await recordGre(server, "apply")
|
||||
return reply.send({ ok: true })
|
||||
} catch (e) {
|
||||
return reply.status(502).send({ error: `RouterOS: ${rosErr(e)}` })
|
||||
}
|
||||
})
|
||||
|
||||
app.delete("/gre/tunnels", async (req, reply) => {
|
||||
const parsed = TunnelKeySchema.safeParse(req.body ?? {})
|
||||
if (!parsed.success) {
|
||||
return reply.status(400).send({ error: "Некорректное тело запроса", details: parsed.error.flatten() })
|
||||
}
|
||||
const body = parsed.data
|
||||
const server = await getEnabledServerById(body.serverId)
|
||||
if (!server) return reply.status(404).send({ error: "Сервер не найден" })
|
||||
try {
|
||||
const state = await fetchGreState(server)
|
||||
const live = state.gre.find((g) =>
|
||||
(body.rosId && g.rosId === body.rosId) || (body.name && g.name === body.name),
|
||||
)
|
||||
if (!live) return reply.status(404).send({ error: "Туннель не найден" })
|
||||
await executeRosOps(state.client, planGreDelete(live.name, { gre: state.gre, addrs: state.addrs }))
|
||||
await recordGre(server, "apply")
|
||||
return reply.send({ ok: true })
|
||||
} catch (e) {
|
||||
return reply.status(502).send({ error: `RouterOS: ${rosErr(e)}` })
|
||||
}
|
||||
})
|
||||
|
||||
app.get("/gre/revisions", async (req, reply) => {
|
||||
const q = req.query as { serverId?: string | number }
|
||||
const serverId = parseDbServerId(q.serverId)
|
||||
if (serverId === null) return reply.status(400).send({ error: "serverId is required" })
|
||||
const revisions = await listRevisions(serverId, "gre")
|
||||
return reply.send({ revisions })
|
||||
})
|
||||
|
||||
app.post("/gre/revisions/:id/restore", {
|
||||
schema: { params: RevisionIdParamSchema },
|
||||
}, async (req, reply) => {
|
||||
const { id } = req.params
|
||||
const body = req.body as { serverId?: string | number } | undefined
|
||||
const loaded = await loadRevisionForRestore({
|
||||
id,
|
||||
section: "gre",
|
||||
requestedServerId: parseDbServerId(body?.serverId),
|
||||
})
|
||||
if (!loaded.ok) return reply.status(loaded.status).send({ error: loaded.error })
|
||||
try {
|
||||
const desired = parseGreSnapshot(loaded.row.payload)
|
||||
const state = await fetchGreState(loaded.server)
|
||||
const ops = planGreRestore(desired, { gre: state.gre, addrs: state.addrs })
|
||||
await executeRosOps(state.client, ops)
|
||||
await recordGre(loaded.server, "rollback")
|
||||
const next = await fetchGreState(loaded.server)
|
||||
return reply.send({ ok: true, tunnels: next.tunnels })
|
||||
} catch (e) {
|
||||
return reply.status(502).send({ error: `RouterOS: ${rosErr(e)}` })
|
||||
}
|
||||
})
|
||||
}
|
||||
|
||||
export default greRoutes
|
||||
@@ -6,9 +6,10 @@ import { MikrotikClient } from "../services/mikrotik.js"
|
||||
import { ServerIdParamSchema, type ServerIdParams } from "../types/server.js"
|
||||
import type {
|
||||
RosOspfNeighbor, RosOspfArea, RosOspfInterfaceTemplate, RosOspfInstance,
|
||||
RosBfdSession,
|
||||
OspfNeighborRead, OspfInterfaceRead, OspfInstanceRead, BfdSessionRead,
|
||||
RosBfdSession, RosIpRoute,
|
||||
OspfNeighborRead, OspfInterfaceRead, OspfInstanceRead, OspfRouteRead, BfdSessionRead,
|
||||
} from "../types/server.js"
|
||||
import { parseOspfGateway, parseOspfRouteType } from "../services/ospf-route-parse.js"
|
||||
import { z } from "zod"
|
||||
|
||||
type ServerRow = typeof servers.$inferSelect
|
||||
@@ -74,14 +75,15 @@ function parseAddrIface(addr: string): { ip: string; iface: string } {
|
||||
/** Fetch all OSPF + BFD data for one server */
|
||||
async function fetchServerOspf(server: ServerRow) {
|
||||
const client = MikrotikClient.fromServer(server)
|
||||
const [neighbors, areas, ifaceTemplates, instances, bfdSessions] = await Promise.all([
|
||||
const [neighbors, areas, ifaceTemplates, instances, bfdSessions, ipRoutes] = await Promise.all([
|
||||
client.getOspfNeighbors(),
|
||||
client.getOspfAreas(),
|
||||
client.getOspfInterfaceTemplates(),
|
||||
client.getOspfInstances(),
|
||||
client.getBfdSessions().catch(() => [] as RosBfdSession[]), // BFD is optional
|
||||
client.getIpRoutes().catch(() => [] as RosIpRoute[]),
|
||||
])
|
||||
return { neighbors, areas, ifaceTemplates, instances, bfdSessions }
|
||||
return { neighbors, areas, ifaceTemplates, instances, bfdSessions, ipRoutes }
|
||||
}
|
||||
|
||||
// ── BFD parser ────────────────────────────────────────────────────────────────
|
||||
@@ -200,6 +202,29 @@ function parseInstances(
|
||||
}))
|
||||
}
|
||||
|
||||
function parseOspfRoutes(server: ServerRow, routes: RosIpRoute[]): OspfRouteRead[] {
|
||||
const out: OspfRouteRead[] = []
|
||||
for (const [idx, r] of routes.entries()) {
|
||||
const type = parseOspfRouteType(r)
|
||||
if (!type) continue
|
||||
const { nextHop, via } = parseOspfGateway(r)
|
||||
const metric = parseInt(r["ospf-metric"] ?? r.distance ?? "0") || 0
|
||||
out.push({
|
||||
id: r[".id"] ?? String(idx),
|
||||
serverId: server.id,
|
||||
serverName: server.name || server.host,
|
||||
serverSite: server.site,
|
||||
destination: r["dst-address"] ?? "",
|
||||
type,
|
||||
cost: metric,
|
||||
nextHop,
|
||||
via,
|
||||
area: r["ospf-area"] ?? "",
|
||||
})
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
function calcRouteScore(pingMs: number, dlMbps: number, ulMbps: number, pingWeight: number) {
|
||||
const pingScore = Math.max(0, 100 - pingMs * 0.6)
|
||||
const speedScore = Math.min(100, (dlMbps + ulMbps) / 18)
|
||||
@@ -584,16 +609,17 @@ const ospfRoutes: FastifyPluginAsyncZod = async (app) => {
|
||||
const perServer = await Promise.all(
|
||||
allServers.map(async (server) => {
|
||||
try {
|
||||
const { neighbors, areas, ifaceTemplates, instances, bfdSessions } = await fetchServerOspf(server)
|
||||
const { neighbors, areas, ifaceTemplates, instances, bfdSessions, ipRoutes } = await fetchServerOspf(server)
|
||||
const areaMap = buildAreaMap(areas)
|
||||
return {
|
||||
neighbors: parseNeighbors(server, neighbors, areaMap),
|
||||
interfaces: parseInterfaces(server, ifaceTemplates, areas, instances, areaMap),
|
||||
instances: parseInstances(server, instances),
|
||||
bfdSessions: parseBfdSessions(server, bfdSessions),
|
||||
routes: parseOspfRoutes(server, ipRoutes),
|
||||
}
|
||||
} catch {
|
||||
return { neighbors: [], interfaces: [], instances: [], bfdSessions: [] }
|
||||
return { neighbors: [], interfaces: [], instances: [], bfdSessions: [], routes: [] }
|
||||
}
|
||||
}),
|
||||
)
|
||||
@@ -603,6 +629,7 @@ const ospfRoutes: FastifyPluginAsyncZod = async (app) => {
|
||||
interfaces: perServer.flatMap(r => r.interfaces),
|
||||
instances: perServer.flatMap(r => r.instances),
|
||||
bfdSessions: perServer.flatMap(r => r.bfdSessions),
|
||||
routes: perServer.flatMap(r => r.routes),
|
||||
})
|
||||
})
|
||||
|
||||
@@ -634,13 +661,14 @@ const ospfRoutes: FastifyPluginAsyncZod = async (app) => {
|
||||
if (!server) return reply.status(404).send({ error: "Server not found" })
|
||||
|
||||
try {
|
||||
const { neighbors, areas, ifaceTemplates, instances, bfdSessions } = await fetchServerOspf(server)
|
||||
const { neighbors, areas, ifaceTemplates, instances, bfdSessions, ipRoutes } = await fetchServerOspf(server)
|
||||
const areaMap = buildAreaMap(areas)
|
||||
return reply.send({
|
||||
neighbors: parseNeighbors(server, neighbors, areaMap),
|
||||
interfaces: parseInterfaces(server, ifaceTemplates, areas, instances, areaMap),
|
||||
instances: parseInstances(server, instances),
|
||||
bfdSessions: parseBfdSessions(server, bfdSessions),
|
||||
routes: parseOspfRoutes(server, ipRoutes),
|
||||
areas: areas.map(a => ({ name: a.name, areaId: a["area-id"] ?? "0.0.0.0", type: a.type, disabled: a.disabled === "true", inactive: a.inactive === "true", instance: a.instance })),
|
||||
})
|
||||
} catch (err) {
|
||||
|
||||
@@ -1,13 +1,23 @@
|
||||
import { asc, eq } from "drizzle-orm"
|
||||
import { z } from "zod"
|
||||
import type { FastifyPluginAsyncZod } from "@fastify/type-provider-zod"
|
||||
import { db } from "../db/index.js"
|
||||
import { recursiveRoutes, servers } from "../db/schema.js"
|
||||
import { MikrotikClient } from "../services/mikrotik.js"
|
||||
import { parseDbServerId } from "../utils/server-id.js"
|
||||
import { managedRecursiveComment } from "../managed-markers.js"
|
||||
import {
|
||||
hasManagedRecursiveComment,
|
||||
managedRecursiveComment,
|
||||
} from "../managed-markers.js"
|
||||
mapRosManagedRoutes,
|
||||
planRecursiveApply,
|
||||
userRecursiveComment,
|
||||
} from "../services/config-apply-plan.js"
|
||||
import {
|
||||
appendRevisionIfChanged,
|
||||
canonicalRecursiveRoutes,
|
||||
getRevisionById,
|
||||
listRevisions,
|
||||
type ConfigRevisionSource,
|
||||
} from "../services/config-revisions.js"
|
||||
|
||||
type ServerRow = typeof servers.$inferSelect
|
||||
|
||||
@@ -51,27 +61,6 @@ interface RecursiveRouteDto {
|
||||
disabled: boolean
|
||||
}
|
||||
|
||||
function isIpGateway(gw: string): boolean {
|
||||
return /^\d{1,3}(\.\d{1,3}){3}(?:%\S+)?$/.test(gw.trim())
|
||||
}
|
||||
|
||||
function isRecursiveRoute(r: RosRoute): boolean {
|
||||
if ((r.static ?? "false") !== "true") return false
|
||||
if ((r.dynamic ?? "false") === "true") return false
|
||||
if ((r.blackhole ?? "false") === "true") return false
|
||||
if ((r.unreachable ?? "false") === "true") return false
|
||||
if ((r.prohibit ?? "false") === "true") return false
|
||||
const dst = r["dst-address"] ?? ""
|
||||
const gw = r.gateway ?? ""
|
||||
if (!dst || !gw) return false
|
||||
return isIpGateway(gw)
|
||||
}
|
||||
|
||||
function hasRecursiveCommentMask(comment: string | undefined): boolean {
|
||||
if (!comment) return false
|
||||
return /^recursive:\s*/i.test(comment.trim())
|
||||
}
|
||||
|
||||
function splitGateway(raw: string): { ip: string; name: string } | null {
|
||||
const v = raw.trim()
|
||||
if (!v) return null
|
||||
@@ -102,6 +91,21 @@ async function mapDbRoutes(serverId: number): Promise<RecursiveRouteDto[]> {
|
||||
}))
|
||||
}
|
||||
|
||||
function mergeCachedCountry(
|
||||
live: RecursiveRouteDto[],
|
||||
cached: RecursiveRouteDto[],
|
||||
): RecursiveRouteDto[] {
|
||||
return live.map((row) => {
|
||||
if (row.country) return row
|
||||
const match = cached.find((c) =>
|
||||
c.dstAddress === row.dstAddress &&
|
||||
c.gateway === row.gateway &&
|
||||
c.distance === row.distance,
|
||||
)
|
||||
return match?.country ? { ...row, country: match.country } : row
|
||||
})
|
||||
}
|
||||
|
||||
function toRouterPayload(route: RecursiveRouteDto): Record<string, string> {
|
||||
return {
|
||||
"dst-address": route.dstAddress,
|
||||
@@ -132,7 +136,7 @@ async function replaceDbRoutes(serverId: number, routes: RecursiveRouteDto[]) {
|
||||
routingTable: r.routingTable || "main",
|
||||
checkGateway: r.checkGateway ?? "",
|
||||
country: r.country ?? "",
|
||||
comment: r.comment ?? "",
|
||||
comment: userRecursiveComment(r.comment),
|
||||
disabled: r.disabled,
|
||||
createdAt: now,
|
||||
updatedAt: now,
|
||||
@@ -140,6 +144,34 @@ async function replaceDbRoutes(serverId: number, routes: RecursiveRouteDto[]) {
|
||||
)
|
||||
}
|
||||
|
||||
async function applyRecursiveToServer(
|
||||
server: ServerRow,
|
||||
routes: RecursiveRouteDto[],
|
||||
source: ConfigRevisionSource,
|
||||
): Promise<{ pushed: number; deleted: number }> {
|
||||
const client = MikrotikClient.fromServer(server)
|
||||
const existing = await client.get<RosRoute[]>("/ip/route")
|
||||
const { deleteIds } = planRecursiveApply(existing)
|
||||
|
||||
for (const route of routes) {
|
||||
await client.post("/ip/route", toRouterPayload(route))
|
||||
}
|
||||
for (const id of deleteIds) {
|
||||
await client.delete(`/ip/route/${encodeURIComponent(id)}`)
|
||||
}
|
||||
|
||||
await replaceDbRoutes(server.id, routes)
|
||||
await appendRevisionIfChanged({
|
||||
serverId: server.id,
|
||||
section: "recursive-routes",
|
||||
source,
|
||||
payload: canonicalRecursiveRoutes(routes),
|
||||
})
|
||||
return { pushed: routes.length, deleted: deleteIds.length }
|
||||
}
|
||||
|
||||
const RevisionIdParamSchema = z.object({ id: z.string().min(1) })
|
||||
|
||||
const recursiveRoutesPlugin: FastifyPluginAsyncZod = async (app) => {
|
||||
app.get("/recursive-routes/gateways", async (req, reply) => {
|
||||
const q = req.query as { serverId?: string | number }
|
||||
@@ -175,79 +207,108 @@ const recursiveRoutesPlugin: FastifyPluginAsyncZod = async (app) => {
|
||||
const q = req.query as { serverId?: string | number }
|
||||
const serverId = parseDbServerId(q.serverId)
|
||||
if (serverId === null) return reply.status(400).send({ error: "serverId is required" })
|
||||
const server = (await db.select().from(servers).where(eq(servers.id, serverId)).limit(1))[0]
|
||||
if (!server) return reply.status(404).send({ error: "Server not found" })
|
||||
return reply.send({ routes: await mapDbRoutes(serverId) })
|
||||
})
|
||||
|
||||
app.put("/recursive-routes", async (req, reply) => {
|
||||
const body = req.body as { serverId?: string | number; routes?: RecursiveRouteDto[] }
|
||||
const serverId = parseDbServerId(body.serverId)
|
||||
if (serverId === null) return reply.status(400).send({ error: "serverId is required" })
|
||||
const server = (await db.select().from(servers).where(eq(servers.id, serverId)).limit(1))[0]
|
||||
if (!server) return reply.status(404).send({ error: "Server not found" })
|
||||
await replaceDbRoutes(serverId, body.routes ?? [])
|
||||
return reply.send({ ok: true })
|
||||
})
|
||||
|
||||
app.post("/recursive-routes/sync/from-router", async (req, reply) => {
|
||||
const body = req.body as { serverId?: string | number } | undefined
|
||||
const serverId = parseDbServerId(body?.serverId)
|
||||
if (serverId === null) return reply.status(400).send({ error: "serverId is required" })
|
||||
const server: ServerRow | undefined = (await db
|
||||
.select().from(servers)
|
||||
.where(eq(servers.id, serverId))
|
||||
.limit(1))[0]
|
||||
if (!server) return reply.status(404).send({ error: "Server not found" })
|
||||
|
||||
const cached = await mapDbRoutes(serverId)
|
||||
try {
|
||||
const client = MikrotikClient.fromServer(server)
|
||||
const rosRoutes = await client.get<RosRoute[]>("/ip/route")
|
||||
const rec = rosRoutes.filter(r =>
|
||||
isRecursiveRoute(r) && hasRecursiveCommentMask(r.comment),
|
||||
)
|
||||
const mapped: RecursiveRouteDto[] = rec.map((r, i) => ({
|
||||
id: r[".id"] ?? `ros-${i}`,
|
||||
dstAddress: r["dst-address"] ?? "",
|
||||
gateway: r.gateway ?? "",
|
||||
distance: Number.parseInt(r.distance ?? "1", 10) || 1,
|
||||
scope: r.scope ? (Number.parseInt(r.scope, 10) || null) : null,
|
||||
targetScope: r["target-scope"] ? (Number.parseInt(r["target-scope"], 10) || null) : null,
|
||||
routingTable: r["routing-table"] ?? "main",
|
||||
checkGateway: r["check-gateway"] ?? "",
|
||||
country: "",
|
||||
comment: r.comment ?? "",
|
||||
disabled: r.disabled === "true",
|
||||
}))
|
||||
await replaceDbRoutes(serverId, mapped)
|
||||
return reply.send({ ok: true, serverId, totalRoutes: mapped.length })
|
||||
const live = mergeCachedCountry(mapRosManagedRoutes(rosRoutes), cached)
|
||||
await replaceDbRoutes(serverId, live)
|
||||
await appendRevisionIfChanged({
|
||||
serverId,
|
||||
section: "recursive-routes",
|
||||
source: "observed",
|
||||
payload: canonicalRecursiveRoutes(live),
|
||||
})
|
||||
const stored = await mapDbRoutes(serverId)
|
||||
return reply.send({ routes: stored, live: true, stale: false })
|
||||
} catch (err) {
|
||||
app.log.warn({ serverId, err: String(err) }, "recursive live GET failed, serving cache")
|
||||
return reply.send({
|
||||
routes: cached,
|
||||
live: false,
|
||||
stale: true,
|
||||
error: String(err),
|
||||
})
|
||||
}
|
||||
})
|
||||
|
||||
app.put("/recursive-routes", async (req, reply) => {
|
||||
const body = req.body as {
|
||||
serverId?: string | number
|
||||
routes?: RecursiveRouteDto[]
|
||||
source?: ConfigRevisionSource
|
||||
}
|
||||
const serverId = parseDbServerId(body.serverId)
|
||||
if (serverId === null) return reply.status(400).send({ error: "serverId is required" })
|
||||
const server = (await db.select().from(servers).where(eq(servers.id, serverId)).limit(1))[0]
|
||||
if (!server) return reply.status(404).send({ error: "Server not found" })
|
||||
const routes = (body.routes ?? []).map((r) => ({
|
||||
...r,
|
||||
comment: userRecursiveComment(r.comment),
|
||||
}))
|
||||
const source: ConfigRevisionSource = body.source === "copy" ? "copy" : "apply"
|
||||
try {
|
||||
const result = await applyRecursiveToServer(server, routes, source)
|
||||
const stored = await mapDbRoutes(serverId)
|
||||
return reply.send({ ok: true, routes: stored, pushedRoutes: result.pushed })
|
||||
} catch (err) {
|
||||
return reply.status(500).send({ error: String(err) })
|
||||
}
|
||||
})
|
||||
|
||||
app.post("/recursive-routes/sync/to-router", async (req, reply) => {
|
||||
const body = req.body as { serverId?: string | number } | undefined
|
||||
const serverId = parseDbServerId(body?.serverId)
|
||||
app.get("/recursive-routes/revisions", async (req, reply) => {
|
||||
const q = req.query as { serverId?: string | number }
|
||||
const serverId = parseDbServerId(q.serverId)
|
||||
if (serverId === null) return reply.status(400).send({ error: "serverId is required" })
|
||||
const server = (await db.select().from(servers).where(eq(servers.id, serverId)).limit(1))[0]
|
||||
const revisions = await listRevisions(serverId, "recursive-routes")
|
||||
return reply.send({ revisions })
|
||||
})
|
||||
|
||||
app.post("/recursive-routes/revisions/:id/restore", {
|
||||
schema: { params: RevisionIdParamSchema },
|
||||
}, async (req, reply) => {
|
||||
const { id } = req.params
|
||||
const body = req.body as { serverId?: string | number } | undefined
|
||||
const rev = await getRevisionById(id)
|
||||
if (!rev) return reply.status(404).send({ error: "Revision not found" })
|
||||
if (rev.section !== "recursive-routes") {
|
||||
return reply.status(400).send({ error: "Revision section mismatch" })
|
||||
}
|
||||
const requested = parseDbServerId(body?.serverId)
|
||||
if (requested !== null && requested !== rev.serverId) {
|
||||
return reply.status(400).send({ error: "Revision belongs to another server" })
|
||||
}
|
||||
const server = (await db.select().from(servers).where(eq(servers.id, rev.serverId)).limit(1))[0]
|
||||
if (!server) return reply.status(404).send({ error: "Server not found" })
|
||||
|
||||
const raw = Array.isArray(rev.payload) ? rev.payload : []
|
||||
const routes: RecursiveRouteDto[] = raw.map((item, idx) => {
|
||||
const r = item as Partial<RecursiveRouteDto>
|
||||
return {
|
||||
id: `rev-${idx}`,
|
||||
dstAddress: r.dstAddress ?? "",
|
||||
gateway: r.gateway ?? "",
|
||||
distance: r.distance ?? 1,
|
||||
scope: r.scope ?? null,
|
||||
targetScope: r.targetScope ?? null,
|
||||
routingTable: r.routingTable || "main",
|
||||
checkGateway: r.checkGateway ?? "",
|
||||
country: r.country ?? "",
|
||||
comment: userRecursiveComment(r.comment),
|
||||
disabled: Boolean(r.disabled),
|
||||
}
|
||||
})
|
||||
|
||||
try {
|
||||
const client = MikrotikClient.fromServer(server)
|
||||
const existing = await client.get<RosRoute[]>("/ip/route")
|
||||
const managed = existing.filter(r => hasManagedRecursiveComment(r.comment ?? ""))
|
||||
for (const r of managed) {
|
||||
if (!r[".id"]) continue
|
||||
await client.delete(`/ip/route/${encodeURIComponent(r[".id"])}`)
|
||||
}
|
||||
|
||||
const dbRows = await mapDbRoutes(serverId)
|
||||
for (const route of dbRows) {
|
||||
await client.post("/ip/route", toRouterPayload(route))
|
||||
}
|
||||
|
||||
return reply.send({ ok: true, serverId, pushedRoutes: dbRows.length })
|
||||
const result = await applyRecursiveToServer(server, routes, "rollback")
|
||||
const stored = await mapDbRoutes(server.id)
|
||||
return reply.send({ ok: true, routes: stored, pushedRoutes: result.pushed })
|
||||
} catch (err) {
|
||||
return reply.status(500).send({ error: String(err) })
|
||||
}
|
||||
|
||||
@@ -2,6 +2,9 @@ import type { FastifyPluginAsyncZod } from "@fastify/type-provider-zod"
|
||||
import { count } from "drizzle-orm"
|
||||
import { listCertificatesFromServers } from "../services/certificates-service.js"
|
||||
import { countWireGuardInterfaces } from "../services/wireguard-live.js"
|
||||
import { countVxlanTunnels } from "../services/vxlan-live.js"
|
||||
import { countContainers } from "../services/containers-live.js"
|
||||
import { countBgpSessions } from "../services/bgp-peers-live.js"
|
||||
import { db } from "../db/index.js"
|
||||
import {
|
||||
filterRules,
|
||||
@@ -24,9 +27,12 @@ const sidebarCountsRoutes: FastifyPluginAsyncZod = async (app) => {
|
||||
const uptimeProbesTotal = await tableCount(uptimeProbes)
|
||||
const uptimeSpeedProbesTotal = await tableCount(uptimeSpeedProbes)
|
||||
const recursiveRoutesTotal = await tableCount(recursiveRoutes)
|
||||
const [certRes, wireguardTotal] = await Promise.all([
|
||||
const [certRes, wireguardTotal, bgpTotal, vxlanTotal, containersTotal] = await Promise.all([
|
||||
listCertificatesFromServers(),
|
||||
countWireGuardInterfaces().catch(() => 0),
|
||||
countBgpSessions().catch(() => 0),
|
||||
countVxlanTunnels().catch(() => 0),
|
||||
countContainers().catch(() => 0),
|
||||
])
|
||||
const certificatesTotal = certRes.certificates.length
|
||||
const usersTotal = (await listUsers()).length
|
||||
@@ -41,6 +47,9 @@ const sidebarCountsRoutes: FastifyPluginAsyncZod = async (app) => {
|
||||
certificates: certificatesTotal,
|
||||
wireguard: wireguardTotal,
|
||||
users: usersTotal,
|
||||
bgpSessions: bgpTotal,
|
||||
vxlan: vxlanTotal,
|
||||
containers: containersTotal,
|
||||
})
|
||||
})
|
||||
}
|
||||
|
||||
@@ -0,0 +1,26 @@
|
||||
import type { FastifyPluginAsyncZod } from "@fastify/type-provider-zod"
|
||||
import { statisticsPivotQuerySchema, statisticsQuerySchema } from "@mmapp/contracts/statistics"
|
||||
import { getStatistics, getStatisticsPivot, pivotDimsConflict } from "../services/statistics-aggregate.js"
|
||||
|
||||
const statisticsRoutes: FastifyPluginAsyncZod = async (app) => {
|
||||
app.get("/statistics", async (req, reply) => {
|
||||
const parsed = statisticsQuerySchema.safeParse(req.query ?? {})
|
||||
if (!parsed.success) {
|
||||
return reply.status(400).send({ error: "Некорректный период или фильтры", details: parsed.error.flatten() })
|
||||
}
|
||||
return reply.send(await getStatistics(parsed.data))
|
||||
})
|
||||
|
||||
app.get("/statistics/pivot", async (req, reply) => {
|
||||
const parsed = statisticsPivotQuerySchema.safeParse(req.query ?? {})
|
||||
if (!parsed.success) {
|
||||
return reply.status(400).send({ error: "Некорректный период или измерения", details: parsed.error.flatten() })
|
||||
}
|
||||
if (pivotDimsConflict(parsed.data.row, parsed.data.col)) {
|
||||
return reply.status(400).send({ error: "Строки и колонки должны отличаться" })
|
||||
}
|
||||
return reply.send(await getStatisticsPivot(parsed.data))
|
||||
})
|
||||
}
|
||||
|
||||
export default statisticsRoutes
|
||||
@@ -27,6 +27,7 @@ import {
|
||||
import { buildFlowMapHops } from "../services/traffic-flow-map-hops.js"
|
||||
import { applyFlowOverlay } from "../services/traffic-flow-overlay.js"
|
||||
import { listTrafficFlowHostFiles } from "../services/traffic-flow-host-files.js"
|
||||
import { rebuildFlowFactsFromBuckets } from "../services/traffic-flow-facts-rebuild.js"
|
||||
import { appendEvent } from "../modules/events/service/events-service.js"
|
||||
|
||||
const LIVE_TICK_MS = 2000
|
||||
@@ -112,6 +113,7 @@ async function applyOverlayHandler(req: FastifyRequest, reply: FastifyReply) {
|
||||
const result = await applyFlowOverlay(parsed.data.serverId, {
|
||||
publicEndpoint: parsed.data.publicEndpoint,
|
||||
requestHost: requestPublicHost(req),
|
||||
disableGreFastPath: parsed.data.disableGreFastPath,
|
||||
})
|
||||
return reply.send(result)
|
||||
} catch (e) {
|
||||
@@ -203,6 +205,27 @@ const trafficFlowRoutes: FastifyPluginAsyncZod = async (app) => {
|
||||
}
|
||||
})
|
||||
|
||||
app.post("/traffic/flow/rebuild-facts", async (_req, reply) => {
|
||||
try {
|
||||
const result = await rebuildFlowFactsFromBuckets()
|
||||
await appendEvent({
|
||||
level: "info",
|
||||
eventType: "traffic.flow.rebuild_facts",
|
||||
sourceModule: "traffic",
|
||||
title: "Пересчитан куб NetFlow",
|
||||
message: `Факты ${result.facts} из ${result.buckets} сессий, дней ${result.days.length}`,
|
||||
entityType: "traffic_flow",
|
||||
entityId: "rebuild-facts",
|
||||
payload: { buckets: result.buckets, facts: result.facts, days: result.days },
|
||||
})
|
||||
return reply.send(result)
|
||||
} catch (err) {
|
||||
const message = err instanceof Error ? err.message : String(err)
|
||||
const status = message.includes("уже выполняется") ? 409 : 500
|
||||
return reply.status(status).send({ error: message })
|
||||
}
|
||||
})
|
||||
|
||||
app.post("/traffic/flow/overlay", applyOverlayHandler)
|
||||
app.post("/traffic/flow-overlay", applyOverlayHandler)
|
||||
|
||||
|
||||
@@ -0,0 +1,23 @@
|
||||
import { eq } from "drizzle-orm"
|
||||
import type { FastifyPluginAsyncZod } from "@fastify/type-provider-zod"
|
||||
import { db } from "../db/index.js"
|
||||
import { servers } from "../db/schema.js"
|
||||
import { listVxlanTunnels, listVxlanTunnelsForServer } from "../services/vxlan-live.js"
|
||||
import { ServerIdParamSchema, type ServerIdParams } from "../types/server.js"
|
||||
|
||||
const vxlanRoutes: FastifyPluginAsyncZod = async (app) => {
|
||||
app.get("/vxlan", async (_req, reply) => {
|
||||
const tunnels = await listVxlanTunnels()
|
||||
return reply.send({ tunnels })
|
||||
})
|
||||
|
||||
app.get("/servers/:id/vxlan", { schema: { params: ServerIdParamSchema } }, async (req, reply) => {
|
||||
const params = req.params as ServerIdParams
|
||||
const server = (await db.select().from(servers).where(eq(servers.id, params.id)).limit(1))[0]
|
||||
if (!server) return reply.status(404).send({ error: "Server not found" })
|
||||
const tunnels = await listVxlanTunnelsForServer(server)
|
||||
return reply.send({ tunnels })
|
||||
})
|
||||
}
|
||||
|
||||
export default vxlanRoutes
|
||||
@@ -18,6 +18,8 @@ import {
|
||||
type WgParsedConfig,
|
||||
} from "../services/wireguard-config.js"
|
||||
import {
|
||||
captureWireguardSnapshot,
|
||||
fetchWireguardRestoreState,
|
||||
getEnabledServerById,
|
||||
listWireGuardInterfaces,
|
||||
} from "../services/wireguard-live.js"
|
||||
@@ -27,6 +29,16 @@ import {
|
||||
putWireguardPeer,
|
||||
toRosBody,
|
||||
} from "../services/wireguard-ros.js"
|
||||
import {
|
||||
captureAndAppendRevision,
|
||||
listRevisions,
|
||||
loadRevisionForRestore,
|
||||
type ConfigRevisionSource,
|
||||
} from "../services/config-revisions.js"
|
||||
import { parseWireguardSnapshot, planWireguardRestore } from "../services/entity-snapshots.js"
|
||||
import { executeRosOps } from "../services/ros-ops.js"
|
||||
import { parseDbServerId } from "../utils/server-id.js"
|
||||
import { z } from "zod"
|
||||
|
||||
function serverIdParam(v: string): string {
|
||||
return decodeURIComponent(v)
|
||||
@@ -137,6 +149,20 @@ function findIface(
|
||||
return list.find((i) => i.serverId === serverId && i.name === interfaceName)
|
||||
}
|
||||
|
||||
async function recordWireguard(
|
||||
server: NonNullable<Awaited<ReturnType<typeof getEnabledServerById>>>,
|
||||
source: ConfigRevisionSource,
|
||||
) {
|
||||
await captureAndAppendRevision({
|
||||
serverId: server.id,
|
||||
section: "wireguard",
|
||||
source,
|
||||
capture: () => captureWireguardSnapshot(server),
|
||||
})
|
||||
}
|
||||
|
||||
const RevisionIdParamSchema = z.object({ id: z.string().min(1) })
|
||||
|
||||
const wireguardRoutes: FastifyPluginAsyncZod = async (app) => {
|
||||
app.get("/wireguard", async (req, reply) => {
|
||||
const q = req.query as { serverId?: string; includePrivateKey?: string }
|
||||
@@ -145,6 +171,11 @@ const wireguardRoutes: FastifyPluginAsyncZod = async (app) => {
|
||||
serverId: q.serverId,
|
||||
includePrivateKey,
|
||||
})
|
||||
const sid = parseDbServerId(q.serverId)
|
||||
if (sid !== null) {
|
||||
const server = await getEnabledServerById(sid)
|
||||
if (server) await recordWireguard(server, "observed")
|
||||
}
|
||||
return reply.send(result)
|
||||
})
|
||||
|
||||
@@ -181,6 +212,7 @@ const wireguardRoutes: FastifyPluginAsyncZod = async (app) => {
|
||||
includePrivateKey: true,
|
||||
})
|
||||
const created = list.interfaces.find((i) => i.name === body.name)
|
||||
await recordWireguard(server, "apply")
|
||||
return reply.status(201).send(created ?? { ok: true, name: body.name })
|
||||
} catch (e) {
|
||||
const msg = e instanceof MikrotikError ? e.message : e instanceof Error ? e.message : String(e)
|
||||
@@ -210,6 +242,7 @@ const wireguardRoutes: FastifyPluginAsyncZod = async (app) => {
|
||||
disabled: d.disabled === true ? "yes" : d.disabled === false ? "no" : undefined,
|
||||
}),
|
||||
)
|
||||
await recordWireguard(server, "apply")
|
||||
return reply.send({ ok: true })
|
||||
} catch (e) {
|
||||
const msg = e instanceof Error ? e.message : String(e)
|
||||
@@ -224,6 +257,7 @@ const wireguardRoutes: FastifyPluginAsyncZod = async (app) => {
|
||||
const client = MikrotikClient.fromServer(server)
|
||||
try {
|
||||
await client.delete(`/interface/wireguard/${encodeURIComponent(rosIdParam(rosId))}`)
|
||||
await recordWireguard(server, "apply")
|
||||
return reply.send({ ok: true })
|
||||
} catch (e) {
|
||||
const msg = e instanceof Error ? e.message : String(e)
|
||||
@@ -242,6 +276,7 @@ const wireguardRoutes: FastifyPluginAsyncZod = async (app) => {
|
||||
const client = MikrotikClient.fromServer(server)
|
||||
try {
|
||||
await putWireguardPeer(client, peerToRosBody(body))
|
||||
await recordWireguard(server, "apply")
|
||||
return reply.status(201).send({ ok: true })
|
||||
} catch (e) {
|
||||
const msg = e instanceof Error ? e.message : String(e)
|
||||
@@ -277,6 +312,7 @@ const wireguardRoutes: FastifyPluginAsyncZod = async (app) => {
|
||||
disabled: d.disabled === true ? "yes" : d.disabled === false ? "no" : undefined,
|
||||
}),
|
||||
)
|
||||
await recordWireguard(server, "apply")
|
||||
return reply.send({ ok: true })
|
||||
} catch (e) {
|
||||
const msg = e instanceof Error ? e.message : String(e)
|
||||
@@ -291,6 +327,7 @@ const wireguardRoutes: FastifyPluginAsyncZod = async (app) => {
|
||||
const client = MikrotikClient.fromServer(server)
|
||||
try {
|
||||
await client.delete(`/interface/wireguard/peers/${encodeURIComponent(rosIdParam(rosId))}`)
|
||||
await recordWireguard(server, "apply")
|
||||
return reply.send({ ok: true })
|
||||
} catch (e) {
|
||||
const msg = e instanceof Error ? e.message : String(e)
|
||||
@@ -320,6 +357,7 @@ const wireguardRoutes: FastifyPluginAsyncZod = async (app) => {
|
||||
const client = MikrotikClient.fromServer(server)
|
||||
try {
|
||||
const applied = await applyParsedConfig(client, config)
|
||||
await recordWireguard(server, "copy")
|
||||
return reply.send({ dryRun: false, preview, applied })
|
||||
} catch (e) {
|
||||
const msg = e instanceof Error ? e.message : String(e)
|
||||
@@ -437,6 +475,46 @@ const wireguardRoutes: FastifyPluginAsyncZod = async (app) => {
|
||||
content,
|
||||
})
|
||||
})
|
||||
|
||||
app.get("/wireguard/revisions", async (req, reply) => {
|
||||
const q = req.query as { serverId?: string | number }
|
||||
const serverId = parseDbServerId(q.serverId)
|
||||
if (serverId === null) return reply.status(400).send({ error: "serverId is required" })
|
||||
const revisions = await listRevisions(serverId, "wireguard")
|
||||
return reply.send({ revisions })
|
||||
})
|
||||
|
||||
app.post("/wireguard/revisions/:id/restore", {
|
||||
schema: { params: RevisionIdParamSchema },
|
||||
}, async (req, reply) => {
|
||||
const { id } = req.params
|
||||
const body = req.body as { serverId?: string | number } | undefined
|
||||
const loaded = await loadRevisionForRestore({
|
||||
id,
|
||||
section: "wireguard",
|
||||
requestedServerId: parseDbServerId(body?.serverId),
|
||||
})
|
||||
if (!loaded.ok) return reply.status(loaded.status).send({ error: loaded.error })
|
||||
try {
|
||||
const desired = parseWireguardSnapshot(loaded.row.payload)
|
||||
const state = await fetchWireguardRestoreState(loaded.server)
|
||||
const ops = planWireguardRestore(desired, {
|
||||
ifaces: state.ifaces,
|
||||
peers: state.peers,
|
||||
addrs: state.addrs,
|
||||
})
|
||||
await executeRosOps(state.client, ops)
|
||||
await recordWireguard(loaded.server, "rollback")
|
||||
const list = await listWireGuardInterfaces({
|
||||
serverId: String(loaded.server.id),
|
||||
includePrivateKey: true,
|
||||
})
|
||||
return reply.send({ ok: true, interfaces: list.interfaces })
|
||||
} catch (e) {
|
||||
const msg = e instanceof Error ? e.message : String(e)
|
||||
return reply.status(502).send({ error: `RouterOS: ${msg}` })
|
||||
}
|
||||
})
|
||||
}
|
||||
|
||||
export default wireguardRoutes
|
||||
|
||||
@@ -0,0 +1,8 @@
|
||||
import { initDatabase } from "../db/bootstrap.js"
|
||||
import { closePool } from "../db/index.js"
|
||||
import { rebuildFlowFactsFromBuckets } from "../services/traffic-flow-facts-rebuild.js"
|
||||
|
||||
await initDatabase()
|
||||
const result = await rebuildFlowFactsFromBuckets()
|
||||
console.log(JSON.stringify(result, null, 2))
|
||||
await closePool()
|
||||
@@ -28,3 +28,16 @@ export async function fetchBgpSessionsForAlerts(): Promise<BgpSessionRead[]> {
|
||||
)
|
||||
return results.flat()
|
||||
}
|
||||
|
||||
export async function countBgpSessions(): Promise<number> {
|
||||
try {
|
||||
const result = await Promise.race([
|
||||
fetchBgpSessionsForAlerts(),
|
||||
new Promise<null>((resolve) => setTimeout(() => resolve(null), 8_000)),
|
||||
])
|
||||
if (!result) return 0
|
||||
return result.length
|
||||
} catch {
|
||||
return 0
|
||||
}
|
||||
}
|
||||
|
||||
@@ -0,0 +1,96 @@
|
||||
import assert from "node:assert/strict"
|
||||
import { hasManagedCommentPrefix, isOwnedRecursiveComment, managedRecursiveComment, stripManagedRecursiveComment } from "../managed-markers.js"
|
||||
import {
|
||||
planBgpInApply,
|
||||
planRecursiveApply,
|
||||
unmanagedRouteIds,
|
||||
} from "./config-apply-plan.js"
|
||||
import {
|
||||
canonicalFilterRules,
|
||||
fingerprintPayload,
|
||||
} from "./config-revisions.js"
|
||||
import { mapRosManagedRoutes } from "./config-apply-plan.js"
|
||||
|
||||
{
|
||||
const fp1 = fingerprintPayload(canonicalFilterRules([
|
||||
{ community: "65001:100", action: "route", gateway: "10.0.0.1", gatewayTunnelId: "gre1", description: "a" },
|
||||
]))
|
||||
const fp2 = fingerprintPayload(canonicalFilterRules([
|
||||
{ community: "65001:100", action: "route", gateway: "10.0.0.1", gatewayTunnelId: "gre1", description: "a" },
|
||||
]))
|
||||
const fp3 = fingerprintPayload(canonicalFilterRules([
|
||||
{ community: "65001:100", action: "route", gateway: "10.0.0.2", gatewayTunnelId: "gre1", description: "a" },
|
||||
]))
|
||||
assert.equal(fp1, fp2)
|
||||
assert.notEqual(fp1, fp3)
|
||||
}
|
||||
|
||||
{
|
||||
const existing = [
|
||||
{ ".id": "*1", chain: "bgp-in", comment: "MikrotikManager: msk", rule: "if (true) { accept; }" },
|
||||
{ ".id": "*2", chain: "bgp-in", comment: "legacy", rule: "if (bgp-communities includes 1:1) { reject; }" },
|
||||
{ ".id": "*3", chain: "bgp-out", comment: "MikrotikManager: other", rule: "if (bgp-communities includes 1:1) { accept; }" },
|
||||
]
|
||||
const patch = planBgpInApply(existing, 3)
|
||||
assert.equal(patch.action, "patch")
|
||||
assert.equal(patch.managedId, "*1")
|
||||
assert.deepEqual(patch.conflictIds, ["*2"])
|
||||
|
||||
const create = planBgpInApply(existing.filter((r) => r[".id"] !== "*1"), 1)
|
||||
assert.equal(create.action, "create")
|
||||
assert.equal(create.managedId, undefined)
|
||||
|
||||
const del = planBgpInApply(existing, 0)
|
||||
assert.equal(del.action, "delete")
|
||||
assert.equal(del.managedId, "*1")
|
||||
|
||||
const noop = planBgpInApply([], 0)
|
||||
assert.equal(noop.action, "noop")
|
||||
}
|
||||
|
||||
{
|
||||
const routes = [
|
||||
{ ".id": "*10", comment: "MikrotikManager:recursive via de", static: "true", "dst-address": "8.8.8.8/32", gateway: "1.1.1.1" },
|
||||
{ ".id": "*11", comment: "user static", static: "true", "dst-address": "1.1.1.1/32", gateway: "9.9.9.9" },
|
||||
{ ".id": "*12", comment: "recursive: old", static: "true", "dst-address": "9.9.9.9/32", gateway: "1.1.1.1" },
|
||||
]
|
||||
const plan = planRecursiveApply(routes)
|
||||
assert.deepEqual(plan.deleteIds, ["*10", "*12"])
|
||||
assert.deepEqual(unmanagedRouteIds(routes), ["*11"])
|
||||
}
|
||||
|
||||
{
|
||||
assert.equal(stripManagedRecursiveComment("MikrotikManager:recursive via de"), "via de")
|
||||
assert.equal(stripManagedRecursiveComment("recursive: old"), "old")
|
||||
assert.equal(managedRecursiveComment("MikrotikManager:recursive via de"), "MikrotikManager:recursive via de")
|
||||
assert.equal(isOwnedRecursiveComment("MikrotikManager:recursive via de"), true)
|
||||
assert.equal(isOwnedRecursiveComment("recursive: x"), true)
|
||||
assert.equal(isOwnedRecursiveComment("user static"), false)
|
||||
assert.equal(hasManagedCommentPrefix("MikrotikManager: msk"), true)
|
||||
}
|
||||
|
||||
{
|
||||
const mapped = mapRosManagedRoutes([
|
||||
{
|
||||
".id": "*1",
|
||||
static: "true",
|
||||
"dst-address": "10.9.9.2/32",
|
||||
gateway: "1.2.3.4",
|
||||
comment: "MikrotikManager:recursive hop-de",
|
||||
distance: "1",
|
||||
},
|
||||
{
|
||||
".id": "*2",
|
||||
static: "true",
|
||||
"dst-address": "10.9.9.3/32",
|
||||
gateway: "1.2.3.4",
|
||||
comment: "not ours",
|
||||
distance: "1",
|
||||
},
|
||||
])
|
||||
assert.equal(mapped.length, 1)
|
||||
assert.equal(mapped[0]?.dstAddress, "10.9.9.2/32")
|
||||
assert.equal(mapped[0]?.comment, "hop-de")
|
||||
}
|
||||
|
||||
console.log("config-apply-plan.test.ts: ok")
|
||||
@@ -0,0 +1,140 @@
|
||||
import {
|
||||
hasManagedCommentPrefix,
|
||||
isOwnedRecursiveComment,
|
||||
stripManagedRecursiveComment,
|
||||
} from "../managed-markers.js"
|
||||
|
||||
export type RosFilterRuleLike = {
|
||||
".id"?: string
|
||||
chain?: string
|
||||
rule?: string
|
||||
comment?: string
|
||||
}
|
||||
|
||||
export type BgpInApplyAction = "patch" | "create" | "delete" | "noop"
|
||||
|
||||
export interface BgpInApplyPlan {
|
||||
action: BgpInApplyAction
|
||||
managedId?: string
|
||||
conflictIds: string[]
|
||||
}
|
||||
|
||||
function isInBgpIn(rule: RosFilterRuleLike): boolean {
|
||||
return (rule.chain ?? "").trim().toLowerCase() === "bgp-in"
|
||||
}
|
||||
|
||||
export function planBgpInApply(
|
||||
existing: RosFilterRuleLike[],
|
||||
rulesCount: number,
|
||||
): BgpInApplyPlan {
|
||||
const managed = existing.find(
|
||||
(r) => isInBgpIn(r) && hasManagedCommentPrefix(r.comment ?? ""),
|
||||
)
|
||||
const conflictIds = existing
|
||||
.filter((r) =>
|
||||
isInBgpIn(r) &&
|
||||
!hasManagedCommentPrefix(r.comment ?? "") &&
|
||||
/bgp-communities/i.test(r.rule ?? ""),
|
||||
)
|
||||
.map((r) => r[".id"])
|
||||
.filter((id): id is string => Boolean(id))
|
||||
|
||||
if (rulesCount > 0) {
|
||||
return {
|
||||
action: managed?.[".id"] ? "patch" : "create",
|
||||
managedId: managed?.[".id"],
|
||||
conflictIds,
|
||||
}
|
||||
}
|
||||
if (managed?.[".id"]) {
|
||||
return { action: "delete", managedId: managed[".id"], conflictIds }
|
||||
}
|
||||
return { action: "noop", conflictIds }
|
||||
}
|
||||
|
||||
export type RosRouteLike = {
|
||||
".id"?: string
|
||||
comment?: string
|
||||
static?: string
|
||||
dynamic?: string
|
||||
blackhole?: string
|
||||
unreachable?: string
|
||||
prohibit?: string
|
||||
"dst-address"?: string
|
||||
gateway?: string
|
||||
}
|
||||
|
||||
export function planRecursiveApply(existing: RosRouteLike[]): { deleteIds: string[] } {
|
||||
return {
|
||||
deleteIds: existing
|
||||
.filter((r) => isOwnedRecursiveComment(r.comment))
|
||||
.map((r) => r[".id"])
|
||||
.filter((id): id is string => Boolean(id)),
|
||||
}
|
||||
}
|
||||
|
||||
export function unmanagedRouteIds(existing: RosRouteLike[]): string[] {
|
||||
return existing
|
||||
.filter((r) => Boolean(r[".id"]) && !isOwnedRecursiveComment(r.comment))
|
||||
.map((r) => r[".id"] as string)
|
||||
}
|
||||
|
||||
export function userRecursiveComment(comment: string | undefined): string {
|
||||
return stripManagedRecursiveComment(comment ?? "")
|
||||
}
|
||||
|
||||
function isIpGateway(gw: string): boolean {
|
||||
return /^\d{1,3}(\.\d{1,3}){3}(?:%\S+)?$/.test(gw.trim())
|
||||
}
|
||||
|
||||
export function isManagedRecursiveRoute(r: RosRouteLike): boolean {
|
||||
if ((r.static ?? "false") !== "true") return false
|
||||
if ((r.dynamic ?? "false") === "true") return false
|
||||
if ((r.blackhole ?? "false") === "true") return false
|
||||
if ((r.unreachable ?? "false") === "true") return false
|
||||
if ((r.prohibit ?? "false") === "true") return false
|
||||
const dst = r["dst-address"] ?? ""
|
||||
const gw = r.gateway ?? ""
|
||||
if (!dst || !gw) return false
|
||||
if (!isIpGateway(gw)) return false
|
||||
return isOwnedRecursiveComment(r.comment)
|
||||
}
|
||||
|
||||
export interface MappedRecursiveRoute {
|
||||
id: string
|
||||
dstAddress: string
|
||||
gateway: string
|
||||
distance: number
|
||||
scope: number | null
|
||||
targetScope: number | null
|
||||
routingTable: string
|
||||
checkGateway: string
|
||||
country: string
|
||||
comment: string
|
||||
disabled: boolean
|
||||
}
|
||||
|
||||
export function mapRosManagedRoutes(
|
||||
rosRoutes: Array<RosRouteLike & {
|
||||
distance?: string
|
||||
scope?: string
|
||||
"target-scope"?: string
|
||||
"routing-table"?: string
|
||||
"check-gateway"?: string
|
||||
disabled?: string
|
||||
}>,
|
||||
): MappedRecursiveRoute[] {
|
||||
return rosRoutes.filter(isManagedRecursiveRoute).map((r, i) => ({
|
||||
id: r[".id"] ?? `ros-${i}`,
|
||||
dstAddress: r["dst-address"] ?? "",
|
||||
gateway: r.gateway ?? "",
|
||||
distance: Number.parseInt(r.distance ?? "1", 10) || 1,
|
||||
scope: r.scope ? (Number.parseInt(r.scope, 10) || null) : null,
|
||||
targetScope: r["target-scope"] ? (Number.parseInt(r["target-scope"], 10) || null) : null,
|
||||
routingTable: r["routing-table"] ?? "main",
|
||||
checkGateway: r["check-gateway"] ?? "",
|
||||
country: "",
|
||||
comment: userRecursiveComment(r.comment),
|
||||
disabled: r.disabled === "true",
|
||||
}))
|
||||
}
|
||||
@@ -0,0 +1,101 @@
|
||||
import assert from "node:assert/strict"
|
||||
import { withPgOrSkip } from "../test/pg.js"
|
||||
import { dbQuery } from "../db/index.js"
|
||||
import {
|
||||
appendRevisionIfChanged,
|
||||
fingerprintPayload,
|
||||
getRevisionById,
|
||||
listRevisions,
|
||||
pruneRevisions,
|
||||
} from "./config-revisions.js"
|
||||
|
||||
if (!(await withPgOrSkip())) {
|
||||
console.log("config-revisions.test.ts: skip")
|
||||
process.exit(0)
|
||||
}
|
||||
|
||||
const tag = `rev-test-${Date.now()}`
|
||||
await dbQuery(`INSERT INTO servers (name, host) VALUES ($1, '127.0.0.1')`, [tag])
|
||||
const { rows } = await dbQuery<{ id: number }>(`SELECT id FROM servers WHERE name = $1 LIMIT 1`, [tag])
|
||||
const serverId = rows[0]?.id
|
||||
assert.ok(serverId)
|
||||
|
||||
try {
|
||||
const payloadA = [{ community: "1:1", action: "route" }]
|
||||
const first = await appendRevisionIfChanged({
|
||||
serverId,
|
||||
section: "filters",
|
||||
source: "apply",
|
||||
payload: payloadA,
|
||||
})
|
||||
assert.equal(first.created, true)
|
||||
assert.equal(first.revision.source, "apply")
|
||||
|
||||
const dup = await appendRevisionIfChanged({
|
||||
serverId,
|
||||
section: "filters",
|
||||
source: "observed",
|
||||
payload: payloadA,
|
||||
})
|
||||
assert.equal(dup.created, false)
|
||||
assert.equal(dup.revision.id, first.revision.id)
|
||||
|
||||
const payloadB = [{ community: "1:2", action: "blackhole" }]
|
||||
const second = await appendRevisionIfChanged({
|
||||
serverId,
|
||||
section: "filters",
|
||||
source: "rollback",
|
||||
payload: payloadB,
|
||||
})
|
||||
assert.equal(second.created, true)
|
||||
assert.equal(second.revision.source, "rollback")
|
||||
assert.notEqual(second.revision.fingerprint, first.revision.fingerprint)
|
||||
|
||||
const listed = await listRevisions(serverId, "filters")
|
||||
assert.equal(listed.length, 2)
|
||||
assert.equal(listed[0]?.source, "rollback")
|
||||
|
||||
const stored = await getRevisionById(second.revision.id)
|
||||
assert.ok(stored)
|
||||
assert.equal(fingerprintPayload(stored.payload), second.revision.fingerprint)
|
||||
|
||||
const objPayload = { rules: [{ chain: "input" }], addressLists: [{ list: "vip" }] }
|
||||
const objRev = await appendRevisionIfChanged({
|
||||
serverId,
|
||||
section: "firewall",
|
||||
source: "apply",
|
||||
payload: objPayload,
|
||||
})
|
||||
assert.equal(objRev.created, true)
|
||||
assert.equal(objRev.revision.itemCount, 2)
|
||||
const objStored = await getRevisionById(objRev.revision.id)
|
||||
assert.ok(objStored)
|
||||
assert.ok(!Array.isArray(objStored.payload))
|
||||
assert.equal(fingerprintPayload(objStored.payload), objRev.revision.fingerprint)
|
||||
|
||||
const objDup = await appendRevisionIfChanged({
|
||||
serverId,
|
||||
section: "firewall",
|
||||
source: "rollback",
|
||||
payload: objPayload,
|
||||
})
|
||||
assert.equal(objDup.created, false)
|
||||
assert.equal(objDup.revision.source, "apply")
|
||||
|
||||
for (let i = 0; i < 4; i++) {
|
||||
await appendRevisionIfChanged({
|
||||
serverId,
|
||||
section: "filters",
|
||||
source: "apply",
|
||||
payload: [{ community: `9:${i}`, action: "route" }],
|
||||
})
|
||||
}
|
||||
const pruned = await pruneRevisions(serverId, "filters", 3)
|
||||
assert.ok(pruned >= 1)
|
||||
const after = await listRevisions(serverId, "filters")
|
||||
assert.equal(after.length, 3)
|
||||
} finally {
|
||||
await dbQuery(`DELETE FROM servers WHERE id = $1`, [serverId])
|
||||
}
|
||||
|
||||
console.log("config-revisions.test.ts: ok")
|
||||
@@ -0,0 +1,236 @@
|
||||
import { createHash, randomUUID } from "node:crypto"
|
||||
import { and, desc, eq } from "drizzle-orm"
|
||||
import { db } from "../db/index.js"
|
||||
import { configRevisions, servers, type ConfigRevisionRow } from "../db/schema.js"
|
||||
|
||||
export const CONFIG_REVISION_KEEP = 50
|
||||
|
||||
export const CONFIG_SECTIONS = [
|
||||
"filters",
|
||||
"recursive-routes",
|
||||
"firewall",
|
||||
"wireguard",
|
||||
"gre",
|
||||
] as const
|
||||
|
||||
export type ConfigSection = (typeof CONFIG_SECTIONS)[number]
|
||||
export type ConfigRevisionSource = "apply" | "rollback" | "observed" | "copy"
|
||||
|
||||
export interface ConfigRevisionDto {
|
||||
id: string
|
||||
serverId: string
|
||||
section: ConfigSection
|
||||
source: ConfigRevisionSource
|
||||
fingerprint: string
|
||||
createdAt: string
|
||||
note: string | null
|
||||
itemCount: number
|
||||
}
|
||||
|
||||
export function stableStringify(value: unknown): string {
|
||||
if (value === null || typeof value !== "object") return JSON.stringify(value)
|
||||
if (Array.isArray(value)) return `[${value.map(stableStringify).join(",")}]`
|
||||
const obj = value as Record<string, unknown>
|
||||
const keys = Object.keys(obj).sort()
|
||||
return `{${keys.map((k) => `${JSON.stringify(k)}:${stableStringify(obj[k])}`).join(",")}}`
|
||||
}
|
||||
|
||||
export function fingerprintPayload(payload: unknown): string {
|
||||
return createHash("sha256").update(stableStringify(payload)).digest("hex")
|
||||
}
|
||||
|
||||
export function revisionItemCount(payload: unknown): number {
|
||||
if (Array.isArray(payload)) return payload.length
|
||||
if (payload && typeof payload === "object") {
|
||||
let n = 0
|
||||
for (const value of Object.values(payload as Record<string, unknown>)) {
|
||||
if (Array.isArray(value)) n += value.length
|
||||
}
|
||||
return n
|
||||
}
|
||||
return 0
|
||||
}
|
||||
|
||||
export function persistablePayload(payload: unknown): unknown {
|
||||
if (payload === undefined) return []
|
||||
return payload
|
||||
}
|
||||
|
||||
export function canonicalFilterRules(
|
||||
rules: Array<{
|
||||
community?: string
|
||||
action?: string
|
||||
gateway?: string
|
||||
gatewayTunnelId?: string
|
||||
description?: string
|
||||
}>,
|
||||
): unknown[] {
|
||||
return rules.map((r) => ({
|
||||
community: (r.community ?? "").trim(),
|
||||
action: r.action === "blackhole" ? "blackhole" : "route",
|
||||
gateway: r.gateway ?? "",
|
||||
gatewayTunnelId: r.gatewayTunnelId ?? "",
|
||||
description: r.description ?? "",
|
||||
}))
|
||||
}
|
||||
|
||||
export function canonicalRecursiveRoutes(
|
||||
routes: Array<{
|
||||
dstAddress?: string
|
||||
gateway?: string
|
||||
distance?: number
|
||||
scope?: number | null
|
||||
targetScope?: number | null
|
||||
routingTable?: string
|
||||
checkGateway?: string
|
||||
comment?: string
|
||||
disabled?: boolean
|
||||
country?: string
|
||||
}>,
|
||||
): unknown[] {
|
||||
return routes.map((r) => ({
|
||||
dstAddress: (r.dstAddress ?? "").trim(),
|
||||
gateway: r.gateway ?? "",
|
||||
distance: r.distance ?? 1,
|
||||
scope: r.scope ?? null,
|
||||
targetScope: r.targetScope ?? null,
|
||||
routingTable: r.routingTable || "main",
|
||||
checkGateway: r.checkGateway ?? "",
|
||||
comment: r.comment ?? "",
|
||||
disabled: Boolean(r.disabled),
|
||||
country: r.country ?? "",
|
||||
}))
|
||||
}
|
||||
|
||||
export function toRevisionDto(row: ConfigRevisionRow): ConfigRevisionDto {
|
||||
return {
|
||||
id: row.id,
|
||||
serverId: String(row.serverId),
|
||||
section: row.section as ConfigSection,
|
||||
source: row.source,
|
||||
fingerprint: row.fingerprint,
|
||||
createdAt: row.createdAt,
|
||||
note: row.note ?? null,
|
||||
itemCount: revisionItemCount(row.payload),
|
||||
}
|
||||
}
|
||||
|
||||
export async function listRevisions(
|
||||
serverId: number,
|
||||
section: ConfigSection,
|
||||
limit = CONFIG_REVISION_KEEP,
|
||||
): Promise<ConfigRevisionDto[]> {
|
||||
const rows = await db
|
||||
.select()
|
||||
.from(configRevisions)
|
||||
.where(and(eq(configRevisions.serverId, serverId), eq(configRevisions.section, section)))
|
||||
.orderBy(desc(configRevisions.createdAt))
|
||||
.limit(limit)
|
||||
return rows.map(toRevisionDto)
|
||||
}
|
||||
|
||||
export async function getRevisionById(id: string): Promise<ConfigRevisionRow | undefined> {
|
||||
return (await db.select().from(configRevisions).where(eq(configRevisions.id, id)).limit(1))[0]
|
||||
}
|
||||
|
||||
export async function pruneRevisions(
|
||||
serverId: number,
|
||||
section: ConfigSection,
|
||||
keep = CONFIG_REVISION_KEEP,
|
||||
): Promise<number> {
|
||||
const rows = await db
|
||||
.select({ id: configRevisions.id })
|
||||
.from(configRevisions)
|
||||
.where(and(eq(configRevisions.serverId, serverId), eq(configRevisions.section, section)))
|
||||
.orderBy(desc(configRevisions.createdAt))
|
||||
const extra = rows.slice(keep)
|
||||
if (extra.length === 0) return 0
|
||||
for (const row of extra) {
|
||||
await db.delete(configRevisions).where(eq(configRevisions.id, row.id))
|
||||
}
|
||||
return extra.length
|
||||
}
|
||||
|
||||
export async function appendRevisionIfChanged(input: {
|
||||
serverId: number
|
||||
section: ConfigSection
|
||||
source: ConfigRevisionSource
|
||||
payload: unknown
|
||||
note?: string | null
|
||||
}): Promise<{ created: boolean; revision: ConfigRevisionDto }> {
|
||||
const payload = persistablePayload(input.payload)
|
||||
const fingerprint = fingerprintPayload(payload)
|
||||
const latest = (await db
|
||||
.select()
|
||||
.from(configRevisions)
|
||||
.where(and(
|
||||
eq(configRevisions.serverId, input.serverId),
|
||||
eq(configRevisions.section, input.section),
|
||||
))
|
||||
.orderBy(desc(configRevisions.createdAt))
|
||||
.limit(1))[0]
|
||||
|
||||
if (latest?.fingerprint === fingerprint) {
|
||||
return { created: false, revision: toRevisionDto(latest) }
|
||||
}
|
||||
|
||||
const now = new Date().toISOString()
|
||||
const id = randomUUID()
|
||||
await db.insert(configRevisions).values({
|
||||
id,
|
||||
serverId: input.serverId,
|
||||
section: input.section,
|
||||
source: input.source,
|
||||
fingerprint,
|
||||
payload,
|
||||
note: input.note ?? null,
|
||||
createdAt: now,
|
||||
})
|
||||
await pruneRevisions(input.serverId, input.section)
|
||||
const row = await getRevisionById(id)
|
||||
if (!row) throw new Error("config-revisions: insert vanished")
|
||||
return { created: true, revision: toRevisionDto(row) }
|
||||
}
|
||||
|
||||
/** После успешного mutate: capture live → append, ошибки snapshot не валят мутацию. */
|
||||
export async function captureAndAppendRevision(input: {
|
||||
serverId: number
|
||||
section: ConfigSection
|
||||
source: ConfigRevisionSource
|
||||
capture: () => Promise<unknown>
|
||||
note?: string | null
|
||||
}): Promise<{ created: boolean; revision: ConfigRevisionDto } | null> {
|
||||
try {
|
||||
const payload = await input.capture()
|
||||
return await appendRevisionIfChanged({
|
||||
serverId: input.serverId,
|
||||
section: input.section,
|
||||
source: input.source,
|
||||
payload,
|
||||
note: input.note,
|
||||
})
|
||||
} catch {
|
||||
return null
|
||||
}
|
||||
}
|
||||
|
||||
export async function loadRevisionForRestore(opts: {
|
||||
id: string
|
||||
section: ConfigSection
|
||||
requestedServerId: number | null
|
||||
}): Promise<
|
||||
| { ok: true; row: ConfigRevisionRow; server: typeof servers.$inferSelect }
|
||||
| { ok: false; status: number; error: string }
|
||||
> {
|
||||
const row = await getRevisionById(opts.id)
|
||||
if (!row) return { ok: false, status: 404, error: "Revision not found" }
|
||||
if (row.section !== opts.section) {
|
||||
return { ok: false, status: 400, error: "Revision section mismatch" }
|
||||
}
|
||||
if (opts.requestedServerId !== null && opts.requestedServerId !== row.serverId) {
|
||||
return { ok: false, status: 400, error: "Revision belongs to another server" }
|
||||
}
|
||||
const server = (await db.select().from(servers).where(eq(servers.id, row.serverId)).limit(1))[0]
|
||||
if (!server) return { ok: false, status: 404, error: "Server not found" }
|
||||
return { ok: true, row, server }
|
||||
}
|
||||
@@ -0,0 +1,46 @@
|
||||
import assert from "node:assert/strict"
|
||||
import { mapContainerRow } from "./containers-live.js"
|
||||
|
||||
const server = {
|
||||
id: 3,
|
||||
name: "mt-spb",
|
||||
host: "10.0.1.1",
|
||||
} as Parameters<typeof mapContainerRow>[0]
|
||||
|
||||
const row = mapContainerRow(
|
||||
server,
|
||||
{
|
||||
".id": "*A",
|
||||
name: "adguard",
|
||||
"remote-image": "adguard/adguardhome:latest",
|
||||
interface: "veth-adguard",
|
||||
envlist: "adguard-envs",
|
||||
mounts: "agh-conf,agh-work",
|
||||
status: "running",
|
||||
"start-on-boot": "true",
|
||||
comment: "DNS",
|
||||
},
|
||||
[
|
||||
{ name: "adguard-envs", key: "FOO", value: "bar" },
|
||||
{ name: "other", key: "SKIP", value: "x" },
|
||||
],
|
||||
[
|
||||
{ name: "agh-conf", dst: "/opt/conf", src: "/disk1/conf" },
|
||||
{ name: "agh-work", dst: "/opt/work" },
|
||||
],
|
||||
0,
|
||||
)
|
||||
|
||||
assert.equal(row.rosId, "*A")
|
||||
assert.equal(row.image, "adguard/adguardhome")
|
||||
assert.equal(row.tag, "latest")
|
||||
assert.equal(row.status, "running")
|
||||
assert.deepEqual(row.interfaces, ["veth-adguard"])
|
||||
assert.deepEqual(row.envs, [{ key: "FOO", value: "bar" }])
|
||||
assert.deepEqual(row.mounts, [
|
||||
{ dst: "/opt/conf", src: "/disk1/conf" },
|
||||
{ dst: "/opt/work", src: undefined },
|
||||
])
|
||||
assert.equal(row.startOnBoot, true)
|
||||
|
||||
console.log("containers-live.test.ts: ok")
|
||||
@@ -0,0 +1,215 @@
|
||||
import { eq } from "drizzle-orm"
|
||||
import { db } from "../db/index.js"
|
||||
import { servers } from "../db/schema.js"
|
||||
import { MikrotikClient, MikrotikError } from "./mikrotik.js"
|
||||
|
||||
type ServerRow = typeof servers.$inferSelect
|
||||
|
||||
interface RosContainer {
|
||||
".id"?: string
|
||||
name?: string
|
||||
tag?: string
|
||||
"remote-image"?: string
|
||||
interface?: string
|
||||
envlist?: string
|
||||
mounts?: string
|
||||
cmd?: string
|
||||
"start-on-boot"?: string
|
||||
comment?: string
|
||||
status?: string
|
||||
"memory-high"?: string
|
||||
cpu?: string
|
||||
}
|
||||
|
||||
interface RosContainerEnv {
|
||||
name?: string
|
||||
key?: string
|
||||
value?: string
|
||||
}
|
||||
|
||||
interface RosContainerMount {
|
||||
name?: string
|
||||
src?: string
|
||||
dst?: string
|
||||
}
|
||||
|
||||
export type ContainerLiveStatus = "running" | "stopped" | "error"
|
||||
|
||||
export type ContainerLive = {
|
||||
id: string
|
||||
rosId: string
|
||||
name: string
|
||||
serverId: string
|
||||
image: string
|
||||
tag: string
|
||||
status: ContainerLiveStatus
|
||||
envs: { key: string; value: string }[]
|
||||
mounts: { dst: string; src?: string }[]
|
||||
interfaces: string[]
|
||||
cmd?: string
|
||||
startOnBoot: boolean
|
||||
comment: string
|
||||
uptime?: string
|
||||
cpu?: number
|
||||
memMb?: number
|
||||
}
|
||||
|
||||
function rosYes(v: string | undefined): boolean {
|
||||
return v === "true" || v === "yes"
|
||||
}
|
||||
|
||||
function mapStatus(raw: string | undefined): ContainerLiveStatus {
|
||||
const s = (raw ?? "").toLowerCase()
|
||||
if (s === "running") return "running"
|
||||
if (s === "error" || s === "failed") return "error"
|
||||
return "stopped"
|
||||
}
|
||||
|
||||
function splitCsv(v: string | undefined): string[] {
|
||||
return (v ?? "")
|
||||
.split(",")
|
||||
.map((x) => x.trim())
|
||||
.filter(Boolean)
|
||||
}
|
||||
|
||||
function parseImageTag(c: RosContainer): { image: string; tag: string } {
|
||||
const remote = (c["remote-image"] ?? "").trim()
|
||||
if (remote) {
|
||||
const idx = remote.lastIndexOf(":")
|
||||
if (idx > 0 && !remote.slice(idx + 1).includes("/")) {
|
||||
return { image: remote.slice(0, idx), tag: remote.slice(idx + 1) }
|
||||
}
|
||||
return { image: remote, tag: (c.tag ?? "latest").trim() || "latest" }
|
||||
}
|
||||
return { image: (c.name ?? "").trim(), tag: (c.tag ?? "latest").trim() || "latest" }
|
||||
}
|
||||
|
||||
function isMissingPackage(err: unknown): boolean {
|
||||
if (err instanceof MikrotikError) {
|
||||
if (err.statusCode === 404) return true
|
||||
const body = err.body.toLowerCase()
|
||||
return body.includes("no such command") || body.includes("not found") || body.includes("unknown")
|
||||
}
|
||||
const msg = err instanceof Error ? err.message.toLowerCase() : String(err).toLowerCase()
|
||||
return msg.includes("no such command") || msg.includes("404")
|
||||
}
|
||||
|
||||
export function mapContainerRow(
|
||||
server: ServerRow,
|
||||
c: RosContainer,
|
||||
envs: RosContainerEnv[],
|
||||
mounts: RosContainerMount[],
|
||||
idx: number,
|
||||
): ContainerLive {
|
||||
const rosId = String(c[".id"] ?? `c-${idx}`)
|
||||
const name = (c.name ?? "").trim() || `container-${idx + 1}`
|
||||
const { image, tag } = parseImageTag(c)
|
||||
const envlist = (c.envlist ?? "").trim()
|
||||
const mountNames = new Set(splitCsv(c.mounts))
|
||||
const envRows = envlist
|
||||
? envs.filter((e) => (e.name ?? "").trim() === envlist && (e.key ?? "").trim())
|
||||
: []
|
||||
const mountRows = mounts.filter((m) => mountNames.has((m.name ?? "").trim()) && (m.dst ?? "").trim())
|
||||
const cpuRaw = Number.parseInt(c.cpu ?? "", 10)
|
||||
const memRaw = Number.parseInt(c["memory-high"] ?? "", 10)
|
||||
return {
|
||||
id: `${server.id}-${rosId}`,
|
||||
rosId,
|
||||
name,
|
||||
serverId: String(server.id),
|
||||
image,
|
||||
tag,
|
||||
status: mapStatus(c.status),
|
||||
envs: envRows.map((e) => ({ key: e.key ?? "", value: e.value ?? "" })),
|
||||
mounts: mountRows.map((m) => ({ dst: m.dst ?? "", src: m.src || undefined })),
|
||||
interfaces: splitCsv(c.interface),
|
||||
cmd: (c.cmd ?? "").trim() || undefined,
|
||||
startOnBoot: rosYes(c["start-on-boot"]),
|
||||
comment: c.comment ?? "",
|
||||
cpu: Number.isFinite(cpuRaw) ? cpuRaw : undefined,
|
||||
memMb: Number.isFinite(memRaw) ? Math.round(memRaw / (1024 * 1024)) || undefined : undefined,
|
||||
}
|
||||
}
|
||||
|
||||
async function fetchContainersForServer(server: ServerRow): Promise<ContainerLive[]> {
|
||||
const client = MikrotikClient.fromServer(server)
|
||||
try {
|
||||
const [raw, envsRaw, mountsRaw] = await Promise.all([
|
||||
client.get<RosContainer[]>("/container"),
|
||||
client.get<RosContainerEnv[]>("/container/envs").catch(() => [] as RosContainerEnv[]),
|
||||
client.get<RosContainerMount[]>("/container/mounts").catch(() => [] as RosContainerMount[]),
|
||||
])
|
||||
const list = Array.isArray(raw) ? raw : []
|
||||
const envs = Array.isArray(envsRaw) ? envsRaw : []
|
||||
const mounts = Array.isArray(mountsRaw) ? mountsRaw : []
|
||||
return list.map((c, idx) => mapContainerRow(server, c, envs, mounts, idx))
|
||||
} catch (err) {
|
||||
if (isMissingPackage(err)) return []
|
||||
throw err
|
||||
}
|
||||
}
|
||||
|
||||
export async function listContainers(): Promise<ContainerLive[]> {
|
||||
const enabledServers = await db.select().from(servers).where(eq(servers.enabled, true))
|
||||
const results = await Promise.all(
|
||||
enabledServers.map(async (server) => {
|
||||
try {
|
||||
return await fetchContainersForServer(server)
|
||||
} catch {
|
||||
return [] as ContainerLive[]
|
||||
}
|
||||
}),
|
||||
)
|
||||
return results.flat()
|
||||
}
|
||||
|
||||
export async function listContainersForServer(server: ServerRow): Promise<ContainerLive[]> {
|
||||
try {
|
||||
return await fetchContainersForServer(server)
|
||||
} catch {
|
||||
return []
|
||||
}
|
||||
}
|
||||
|
||||
export async function countContainers(): Promise<number> {
|
||||
try {
|
||||
const result = await Promise.race([
|
||||
listContainers(),
|
||||
new Promise<null>((resolve) => setTimeout(() => resolve(null), 8_000)),
|
||||
])
|
||||
if (!result) return 0
|
||||
return result.length
|
||||
} catch {
|
||||
return 0
|
||||
}
|
||||
}
|
||||
|
||||
function encodeRosId(rosId: string): string {
|
||||
return encodeURIComponent(rosId)
|
||||
}
|
||||
|
||||
export async function startContainer(server: ServerRow, rosId: string): Promise<void> {
|
||||
const client = MikrotikClient.fromServer(server)
|
||||
await client.post("/container/start", { ".id": rosId })
|
||||
}
|
||||
|
||||
export async function stopContainer(server: ServerRow, rosId: string): Promise<void> {
|
||||
const client = MikrotikClient.fromServer(server)
|
||||
await client.post("/container/stop", { ".id": rosId })
|
||||
}
|
||||
|
||||
export async function restartContainer(server: ServerRow, rosId: string): Promise<void> {
|
||||
await stopContainer(server, rosId)
|
||||
await startContainer(server, rosId)
|
||||
}
|
||||
|
||||
export async function removeContainer(server: ServerRow, rosId: string): Promise<void> {
|
||||
const client = MikrotikClient.fromServer(server)
|
||||
await client.delete(`/container/${encodeRosId(rosId)}`)
|
||||
}
|
||||
|
||||
export async function getEnabledServerById(serverId: string | number) {
|
||||
const id = typeof serverId === "number" ? serverId : Number.parseInt(String(serverId), 10)
|
||||
if (!Number.isFinite(id)) return null
|
||||
return (await db.select().from(servers).where(eq(servers.id, id)).limit(1))[0] ?? null
|
||||
}
|
||||
@@ -0,0 +1,164 @@
|
||||
import assert from "node:assert/strict"
|
||||
import {
|
||||
canonicalFirewallSnapshot,
|
||||
canonicalGreSnapshot,
|
||||
canonicalWireguardSnapshot,
|
||||
opsPaths,
|
||||
opsTouchOnly,
|
||||
planFirewallRestore,
|
||||
planGreCreate,
|
||||
planGreDelete,
|
||||
planGreRestore,
|
||||
planWireguardRestore,
|
||||
} from "./entity-snapshots.js"
|
||||
import { fingerprintPayload, revisionItemCount } from "./config-revisions.js"
|
||||
|
||||
{
|
||||
const a = canonicalFirewallSnapshot({
|
||||
rules: [{ family: "ip", table: "filter", chain: "input", action: "accept", comment: "ssh" }],
|
||||
addressLists: [{ family: "ip", list: "vip", address: "1.1.1.1" }],
|
||||
})
|
||||
const b = canonicalFirewallSnapshot({
|
||||
rules: [{ family: "ip", table: "filter", chain: "input", action: "accept", comment: "ssh" }],
|
||||
addressLists: [{ family: "ip", list: "vip", address: "1.1.1.1" }],
|
||||
})
|
||||
assert.equal(fingerprintPayload(a), fingerprintPayload(b))
|
||||
assert.equal(revisionItemCount(a), 2)
|
||||
}
|
||||
|
||||
{
|
||||
const desired = canonicalFirewallSnapshot({
|
||||
rules: [{ family: "ip", table: "filter", chain: "input", action: "accept", comment: "keep" }],
|
||||
addressLists: [],
|
||||
})
|
||||
const ops = planFirewallRestore(desired, {
|
||||
rules: [
|
||||
{
|
||||
...desired.rules[0]!,
|
||||
rosId: "*1",
|
||||
dynamic: false,
|
||||
},
|
||||
{
|
||||
family: "ip",
|
||||
table: "filter",
|
||||
chain: "forward",
|
||||
action: "drop",
|
||||
protocol: "",
|
||||
srcAddress: "",
|
||||
dstAddress: "",
|
||||
srcAddressList: "",
|
||||
dstAddressList: "",
|
||||
srcPort: "",
|
||||
dstPort: "",
|
||||
inInterface: "",
|
||||
outInterface: "",
|
||||
connectionState: "",
|
||||
comment: "extra",
|
||||
disabled: false,
|
||||
log: false,
|
||||
logPrefix: "",
|
||||
tlsHost: "",
|
||||
layer7Proto: "",
|
||||
rosId: "*2",
|
||||
dynamic: false,
|
||||
},
|
||||
],
|
||||
addressLists: [],
|
||||
})
|
||||
assert.ok(ops.some((op) => op.op === "delete" && op.path.includes("/ip/firewall/filter/")))
|
||||
assert.equal(opsTouchOnly(ops, ["/ip/firewall", "/ipv6/firewall"]), true)
|
||||
assert.equal(opsPaths(ops).some((p) => p.startsWith("/ip/route") || p.startsWith("/interface/wireguard")), false)
|
||||
}
|
||||
|
||||
{
|
||||
const snap = canonicalWireguardSnapshot({
|
||||
interfaces: [{
|
||||
name: "wg0",
|
||||
privateKey: "abc",
|
||||
address: "10.8.0.1/24",
|
||||
peers: [{ publicKey: "pk", allowedAddresses: ["10.8.0.2/32"] }],
|
||||
}],
|
||||
})
|
||||
const ops = planWireguardRestore(snap, {
|
||||
ifaces: [{ name: "wg0", rosId: "*w", listenPort: 13231, mtu: 1420, privateKey: "abc", comment: "", disabled: false }],
|
||||
peers: [{
|
||||
rosId: "*p",
|
||||
interfaceName: "wg0",
|
||||
publicKey: "old",
|
||||
allowedAddresses: ["0.0.0.0/0"],
|
||||
endpointAddress: "",
|
||||
endpointPort: "",
|
||||
persistentKeepalive: null,
|
||||
comment: "",
|
||||
name: "",
|
||||
disabled: false,
|
||||
privateKey: "",
|
||||
clientAddress: "",
|
||||
clientDns: "",
|
||||
clientEndpoint: "",
|
||||
}],
|
||||
addrs: [{ rosId: "*a", interfaceName: "wg0", address: "10.8.0.1/24" }],
|
||||
})
|
||||
assert.ok(ops.some((op) => op.op === "delete" && op.path.includes("/interface/wireguard/peers/")))
|
||||
assert.ok(ops.some((op) => op.op === "put" && op.path === "/interface/wireguard/peers"))
|
||||
assert.equal(opsTouchOnly(ops, ["/interface/wireguard", "/ip/address"]), true)
|
||||
assert.equal(opsPaths(ops).some((p) => p.startsWith("/interface/gre") || p.startsWith("/ip/route")), false)
|
||||
}
|
||||
|
||||
{
|
||||
const tunnel = canonicalGreSnapshot({
|
||||
tunnels: [{
|
||||
name: "gre-a",
|
||||
remoteAddress: "203.0.113.1",
|
||||
localInnerIp: "10.200.0.1/30",
|
||||
ipsecSecret: "psk-secret",
|
||||
}],
|
||||
}).tunnels[0]!
|
||||
const create = planGreCreate(tunnel)
|
||||
assert.deepEqual(create.map((op) => op.op), ["put", "put"])
|
||||
assert.equal(create[0]?.path, "/interface/gre")
|
||||
assert.equal(create[1]?.path, "/ip/address")
|
||||
assert.equal(create[1] && create[1].op === "put" ? create[1].body.interface : "", "gre-a")
|
||||
assert.equal(opsPaths(create).some((p) => p.includes("gre-b")), false)
|
||||
|
||||
const del = planGreDelete("gre-a", {
|
||||
gre: [
|
||||
{ name: "gre-a", rosId: "*1", localAddress: "", remoteAddress: "203.0.113.1", comment: "", disabled: false, mtu: 1476, keepalive: "0", dscp: "inherit", clampTcpMss: true, allowFastPath: true, ipsecSecret: "" },
|
||||
{ name: "gre-b", rosId: "*2", localAddress: "", remoteAddress: "203.0.113.2", comment: "", disabled: false, mtu: 1476, keepalive: "0", dscp: "inherit", clampTcpMss: true, allowFastPath: true, ipsecSecret: "" },
|
||||
],
|
||||
addrs: [
|
||||
{ rosId: "*a1", interfaceName: "gre-a", address: "10.200.0.1/30" },
|
||||
{ rosId: "*a2", interfaceName: "gre-b", address: "10.200.0.5/30" },
|
||||
],
|
||||
})
|
||||
assert.ok(del.some((op) => op.path === "/ip/address/*a1"))
|
||||
assert.ok(del.some((op) => op.path === "/interface/gre/*1"))
|
||||
assert.equal(opsPaths(del).some((p) => p.includes("*2") || p.includes("*a2")), false)
|
||||
|
||||
const restore = planGreRestore(
|
||||
canonicalGreSnapshot({ tunnels: [tunnel] }),
|
||||
{
|
||||
gre: [
|
||||
{ name: "gre-a", rosId: "*1", localAddress: "", remoteAddress: "203.0.113.1", comment: "", disabled: false, mtu: 1476, keepalive: "0", dscp: "inherit", clampTcpMss: true, allowFastPath: true, ipsecSecret: "psk-secret" },
|
||||
{ name: "gre-b", rosId: "*2", localAddress: "", remoteAddress: "203.0.113.2", comment: "", disabled: false, mtu: 1476, keepalive: "0", dscp: "inherit", clampTcpMss: true, allowFastPath: true, ipsecSecret: "" },
|
||||
],
|
||||
addrs: [
|
||||
{ rosId: "*a1", interfaceName: "gre-a", address: "10.200.0.1/30" },
|
||||
{ rosId: "*a2", interfaceName: "gre-b", address: "10.200.0.5/30" },
|
||||
],
|
||||
},
|
||||
)
|
||||
assert.ok(restore.some((op) => op.path === "/interface/gre/*2"))
|
||||
assert.ok(restore.some((op) => op.path === "/ip/address/*a2"))
|
||||
assert.equal(opsTouchOnly(restore, ["/interface/gre", "/ip/address"]), true)
|
||||
}
|
||||
|
||||
{
|
||||
const p1 = fingerprintPayload({ tunnels: [{ name: "gre-a", mtu: 1476 }] })
|
||||
const p2 = fingerprintPayload({ tunnels: [{ name: "gre-a", mtu: 1476 }] })
|
||||
const p3 = fingerprintPayload({ tunnels: [{ name: "gre-a", mtu: 1400 }] })
|
||||
assert.equal(p1, p2)
|
||||
assert.notEqual(p1, p3)
|
||||
}
|
||||
|
||||
console.log("entity-snapshots.test.ts: ok")
|
||||
@@ -0,0 +1,649 @@
|
||||
/** Канонические снапшоты и планы restore для firewall / WireGuard / GRE. */
|
||||
|
||||
export type FirewallFamily = "ip" | "ip6"
|
||||
export type FirewallTable = "filter" | "nat" | "mangle" | "raw"
|
||||
|
||||
export type RosWriteOp =
|
||||
| { op: "put"; path: string; body: Record<string, string> }
|
||||
| { op: "post"; path: string; body: Record<string, string> }
|
||||
| { op: "patch"; path: string; body: Record<string, string> }
|
||||
| { op: "delete"; path: string }
|
||||
| { op: "move"; path: string; body: Record<string, string> }
|
||||
|
||||
export interface FirewallSnapshotRule {
|
||||
family: FirewallFamily
|
||||
table: FirewallTable
|
||||
chain: string
|
||||
action: string
|
||||
protocol: string
|
||||
srcAddress: string
|
||||
dstAddress: string
|
||||
srcAddressList: string
|
||||
dstAddressList: string
|
||||
srcPort: string
|
||||
dstPort: string
|
||||
inInterface: string
|
||||
outInterface: string
|
||||
connectionState: string
|
||||
comment: string
|
||||
disabled: boolean
|
||||
log: boolean
|
||||
logPrefix: string
|
||||
tlsHost: string
|
||||
layer7Proto: string
|
||||
}
|
||||
|
||||
export interface FirewallSnapshotList {
|
||||
family: FirewallFamily
|
||||
list: string
|
||||
address: string
|
||||
comment: string
|
||||
disabled: boolean
|
||||
timeout: string
|
||||
}
|
||||
|
||||
export interface FirewallSnapshot {
|
||||
rules: FirewallSnapshotRule[]
|
||||
addressLists: FirewallSnapshotList[]
|
||||
}
|
||||
|
||||
export interface FirewallLiveRule extends FirewallSnapshotRule {
|
||||
rosId: string
|
||||
dynamic: boolean
|
||||
}
|
||||
|
||||
export interface FirewallLiveList extends FirewallSnapshotList {
|
||||
rosId: string
|
||||
dynamic: boolean
|
||||
}
|
||||
|
||||
export interface WgSnapshotPeer {
|
||||
publicKey: string
|
||||
allowedAddresses: string[]
|
||||
endpointAddress: string
|
||||
endpointPort: string
|
||||
persistentKeepalive: number | null
|
||||
comment: string
|
||||
name: string
|
||||
disabled: boolean
|
||||
privateKey: string
|
||||
clientAddress: string
|
||||
clientDns: string
|
||||
clientEndpoint: string
|
||||
}
|
||||
|
||||
export interface WgSnapshotIface {
|
||||
name: string
|
||||
listenPort: number
|
||||
mtu: number
|
||||
privateKey: string
|
||||
address: string
|
||||
comment: string
|
||||
disabled: boolean
|
||||
peers: WgSnapshotPeer[]
|
||||
}
|
||||
|
||||
export interface WgSnapshot {
|
||||
interfaces: WgSnapshotIface[]
|
||||
}
|
||||
|
||||
export interface WgLiveIface {
|
||||
name: string
|
||||
rosId: string
|
||||
listenPort: number
|
||||
mtu: number
|
||||
privateKey: string
|
||||
comment: string
|
||||
disabled: boolean
|
||||
}
|
||||
|
||||
export interface WgLivePeer {
|
||||
rosId: string
|
||||
interfaceName: string
|
||||
publicKey: string
|
||||
allowedAddresses: string[]
|
||||
endpointAddress: string
|
||||
endpointPort: string
|
||||
persistentKeepalive: number | null
|
||||
comment: string
|
||||
name: string
|
||||
disabled: boolean
|
||||
privateKey: string
|
||||
clientAddress: string
|
||||
clientDns: string
|
||||
clientEndpoint: string
|
||||
}
|
||||
|
||||
export interface WgLiveAddr {
|
||||
rosId: string
|
||||
interfaceName: string
|
||||
address: string
|
||||
}
|
||||
|
||||
export interface GreSnapshotTunnel {
|
||||
name: string
|
||||
localAddress: string
|
||||
remoteAddress: string
|
||||
localInnerIp: string
|
||||
remoteInnerIp: string
|
||||
comment: string
|
||||
disabled: boolean
|
||||
mtu: number
|
||||
keepalive: string
|
||||
dscp: string
|
||||
clampTcpMss: boolean
|
||||
allowFastPath: boolean
|
||||
ipsecSecret: string
|
||||
}
|
||||
|
||||
export interface GreSnapshot {
|
||||
tunnels: GreSnapshotTunnel[]
|
||||
}
|
||||
|
||||
export interface GreLiveIface {
|
||||
name: string
|
||||
rosId: string
|
||||
localAddress: string
|
||||
remoteAddress: string
|
||||
comment: string
|
||||
disabled: boolean
|
||||
mtu: number
|
||||
keepalive: string
|
||||
dscp: string
|
||||
clampTcpMss: boolean
|
||||
allowFastPath: boolean
|
||||
ipsecSecret: string
|
||||
}
|
||||
|
||||
export interface GreLiveAddr {
|
||||
rosId: string
|
||||
interfaceName: string
|
||||
address: string
|
||||
}
|
||||
|
||||
function str(v: unknown): string {
|
||||
return String(v ?? "").trim()
|
||||
}
|
||||
|
||||
function bool(v: unknown): boolean {
|
||||
if (typeof v === "boolean") return v
|
||||
const s = str(v).toLowerCase()
|
||||
return s === "true" || s === "yes" || s === "1"
|
||||
}
|
||||
|
||||
function num(v: unknown, fallback: number): number {
|
||||
const n = typeof v === "number" ? v : Number.parseInt(str(v), 10)
|
||||
return Number.isFinite(n) ? n : fallback
|
||||
}
|
||||
|
||||
export function isHiddenSecret(value: string | undefined): boolean {
|
||||
const s = str(value)
|
||||
if (!s) return true
|
||||
if (s === "(hidden)") return true
|
||||
return /^\*+$/.test(s)
|
||||
}
|
||||
|
||||
export function firewallRestPath(
|
||||
family: FirewallFamily,
|
||||
table: FirewallTable | "address-list",
|
||||
): string {
|
||||
const root = family === "ip6" ? "/ipv6/firewall" : "/ip/firewall"
|
||||
return `${root}/${table}`
|
||||
}
|
||||
|
||||
function rosYesNo(v: boolean | undefined): string | undefined {
|
||||
if (v === true) return "yes"
|
||||
if (v === false) return "no"
|
||||
return undefined
|
||||
}
|
||||
|
||||
function compactBody(obj: Record<string, string | undefined>): Record<string, string> {
|
||||
const out: Record<string, string> = {}
|
||||
for (const [k, v] of Object.entries(obj)) {
|
||||
if (v !== undefined && v !== "") out[k] = v
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
export function canonicalFirewallSnapshot(input: {
|
||||
rules?: Array<Partial<FirewallSnapshotRule>>
|
||||
addressLists?: Array<Partial<FirewallSnapshotList>>
|
||||
}): FirewallSnapshot {
|
||||
const rules = (input.rules ?? []).map((r) => ({
|
||||
family: r.family === "ip6" ? "ip6" as const : "ip" as const,
|
||||
table: (["filter", "nat", "mangle", "raw"] as const).includes(r.table as FirewallTable)
|
||||
? (r.table as FirewallTable)
|
||||
: "filter",
|
||||
chain: str(r.chain),
|
||||
action: str(r.action),
|
||||
protocol: str(r.protocol),
|
||||
srcAddress: str(r.srcAddress),
|
||||
dstAddress: str(r.dstAddress),
|
||||
srcAddressList: str(r.srcAddressList),
|
||||
dstAddressList: str(r.dstAddressList),
|
||||
srcPort: str(r.srcPort),
|
||||
dstPort: str(r.dstPort),
|
||||
inInterface: str(r.inInterface),
|
||||
outInterface: str(r.outInterface),
|
||||
connectionState: str(r.connectionState),
|
||||
comment: str(r.comment),
|
||||
disabled: Boolean(r.disabled),
|
||||
log: Boolean(r.log),
|
||||
logPrefix: str(r.logPrefix),
|
||||
tlsHost: str(r.tlsHost),
|
||||
layer7Proto: str(r.layer7Proto),
|
||||
}))
|
||||
const addressLists = (input.addressLists ?? []).map((e) => ({
|
||||
family: e.family === "ip6" ? "ip6" as const : "ip" as const,
|
||||
list: str(e.list),
|
||||
address: str(e.address),
|
||||
comment: str(e.comment),
|
||||
disabled: Boolean(e.disabled),
|
||||
timeout: str(e.timeout),
|
||||
}))
|
||||
return { rules, addressLists }
|
||||
}
|
||||
|
||||
export function parseFirewallSnapshot(payload: unknown): FirewallSnapshot {
|
||||
if (!payload || typeof payload !== "object" || Array.isArray(payload)) {
|
||||
return { rules: [], addressLists: [] }
|
||||
}
|
||||
const o = payload as Record<string, unknown>
|
||||
return canonicalFirewallSnapshot({
|
||||
rules: Array.isArray(o.rules) ? o.rules as Partial<FirewallSnapshotRule>[] : [],
|
||||
addressLists: Array.isArray(o.addressLists) ? o.addressLists as Partial<FirewallSnapshotList>[] : [],
|
||||
})
|
||||
}
|
||||
|
||||
function firewallRuleKey(r: FirewallSnapshotRule): string {
|
||||
return [
|
||||
r.family, r.table, r.chain, r.action, r.protocol,
|
||||
r.srcAddress, r.dstAddress, r.srcAddressList, r.dstAddressList,
|
||||
r.srcPort, r.dstPort, r.inInterface, r.outInterface, r.connectionState,
|
||||
r.comment, r.disabled ? "1" : "0", r.log ? "1" : "0", r.logPrefix, r.tlsHost, r.layer7Proto,
|
||||
].join("\0")
|
||||
}
|
||||
|
||||
function firewallListKey(e: FirewallSnapshotList): string {
|
||||
return [e.family, e.list, e.address, e.comment, e.disabled ? "1" : "0", e.timeout].join("\0")
|
||||
}
|
||||
|
||||
function firewallRuleBody(r: FirewallSnapshotRule): Record<string, string> {
|
||||
return compactBody({
|
||||
chain: r.chain,
|
||||
action: r.action,
|
||||
protocol: r.protocol && r.protocol !== "all" ? r.protocol : undefined,
|
||||
"src-address": r.srcAddress,
|
||||
"dst-address": r.dstAddress,
|
||||
"src-address-list": r.srcAddressList,
|
||||
"dst-address-list": r.dstAddressList,
|
||||
"src-port": r.srcPort,
|
||||
"dst-port": r.dstPort,
|
||||
"in-interface": r.inInterface,
|
||||
"out-interface": r.outInterface,
|
||||
"connection-state": r.connectionState,
|
||||
comment: r.comment,
|
||||
disabled: rosYesNo(r.disabled),
|
||||
log: rosYesNo(r.log),
|
||||
"log-prefix": r.logPrefix,
|
||||
"tls-host": r.tlsHost,
|
||||
"layer7-protocol": r.layer7Proto,
|
||||
})
|
||||
}
|
||||
|
||||
export function planFirewallRestore(
|
||||
desiredInput: FirewallSnapshot,
|
||||
current: { rules: FirewallLiveRule[]; addressLists: FirewallLiveList[] },
|
||||
): RosWriteOp[] {
|
||||
const desired = canonicalFirewallSnapshot(desiredInput)
|
||||
const ops: RosWriteOp[] = []
|
||||
const usedRules = new Set<string>()
|
||||
const usedLists = new Set<string>()
|
||||
|
||||
for (const live of current.rules) {
|
||||
if (live.dynamic) continue
|
||||
const key = firewallRuleKey(live)
|
||||
const stillWanted = desired.rules.some((d) => firewallRuleKey(d) === key)
|
||||
if (!stillWanted) {
|
||||
ops.push({
|
||||
op: "delete",
|
||||
path: `${firewallRestPath(live.family, live.table)}/${live.rosId}`,
|
||||
})
|
||||
} else {
|
||||
usedRules.add(key)
|
||||
}
|
||||
}
|
||||
|
||||
for (const live of current.addressLists) {
|
||||
if (live.dynamic) continue
|
||||
const key = firewallListKey(live)
|
||||
const stillWanted = desired.addressLists.some((d) => firewallListKey(d) === key)
|
||||
if (!stillWanted) {
|
||||
ops.push({
|
||||
op: "delete",
|
||||
path: `${firewallRestPath(live.family, "address-list")}/${live.rosId}`,
|
||||
})
|
||||
} else {
|
||||
usedLists.add(key)
|
||||
}
|
||||
}
|
||||
|
||||
for (const rule of desired.rules) {
|
||||
if (usedRules.has(firewallRuleKey(rule))) continue
|
||||
ops.push({
|
||||
op: "put",
|
||||
path: firewallRestPath(rule.family, rule.table),
|
||||
body: firewallRuleBody(rule),
|
||||
})
|
||||
}
|
||||
|
||||
for (const entry of desired.addressLists) {
|
||||
if (usedLists.has(firewallListKey(entry))) continue
|
||||
ops.push({
|
||||
op: "put",
|
||||
path: firewallRestPath(entry.family, "address-list"),
|
||||
body: compactBody({
|
||||
list: entry.list,
|
||||
address: entry.address,
|
||||
comment: entry.comment,
|
||||
timeout: entry.timeout,
|
||||
disabled: rosYesNo(entry.disabled),
|
||||
}),
|
||||
})
|
||||
}
|
||||
|
||||
return ops
|
||||
}
|
||||
|
||||
function canonicalPeer(p: Partial<WgSnapshotPeer>): WgSnapshotPeer {
|
||||
const allowed = Array.isArray(p.allowedAddresses)
|
||||
? p.allowedAddresses.map((a) => str(a)).filter(Boolean)
|
||||
: str((p as { allowedIps?: unknown }).allowedIps)
|
||||
.split(",")
|
||||
.map((s) => s.trim())
|
||||
.filter(Boolean)
|
||||
return {
|
||||
publicKey: str(p.publicKey),
|
||||
allowedAddresses: allowed,
|
||||
endpointAddress: str(p.endpointAddress),
|
||||
endpointPort: str(p.endpointPort),
|
||||
persistentKeepalive: p.persistentKeepalive == null ? null : num(p.persistentKeepalive, 0) || null,
|
||||
comment: str(p.comment),
|
||||
name: str(p.name),
|
||||
disabled: Boolean(p.disabled),
|
||||
privateKey: str(p.privateKey),
|
||||
clientAddress: str(p.clientAddress),
|
||||
clientDns: str(p.clientDns),
|
||||
clientEndpoint: str(p.clientEndpoint),
|
||||
}
|
||||
}
|
||||
|
||||
export function canonicalWireguardSnapshot(input: {
|
||||
interfaces?: Array<Partial<WgSnapshotIface> & { peers?: Array<Partial<WgSnapshotPeer>> }>
|
||||
}): WgSnapshot {
|
||||
const interfaces = (input.interfaces ?? [])
|
||||
.map((iface) => ({
|
||||
name: str(iface.name),
|
||||
listenPort: num(iface.listenPort, 13231),
|
||||
mtu: num(iface.mtu, 1420),
|
||||
privateKey: str(iface.privateKey),
|
||||
address: str(iface.address),
|
||||
comment: str(iface.comment),
|
||||
disabled: Boolean(iface.disabled),
|
||||
peers: (iface.peers ?? []).map(canonicalPeer).sort((a, b) => a.publicKey.localeCompare(b.publicKey)),
|
||||
}))
|
||||
.filter((i) => i.name)
|
||||
.sort((a, b) => a.name.localeCompare(b.name))
|
||||
return { interfaces }
|
||||
}
|
||||
|
||||
export function parseWireguardSnapshot(payload: unknown): WgSnapshot {
|
||||
if (!payload || typeof payload !== "object" || Array.isArray(payload)) {
|
||||
return { interfaces: [] }
|
||||
}
|
||||
const o = payload as Record<string, unknown>
|
||||
return canonicalWireguardSnapshot({
|
||||
interfaces: Array.isArray(o.interfaces)
|
||||
? o.interfaces as Array<Partial<WgSnapshotIface> & { peers?: Array<Partial<WgSnapshotPeer>> }>
|
||||
: [],
|
||||
})
|
||||
}
|
||||
|
||||
function peerBody(interfaceName: string, p: WgSnapshotPeer): Record<string, string> {
|
||||
return compactBody({
|
||||
interface: interfaceName,
|
||||
"public-key": p.publicKey,
|
||||
"allowed-address": p.allowedAddresses.join(","),
|
||||
"endpoint-address": p.endpointAddress,
|
||||
"endpoint-port": p.endpointPort,
|
||||
"persistent-keepalive": p.persistentKeepalive != null ? String(p.persistentKeepalive) : undefined,
|
||||
comment: p.comment,
|
||||
name: p.name,
|
||||
"private-key": isHiddenSecret(p.privateKey) ? undefined : p.privateKey,
|
||||
"client-address": p.clientAddress,
|
||||
"client-dns": p.clientDns,
|
||||
"client-endpoint": p.clientEndpoint,
|
||||
disabled: rosYesNo(p.disabled),
|
||||
})
|
||||
}
|
||||
|
||||
export function planWireguardRestore(
|
||||
desiredInput: WgSnapshot,
|
||||
current: { ifaces: WgLiveIface[]; peers: WgLivePeer[]; addrs: WgLiveAddr[] },
|
||||
): RosWriteOp[] {
|
||||
const desired = canonicalWireguardSnapshot(desiredInput)
|
||||
const wantedNames = new Set(desired.interfaces.map((i) => i.name))
|
||||
const ops: RosWriteOp[] = []
|
||||
|
||||
for (const peer of current.peers) {
|
||||
const iface = desired.interfaces.find((i) => i.name === peer.interfaceName)
|
||||
const keep = iface?.peers.some((p) => p.publicKey === peer.publicKey)
|
||||
if (!keep) {
|
||||
ops.push({ op: "delete", path: `/interface/wireguard/peers/${peer.rosId}` })
|
||||
}
|
||||
}
|
||||
|
||||
for (const addr of current.addrs) {
|
||||
if (!wantedNames.has(addr.interfaceName)) {
|
||||
ops.push({ op: "delete", path: `/ip/address/${addr.rosId}` })
|
||||
}
|
||||
}
|
||||
|
||||
for (const iface of current.ifaces) {
|
||||
if (!wantedNames.has(iface.name)) {
|
||||
ops.push({ op: "delete", path: `/interface/wireguard/${iface.rosId}` })
|
||||
}
|
||||
}
|
||||
|
||||
for (const want of desired.interfaces) {
|
||||
const live = current.ifaces.find((i) => i.name === want.name)
|
||||
const ifaceBody = compactBody({
|
||||
name: want.name,
|
||||
"listen-port": String(want.listenPort),
|
||||
mtu: String(want.mtu),
|
||||
"private-key": isHiddenSecret(want.privateKey) ? undefined : want.privateKey,
|
||||
comment: want.comment,
|
||||
disabled: rosYesNo(want.disabled),
|
||||
})
|
||||
if (!live) {
|
||||
ops.push({ op: "put", path: "/interface/wireguard", body: ifaceBody })
|
||||
} else {
|
||||
ops.push({
|
||||
op: "patch",
|
||||
path: `/interface/wireguard/${live.rosId}`,
|
||||
body: ifaceBody,
|
||||
})
|
||||
}
|
||||
|
||||
const liveAddr = current.addrs.find((a) => a.interfaceName === want.name)
|
||||
if (want.address) {
|
||||
if (!liveAddr) {
|
||||
ops.push({ op: "put", path: "/ip/address", body: { address: want.address, interface: want.name } })
|
||||
} else if (liveAddr.address !== want.address) {
|
||||
ops.push({ op: "delete", path: `/ip/address/${liveAddr.rosId}` })
|
||||
ops.push({ op: "put", path: "/ip/address", body: { address: want.address, interface: want.name } })
|
||||
}
|
||||
} else if (liveAddr) {
|
||||
ops.push({ op: "delete", path: `/ip/address/${liveAddr.rosId}` })
|
||||
}
|
||||
|
||||
for (const peer of want.peers) {
|
||||
if (!peer.publicKey) continue
|
||||
const livePeer = current.peers.find(
|
||||
(p) => p.interfaceName === want.name && p.publicKey === peer.publicKey,
|
||||
)
|
||||
const body = peerBody(want.name, peer)
|
||||
if (!livePeer) {
|
||||
ops.push({ op: "put", path: "/interface/wireguard/peers", body })
|
||||
} else {
|
||||
ops.push({
|
||||
op: "patch",
|
||||
path: `/interface/wireguard/peers/${livePeer.rosId}`,
|
||||
body,
|
||||
})
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
return ops
|
||||
}
|
||||
|
||||
export function canonicalGreSnapshot(input: {
|
||||
tunnels?: Array<Partial<GreSnapshotTunnel>>
|
||||
}): GreSnapshot {
|
||||
const tunnels = (input.tunnels ?? [])
|
||||
.map((t) => ({
|
||||
name: str(t.name),
|
||||
localAddress: str(t.localAddress),
|
||||
remoteAddress: str(t.remoteAddress),
|
||||
localInnerIp: str(t.localInnerIp),
|
||||
remoteInnerIp: str(t.remoteInnerIp),
|
||||
comment: str(t.comment),
|
||||
disabled: Boolean(t.disabled),
|
||||
mtu: num(t.mtu, 1476),
|
||||
keepalive: str(t.keepalive) || "0",
|
||||
dscp: str(t.dscp) || "inherit",
|
||||
clampTcpMss: t.clampTcpMss !== false,
|
||||
allowFastPath: t.allowFastPath !== false,
|
||||
ipsecSecret: str(t.ipsecSecret),
|
||||
}))
|
||||
.filter((t) => t.name)
|
||||
.sort((a, b) => a.name.localeCompare(b.name))
|
||||
return { tunnels }
|
||||
}
|
||||
|
||||
export function parseGreSnapshot(payload: unknown): GreSnapshot {
|
||||
if (!payload || typeof payload !== "object" || Array.isArray(payload)) {
|
||||
return { tunnels: [] }
|
||||
}
|
||||
const o = payload as Record<string, unknown>
|
||||
return canonicalGreSnapshot({
|
||||
tunnels: Array.isArray(o.tunnels) ? o.tunnels as Array<Partial<GreSnapshotTunnel>> : [],
|
||||
})
|
||||
}
|
||||
|
||||
export function greInterfaceBody(t: GreSnapshotTunnel): Record<string, string> {
|
||||
return compactBody({
|
||||
name: t.name,
|
||||
"local-address": t.localAddress && t.localAddress !== "0.0.0.0" ? t.localAddress : undefined,
|
||||
"remote-address": t.remoteAddress,
|
||||
mtu: String(t.mtu),
|
||||
keepalive: t.keepalive,
|
||||
dscp: t.dscp,
|
||||
"clamp-tcp-mss": t.clampTcpMss ? "yes" : "no",
|
||||
"allow-fast-path": t.allowFastPath ? "yes" : "no",
|
||||
comment: t.comment,
|
||||
disabled: rosYesNo(t.disabled),
|
||||
"ipsec-secret": isHiddenSecret(t.ipsecSecret) ? undefined : t.ipsecSecret,
|
||||
})
|
||||
}
|
||||
|
||||
export function planGreCreate(tunnel: GreSnapshotTunnel): RosWriteOp[] {
|
||||
const t = canonicalGreSnapshot({ tunnels: [tunnel] }).tunnels[0]
|
||||
if (!t) return []
|
||||
const ops: RosWriteOp[] = [
|
||||
{ op: "put", path: "/interface/gre", body: greInterfaceBody(t) },
|
||||
]
|
||||
if (t.localInnerIp) {
|
||||
ops.push({
|
||||
op: "put",
|
||||
path: "/ip/address",
|
||||
body: { address: t.localInnerIp, interface: t.name },
|
||||
})
|
||||
}
|
||||
return ops
|
||||
}
|
||||
|
||||
export function planGreDelete(
|
||||
name: string,
|
||||
current: { gre: GreLiveIface[]; addrs: GreLiveAddr[] },
|
||||
): RosWriteOp[] {
|
||||
const want = str(name)
|
||||
const ops: RosWriteOp[] = []
|
||||
for (const addr of current.addrs) {
|
||||
if (addr.interfaceName === want) {
|
||||
ops.push({ op: "delete", path: `/ip/address/${addr.rosId}` })
|
||||
}
|
||||
}
|
||||
for (const gre of current.gre) {
|
||||
if (gre.name === want) {
|
||||
ops.push({ op: "delete", path: `/interface/gre/${gre.rosId}` })
|
||||
}
|
||||
}
|
||||
return ops
|
||||
}
|
||||
|
||||
export function planGreRestore(
|
||||
desiredInput: GreSnapshot,
|
||||
current: { gre: GreLiveIface[]; addrs: GreLiveAddr[] },
|
||||
): RosWriteOp[] {
|
||||
const desired = canonicalGreSnapshot(desiredInput)
|
||||
const wanted = new Set(desired.tunnels.map((t) => t.name))
|
||||
const ops: RosWriteOp[] = []
|
||||
|
||||
for (const gre of current.gre) {
|
||||
if (!wanted.has(gre.name)) {
|
||||
ops.push(...planGreDelete(gre.name, current))
|
||||
}
|
||||
}
|
||||
|
||||
for (const want of desired.tunnels) {
|
||||
const live = current.gre.find((g) => g.name === want.name)
|
||||
const body = greInterfaceBody(want)
|
||||
if (!live) {
|
||||
ops.push({ op: "put", path: "/interface/gre", body })
|
||||
} else {
|
||||
ops.push({ op: "patch", path: `/interface/gre/${live.rosId}`, body })
|
||||
}
|
||||
|
||||
const liveAddr = current.addrs.find((a) => a.interfaceName === want.name)
|
||||
if (want.localInnerIp) {
|
||||
if (!liveAddr) {
|
||||
ops.push({
|
||||
op: "put",
|
||||
path: "/ip/address",
|
||||
body: { address: want.localInnerIp, interface: want.name },
|
||||
})
|
||||
} else if (liveAddr.address !== want.localInnerIp) {
|
||||
ops.push({ op: "delete", path: `/ip/address/${liveAddr.rosId}` })
|
||||
ops.push({
|
||||
op: "put",
|
||||
path: "/ip/address",
|
||||
body: { address: want.localInnerIp, interface: want.name },
|
||||
})
|
||||
}
|
||||
} else if (liveAddr) {
|
||||
ops.push({ op: "delete", path: `/ip/address/${liveAddr.rosId}` })
|
||||
}
|
||||
}
|
||||
|
||||
return ops
|
||||
}
|
||||
|
||||
export function opsPaths(ops: RosWriteOp[]): string[] {
|
||||
return ops.map((op) => op.path)
|
||||
}
|
||||
|
||||
export function opsTouchOnly(ops: RosWriteOp[], prefixes: string[]): boolean {
|
||||
return ops.every((op) => prefixes.some((p) => op.path === p || op.path.startsWith(`${p}/`)))
|
||||
}
|
||||
@@ -5,12 +5,21 @@ import {
|
||||
MikrotikClient,
|
||||
firewallRestPath,
|
||||
} from "./mikrotik.js"
|
||||
import {
|
||||
captureAndAppendRevision,
|
||||
} from "./config-revisions.js"
|
||||
import type {
|
||||
FirewallFamily,
|
||||
FirewallTable,
|
||||
RosFirewallAddressList,
|
||||
RosFirewallFilter,
|
||||
} from "../types/server.js"
|
||||
import {
|
||||
canonicalFirewallSnapshot,
|
||||
type FirewallLiveList,
|
||||
type FirewallLiveRule,
|
||||
type FirewallSnapshot,
|
||||
} from "./entity-snapshots.js"
|
||||
|
||||
type ServerRow = typeof servers.$inferSelect
|
||||
|
||||
@@ -150,29 +159,121 @@ async function safeGet<T>(fn: () => Promise<T[]>, fallback: T[] = []): Promise<T
|
||||
}
|
||||
}
|
||||
|
||||
export async function fetchServerFirewall(server: ServerRow): Promise<{
|
||||
function rosYes(v: string | undefined): boolean {
|
||||
return v === "true" || v === "yes"
|
||||
}
|
||||
|
||||
export function mapFirewallSnapshotRule(
|
||||
family: FirewallFamily,
|
||||
table: FirewallTable,
|
||||
raw: RosFirewallFilter,
|
||||
): FirewallLiveRule {
|
||||
return {
|
||||
rosId: raw[".id"] || "",
|
||||
dynamic: rosYes(raw.dynamic),
|
||||
family,
|
||||
table,
|
||||
chain: raw.chain || "",
|
||||
action: raw.action || "",
|
||||
protocol: raw.protocol || "",
|
||||
srcAddress: raw["src-address"] ?? "",
|
||||
dstAddress: raw["dst-address"] ?? "",
|
||||
srcAddressList: raw["src-address-list"] ?? "",
|
||||
dstAddressList: raw["dst-address-list"] ?? "",
|
||||
srcPort: raw["src-port"] ?? "",
|
||||
dstPort: raw["dst-port"] ?? "",
|
||||
inInterface: raw["in-interface"] ?? "",
|
||||
outInterface: raw["out-interface"] ?? "",
|
||||
connectionState: raw["connection-state"] ?? "",
|
||||
comment: raw.comment ?? "",
|
||||
disabled: rosDisabled(raw.disabled),
|
||||
log: rosYes(raw.log),
|
||||
logPrefix: raw["log-prefix"] ?? "",
|
||||
tlsHost: raw["tls-host"] ?? "",
|
||||
layer7Proto: raw["layer7-protocol"] ?? "",
|
||||
}
|
||||
}
|
||||
|
||||
export function mapFirewallSnapshotList(
|
||||
family: FirewallFamily,
|
||||
raw: RosFirewallAddressList,
|
||||
): FirewallLiveList {
|
||||
return {
|
||||
rosId: raw[".id"] || "",
|
||||
dynamic: rosYes(raw.dynamic),
|
||||
family,
|
||||
list: raw.list || "",
|
||||
address: raw.address || "",
|
||||
comment: raw.comment ?? "",
|
||||
disabled: rosDisabled(raw.disabled),
|
||||
timeout: raw.timeout ?? "",
|
||||
}
|
||||
}
|
||||
|
||||
export async function fetchFirewallState(server: ServerRow): Promise<{
|
||||
rules: FirewallRuleDto[]
|
||||
addressLists: FirewallAddressListDto[]
|
||||
liveRules: FirewallLiveRule[]
|
||||
liveLists: FirewallLiveList[]
|
||||
snapshot: FirewallSnapshot
|
||||
}> {
|
||||
const client = MikrotikClient.fromServer(server)
|
||||
const ruleJobs = FAMILIES.flatMap((family) =>
|
||||
TABLES.map(async (table) => {
|
||||
const raw = await safeGet(() => client.getFirewallRules(family, table))
|
||||
return raw.map((row, idx) => mapFirewallRule(server, family, table, row, idx))
|
||||
return { family, table, raw }
|
||||
}),
|
||||
)
|
||||
const listJobs = FAMILIES.map(async (family) => {
|
||||
const raw = await safeGet(() => client.getFirewallAddressList(family))
|
||||
return raw.map((row, idx) => mapAddressList(server, family, row, idx))
|
||||
return { family, raw }
|
||||
})
|
||||
const [ruleChunks, listChunks] = await Promise.all([
|
||||
Promise.all(ruleJobs),
|
||||
Promise.all(listJobs),
|
||||
])
|
||||
return {
|
||||
rules: ruleChunks.flat(),
|
||||
addressLists: listChunks.flat(),
|
||||
|
||||
const rules: FirewallRuleDto[] = []
|
||||
const liveRules: FirewallLiveRule[] = []
|
||||
for (const chunk of ruleChunks) {
|
||||
chunk.raw.forEach((row, idx) => {
|
||||
rules.push(mapFirewallRule(server, chunk.family, chunk.table, row, idx))
|
||||
liveRules.push(mapFirewallSnapshotRule(chunk.family, chunk.table, row))
|
||||
})
|
||||
}
|
||||
|
||||
const addressLists: FirewallAddressListDto[] = []
|
||||
const liveLists: FirewallLiveList[] = []
|
||||
for (const chunk of listChunks) {
|
||||
chunk.raw.forEach((row, idx) => {
|
||||
addressLists.push(mapAddressList(server, chunk.family, row, idx))
|
||||
liveLists.push(mapFirewallSnapshotList(chunk.family, row))
|
||||
})
|
||||
}
|
||||
|
||||
return {
|
||||
rules,
|
||||
addressLists,
|
||||
liveRules,
|
||||
liveLists,
|
||||
snapshot: canonicalFirewallSnapshot({
|
||||
rules: liveRules.filter((r) => !r.dynamic),
|
||||
addressLists: liveLists.filter((e) => !e.dynamic),
|
||||
}),
|
||||
}
|
||||
}
|
||||
|
||||
export async function fetchServerFirewall(server: ServerRow): Promise<{
|
||||
rules: FirewallRuleDto[]
|
||||
addressLists: FirewallAddressListDto[]
|
||||
}> {
|
||||
const state = await fetchFirewallState(server)
|
||||
return { rules: state.rules, addressLists: state.addressLists }
|
||||
}
|
||||
|
||||
export async function captureFirewallSnapshot(server: ServerRow): Promise<FirewallSnapshot> {
|
||||
const state = await fetchFirewallState(server)
|
||||
return state.snapshot
|
||||
}
|
||||
|
||||
export async function listFirewallAll(): Promise<{
|
||||
@@ -183,7 +284,14 @@ export async function listFirewallAll(): Promise<{
|
||||
const perServer = await Promise.all(
|
||||
allServers.map(async (server) => {
|
||||
try {
|
||||
return await fetchServerFirewall(server)
|
||||
const state = await fetchFirewallState(server)
|
||||
await captureAndAppendRevision({
|
||||
serverId: server.id,
|
||||
section: "firewall",
|
||||
source: "observed",
|
||||
capture: async () => state.snapshot,
|
||||
})
|
||||
return { rules: state.rules, addressLists: state.addressLists }
|
||||
} catch {
|
||||
return { rules: [] as FirewallRuleDto[], addressLists: [] as FirewallAddressListDto[] }
|
||||
}
|
||||
|
||||
@@ -0,0 +1,253 @@
|
||||
import { eq } from "drizzle-orm"
|
||||
import { db } from "../db/index.js"
|
||||
import { servers } from "../db/schema.js"
|
||||
import { MikrotikClient } from "./mikrotik.js"
|
||||
import {
|
||||
canonicalGreSnapshot,
|
||||
type GreLiveAddr,
|
||||
type GreLiveIface,
|
||||
type GreSnapshot,
|
||||
} from "./entity-snapshots.js"
|
||||
import { captureAndAppendRevision } from "./config-revisions.js"
|
||||
|
||||
type ServerRow = typeof servers.$inferSelect
|
||||
|
||||
export interface RosGre {
|
||||
".id"?: string
|
||||
name?: string
|
||||
"local-address"?: string
|
||||
"remote-address"?: string
|
||||
"allow-fast-path"?: string
|
||||
"clamp-tcp-mss"?: string
|
||||
mtu?: string
|
||||
keepalive?: string
|
||||
dscp?: string
|
||||
running?: string
|
||||
disabled?: string
|
||||
comment?: string
|
||||
"ipsec-secret"?: string
|
||||
}
|
||||
|
||||
interface RosIpAddress {
|
||||
".id"?: string
|
||||
address?: string
|
||||
interface?: string
|
||||
disabled?: string
|
||||
network?: string
|
||||
}
|
||||
|
||||
export interface LiveGreTunnel {
|
||||
id: string
|
||||
rosId: string
|
||||
name: string
|
||||
serverId: string
|
||||
localAddress: string
|
||||
remoteAddress: string
|
||||
localInnerIp: string
|
||||
remoteInnerIp: string
|
||||
poolId: string
|
||||
ipsec: { secret: string } | null
|
||||
mtu: number
|
||||
keepaliveInterval: number
|
||||
keepaliveRetries: number
|
||||
dscp: "inherit" | number
|
||||
clampTcpMss: boolean
|
||||
allowFastPath: boolean
|
||||
comment: string
|
||||
enabled: boolean
|
||||
status: "up" | "down" | "degraded"
|
||||
}
|
||||
|
||||
export function parseKeepalive(value: string | undefined): { interval: number; retries: number } {
|
||||
if (!value || value.toLowerCase() === "none") return { interval: 0, retries: 0 }
|
||||
const [intervalRaw, retriesRaw] = value.split(",")
|
||||
const interval = Number.parseInt((intervalRaw ?? "").trim(), 10)
|
||||
const retries = Number.parseInt((retriesRaw ?? "").trim(), 10)
|
||||
return {
|
||||
interval: Number.isFinite(interval) ? interval : 0,
|
||||
retries: Number.isFinite(retries) ? retries : 0,
|
||||
}
|
||||
}
|
||||
|
||||
export function formatKeepalive(interval: number, retries: number): string {
|
||||
if (!interval || interval <= 0) return "0"
|
||||
return `${interval}s,${retries > 0 ? retries : 10}`
|
||||
}
|
||||
|
||||
function parseDscp(value: string | undefined): "inherit" | number {
|
||||
if (!value || value === "inherit") return "inherit"
|
||||
const n = Number.parseInt(value, 10)
|
||||
return Number.isFinite(n) ? n : "inherit"
|
||||
}
|
||||
|
||||
function parseInnerFromComment(comment: string | undefined): { localInnerIp: string; remoteInnerIp: string } {
|
||||
if (!comment) return { localInnerIp: "", remoteInnerIp: "" }
|
||||
const local = comment.match(/address\s*=\s*([0-9.]+\/\d+)/)?.[1] ?? ""
|
||||
const remote = comment.match(/(?:network|gateway)\s*=\s*([0-9.]+\/\d+)/)?.[1] ?? ""
|
||||
return { localInnerIp: local, remoteInnerIp: remote }
|
||||
}
|
||||
|
||||
function rosDisabled(v: string | undefined): boolean {
|
||||
return v === "true" || v === "yes"
|
||||
}
|
||||
|
||||
export function mapGreLive(
|
||||
server: ServerRow,
|
||||
greRaw: RosGre[],
|
||||
addrsRaw: RosIpAddress[],
|
||||
): {
|
||||
tunnels: LiveGreTunnel[]
|
||||
snapshot: GreSnapshot
|
||||
gre: GreLiveIface[]
|
||||
addrs: GreLiveAddr[]
|
||||
} {
|
||||
const addrsByIface = new Map<string, { address: string; rosId: string }[]>()
|
||||
for (const a of addrsRaw) {
|
||||
if (rosDisabled(a.disabled)) continue
|
||||
const iface = (a.interface ?? "").trim()
|
||||
const address = (a.address ?? "").trim()
|
||||
const rosId = String(a[".id"] ?? "")
|
||||
if (!iface || !address || !rosId) continue
|
||||
const list = addrsByIface.get(iface) ?? []
|
||||
list.push({ address, rosId })
|
||||
addrsByIface.set(iface, list)
|
||||
}
|
||||
|
||||
const gre: GreLiveIface[] = []
|
||||
const addrs: GreLiveAddr[] = []
|
||||
const tunnels: LiveGreTunnel[] = []
|
||||
|
||||
greRaw.forEach((g, idx) => {
|
||||
const rosId = String(g[".id"] ?? g.name ?? `gre-${idx}`)
|
||||
const name = (g.name ?? "").trim() || `gre-${idx + 1}`
|
||||
const keepalive = parseKeepalive(g.keepalive)
|
||||
const fromComment = parseInnerFromComment(g.comment)
|
||||
const ifaceAddrs = addrsByIface.get(name) ?? []
|
||||
const localInnerIp = ifaceAddrs[0]?.address || fromComment.localInnerIp
|
||||
const secret = (g["ipsec-secret"] ?? "").trim()
|
||||
const disabled = rosDisabled(g.disabled)
|
||||
const running = g.running === "true" || g.running === "yes"
|
||||
|
||||
gre.push({
|
||||
name,
|
||||
rosId,
|
||||
localAddress: g["local-address"] ?? "",
|
||||
remoteAddress: g["remote-address"] ?? "",
|
||||
comment: g.comment ?? "",
|
||||
disabled,
|
||||
mtu: Number.parseInt(g.mtu ?? "1476", 10) || 1476,
|
||||
keepalive: g.keepalive ?? "0",
|
||||
dscp: g.dscp ?? "inherit",
|
||||
clampTcpMss: g["clamp-tcp-mss"] !== "false" && g["clamp-tcp-mss"] !== "no",
|
||||
allowFastPath: g["allow-fast-path"] !== "false" && g["allow-fast-path"] !== "no",
|
||||
ipsecSecret: secret,
|
||||
})
|
||||
|
||||
for (const a of ifaceAddrs) {
|
||||
addrs.push({ rosId: a.rosId, interfaceName: name, address: a.address })
|
||||
}
|
||||
|
||||
tunnels.push({
|
||||
id: rosId || `${server.id}:${name}`,
|
||||
rosId,
|
||||
name,
|
||||
serverId: String(server.id),
|
||||
localAddress: g["local-address"] ?? "",
|
||||
remoteAddress: g["remote-address"] ?? "",
|
||||
localInnerIp,
|
||||
remoteInnerIp: fromComment.remoteInnerIp,
|
||||
poolId: "live",
|
||||
ipsec: secret ? { secret } : null,
|
||||
mtu: Number.parseInt(g.mtu ?? "1476", 10) || 1476,
|
||||
keepaliveInterval: keepalive.interval,
|
||||
keepaliveRetries: keepalive.retries,
|
||||
dscp: parseDscp(g.dscp),
|
||||
clampTcpMss: g["clamp-tcp-mss"] !== "false" && g["clamp-tcp-mss"] !== "no",
|
||||
allowFastPath: g["allow-fast-path"] !== "false" && g["allow-fast-path"] !== "no",
|
||||
comment: g.comment ?? "",
|
||||
enabled: !disabled,
|
||||
status: disabled ? "down" : running ? "up" : "degraded",
|
||||
})
|
||||
})
|
||||
|
||||
return {
|
||||
tunnels,
|
||||
snapshot: canonicalGreSnapshot({
|
||||
tunnels: gre.map((g) => ({
|
||||
name: g.name,
|
||||
localAddress: g.localAddress,
|
||||
remoteAddress: g.remoteAddress,
|
||||
localInnerIp: addrs.find((a) => a.interfaceName === g.name)?.address ?? "",
|
||||
remoteInnerIp: "",
|
||||
comment: g.comment,
|
||||
disabled: g.disabled,
|
||||
mtu: g.mtu,
|
||||
keepalive: g.keepalive,
|
||||
dscp: g.dscp,
|
||||
clampTcpMss: g.clampTcpMss,
|
||||
allowFastPath: g.allowFastPath,
|
||||
ipsecSecret: g.ipsecSecret,
|
||||
})),
|
||||
}),
|
||||
gre,
|
||||
addrs,
|
||||
}
|
||||
}
|
||||
|
||||
export async function fetchGreState(server: ServerRow) {
|
||||
const client = MikrotikClient.fromServer(server)
|
||||
const [greRaw, addrsRaw] = await Promise.all([
|
||||
client.get<RosGre[]>("/interface/gre"),
|
||||
client.get<RosIpAddress[]>("/ip/address").catch(() => [] as RosIpAddress[]),
|
||||
])
|
||||
return {
|
||||
client,
|
||||
...mapGreLive(server, Array.isArray(greRaw) ? greRaw : [], Array.isArray(addrsRaw) ? addrsRaw : []),
|
||||
}
|
||||
}
|
||||
|
||||
export async function captureGreSnapshot(server: ServerRow): Promise<GreSnapshot> {
|
||||
const state = await fetchGreState(server)
|
||||
return state.snapshot
|
||||
}
|
||||
|
||||
export async function listGreTunnels(opts?: { serverId?: string }): Promise<{
|
||||
tunnels: LiveGreTunnel[]
|
||||
failures: Array<{ serverId: string; serverName?: string; error: string }>
|
||||
}> {
|
||||
let serverRows: ServerRow[]
|
||||
if (opts?.serverId) {
|
||||
const id = Number.parseInt(String(opts.serverId), 10)
|
||||
if (!Number.isFinite(id)) {
|
||||
return { tunnels: [], failures: [{ serverId: String(opts.serverId), error: "Некорректный serverId" }] }
|
||||
}
|
||||
const row = (await db.select().from(servers).where(eq(servers.id, id)).limit(1))[0]
|
||||
serverRows = row ? [row] : []
|
||||
} else {
|
||||
serverRows = await db.select().from(servers).where(eq(servers.enabled, true))
|
||||
}
|
||||
|
||||
const failures: Array<{ serverId: string; serverName?: string; error: string }> = []
|
||||
const chunks = await Promise.all(
|
||||
serverRows.map(async (server) => {
|
||||
try {
|
||||
const state = await fetchGreState(server)
|
||||
await captureAndAppendRevision({
|
||||
serverId: server.id,
|
||||
section: "gre",
|
||||
source: "observed",
|
||||
capture: async () => state.snapshot,
|
||||
})
|
||||
return state.tunnels
|
||||
} catch (e) {
|
||||
failures.push({
|
||||
serverId: String(server.id),
|
||||
serverName: server.name ?? undefined,
|
||||
error: e instanceof Error ? e.message : String(e),
|
||||
})
|
||||
return [] as LiveGreTunnel[]
|
||||
}
|
||||
}),
|
||||
)
|
||||
return { tunnels: chunks.flat(), failures }
|
||||
}
|
||||
@@ -0,0 +1,22 @@
|
||||
import assert from "node:assert/strict"
|
||||
import type { RosIpRoute } from "../types/server.js"
|
||||
import { parseOspfGateway, parseOspfRouteType } from "./ospf-route-parse.js"
|
||||
|
||||
function route(partial: Partial<RosIpRoute>): RosIpRoute {
|
||||
return { ".id": "*1", "dst-address": "10.0.0.0/8", ...partial }
|
||||
}
|
||||
|
||||
assert.equal(parseOspfRouteType(route({ static: "true" })), null)
|
||||
assert.equal(parseOspfRouteType(route({ bgp: "true" })), null)
|
||||
assert.equal(parseOspfRouteType(route({ ospf: "true" })), "O")
|
||||
assert.equal(parseOspfRouteType(route({ "ospf-type": "intra-area" })), "O")
|
||||
assert.equal(parseOspfRouteType(route({ ospf: "true", "ospf-type": "inter-area" })), "O IA")
|
||||
assert.equal(parseOspfRouteType(route({ ospf: "true", "ospf-type": "ext-type-1" })), "O E1")
|
||||
assert.equal(parseOspfRouteType(route({ ospf: "true", "ospf-type": "type-2" })), "O E2")
|
||||
|
||||
assert.deepEqual(parseOspfGateway(route({ gateway: "10.200.0.1%gre-msk-spb" })), {
|
||||
nextHop: "10.200.0.1",
|
||||
via: "gre-msk-spb",
|
||||
})
|
||||
|
||||
console.log("ospf-route-parse.test.ts: ok")
|
||||
@@ -0,0 +1,28 @@
|
||||
import type { RosIpRoute } from "../types/server.js"
|
||||
|
||||
export type OspfRouteKind = "O" | "O IA" | "O E1" | "O E2"
|
||||
|
||||
/** RouterOS /ip/route → тип OSPF-маршрута UI, либо null если маршрут не OSPF. */
|
||||
export function parseOspfRouteType(r: RosIpRoute): OspfRouteKind | null {
|
||||
const ospfFlag = r.ospf === "true" || r.ospf === "yes"
|
||||
const raw = `${r["ospf-type"] ?? ""} ${r.type ?? ""}`.toLowerCase()
|
||||
const looksOspf = ospfFlag || raw.includes("ospf") || Boolean(r["ospf-type"])
|
||||
if (!looksOspf) return null
|
||||
if (raw.includes("inter")) return "O IA"
|
||||
if (raw.includes("e1") || raw.includes("type-1") || raw.includes("ext-1") || raw.includes("nssa-ext-type-1")) {
|
||||
return "O E1"
|
||||
}
|
||||
if (raw.includes("e2") || raw.includes("type-2") || raw.includes("ext-2") || raw.includes("nssa-ext-type-2")) {
|
||||
return "O E2"
|
||||
}
|
||||
return "O"
|
||||
}
|
||||
|
||||
export function parseOspfGateway(r: RosIpRoute): { nextHop: string; via: string } {
|
||||
const gw = (r.gateway ?? r["immediate-gw"] ?? "").trim()
|
||||
const [ip, iface = ""] = gw.split("%")
|
||||
return {
|
||||
nextHop: ip || gw || "—",
|
||||
via: iface || (r.interface ?? "—"),
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,32 @@
|
||||
import type { MikrotikClient } from "./mikrotik.js"
|
||||
import type { RosWriteOp } from "./entity-snapshots.js"
|
||||
|
||||
function encodeIdSegment(path: string): string {
|
||||
const i = path.lastIndexOf("/")
|
||||
if (i < 0) return path
|
||||
const last = path.slice(i + 1)
|
||||
if (!last.startsWith("*")) return path
|
||||
return `${path.slice(0, i + 1)}${encodeURIComponent(last)}`
|
||||
}
|
||||
|
||||
export async function executeRosOps(client: MikrotikClient, ops: RosWriteOp[]): Promise<void> {
|
||||
for (const op of ops) {
|
||||
if (op.op === "put") {
|
||||
await client.put(op.path, op.body)
|
||||
continue
|
||||
}
|
||||
if (op.op === "post") {
|
||||
await client.post(encodeIdSegment(op.path), op.body)
|
||||
continue
|
||||
}
|
||||
if (op.op === "patch") {
|
||||
await client.patch(encodeIdSegment(op.path), op.body)
|
||||
continue
|
||||
}
|
||||
if (op.op === "delete") {
|
||||
await client.delete(encodeIdSegment(op.path))
|
||||
continue
|
||||
}
|
||||
await client.post(encodeIdSegment(op.path), op.body)
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,285 @@
|
||||
import assert from "node:assert/strict"
|
||||
import { getStatistics, getStatisticsPivot, parseStatisticsPeriod, pivotDimsConflict } from "./statistics-aggregate.js"
|
||||
import { rememberServerIfaces, resetIfaceCacheForTests } from "./traffic-flow-ifindex.js"
|
||||
import { setRefreshIfacesForTests } from "./traffic-flow-ifaces.js"
|
||||
import { invalidateFlowCatalogCache } from "./traffic-flow-topology.js"
|
||||
import { withPgOrSkip } from "../test/pg.js"
|
||||
import { dbQuery } from "../db/index.js"
|
||||
import { ensurePartitionFor } from "../db/partitions.js"
|
||||
import { pool } from "../db/index.js"
|
||||
import { STATISTICS_UNBOUND_USER_ID } from "@mmapp/contracts/statistics"
|
||||
|
||||
{
|
||||
const sameDay = parseStatisticsPeriod("2026-09-10", "2026-09-10")
|
||||
assert.ok(sameDay)
|
||||
assert.equal(sameDay.fromDay, "2026-09-10")
|
||||
assert.equal(sameDay.toDayExclusive, "2026-09-11")
|
||||
assert.equal(sameDay.grain, "hour")
|
||||
const month = parseStatisticsPeriod("2026-08-01", "2026-08-31")
|
||||
assert.ok(month)
|
||||
assert.equal(month.grain, "day")
|
||||
assert.equal(month.toDayExclusive, "2026-09-01")
|
||||
assert.equal(parseStatisticsPeriod("2026-09-10", "2026-09-09"), null)
|
||||
assert.equal(pivotDimsConflict("country", "country"), true)
|
||||
assert.equal(pivotDimsConflict("country", "service"), false)
|
||||
}
|
||||
|
||||
if (!(await withPgOrSkip())) {
|
||||
console.log("statistics-aggregate.test.ts: skip")
|
||||
process.exit(0)
|
||||
}
|
||||
|
||||
const inserted = await dbQuery<{ id: number }>(`
|
||||
INSERT INTO servers (name, host, type, wan_uplinks)
|
||||
VALUES ('stats-cube', '127.0.0.1', 'jump-host', '[{"iface":"wan1"}]'::jsonb)
|
||||
RETURNING id
|
||||
`)
|
||||
const serverId = inserted.rows[0]?.id
|
||||
if (serverId == null) throw new Error("no server")
|
||||
|
||||
const enInserted = await dbQuery<{ id: number }>(`
|
||||
INSERT INTO servers (name, host, type)
|
||||
VALUES ('stats-en', '198.51.100.1', 'exit-node')
|
||||
RETURNING id
|
||||
`)
|
||||
const enId = enInserted.rows[0]?.id
|
||||
if (enId == null) throw new Error("no en server")
|
||||
|
||||
await ensurePartitionFor(pool, "flow_daily_facts", "month", new Date("2026-09-01T00:00:00Z"))
|
||||
await ensurePartitionFor(pool, "flow_hour_facts", "day", new Date("2026-09-10T00:00:00Z"))
|
||||
await dbQuery(`DELETE FROM flow_daily_facts WHERE server_id IN ($1, $2)`, [serverId, enId])
|
||||
await dbQuery(`DELETE FROM flow_hour_facts WHERE server_id IN ($1, $2)`, [serverId, enId])
|
||||
await dbQuery(`DELETE FROM user_interface_bindings WHERE server_id = $1`, [serverId])
|
||||
await dbQuery(`DELETE FROM server_snapshots WHERE server_id IN ($1, $2)`, [serverId, enId])
|
||||
await dbQuery(`DELETE FROM app_users WHERE id = 'u-stats-1'`)
|
||||
|
||||
await dbQuery(`
|
||||
INSERT INTO app_users (id, name, login, role, active)
|
||||
VALUES ('u-stats-1', 'Клиент', 'stats-user', 'viewer', TRUE)
|
||||
ON CONFLICT (id) DO NOTHING
|
||||
`)
|
||||
await dbQuery(`
|
||||
INSERT INTO user_interface_bindings (id, user_id, server_id, interface_name, interface_type)
|
||||
VALUES ('bind-stats-1', 'u-stats-1', $1, 'gre-client', 'gre')
|
||||
`, [serverId])
|
||||
|
||||
await dbQuery(`
|
||||
INSERT INTO server_snapshots (server_id, polled_at, status, raw_interfaces)
|
||||
VALUES
|
||||
($1, '2026-09-10T12:00:00Z', 'online', $3::jsonb),
|
||||
($2, '2026-09-10T12:00:00Z', 'online', $4::jsonb)
|
||||
`, [
|
||||
serverId,
|
||||
enId,
|
||||
JSON.stringify([
|
||||
{ name: "gre-client", type: "gre-tunnel" },
|
||||
{ name: "wan1", type: "ether" },
|
||||
{ name: "gre-en", type: "gre-tunnel" },
|
||||
{ name: "NSK-SERVHOST-RTK", type: "gre-tunnel" },
|
||||
{ name: "wg-mesh", type: "wg" },
|
||||
{ name: "wg-server", type: "wg" },
|
||||
{ name: "wg-flow", type: "wg" },
|
||||
]),
|
||||
JSON.stringify([
|
||||
{ name: "ether1", type: "ether" },
|
||||
{ name: "gre-jh", type: "gre-tunnel" },
|
||||
]),
|
||||
])
|
||||
|
||||
resetIfaceCacheForTests()
|
||||
rememberServerIfaces(serverId, [
|
||||
{ name: "gre-client", ifindex: "2" },
|
||||
{ name: "wan1", ifindex: "8" },
|
||||
{ name: "gre-en", ifindex: "9" },
|
||||
{ name: "NSK-SERVHOST-RTK" },
|
||||
{ name: "wg-mesh" },
|
||||
{ name: "wg-server" },
|
||||
{ name: "wg-flow" },
|
||||
])
|
||||
rememberServerIfaces(enId, [
|
||||
{ name: "ether1", ifindex: "2" },
|
||||
{ name: "gre-jh", ifindex: "5" },
|
||||
])
|
||||
setRefreshIfacesForTests(async () => {})
|
||||
invalidateFlowCatalogCache()
|
||||
|
||||
await dbQuery(`
|
||||
INSERT INTO flow_daily_facts (server_id, day, iface, country, service, asn, bytes, packets)
|
||||
VALUES
|
||||
($1, '2026-09-10', '2', 'US', 'https', 15169, 800, 10),
|
||||
($1, '2026-09-10', '2', 'DE', 'dns', 15133, 200, 4),
|
||||
($1, '2026-09-10', 'wan1', 'NL', 'other', 0, 70, 1),
|
||||
($1, '2026-09-10', '0', 'US', 'https', 0, 999, 3),
|
||||
($1, '2026-09-10', 'gre-en', 'US', 'https', 15169, 400, 2),
|
||||
($1, '2026-09-10', 'NSK-SERVHOST-RTK', 'US', 'https', 15169, 300, 2),
|
||||
($1, '2026-09-10', 'wg-mesh', 'US', 'https', 0, 250, 2),
|
||||
($1, '2026-09-10', 'wg-flow', 'US', 'https', 0, 80, 1),
|
||||
($2, '2026-09-10', 'gre-jh', 'US', 'https', 15169, 500, 5),
|
||||
($2, '2026-09-10', 'ether1', 'US', 'https', 15169, 200, 2)
|
||||
`, [serverId, enId])
|
||||
|
||||
try {
|
||||
const unique = await getStatistics({ from: "2026-09-01", to: "2026-09-30", planes: "unique" })
|
||||
assert.equal(unique.grain, "day")
|
||||
assert.equal(unique.kpis.bytes, 1000)
|
||||
const uniqueAsnSum = unique.asns.reduce((s, r) => s + r.bytes, 0)
|
||||
assert.equal(uniqueAsnSum, unique.kpis.bytes, "unique KPI = SUM dest ASN")
|
||||
assert.equal(unique.kpis.users, 1)
|
||||
assert.ok(unique.countries.some((r) => r.id === "US"))
|
||||
assert.ok(unique.users.some((r) => r.id === "u-stats-1"))
|
||||
assert.equal(unique.users.find((r) => r.id === STATISTICS_UNBOUND_USER_ID), undefined)
|
||||
assert.ok(unique.servers.some((r) => r.id === String(serverId)))
|
||||
assert.ok(!unique.servers.some((r) => r.id === String(enId)), "EN-транзит не в сетевом KPI")
|
||||
const greIface = unique.interfaces.find((r) => r.label.includes("gre-client"))
|
||||
assert.ok(greIface)
|
||||
assert.equal(greIface.bytes, 1000)
|
||||
assert.equal(greIface.id, `${serverId}:gre-client`)
|
||||
assert.ok(!unique.interfaces.some((r) => /· (?:#)?\d+$/.test(r.label)))
|
||||
assert.ok(!unique.interfaces.some((r) => r.label.includes(" · —") || r.label.endsWith("· —")))
|
||||
assert.ok(!unique.interfaces.some((r) => r.label.includes("gre-en")))
|
||||
assert.ok(!unique.interfaces.some((r) => r.label.includes("NSK-SERVHOST-RTK")))
|
||||
assert.ok(!unique.interfaces.some((r) => r.label.includes("wg-mesh")))
|
||||
assert.ok(!unique.interfaces.some((r) => r.label.includes("wg-flow")))
|
||||
assert.equal(unique.interfaces.find((r) => r.id === `${serverId}:wan1`), undefined, "unique без WAN")
|
||||
|
||||
const allPlanes = await getStatistics({ from: "2026-09-01", to: "2026-09-30", planes: "all" })
|
||||
assert.equal(allPlanes.kpis.bytes, 1000, "KPI unique и all одинаковый")
|
||||
const wanRow = allPlanes.interfaces.find((r) => r.id === `${serverId}:wan1`)
|
||||
assert.ok(wanRow)
|
||||
assert.ok(wanRow.label.includes("WAN · интернет"))
|
||||
assert.equal(wanRow.bytes, 70)
|
||||
assert.equal(wanRow.percent, 0)
|
||||
const overlayGre = allPlanes.interfaces.find((r) => r.id === `${serverId}:gre-en`)
|
||||
assert.ok(overlayGre)
|
||||
assert.ok(overlayGre.label.includes("дубль"))
|
||||
assert.equal(overlayGre.percent, 0)
|
||||
const overlayCustom = allPlanes.interfaces.find((r) => r.label.includes("NSK-SERVHOST-RTK"))
|
||||
assert.ok(overlayCustom)
|
||||
assert.ok(overlayCustom.label.includes("дубль"))
|
||||
const overlayWg = allPlanes.interfaces.find((r) => r.label.includes("wg-mesh"))
|
||||
assert.ok(overlayWg)
|
||||
assert.ok(overlayWg.label.includes("дубль"))
|
||||
assert.ok(!allPlanes.interfaces.some((r) => r.label.includes("wg-flow")))
|
||||
|
||||
const wanSlice = await getStatistics({
|
||||
from: "2026-09-01",
|
||||
to: "2026-09-30",
|
||||
serverId,
|
||||
iface: "wan1",
|
||||
})
|
||||
assert.equal(wanSlice.kpis.bytes, 70)
|
||||
|
||||
const nodeSlice = await getStatistics({
|
||||
from: "2026-09-01",
|
||||
to: "2026-09-30",
|
||||
serverId,
|
||||
planes: "unique",
|
||||
})
|
||||
assert.equal(nodeSlice.kpis.bytes, 1000)
|
||||
assert.equal(nodeSlice.interfaces.find((r) => r.id === `${serverId}:wan1`), undefined)
|
||||
assert.ok(!nodeSlice.users.some((r) => r.id === STATISTICS_UNBOUND_USER_ID))
|
||||
|
||||
const nodeAll = await getStatistics({
|
||||
from: "2026-09-01",
|
||||
to: "2026-09-30",
|
||||
serverId,
|
||||
planes: "all",
|
||||
})
|
||||
assert.equal(nodeAll.kpis.bytes, 1000)
|
||||
const nodeWan = nodeAll.interfaces.find((r) => r.id === `${serverId}:wan1`)
|
||||
assert.ok(nodeWan)
|
||||
assert.equal(nodeWan.percent, 0)
|
||||
assert.ok(nodeWan.label.includes("WAN · интернет"))
|
||||
|
||||
const enSlice = await getStatistics({
|
||||
from: "2026-09-01",
|
||||
to: "2026-09-30",
|
||||
serverId: enId,
|
||||
planes: "unique",
|
||||
})
|
||||
assert.equal(enSlice.kpis.bytes, 0)
|
||||
assert.ok(!enSlice.interfaces.some((r) => r.label.includes("gre-jh")))
|
||||
assert.ok(!enSlice.interfaces.some((r) => r.label.includes("WAN · интернет")))
|
||||
|
||||
const enAll = await getStatistics({
|
||||
from: "2026-09-01",
|
||||
to: "2026-09-30",
|
||||
serverId: enId,
|
||||
planes: "all",
|
||||
})
|
||||
assert.equal(enAll.kpis.bytes, 0)
|
||||
assert.ok(enAll.interfaces.some((r) => r.label.includes("WAN · интернет") && r.label.includes("ether1") && r.percent === 0))
|
||||
assert.ok(enAll.interfaces.some((r) => r.label.includes("gre-jh") && r.label.includes("дубль")))
|
||||
|
||||
const sliced = await getStatistics({
|
||||
from: "2026-09-01",
|
||||
to: "2026-09-30",
|
||||
country: "US",
|
||||
service: "https",
|
||||
asn: 15169,
|
||||
})
|
||||
assert.equal(sliced.kpis.bytes, 800)
|
||||
assert.equal(sliced.countries.length, 1)
|
||||
assert.equal(sliced.countries[0]?.id, "US")
|
||||
assert.ok(sliced.users.some((r) => r.id === "u-stats-1"))
|
||||
|
||||
const byUser = await getStatistics({
|
||||
from: "2026-09-01",
|
||||
to: "2026-09-30",
|
||||
userId: "u-stats-1",
|
||||
})
|
||||
assert.equal(byUser.kpis.bytes, 1000)
|
||||
|
||||
const pivot = await getStatisticsPivot({
|
||||
from: "2026-09-01",
|
||||
to: "2026-09-30",
|
||||
row: "country",
|
||||
col: "service",
|
||||
metric: "bytes",
|
||||
})
|
||||
const us = pivot.rows.find((r) => r.id === "US")
|
||||
const de = pivot.rows.find((r) => r.id === "DE")
|
||||
assert.ok(us)
|
||||
assert.ok(de)
|
||||
assert.equal(us.cells.https, 800)
|
||||
assert.equal(de.cells.dns, 200)
|
||||
|
||||
await dbQuery(`
|
||||
INSERT INTO user_interface_bindings (id, user_id, server_id, interface_name, interface_type)
|
||||
VALUES ('bind-stats-wg', 'u-stats-1', $1, 'wg-server', 'wg')
|
||||
`, [serverId])
|
||||
await dbQuery(`
|
||||
INSERT INTO flow_daily_facts (server_id, day, iface, country, service, asn, bytes, packets)
|
||||
VALUES ($1, '2026-09-10', 'wg-server', 'US', 'https', 15169, 150, 2)
|
||||
`, [serverId])
|
||||
invalidateFlowCatalogCache()
|
||||
|
||||
const withWg = await getStatistics({ from: "2026-09-01", to: "2026-09-30", planes: "unique" })
|
||||
assert.equal(withWg.kpis.bytes, 1150)
|
||||
assert.ok(withWg.interfaces.some((r) => r.label.includes("wg-server") && r.bytes === 150))
|
||||
assert.ok(!withWg.interfaces.some((r) => r.label.includes("wg-flow")))
|
||||
assert.ok(!withWg.interfaces.some((r) => r.label.includes("wg-mesh")))
|
||||
|
||||
await dbQuery(`
|
||||
INSERT INTO flow_hour_facts (server_id, bucket_at, iface, country, service, asn, bytes, packets)
|
||||
VALUES ($1, '2026-09-10T10:00:00Z', '2', 'US', 'https', 15169, 40, 2)
|
||||
`, [serverId])
|
||||
const hourly = await getStatistics({
|
||||
from: "2026-09-10T00:00:00.000Z",
|
||||
to: "2026-09-10T23:00:00.000Z",
|
||||
})
|
||||
assert.equal(hourly.grain, "hour")
|
||||
assert.equal(hourly.kpis.bytes, 40)
|
||||
assert.ok(hourly.users.some((r) => r.id === "u-stats-1"))
|
||||
} finally {
|
||||
setRefreshIfacesForTests(null)
|
||||
resetIfaceCacheForTests()
|
||||
invalidateFlowCatalogCache()
|
||||
await dbQuery(`DELETE FROM flow_daily_facts WHERE server_id IN ($1, $2)`, [serverId, enId])
|
||||
await dbQuery(`DELETE FROM flow_hour_facts WHERE server_id IN ($1, $2)`, [serverId, enId])
|
||||
await dbQuery(`DELETE FROM user_interface_bindings WHERE server_id IN ($1, $2)`, [serverId, enId])
|
||||
await dbQuery(`DELETE FROM server_snapshots WHERE server_id IN ($1, $2)`, [serverId, enId])
|
||||
await dbQuery(`DELETE FROM servers WHERE id IN ($1, $2)`, [serverId, enId])
|
||||
}
|
||||
|
||||
console.log("statistics-aggregate.test.ts: ok")
|
||||
@@ -0,0 +1,885 @@
|
||||
import { eq } from "drizzle-orm"
|
||||
import { db, dbAll } from "../db/index.js"
|
||||
import { appUsers, flowAsnMeta, servers, userInterfaceBindings } from "../db/schema.js"
|
||||
import {
|
||||
STATISTICS_UNBOUND_USER_ID,
|
||||
type StatisticsBreakdownRow,
|
||||
type StatisticsDto,
|
||||
type StatisticsPivotDim,
|
||||
type StatisticsPivotDto,
|
||||
type StatisticsPivotQuery,
|
||||
type StatisticsQuery,
|
||||
} from "@mmapp/contracts/statistics"
|
||||
import {
|
||||
collapseServerIfaceRows,
|
||||
displayFactIface,
|
||||
expandBindingIfaces,
|
||||
factIfaceAliases,
|
||||
listCachedIfaceNames,
|
||||
} from "./traffic-flow-ifindex.js"
|
||||
import { refreshServerIfaces } from "./traffic-flow-ifaces.js"
|
||||
import {
|
||||
isDashDisplayIface,
|
||||
isJunkFactIface,
|
||||
isOverlayTunnelIface,
|
||||
isWanFactIface,
|
||||
overlayDupLabel,
|
||||
wanIfaceLabel,
|
||||
} from "./traffic-flow-facts-filter.js"
|
||||
import { getServerCatalog, loadFlowTopology, type FlowTopology } from "./traffic-flow-topology.js"
|
||||
|
||||
const TOP_N = 200
|
||||
const HOUR_WINDOW_MS = 48 * 3600_000
|
||||
const PIVOT_ROW_CAP = 50
|
||||
const PIVOT_COL_CAP = 15
|
||||
const PIVOT_OTHER_ID = "__other__"
|
||||
|
||||
export interface ParsedPeriod {
|
||||
fromIso: string
|
||||
toIso: string
|
||||
fromDay: string
|
||||
toDayExclusive: string
|
||||
grain: "hour" | "day"
|
||||
windowSec: number
|
||||
}
|
||||
|
||||
function pad2(n: number): string {
|
||||
return String(n).padStart(2, "0")
|
||||
}
|
||||
|
||||
function toUtcDay(d: Date): string {
|
||||
return `${d.getUTCFullYear()}-${pad2(d.getUTCMonth() + 1)}-${pad2(d.getUTCDate())}`
|
||||
}
|
||||
|
||||
function addUtcDays(day: string, n: number): string {
|
||||
const d = new Date(`${day}T00:00:00Z`)
|
||||
d.setUTCDate(d.getUTCDate() + n)
|
||||
return toUtcDay(d)
|
||||
}
|
||||
|
||||
/** Parse from/to. Date-only `to` is inclusive (end of that UTC day). */
|
||||
export function parseStatisticsPeriod(fromRaw: string, toRaw: string): ParsedPeriod | null {
|
||||
const from = Date.parse(fromRaw.includes("T") ? fromRaw : `${fromRaw}T00:00:00Z`)
|
||||
const toHasTime = toRaw.includes("T")
|
||||
const to = Date.parse(toHasTime ? toRaw : `${toRaw}T00:00:00Z`)
|
||||
if (!Number.isFinite(from) || !Number.isFinite(to)) return null
|
||||
const fromDate = new Date(from)
|
||||
let toDate = new Date(to)
|
||||
let toDayExclusive: string
|
||||
if (toHasTime) {
|
||||
toDayExclusive = toUtcDay(toDate)
|
||||
if (toDate.getUTCHours() !== 0 || toDate.getUTCMinutes() !== 0 || toDate.getUTCSeconds() !== 0) {
|
||||
toDayExclusive = addUtcDays(toDayExclusive, 1)
|
||||
}
|
||||
} else {
|
||||
toDayExclusive = addUtcDays(toUtcDay(toDate), 1)
|
||||
toDate = new Date(`${toDayExclusive}T00:00:00Z`)
|
||||
}
|
||||
if (toDate.getTime() <= from) return null
|
||||
const windowSec = Math.max(1, Math.round((toDate.getTime() - from) / 1000))
|
||||
const grain: "hour" | "day" = toDate.getTime() - from <= HOUR_WINDOW_MS ? "hour" : "day"
|
||||
return {
|
||||
fromIso: fromDate.toISOString(),
|
||||
toIso: toDate.toISOString(),
|
||||
fromDay: toUtcDay(fromDate),
|
||||
toDayExclusive,
|
||||
grain,
|
||||
windowSec,
|
||||
}
|
||||
}
|
||||
|
||||
type FactScope = "unique" | "wan" | "overlay"
|
||||
|
||||
interface FilterCtx {
|
||||
fromIso: string
|
||||
toIso: string
|
||||
fromDay: string
|
||||
toDayExclusive: string
|
||||
serverId?: number
|
||||
iface?: string
|
||||
country?: string
|
||||
service?: string
|
||||
asn?: number
|
||||
planes: "unique" | "all"
|
||||
userIfaces: Array<{ serverId: number; iface: string }> | null
|
||||
unboundOnly: boolean
|
||||
boundIfaces: Array<{ serverId: number; iface: string }>
|
||||
overlayIfaces: Array<{ serverId: number; iface: string }>
|
||||
wanIfaces: Array<{ serverId: number; iface: string }>
|
||||
excludeServerIds: number[]
|
||||
topo: FlowTopology | null
|
||||
}
|
||||
|
||||
function ifaceFilterAliases(iface: string, serverId?: number): string[] {
|
||||
return factIfaceAliases(iface.trim(), serverId)
|
||||
}
|
||||
|
||||
function looksLikeIfIndex(iface: string): boolean {
|
||||
const raw = iface.trim()
|
||||
return /^\d+$/.test(raw) || /^#\d+$/.test(raw)
|
||||
}
|
||||
|
||||
async function warmIfaceCache(ids: Iterable<number>): Promise<void> {
|
||||
const uniq = [...new Set(ids)].filter((id) => Number.isFinite(id) && id > 0)
|
||||
if (!uniq.length) return
|
||||
await Promise.all(uniq.map((id) => refreshServerIfaces(id)))
|
||||
}
|
||||
|
||||
async function warmBindingIfaceCache(): Promise<void> {
|
||||
const rows = await db.select({ serverId: userInterfaceBindings.serverId }).from(userInterfaceBindings)
|
||||
await warmIfaceCache(rows.map((r) => r.serverId))
|
||||
}
|
||||
|
||||
function canonicalIfaceDimId(id: string): string {
|
||||
const colon = id.indexOf(":")
|
||||
if (colon < 0) return id
|
||||
const sid = Number(id.slice(0, colon))
|
||||
if (!Number.isFinite(sid)) return id
|
||||
return `${sid}:${displayFactIface(sid, id.slice(colon + 1))}`
|
||||
}
|
||||
|
||||
function pushIfaceTuples(
|
||||
parts: string[],
|
||||
params: unknown[],
|
||||
alias: string,
|
||||
tuples: Array<{ serverId: number; iface: string }>,
|
||||
op: "IN" | "NOT IN",
|
||||
): void {
|
||||
if (!tuples.length) {
|
||||
if (op === "IN") parts.push("FALSE")
|
||||
return
|
||||
}
|
||||
const sql = tuples.map(() => "(?, ?)").join(", ")
|
||||
parts.push(`(${alias}.server_id, ${alias}.iface) ${op} (${sql})`)
|
||||
for (const t of tuples) {
|
||||
params.push(t.serverId, t.iface)
|
||||
}
|
||||
}
|
||||
|
||||
function factWhere(
|
||||
alias: string,
|
||||
grain: "hour" | "day",
|
||||
ctx: FilterCtx,
|
||||
scope: FactScope = "unique",
|
||||
): { sql: string; params: unknown[] } {
|
||||
const params: unknown[] = []
|
||||
const parts: string[] = []
|
||||
if (grain === "hour") {
|
||||
params.push(ctx.fromIso, ctx.toIso)
|
||||
parts.push(`${alias}.bucket_at >= ? AND ${alias}.bucket_at < ?`)
|
||||
} else {
|
||||
params.push(ctx.fromDay, ctx.toDayExclusive)
|
||||
parts.push(`${alias}.day >= ? AND ${alias}.day < ?`)
|
||||
}
|
||||
if (ctx.serverId != null) {
|
||||
parts.push(`${alias}.server_id = ?`)
|
||||
params.push(ctx.serverId)
|
||||
}
|
||||
if (ctx.country) {
|
||||
parts.push(`${alias}.country = ?`)
|
||||
params.push(ctx.country.toUpperCase())
|
||||
}
|
||||
if (ctx.service) {
|
||||
parts.push(`${alias}.service = ?`)
|
||||
params.push(ctx.service)
|
||||
}
|
||||
if (ctx.asn != null) {
|
||||
parts.push(`${alias}.asn = ?`)
|
||||
params.push(ctx.asn)
|
||||
}
|
||||
parts.push(`${alias}.iface NOT IN ('0', '—', '__unknown__', 'wg-flow', '')`)
|
||||
|
||||
if (scope === "wan") {
|
||||
pushIfaceTuples(parts, params, alias, ctx.wanIfaces, "IN")
|
||||
return { sql: parts.join(" AND "), params }
|
||||
}
|
||||
if (scope === "overlay") {
|
||||
pushIfaceTuples(parts, params, alias, ctx.overlayIfaces, "IN")
|
||||
return { sql: parts.join(" AND "), params }
|
||||
}
|
||||
|
||||
if (ctx.iface) {
|
||||
const aliases = ifaceFilterAliases(ctx.iface, ctx.serverId)
|
||||
if (aliases.length <= 1) {
|
||||
parts.push(`${alias}.iface = ?`)
|
||||
params.push(aliases[0] ?? ctx.iface)
|
||||
} else {
|
||||
parts.push(`${alias}.iface IN (${aliases.map(() => "?").join(", ")})`)
|
||||
params.push(...aliases)
|
||||
}
|
||||
return { sql: parts.join(" AND "), params }
|
||||
}
|
||||
if (ctx.userIfaces) {
|
||||
pushIfaceTuples(parts, params, alias, ctx.userIfaces, "IN")
|
||||
return { sql: parts.join(" AND "), params }
|
||||
}
|
||||
if (ctx.unboundOnly) {
|
||||
parts.push("FALSE")
|
||||
return { sql: parts.join(" AND "), params }
|
||||
}
|
||||
pushIfaceTuples(parts, params, alias, ctx.boundIfaces, "IN")
|
||||
if (ctx.excludeServerIds.length) {
|
||||
parts.push(`${alias}.server_id NOT IN (${ctx.excludeServerIds.map(() => "?").join(", ")})`)
|
||||
params.push(...ctx.excludeServerIds)
|
||||
}
|
||||
return { sql: parts.join(" AND "), params }
|
||||
}
|
||||
|
||||
function emptyDto(period: ParsedPeriod): StatisticsDto {
|
||||
return {
|
||||
from: period.fromIso,
|
||||
to: period.toIso,
|
||||
grain: period.grain,
|
||||
kpis: {
|
||||
bytes: 0,
|
||||
packets: 0,
|
||||
avgBps: 0,
|
||||
users: 0,
|
||||
servers: 0,
|
||||
ifaces: 0,
|
||||
topCountry: "—",
|
||||
topService: "—",
|
||||
},
|
||||
series: [],
|
||||
users: [],
|
||||
servers: [],
|
||||
interfaces: [],
|
||||
countries: [],
|
||||
services: [],
|
||||
asns: [],
|
||||
}
|
||||
}
|
||||
|
||||
function toBreakdown(
|
||||
rows: Array<{ id: string; label: string; bytes: number; packets: number }>,
|
||||
totalBytes: number,
|
||||
windowSec: number,
|
||||
): StatisticsBreakdownRow[] {
|
||||
const denom = totalBytes || 1
|
||||
return rows
|
||||
.sort((a, b) => b.bytes - a.bytes)
|
||||
.slice(0, TOP_N)
|
||||
.map((r) => ({
|
||||
id: r.id,
|
||||
label: r.label,
|
||||
bytes: r.bytes,
|
||||
packets: r.packets,
|
||||
bps: (r.bytes * 8) / windowSec,
|
||||
percent: (r.bytes / denom) * 100,
|
||||
}))
|
||||
}
|
||||
|
||||
interface UserBindTuple {
|
||||
userId: string
|
||||
serverId: number
|
||||
iface: string
|
||||
}
|
||||
|
||||
async function loadBindUserTuples(): Promise<UserBindTuple[]> {
|
||||
const binds = await db.select().from(userInterfaceBindings)
|
||||
const seen = new Set<string>()
|
||||
const out: UserBindTuple[] = []
|
||||
for (const b of binds) {
|
||||
for (const iface of factIfaceAliases(b.interfaceName, b.serverId)) {
|
||||
const k = `${b.userId}\0${b.serverId}\0${iface}`
|
||||
if (seen.has(k)) continue
|
||||
seen.add(k)
|
||||
out.push({ userId: b.userId, serverId: b.serverId, iface })
|
||||
}
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
function uniqueBoundIfaces(tuples: UserBindTuple[]): Array<{ serverId: number; iface: string }> {
|
||||
const seen = new Set<string>()
|
||||
const out: Array<{ serverId: number; iface: string }> = []
|
||||
for (const t of tuples) {
|
||||
const k = `${t.serverId}\0${t.iface}`
|
||||
if (seen.has(k)) continue
|
||||
seen.add(k)
|
||||
out.push({ serverId: t.serverId, iface: t.iface })
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
async function resolveUserIfaces(userId?: string): Promise<Array<{ serverId: number; iface: string }> | null> {
|
||||
if (!userId || userId === STATISTICS_UNBOUND_USER_ID) return null
|
||||
const binds = await db.select().from(userInterfaceBindings).where(eq(userInterfaceBindings.userId, userId))
|
||||
return expandBindingIfaces(binds.map((b) => ({ serverId: b.serverId, iface: b.interfaceName })))
|
||||
}
|
||||
|
||||
function userBindJoinSql(tuples: UserBindTuple[]): { sql: string; params: unknown[] } {
|
||||
const values = tuples.map(() => "(?::text, ?::int, ?::text)").join(", ")
|
||||
const params = tuples.flatMap((t) => [t.userId, t.serverId, t.iface])
|
||||
return {
|
||||
sql: `JOIN (VALUES ${values}) AS b(user_id, server_id, iface) ON b.server_id = f.server_id AND b.iface = f.iface`,
|
||||
params,
|
||||
}
|
||||
}
|
||||
|
||||
function expandIfaceTuples(
|
||||
items: Array<{ serverId: number; iface: string }>,
|
||||
): Array<{ serverId: number; iface: string }> {
|
||||
const seen = new Set<string>()
|
||||
const out: Array<{ serverId: number; iface: string }> = []
|
||||
for (const t of items) {
|
||||
for (const iface of factIfaceAliases(t.iface, t.serverId)) {
|
||||
const k = `${t.serverId}\0${iface}`
|
||||
if (seen.has(k)) continue
|
||||
seen.add(k)
|
||||
out.push({ serverId: t.serverId, iface })
|
||||
}
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
async function loadPayloadScope(serverId?: number): Promise<{
|
||||
overlayIfaces: Array<{ serverId: number; iface: string }>
|
||||
wanIfaces: Array<{ serverId: number; iface: string }>
|
||||
excludeServerIds: number[]
|
||||
topo: FlowTopology
|
||||
}> {
|
||||
const topo = await loadFlowTopology()
|
||||
const catalog = await getServerCatalog()
|
||||
await warmIfaceCache(catalog.list.map((s) => s.id))
|
||||
const overlayRaw: Array<{ serverId: number; iface: string }> = []
|
||||
const wanRaw: Array<{ serverId: number; iface: string }> = []
|
||||
for (const s of catalog.list) {
|
||||
if (serverId != null && s.id !== serverId) continue
|
||||
const wanSet = topo.wanIfaces.get(s.id)
|
||||
const wanNames = wanSet && wanSet.size > 0
|
||||
? [...wanSet]
|
||||
: s.type === "home-router" ? [] : ["ether1"]
|
||||
for (const name of wanNames) wanRaw.push({ serverId: s.id, iface: name })
|
||||
const names = new Set(listCachedIfaceNames(s.id))
|
||||
for (const name of topo.tunnelIfaces?.get(s.id) ?? []) names.add(name)
|
||||
for (const name of names) {
|
||||
if (isOverlayTunnelIface(topo, s.id, name)) overlayRaw.push({ serverId: s.id, iface: name })
|
||||
}
|
||||
}
|
||||
return {
|
||||
overlayIfaces: expandIfaceTuples(overlayRaw),
|
||||
wanIfaces: expandIfaceTuples(wanRaw),
|
||||
excludeServerIds: serverId != null
|
||||
? []
|
||||
: catalog.list.filter((s) => s.type === "exit-node").map((s) => s.id),
|
||||
topo,
|
||||
}
|
||||
}
|
||||
|
||||
async function buildFilterCtx(query: StatisticsQuery, period: ParsedPeriod): Promise<FilterCtx | null> {
|
||||
const bindTuples = await loadBindUserTuples()
|
||||
const boundIfaces = uniqueBoundIfaces(bindTuples)
|
||||
const unboundOnly = query.userId === STATISTICS_UNBOUND_USER_ID
|
||||
const userIfaces = unboundOnly ? null : await resolveUserIfaces(query.userId)
|
||||
if (userIfaces && userIfaces.length === 0) return null
|
||||
if (unboundOnly) return null
|
||||
const scope = await loadPayloadScope(query.serverId)
|
||||
return {
|
||||
...period,
|
||||
serverId: query.serverId,
|
||||
iface: query.iface,
|
||||
country: query.country,
|
||||
service: query.service,
|
||||
asn: query.asn,
|
||||
planes: query.planes ?? "unique",
|
||||
userIfaces,
|
||||
unboundOnly,
|
||||
boundIfaces,
|
||||
overlayIfaces: scope.overlayIfaces,
|
||||
wanIfaces: scope.wanIfaces,
|
||||
excludeServerIds: scope.excludeServerIds,
|
||||
topo: scope.topo,
|
||||
}
|
||||
}
|
||||
|
||||
export async function getStatistics(query: StatisticsQuery): Promise<StatisticsDto> {
|
||||
const period = parseStatisticsPeriod(query.from, query.to)
|
||||
if (!period) return emptyDto({
|
||||
fromIso: query.from,
|
||||
toIso: query.to,
|
||||
fromDay: query.from.slice(0, 10),
|
||||
toDayExclusive: query.to.slice(0, 10),
|
||||
grain: "day",
|
||||
windowSec: 1,
|
||||
})
|
||||
|
||||
await warmBindingIfaceCache()
|
||||
if (query.serverId) await warmIfaceCache([query.serverId])
|
||||
const bindTuples = await loadBindUserTuples()
|
||||
const ctx = await buildFilterCtx(query, period)
|
||||
if (!ctx) return emptyDto(period)
|
||||
|
||||
const table = period.grain === "hour" ? "flow_hour_facts" : "flow_daily_facts"
|
||||
const timeCol = period.grain === "hour" ? "bucket_at" : "day"
|
||||
const where = factWhere("f", period.grain, ctx)
|
||||
|
||||
const totals = await dbAll<{ bytes: number; packets: number; servers: number }>(`
|
||||
SELECT
|
||||
COALESCE(SUM(f.bytes), 0) AS bytes,
|
||||
COALESCE(SUM(f.packets), 0) AS packets,
|
||||
COUNT(DISTINCT f.server_id)::int AS servers
|
||||
FROM ${table} f
|
||||
WHERE ${where.sql}
|
||||
`, where.params)
|
||||
|
||||
const bytes = Number(totals[0]?.bytes) || 0
|
||||
const packets = Number(totals[0]?.packets) || 0
|
||||
const serverCount = Number(totals[0]?.servers) || 0
|
||||
|
||||
const seriesRows = await dbAll<{ t: string; bytes: number }>(`
|
||||
SELECT ${timeCol}::text AS t, SUM(f.bytes) AS bytes
|
||||
FROM ${table} f
|
||||
WHERE ${where.sql}
|
||||
GROUP BY ${timeCol}
|
||||
ORDER BY ${timeCol}
|
||||
`, where.params)
|
||||
|
||||
const countryRows = await dbAll<{ id: string; bytes: number; packets: number }>(`
|
||||
SELECT f.country AS id, SUM(f.bytes) AS bytes, SUM(f.packets) AS packets
|
||||
FROM ${table} f
|
||||
WHERE ${where.sql}
|
||||
GROUP BY f.country
|
||||
`, where.params)
|
||||
|
||||
const serviceRows = await dbAll<{ id: string; bytes: number; packets: number }>(`
|
||||
SELECT f.service AS id, SUM(f.bytes) AS bytes, SUM(f.packets) AS packets
|
||||
FROM ${table} f
|
||||
WHERE ${where.sql}
|
||||
GROUP BY f.service
|
||||
`, where.params)
|
||||
|
||||
const asnRows = await dbAll<{ id: number; bytes: number; packets: number }>(`
|
||||
SELECT f.asn AS id, SUM(f.bytes) AS bytes, SUM(f.packets) AS packets
|
||||
FROM ${table} f
|
||||
WHERE ${where.sql}
|
||||
GROUP BY f.asn
|
||||
`, where.params)
|
||||
|
||||
const serverRows = await dbAll<{ id: number; bytes: number; packets: number }>(`
|
||||
SELECT f.server_id AS id, SUM(f.bytes) AS bytes, SUM(f.packets) AS packets
|
||||
FROM ${table} f
|
||||
WHERE ${where.sql}
|
||||
GROUP BY f.server_id
|
||||
`, where.params)
|
||||
|
||||
const ifaceRowsRaw = await dbAll<{ serverId: number; iface: string; bytes: number; packets: number }>(`
|
||||
SELECT f.server_id AS "serverId", f.iface AS iface, SUM(f.bytes) AS bytes, SUM(f.packets) AS packets
|
||||
FROM ${table} f
|
||||
WHERE ${where.sql}
|
||||
GROUP BY f.server_id, f.iface
|
||||
`, where.params)
|
||||
await warmIfaceCache(ifaceRowsRaw.filter((r) => looksLikeIfIndex(r.iface)).map((r) => r.serverId))
|
||||
const ifaceRows = collapseServerIfaceRows(ifaceRowsRaw).filter((r) => {
|
||||
if (isJunkFactIface(r.iface) || isDashDisplayIface(r.iface)) return false
|
||||
if (ctx.iface) return true
|
||||
if (ctx.topo && isOverlayTunnelIface(ctx.topo, r.serverId, r.iface)) return false
|
||||
if (ctx.topo && isWanFactIface(ctx.topo, r.serverId, r.iface)) return false
|
||||
return true
|
||||
})
|
||||
const ifaceCount = ifaceRows.length
|
||||
|
||||
let dupeIfaceRows: Array<{ serverId: number; iface: string; bytes: number; packets: number; kind: "wan" | "overlay" }> = []
|
||||
if (ctx.planes === "all" && !ctx.iface) {
|
||||
const wanWhere = factWhere("f", period.grain, ctx, "wan")
|
||||
const overlayWhere = factWhere("f", period.grain, ctx, "overlay")
|
||||
const [wanRaw, overlayRaw] = await Promise.all([
|
||||
dbAll<{ serverId: number; iface: string; bytes: number; packets: number }>(`
|
||||
SELECT f.server_id AS "serverId", f.iface AS iface, SUM(f.bytes) AS bytes, SUM(f.packets) AS packets
|
||||
FROM ${table} f
|
||||
WHERE ${wanWhere.sql}
|
||||
GROUP BY f.server_id, f.iface
|
||||
`, wanWhere.params),
|
||||
dbAll<{ serverId: number; iface: string; bytes: number; packets: number }>(`
|
||||
SELECT f.server_id AS "serverId", f.iface AS iface, SUM(f.bytes) AS bytes, SUM(f.packets) AS packets
|
||||
FROM ${table} f
|
||||
WHERE ${overlayWhere.sql}
|
||||
GROUP BY f.server_id, f.iface
|
||||
`, overlayWhere.params),
|
||||
])
|
||||
await warmIfaceCache([
|
||||
...wanRaw.filter((r) => looksLikeIfIndex(r.iface)).map((r) => r.serverId),
|
||||
...overlayRaw.filter((r) => looksLikeIfIndex(r.iface)).map((r) => r.serverId),
|
||||
])
|
||||
const seen = new Set(ifaceRows.map((r) => `${r.serverId}:${r.iface}`))
|
||||
for (const r of collapseServerIfaceRows(wanRaw)) {
|
||||
if (isJunkFactIface(r.iface) || isDashDisplayIface(r.iface)) continue
|
||||
const key = `${r.serverId}:${r.iface}`
|
||||
if (seen.has(key)) continue
|
||||
seen.add(key)
|
||||
dupeIfaceRows.push({ ...r, kind: "wan" })
|
||||
}
|
||||
for (const r of collapseServerIfaceRows(overlayRaw)) {
|
||||
if (isJunkFactIface(r.iface) || isDashDisplayIface(r.iface)) continue
|
||||
const key = `${r.serverId}:${r.iface}`
|
||||
if (seen.has(key)) continue
|
||||
seen.add(key)
|
||||
dupeIfaceRows.push({ ...r, kind: "overlay" })
|
||||
}
|
||||
}
|
||||
|
||||
let userRows: Array<{ id: string; bytes: number; packets: number }> = []
|
||||
if (bindTuples.length && !ctx.unboundOnly) {
|
||||
const join = userBindJoinSql(bindTuples)
|
||||
userRows = await dbAll<{ id: string; bytes: number; packets: number }>(`
|
||||
SELECT b.user_id AS id, SUM(f.bytes) AS bytes, SUM(f.packets) AS packets
|
||||
FROM ${table} f
|
||||
${join.sql}
|
||||
WHERE ${where.sql}
|
||||
GROUP BY b.user_id
|
||||
`, [...join.params, ...where.params])
|
||||
}
|
||||
|
||||
const serverNames = new Map<number, string>()
|
||||
const allServers = await db.select({ id: servers.id, name: servers.name, host: servers.host }).from(servers)
|
||||
for (const s of allServers) serverNames.set(s.id, s.name || s.host)
|
||||
|
||||
const userNames = new Map<string, string>()
|
||||
const allUsers = await db.select({ id: appUsers.id, name: appUsers.name, login: appUsers.login }).from(appUsers)
|
||||
for (const u of allUsers) userNames.set(u.id, u.name || u.login)
|
||||
|
||||
const asnHolders = new Map<number, string>()
|
||||
const asnMeta = await db.select({ asn: flowAsnMeta.asn, holder: flowAsnMeta.holder }).from(flowAsnMeta)
|
||||
for (const a of asnMeta) asnHolders.set(a.asn, a.holder)
|
||||
|
||||
const countries = toBreakdown(
|
||||
countryRows.map((r) => ({
|
||||
id: r.id,
|
||||
label: r.id === "XX" ? "Неизвестно" : r.id,
|
||||
bytes: Number(r.bytes) || 0,
|
||||
packets: Number(r.packets) || 0,
|
||||
})),
|
||||
bytes,
|
||||
period.windowSec,
|
||||
)
|
||||
const services = toBreakdown(
|
||||
serviceRows.map((r) => ({
|
||||
id: r.id,
|
||||
label: r.id,
|
||||
bytes: Number(r.bytes) || 0,
|
||||
packets: Number(r.packets) || 0,
|
||||
})),
|
||||
bytes,
|
||||
period.windowSec,
|
||||
)
|
||||
const asns = toBreakdown(
|
||||
asnRows.map((r) => {
|
||||
const id = Number(r.id) || 0
|
||||
const holder = asnHolders.get(id)
|
||||
return {
|
||||
id: String(id),
|
||||
label: id === 0 ? "other" : holder ? `AS${id} · ${holder}` : `AS${id}`,
|
||||
bytes: Number(r.bytes) || 0,
|
||||
packets: Number(r.packets) || 0,
|
||||
}
|
||||
}),
|
||||
bytes,
|
||||
period.windowSec,
|
||||
)
|
||||
const serverBreakdown = toBreakdown(
|
||||
serverRows.map((r) => ({
|
||||
id: String(r.id),
|
||||
label: serverNames.get(r.id) || String(r.id),
|
||||
bytes: Number(r.bytes) || 0,
|
||||
packets: Number(r.packets) || 0,
|
||||
})),
|
||||
bytes,
|
||||
period.windowSec,
|
||||
)
|
||||
const uniqueInterfaces = toBreakdown(
|
||||
ifaceRows.map((r) => {
|
||||
const serverName = serverNames.get(r.serverId) || String(r.serverId)
|
||||
const wan = ctx.topo ? isWanFactIface(ctx.topo, r.serverId, r.iface) : false
|
||||
return {
|
||||
id: `${r.serverId}:${r.iface}`,
|
||||
label: wan ? wanIfaceLabel(serverName, r.iface) : `${serverName} · ${r.iface}`,
|
||||
bytes: Number(r.bytes) || 0,
|
||||
packets: Number(r.packets) || 0,
|
||||
}
|
||||
}),
|
||||
bytes,
|
||||
period.windowSec,
|
||||
)
|
||||
const dupeInterfaces: StatisticsBreakdownRow[] = dupeIfaceRows.map((r) => {
|
||||
const serverName = serverNames.get(r.serverId) || String(r.serverId)
|
||||
const rowBytes = Number(r.bytes) || 0
|
||||
const rowPackets = Number(r.packets) || 0
|
||||
return {
|
||||
id: `${r.serverId}:${r.iface}`,
|
||||
label: r.kind === "wan" ? wanIfaceLabel(serverName, r.iface) : overlayDupLabel(serverName, r.iface),
|
||||
bytes: rowBytes,
|
||||
packets: rowPackets,
|
||||
bps: (rowBytes * 8) / period.windowSec,
|
||||
percent: 0,
|
||||
}
|
||||
})
|
||||
const interfaces = [...uniqueInterfaces, ...dupeInterfaces]
|
||||
const matchedUsers = toBreakdown(
|
||||
userRows.map((r) => ({
|
||||
id: r.id,
|
||||
label: userNames.get(r.id) || r.id,
|
||||
bytes: Number(r.bytes) || 0,
|
||||
packets: Number(r.packets) || 0,
|
||||
})),
|
||||
bytes,
|
||||
period.windowSec,
|
||||
)
|
||||
|
||||
const users = [...matchedUsers]
|
||||
|
||||
return {
|
||||
from: period.fromIso,
|
||||
to: period.toIso,
|
||||
grain: period.grain,
|
||||
kpis: {
|
||||
bytes,
|
||||
packets,
|
||||
avgBps: (bytes * 8) / period.windowSec,
|
||||
users: matchedUsers.length,
|
||||
servers: serverCount,
|
||||
ifaces: ifaceCount,
|
||||
topCountry: countries[0]?.label || "—",
|
||||
topService: services[0]?.label || "—",
|
||||
},
|
||||
series: seriesRows.map((r) => ({ t: r.t, bytes: Number(r.bytes) || 0 })),
|
||||
users,
|
||||
servers: serverBreakdown,
|
||||
interfaces,
|
||||
countries,
|
||||
services,
|
||||
asns,
|
||||
}
|
||||
}
|
||||
|
||||
function dimSql(dim: StatisticsPivotDim, factAlias: string, bindAlias: string): string {
|
||||
if (dim === "country") return `${factAlias}.country`
|
||||
if (dim === "service") return `${factAlias}.service`
|
||||
if (dim === "asn") return `${factAlias}.asn::text`
|
||||
if (dim === "server") return `${factAlias}.server_id::text`
|
||||
if (dim === "iface") return `(${factAlias}.server_id::text || ':' || ${factAlias}.iface)`
|
||||
return `${bindAlias}.user_id`
|
||||
}
|
||||
|
||||
function emptyPivot(query: StatisticsPivotQuery): StatisticsPivotDto {
|
||||
return {
|
||||
rowDim: query.row,
|
||||
colDim: query.col,
|
||||
metric: query.metric,
|
||||
columns: [],
|
||||
rows: [],
|
||||
otherBytes: 0,
|
||||
}
|
||||
}
|
||||
|
||||
export function pivotDimsConflict(row: StatisticsPivotDim, col: StatisticsPivotDim): boolean {
|
||||
return row === col
|
||||
}
|
||||
|
||||
export async function getStatisticsPivot(query: StatisticsPivotQuery): Promise<StatisticsPivotDto> {
|
||||
if (pivotDimsConflict(query.row, query.col)) return emptyPivot(query)
|
||||
const period = parseStatisticsPeriod(query.from, query.to)
|
||||
if (!period) return emptyPivot(query)
|
||||
await warmBindingIfaceCache()
|
||||
if (query.serverId) await warmIfaceCache([query.serverId])
|
||||
const bindTuples = await loadBindUserTuples()
|
||||
const ctx = await buildFilterCtx(query, period)
|
||||
if (!ctx) return emptyPivot(query)
|
||||
const needsUser = query.row === "user" || query.col === "user"
|
||||
if (needsUser && bindTuples.length === 0) return emptyPivot(query)
|
||||
|
||||
const table = period.grain === "hour" ? "flow_hour_facts" : "flow_daily_facts"
|
||||
const where = factWhere("f", period.grain, ctx)
|
||||
const rowExpr = dimSql(query.row, "f", "b")
|
||||
const colExpr = dimSql(query.col, "f", "b")
|
||||
const join = needsUser ? userBindJoinSql(bindTuples) : { sql: "", params: [] as unknown[] }
|
||||
|
||||
const raw = await dbAll<{ row_id: string; col_id: string; bytes: number; packets: number }>(`
|
||||
SELECT ${rowExpr} AS row_id, ${colExpr} AS col_id,
|
||||
SUM(f.bytes) AS bytes, SUM(f.packets) AS packets
|
||||
FROM ${table} f
|
||||
${join.sql}
|
||||
WHERE ${where.sql}
|
||||
GROUP BY 1, 2
|
||||
`, [...join.params, ...where.params])
|
||||
|
||||
if (query.row === "iface" || query.col === "iface") {
|
||||
const ifaceServerIds: number[] = []
|
||||
for (const r of raw) {
|
||||
for (const dim of [query.row, query.col] as const) {
|
||||
if (dim !== "iface") continue
|
||||
const id = dim === query.row ? String(r.row_id ?? "") : String(r.col_id ?? "")
|
||||
const colon = id.indexOf(":")
|
||||
if (colon < 0) continue
|
||||
const sid = Number(id.slice(0, colon))
|
||||
if (looksLikeIfIndex(id.slice(colon + 1)) && Number.isFinite(sid)) ifaceServerIds.push(sid)
|
||||
}
|
||||
}
|
||||
await warmIfaceCache(ifaceServerIds)
|
||||
for (const r of raw) {
|
||||
if (query.row === "iface") r.row_id = canonicalIfaceDimId(String(r.row_id ?? ""))
|
||||
if (query.col === "iface") r.col_id = canonicalIfaceDimId(String(r.col_id ?? ""))
|
||||
}
|
||||
}
|
||||
|
||||
const metric = query.metric
|
||||
type Acc = { bytes: number; packets: number }
|
||||
const cell = new Map<string, Map<string, Acc>>()
|
||||
const colTotals = new Map<string, number>()
|
||||
for (const r of raw) {
|
||||
const rid = String(r.row_id ?? "")
|
||||
const cid = String(r.col_id ?? "")
|
||||
const acc: Acc = { bytes: Number(r.bytes) || 0, packets: Number(r.packets) || 0 }
|
||||
const val = metric === "packets" ? acc.packets : acc.bytes
|
||||
let rowMap = cell.get(rid)
|
||||
if (!rowMap) {
|
||||
rowMap = new Map()
|
||||
cell.set(rid, rowMap)
|
||||
}
|
||||
const prev = rowMap.get(cid)
|
||||
if (prev) {
|
||||
prev.bytes += acc.bytes
|
||||
prev.packets += acc.packets
|
||||
} else {
|
||||
rowMap.set(cid, acc)
|
||||
}
|
||||
colTotals.set(cid, (colTotals.get(cid) ?? 0) + val)
|
||||
}
|
||||
|
||||
const topCols = [...colTotals.entries()]
|
||||
.sort((a, b) => b[1] - a[1])
|
||||
.slice(0, PIVOT_COL_CAP)
|
||||
.map(([id]) => id)
|
||||
const topColSet = new Set(topCols)
|
||||
const folded = new Map<string, Map<string, number>>()
|
||||
const foldedColTotals = new Map<string, number>()
|
||||
let otherBytes = 0
|
||||
for (const [rid, cols] of cell) {
|
||||
const rowMap = new Map<string, number>()
|
||||
for (const [cid, acc] of cols) {
|
||||
const val = metric === "packets" ? acc.packets : acc.bytes
|
||||
const dest = topColSet.has(cid) ? cid : PIVOT_OTHER_ID
|
||||
if (dest === PIVOT_OTHER_ID) otherBytes += val
|
||||
rowMap.set(dest, (rowMap.get(dest) ?? 0) + val)
|
||||
foldedColTotals.set(dest, (foldedColTotals.get(dest) ?? 0) + val)
|
||||
}
|
||||
folded.set(rid, rowMap)
|
||||
}
|
||||
|
||||
const rowTotals = new Map<string, number>()
|
||||
for (const [rid, cols] of folded) {
|
||||
let t = 0
|
||||
for (const v of cols.values()) t += v
|
||||
rowTotals.set(rid, t)
|
||||
}
|
||||
const topRows = [...rowTotals.entries()]
|
||||
.sort((a, b) => b[1] - a[1])
|
||||
.slice(0, PIVOT_ROW_CAP)
|
||||
.map(([id]) => id)
|
||||
const topRowSet = new Set(topRows)
|
||||
const finalRows = new Map<string, Map<string, number>>()
|
||||
const finalRowTotals = new Map<string, number>()
|
||||
for (const [rid, cols] of folded) {
|
||||
const dest = topRowSet.has(rid) ? rid : PIVOT_OTHER_ID
|
||||
if (dest === PIVOT_OTHER_ID) {
|
||||
for (const [cid, v] of cols) {
|
||||
if (cid !== PIVOT_OTHER_ID) otherBytes += v
|
||||
}
|
||||
}
|
||||
let rowMap = finalRows.get(dest)
|
||||
if (!rowMap) {
|
||||
rowMap = new Map()
|
||||
finalRows.set(dest, rowMap)
|
||||
}
|
||||
for (const [cid, v] of cols) {
|
||||
rowMap.set(cid, (rowMap.get(cid) ?? 0) + v)
|
||||
}
|
||||
}
|
||||
for (const [rid, cols] of finalRows) {
|
||||
let t = 0
|
||||
for (const v of cols.values()) t += v
|
||||
finalRowTotals.set(rid, t)
|
||||
}
|
||||
|
||||
const colIds = [...topCols]
|
||||
if (foldedColTotals.has(PIVOT_OTHER_ID)) colIds.push(PIVOT_OTHER_ID)
|
||||
const rowIds = [...topRows]
|
||||
if (finalRows.has(PIVOT_OTHER_ID) && !topRowSet.has(PIVOT_OTHER_ID)) rowIds.push(PIVOT_OTHER_ID)
|
||||
|
||||
const labels = await loadPivotLabels(query.row, query.col, rowIds, colIds)
|
||||
|
||||
return {
|
||||
rowDim: query.row,
|
||||
colDim: query.col,
|
||||
metric,
|
||||
columns: colIds.map((id) => ({
|
||||
id,
|
||||
label: labels.col.get(id) ?? (id === PIVOT_OTHER_ID ? "Прочие" : id),
|
||||
total: foldedColTotals.get(id) ?? 0,
|
||||
})),
|
||||
rows: rowIds.map((id) => {
|
||||
const cols = finalRows.get(id) ?? new Map()
|
||||
const cells: Record<string, number> = {}
|
||||
for (const cid of colIds) cells[cid] = cols.get(cid) ?? 0
|
||||
return {
|
||||
id,
|
||||
label: labels.row.get(id) ?? (id === PIVOT_OTHER_ID ? "Прочие" : id),
|
||||
total: finalRowTotals.get(id) ?? 0,
|
||||
cells,
|
||||
}
|
||||
}),
|
||||
otherBytes,
|
||||
}
|
||||
}
|
||||
|
||||
async function loadPivotLabels(
|
||||
rowDim: StatisticsPivotDim,
|
||||
colDim: StatisticsPivotDim,
|
||||
rowIds: string[],
|
||||
colIds: string[],
|
||||
): Promise<{ row: Map<string, string>; col: Map<string, string> }> {
|
||||
const topo = await loadFlowTopology()
|
||||
const serverNames = new Map<string, string>()
|
||||
const allServers = await db.select({ id: servers.id, name: servers.name, host: servers.host }).from(servers)
|
||||
for (const s of allServers) serverNames.set(String(s.id), s.name || s.host)
|
||||
const userNames = new Map<string, string>()
|
||||
const allUsers = await db.select({ id: appUsers.id, name: appUsers.name, login: appUsers.login }).from(appUsers)
|
||||
for (const u of allUsers) userNames.set(u.id, u.name || u.login)
|
||||
const asnHolders = new Map<string, string>()
|
||||
const asnMeta = await db.select({ asn: flowAsnMeta.asn, holder: flowAsnMeta.holder }).from(flowAsnMeta)
|
||||
for (const a of asnMeta) asnHolders.set(String(a.asn), a.holder)
|
||||
|
||||
function label(dim: StatisticsPivotDim, id: string): string {
|
||||
if (id === PIVOT_OTHER_ID) return "Прочие"
|
||||
if (dim === "country") return id === "XX" ? "Неизвестно" : id
|
||||
if (dim === "server") return serverNames.get(id) || id
|
||||
if (dim === "user") return userNames.get(id) || id
|
||||
if (dim === "asn") {
|
||||
if (id === "0") return "other"
|
||||
const holder = asnHolders.get(id)
|
||||
return holder ? `AS${id} · ${holder}` : `AS${id}`
|
||||
}
|
||||
if (dim === "iface") {
|
||||
const colon = id.indexOf(":")
|
||||
if (colon < 0) return id
|
||||
const sid = id.slice(0, colon)
|
||||
const iface = id.slice(colon + 1)
|
||||
const sidNum = Number(sid)
|
||||
const name = Number.isFinite(sidNum) ? displayFactIface(sidNum, iface) : iface
|
||||
const serverName = serverNames.get(sid) || sid
|
||||
if (Number.isFinite(sidNum) && isWanFactIface(topo, sidNum, name)) {
|
||||
return wanIfaceLabel(serverName, name)
|
||||
}
|
||||
if (Number.isFinite(sidNum) && isOverlayTunnelIface(topo, sidNum, name)) {
|
||||
return overlayDupLabel(serverName, name)
|
||||
}
|
||||
return `${serverName} · ${name}`
|
||||
}
|
||||
return id
|
||||
}
|
||||
|
||||
const row = new Map<string, string>()
|
||||
const col = new Map<string, string>()
|
||||
for (const id of rowIds) row.set(id, label(rowDim, id))
|
||||
for (const id of colIds) col.set(id, label(colDim, id))
|
||||
return { row, col }
|
||||
}
|
||||
@@ -388,8 +388,8 @@ try {
|
||||
assert.equal(def.excludeOverlayApplied, true)
|
||||
assert.equal(def.excludeMeshApplied, true)
|
||||
assert.ok(!def.conversationsList.some((r) => r.proto === 47))
|
||||
assert.equal(def.conversationsList[0]?.service, "Google")
|
||||
assert.equal(def.conversationsList[0]?.category, "Веб")
|
||||
assert.equal(def.conversationsList[0]?.service, "YouTube")
|
||||
assert.equal(def.conversationsList[0]?.category, "Видео / стриминг")
|
||||
assert.equal(def.conversationsList[0]?.clientName, "Alice")
|
||||
assert.equal(def.conversationsList[0]?.enName, "NSK-SERVHOST-RTK")
|
||||
assert.equal(def.conversationsList[0]?.plane, "payload")
|
||||
@@ -416,14 +416,44 @@ try {
|
||||
resetIfaceCacheForTests()
|
||||
resetRipeCacheForTests()
|
||||
disableRipeEnqueueForTests()
|
||||
seedRipeCacheForTests({
|
||||
prefix: "74.125.0.0/16",
|
||||
asn: 15169,
|
||||
country: "US",
|
||||
lat: null,
|
||||
lng: null,
|
||||
holder: "GOOGLE",
|
||||
ok: true,
|
||||
fetchedAt: Date.now(),
|
||||
})
|
||||
seedRipeCacheForTests({
|
||||
prefix: "104.18.0.0/16",
|
||||
asn: 13335,
|
||||
country: "US",
|
||||
lat: null,
|
||||
lng: null,
|
||||
holder: "CLOUDFLARENET",
|
||||
ok: true,
|
||||
fetchedAt: Date.now(),
|
||||
})
|
||||
seedRipeCacheForTests({
|
||||
prefix: "146.75.0.0/16",
|
||||
asn: 54113,
|
||||
country: "US",
|
||||
lat: null,
|
||||
lng: null,
|
||||
holder: "FASTLY",
|
||||
ok: true,
|
||||
fetchedAt: Date.now(),
|
||||
})
|
||||
rememberServerIfaces(7, [{ ".id": "*2", name: "ether1" }])
|
||||
ingestParsedFlowsForServerForTests(7, [
|
||||
{
|
||||
src: "173.194.151.65",
|
||||
dst: "10.200.100.53",
|
||||
proto: 6,
|
||||
src: "74.125.104.196/32",
|
||||
dst: "10.200.100.53/32",
|
||||
proto: 17,
|
||||
srcPort: 443,
|
||||
dstPort: 57182,
|
||||
dstPort: 62598,
|
||||
bytes: 12_000,
|
||||
packets: 10,
|
||||
inIface: "2",
|
||||
@@ -440,15 +470,40 @@ try {
|
||||
inIface: "2",
|
||||
outIface: "2",
|
||||
},
|
||||
{
|
||||
src: "146.75.118.132/32",
|
||||
dst: "10.200.100.53/32",
|
||||
proto: 6,
|
||||
srcPort: 80,
|
||||
dstPort: 35026,
|
||||
bytes: 4_000,
|
||||
packets: 5,
|
||||
inIface: "2",
|
||||
outIface: "2",
|
||||
},
|
||||
])
|
||||
try {
|
||||
const rev = await buildFlowAnalytics({ minutes: 5, serverId: 7 })
|
||||
const google = rev.conversationsList.find((r) => r.src === "173.194.151.65")
|
||||
const google = rev.conversationsList.find((r) => r.src === "74.125.104.196")
|
||||
const cf = rev.conversationsList.find((r) => r.src === "104.18.35.51")
|
||||
assert.equal(google?.service, "Google")
|
||||
assert.equal(google?.category, "Веб")
|
||||
const fastly = rev.conversationsList.find((r) => r.src === "146.75.118.132")
|
||||
assert.equal(google?.service, "YouTube")
|
||||
assert.equal(google?.category, "Видео / стриминг")
|
||||
assert.equal(google?.internetPeer, "74.125.104.196")
|
||||
assert.equal(google?.internetPeerPort, 443)
|
||||
assert.equal(google?.clientIp, "10.200.100.53")
|
||||
assert.equal(google?.direction, "to_client")
|
||||
assert.equal(google?.dstAsn, 15169)
|
||||
assert.equal(google?.dstCountry, "US")
|
||||
assert.ok(!String(google?.src).includes("/"), "DTO src без /32")
|
||||
assert.equal(cf?.service, "Cloudflare")
|
||||
assert.equal(cf?.category, "CDN")
|
||||
assert.equal(fastly?.service, "Fastly")
|
||||
assert.equal(fastly?.dstAsn, 54113)
|
||||
assert.equal(fastly?.dstCountry, "US")
|
||||
assert.ok(rev.asns?.some((r) => r.id === "54113"))
|
||||
assert.ok(rev.countries?.some((r) => r.id === "US"))
|
||||
assert.ok(!rev.services?.every((s) => s.label === "Прочее"), "сервисы не схлопнуты в Прочее")
|
||||
} finally {
|
||||
resetFlowRingsForTests()
|
||||
resetIfaceCacheForTests()
|
||||
|
||||
@@ -27,11 +27,9 @@ import { getTrafficFlowSettingsRow, listHostPeers } from "./traffic-flow-setting
|
||||
import { applicationName, flowRowMatchesFilter } from "./traffic-flow-apps.js"
|
||||
import { dedupFlowRowsMaxBytes, flowTupleKey } from "./traffic-flow-dedup.js"
|
||||
import { enqueueRipeMisses } from "./traffic-flow-ripe.js"
|
||||
import { resolveFlowIp } from "./traffic-flow-geoip.js"
|
||||
import { classifyFlowDst, refreshFlowCatalogInBackground } from "./traffic-flow-classify.js"
|
||||
import { isIsoCountry } from "./traffic-flow-brands.js"
|
||||
import { classifyFlowPlane, flowBps, shouldKeepPlane } from "./traffic-flow-planes.js"
|
||||
import { pickInternetPeer } from "./traffic-flow-ip.js"
|
||||
import { resolveInternetDest } from "./traffic-flow-dest.js"
|
||||
import { refreshFlowCatalogInBackground } from "./traffic-flow-classify.js"
|
||||
import {
|
||||
enGreIfaceNames,
|
||||
getServerCatalog,
|
||||
@@ -253,11 +251,25 @@ async function buildFlowAnalyticsUncached(q: FlowAnalyticsQuery): Promise<FlowAn
|
||||
totalPackets += r.packets
|
||||
srcs.add(r.src)
|
||||
dsts.add(r.dst)
|
||||
const peer = pickInternetPeer(r.src, r.dst, r.srcPort, r.dstPort)
|
||||
peers.add(peer)
|
||||
const app = applicationName(r.proto, r.dstPort, r.srcPort)
|
||||
const ripe = resolveFlowIp(peer)
|
||||
const classified = classifyFlowDst(peer, r.proto, r.dstPort, r.srcPort, ripe)
|
||||
const destMeta = resolveInternetDest({
|
||||
src: r.src,
|
||||
dst: r.dst,
|
||||
proto: r.proto,
|
||||
srcPort: r.srcPort,
|
||||
dstPort: r.dstPort,
|
||||
serverId: r.serverId,
|
||||
inIface: resolved.name,
|
||||
topo,
|
||||
natSrc: r.natSrc,
|
||||
natDst: r.natDst,
|
||||
natSrcPort: r.natSrcPort,
|
||||
natDstPort: r.natDstPort,
|
||||
})
|
||||
const ep = destMeta.endpoints
|
||||
if (destMeta.dest) peers.add(destMeta.dest)
|
||||
const app = applicationName(r.proto, ep.peerPort || r.dstPort, ep.otherPort || r.srcPort)
|
||||
const ripe = destMeta.ripe
|
||||
const classified = destMeta.classified
|
||||
bump(applications, app, r.bytes, r.packets)
|
||||
bump(protocols, protoName(r.proto), r.bytes, r.packets)
|
||||
bump(sources, r.src, r.bytes, r.packets)
|
||||
@@ -269,7 +281,7 @@ async function buildFlowAnalyticsUncached(q: FlowAnalyticsQuery): Promise<FlowAn
|
||||
const asnLabel = ripe.holder ? `AS${ripe.asn} ${ripe.holder}` : `AS${ripe.asn}`
|
||||
bump(asns, asnId, r.bytes, r.packets, asnLabel)
|
||||
}
|
||||
const dstCountry = ripe?.ok && isIsoCountry(ripe.country) ? ripe.country : ""
|
||||
const dstCountry = destMeta.country && destMeta.country !== "unknown" ? destMeta.country : ""
|
||||
if (dstCountry) {
|
||||
bump(countries, dstCountry, r.bytes, r.packets)
|
||||
}
|
||||
@@ -301,8 +313,8 @@ async function buildFlowAnalyticsUncached(q: FlowAnalyticsQuery): Promise<FlowAn
|
||||
conv.set(ckey, {
|
||||
serverId: String(r.serverId),
|
||||
serverName: nameById.get(r.serverId) ?? String(r.serverId),
|
||||
src: r.src,
|
||||
dst: r.dst,
|
||||
src: ep.packetSrc || r.src,
|
||||
dst: ep.packetDst || r.dst,
|
||||
proto: r.proto,
|
||||
protoName: protoName(r.proto),
|
||||
srcPort: r.srcPort,
|
||||
@@ -321,6 +333,10 @@ async function buildFlowAnalyticsUncached(q: FlowAnalyticsQuery): Promise<FlowAn
|
||||
dstAsn: ripe?.asn || undefined,
|
||||
clientId: client?.userId,
|
||||
clientName: client?.name,
|
||||
clientIp: ep.clientIp || undefined,
|
||||
internetPeer: ep.internetPeer || undefined,
|
||||
internetPeerPort: ep.peerPort || undefined,
|
||||
direction: ep.direction,
|
||||
enId: en ? String(en.id) : undefined,
|
||||
enName: en?.name,
|
||||
plane,
|
||||
|
||||
@@ -8,6 +8,8 @@ import {
|
||||
OTHER_SERVICE,
|
||||
isNamedInternetService,
|
||||
mapServiceNodeId,
|
||||
mapCountryNodeId,
|
||||
mapCountryServiceNodeId,
|
||||
resolveFlowBrand,
|
||||
resolveRipeCountry,
|
||||
} from "./traffic-flow-brands.js"
|
||||
@@ -32,6 +34,8 @@ assert.equal(brandByAsn(401115)?.service, "ChatGPT")
|
||||
assert.equal(lookupBrand("1.1.1.1", 13335)?.service, "Cloudflare")
|
||||
assert.equal(lookupBrand("104.18.35.51", 0)?.service, "Cloudflare")
|
||||
assert.equal(lookupBrand("173.194.151.65", 0)?.service, "Google")
|
||||
assert.equal(lookupBrand("64.233.161.1", 0)?.service, "Google")
|
||||
assert.equal(lookupBrand("142.250.1.10", 0)?.service, "Google")
|
||||
assert.equal(lookupBrand("8.8.8.8", 0)?.service, "Google")
|
||||
assert.equal(lookupBrand("203.0.113.9", 64500), null)
|
||||
assert.equal(OTHER_SERVICE, "Прочее")
|
||||
@@ -41,6 +45,14 @@ assert.equal(isNamedInternetService("GRE", "Туннель"), false)
|
||||
assert.equal(isNamedInternetService("DNS", "DNS"), false)
|
||||
assert.equal(mapServiceNodeId("AWS"), "svc:aws")
|
||||
assert.equal(mapServiceNodeId("Cloudflare"), "svc:cloudflare")
|
||||
assert.equal(mapServiceNodeId("Прочее"), "svc:other")
|
||||
assert.equal(mapCountryNodeId("US"), "cc:us")
|
||||
assert.equal(mapCountryNodeId("nl"), "cc:nl")
|
||||
assert.equal(mapCountryNodeId(""), "cc:other")
|
||||
assert.equal(mapCountryNodeId("Прочее"), "cc:other")
|
||||
assert.equal(mapCountryNodeId("EU"), "cc:other")
|
||||
assert.equal(mapCountryServiceNodeId("cc:us", "svc:google"), "cc:us|svc:google")
|
||||
assert.equal(mapCountryServiceNodeId("cc:other", "svc:other"), "cc:other|svc:other")
|
||||
|
||||
assert.equal(brandByAsn(714)?.service, "Apple")
|
||||
assert.equal(brandByAsn(714)?.category, "CDN")
|
||||
@@ -70,6 +82,39 @@ assert.equal(isSteamGamePort(6, 443, 50000), false)
|
||||
|
||||
assert.equal(resolveFlowBrand("104.18.35.51", 32590, "VALVE-CORPORATION", 6, 443, 1)?.service, "Cloudflare")
|
||||
assert.equal(resolveFlowBrand("203.0.113.9", 32590, "", 17, 27015, 50000)?.service, "Steam")
|
||||
assert.equal(resolveFlowBrand("8.8.8.8", 15169, "GOOGLE", 6, 443, 51234)?.service, "Google")
|
||||
assert.equal(resolveFlowBrand("173.194.160.163", 15169, "GOOGLE", 6, 443, 51234)?.service, "YouTube")
|
||||
assert.equal(resolveFlowBrand("64.233.161.1", 0, "", 17, 443, 50000)?.service, "YouTube")
|
||||
assert.equal(resolveFlowBrand("64.233.161.1", 0, "", 6, 80, 50000)?.service, "Google")
|
||||
assert.equal(resolveFlowBrand("2001:4860:4860::8888", 15169, "GOOGLE", 17, 53, 53000)?.service, "Google")
|
||||
assert.equal(resolveFlowBrand("2001:4860:4860::8888", 15169, "GOOGLE", 17, 443, 50000)?.service, "YouTube")
|
||||
|
||||
assert.equal(brandByAsn(32934)?.service, "Meta")
|
||||
assert.equal(lookupBrand("157.240.12.52", 0)?.service, "Meta")
|
||||
assert.equal(lookupBrand("57.144.22.192", 0)?.service, "Meta")
|
||||
assert.equal(brandByHolder("Instagram LLC")?.service, "Instagram")
|
||||
assert.equal(mapServiceNodeId("Instagram"), "svc:instagram")
|
||||
assert.equal(isNamedInternetService("Instagram", "Видео / стриминг"), true)
|
||||
assert.equal(
|
||||
resolveFlowBrand("157.240.12.52", 32934, "FACEBOOK", 6, 443, 51234)?.service,
|
||||
"Instagram",
|
||||
"HTTPS на Meta front → Instagram, как YouTube на Google",
|
||||
)
|
||||
assert.equal(
|
||||
resolveFlowBrand("57.144.22.192", 0, "", 17, 443, 50000)?.service,
|
||||
"Instagram",
|
||||
"cdninstagram CIDR :443 без ASN → Instagram",
|
||||
)
|
||||
assert.equal(
|
||||
resolveFlowBrand("157.240.12.52", 32934, "FACEBOOK", 6, 80, 50000)?.service,
|
||||
"Meta",
|
||||
":80 на Meta остаётся Meta",
|
||||
)
|
||||
assert.equal(
|
||||
resolveFlowBrand("157.240.1.1", 54115, "WHATSAPP", 6, 443, 1)?.service,
|
||||
"Meta",
|
||||
"AS54115 WhatsApp не становится Instagram",
|
||||
)
|
||||
assert.equal(resolveRipeCountry("", 9059, ""), "IE")
|
||||
assert.equal(resolveRipeCountry("", 24940, ""), "DE")
|
||||
|
||||
|
||||
@@ -39,6 +39,7 @@ const TIMEWEB: BrandHit = { service: "Timeweb", ...CDN }
|
||||
const BEGET: BrandHit = { service: "Beget", ...CDN }
|
||||
const DDOS_GUARD: BrandHit = { service: "DDoS-Guard", ...CDN }
|
||||
const META: BrandHit = { service: "Meta", ...CDN }
|
||||
const INSTAGRAM: BrandHit = { service: "Instagram", ...VIDEO }
|
||||
|
||||
const GOOGLE: BrandHit = { service: "Google", ...WEB }
|
||||
const GITHUB: BrandHit = { service: "GitHub", ...WEB }
|
||||
@@ -174,12 +175,37 @@ const CIDR_BRANDS: Array<{ cidr: string; prefixLen: number; hit: BrandHit }> = [
|
||||
{ cidr: "172.217.0.0/16", prefixLen: 16, hit: GOOGLE },
|
||||
{ cidr: "74.125.0.0/16", prefixLen: 16, hit: GOOGLE },
|
||||
{ cidr: "142.250.0.0/15", prefixLen: 15, hit: GOOGLE },
|
||||
{ cidr: "64.233.0.0/16", prefixLen: 16, hit: GOOGLE },
|
||||
{ cidr: "66.102.0.0/16", prefixLen: 16, hit: GOOGLE },
|
||||
{ cidr: "66.249.64.0/19", prefixLen: 19, hit: GOOGLE },
|
||||
{ cidr: "72.14.192.0/18", prefixLen: 18, hit: GOOGLE },
|
||||
{ cidr: "108.177.0.0/16", prefixLen: 16, hit: GOOGLE },
|
||||
{ cidr: "209.85.128.0/17", prefixLen: 17, hit: GOOGLE },
|
||||
{ cidr: "216.58.192.0/19", prefixLen: 19, hit: GOOGLE },
|
||||
{ cidr: "216.239.32.0/19", prefixLen: 19, hit: GOOGLE },
|
||||
{ cidr: "208.65.152.0/22", prefixLen: 22, hit: YOUTUBE },
|
||||
{ cidr: "208.117.224.0/19", prefixLen: 19, hit: YOUTUBE },
|
||||
{ cidr: "31.13.64.0/18", prefixLen: 18, hit: META },
|
||||
{ cidr: "57.141.0.0/16", prefixLen: 16, hit: META },
|
||||
{ cidr: "57.142.0.0/15", prefixLen: 15, hit: META },
|
||||
{ cidr: "57.144.0.0/14", prefixLen: 14, hit: META },
|
||||
{ cidr: "57.148.0.0/15", prefixLen: 15, hit: META },
|
||||
{ cidr: "66.220.144.0/20", prefixLen: 20, hit: META },
|
||||
{ cidr: "69.63.176.0/20", prefixLen: 20, hit: META },
|
||||
{ cidr: "69.171.224.0/19", prefixLen: 19, hit: META },
|
||||
{ cidr: "74.119.76.0/22", prefixLen: 22, hit: META },
|
||||
{ cidr: "129.134.0.0/16", prefixLen: 16, hit: META },
|
||||
{ cidr: "157.240.0.0/16", prefixLen: 16, hit: META },
|
||||
{ cidr: "173.252.64.0/18", prefixLen: 18, hit: META },
|
||||
{ cidr: "179.60.192.0/22", prefixLen: 22, hit: META },
|
||||
{ cidr: "185.60.216.0/22", prefixLen: 22, hit: META },
|
||||
{ cidr: "199.201.64.0/22", prefixLen: 22, hit: META },
|
||||
{ cidr: "204.15.20.0/22", prefixLen: 22, hit: META },
|
||||
].sort((a, b) => b.prefixLen - a.prefixLen)
|
||||
|
||||
const HOLDER_BRANDS: Array<{ re: RegExp; hit: BrandHit }> = [
|
||||
{ re: /youtube/i, hit: YOUTUBE },
|
||||
{ re: /instagram/i, hit: INSTAGRAM },
|
||||
{ re: /valve|\bsteam\b/i, hit: STEAM },
|
||||
{ re: /blizzard|battle.?net/i, hit: BLIZZARD },
|
||||
{ re: /openai/i, hit: CHATGPT },
|
||||
@@ -196,6 +222,19 @@ const HOLDER_BRANDS: Array<{ re: RegExp; hit: BrandHit }> = [
|
||||
const NON_ISO = new Set(["EU", "AP", "ZZ", "XX", "A1", "A2", "O1"])
|
||||
|
||||
const STEAM_ASN = 32590
|
||||
const GOOGLE_FRONT_ASN = new Set([15169, 396982])
|
||||
/** AS32934 / AS63293 — Meta front (Facebook + Instagram CDN). AS54115 — WhatsApp, не Instagram. */
|
||||
const META_FRONT_ASN = new Set([32934, 63293])
|
||||
const WHATSAPP_ASN = 54115
|
||||
|
||||
function isGooglePublicDns(ip: string): boolean {
|
||||
return ipInCidrV4(ip, "8.8.8.0/24") || ipInCidrV4(ip, "8.8.4.0/24")
|
||||
}
|
||||
|
||||
function isHttpsOrQuic(proto: number, dstPort: number, srcPort: number): boolean {
|
||||
if (proto !== 6 && proto !== 17) return false
|
||||
return dstPort === 443 || srcPort === 443
|
||||
}
|
||||
|
||||
export function isIsoCountry(code: string): boolean {
|
||||
const c = String(code ?? "").trim().toUpperCase()
|
||||
@@ -256,9 +295,19 @@ export function lookupBrand(ip: string, asn: number): BrandHit | null {
|
||||
return brandByCidr(ip) || brandByAsn(asn)
|
||||
}
|
||||
|
||||
function isGoogleFront(asn: number, cidrBrand: BrandHit | null, asnBrand: BrandHit | null): boolean {
|
||||
return GOOGLE_FRONT_ASN.has(asn) || cidrBrand?.service === "Google" || asnBrand?.service === "Google"
|
||||
}
|
||||
|
||||
function isInstagramFront(asn: number, cidrBrand: BrandHit | null, asnBrand: BrandHit | null): boolean {
|
||||
if (asn === WHATSAPP_ASN) return false
|
||||
return META_FRONT_ASN.has(asn) || cidrBrand?.service === "Meta" || asnBrand?.service === "Meta"
|
||||
}
|
||||
|
||||
/**
|
||||
* Cloudflare CIDR бьёт holder (витрина на CF не становится Steam).
|
||||
* Holder (YouTube и др.) бьёт остальные CIDR/ASN.
|
||||
* Holder (YouTube / Instagram и др.) бьёт остальные CIDR/ASN.
|
||||
* HTTPS/QUIC на Google front → YouTube (кроме 8.8.8.8); на Meta front → Instagram (кроме WhatsApp ASN).
|
||||
* Порты Steam — только AS32590 и не выше Cloudflare CIDR.
|
||||
*/
|
||||
export function resolveFlowBrand(
|
||||
@@ -273,7 +322,14 @@ export function resolveFlowBrand(
|
||||
if (cidrBrand?.service === "Cloudflare") return cidrBrand
|
||||
const holderBrand = brandByHolder(holder)
|
||||
if (holderBrand) return holderBrand
|
||||
const fromLookup = cidrBrand || brandByAsn(asn)
|
||||
const asnBrand = brandByAsn(asn)
|
||||
if (!isGooglePublicDns(ip) && isHttpsOrQuic(proto, dstPort, srcPort) && isGoogleFront(asn, cidrBrand, asnBrand)) {
|
||||
return YOUTUBE
|
||||
}
|
||||
if (isHttpsOrQuic(proto, dstPort, srcPort) && isInstagramFront(asn, cidrBrand, asnBrand)) {
|
||||
return INSTAGRAM
|
||||
}
|
||||
const fromLookup = cidrBrand || asnBrand
|
||||
if (fromLookup) return fromLookup
|
||||
if (asn === STEAM_ASN && isSteamGamePort(proto, dstPort, srcPort)) return STEAM
|
||||
return null
|
||||
@@ -300,10 +356,23 @@ export function isNamedInternetService(service: string, category: string): boole
|
||||
}
|
||||
|
||||
export function mapServiceNodeId(label: string): string {
|
||||
const slug = label
|
||||
.trim()
|
||||
const raw = label.trim()
|
||||
if (raw === OTHER_SERVICE) return "svc:other"
|
||||
const slug = raw
|
||||
.toLowerCase()
|
||||
.replace(/[^a-z0-9]+/g, "-")
|
||||
.replace(/^-+|-+$/g, "")
|
||||
return `svc:${slug || "unknown"}`
|
||||
}
|
||||
|
||||
/** `US` → `cc:us`; неизвестная / пустая → `cc:other`. */
|
||||
export function mapCountryNodeId(code: string): string {
|
||||
const iso = normalizeIsoCountry(code)
|
||||
if (!iso) return "cc:other"
|
||||
return `cc:${iso.toLowerCase()}`
|
||||
}
|
||||
|
||||
/** id сервиса внутри страны: `cc:us|svc:google` — Google в US и NL не смешиваются в одном payload. */
|
||||
export function mapCountryServiceNodeId(countryId: string, serviceId: string): string {
|
||||
return `${countryId}|${serviceId}`
|
||||
}
|
||||
|
||||
@@ -33,10 +33,10 @@ const google = classifyFlowDst("173.194.160.163", 6, 443, 1, {
|
||||
ok: true,
|
||||
fetchedAt: Date.now(),
|
||||
})
|
||||
assert.equal(google.service, "Google")
|
||||
assert.equal(google.category, "Веб")
|
||||
assert.equal(google.service, "YouTube")
|
||||
assert.equal(google.category, "Видео / стриминг")
|
||||
|
||||
const googleCidr = classifyFlowDst("173.194.151.65", 6, 57182, 443, null)
|
||||
const googleCidr = classifyFlowDst("173.194.151.65", 6, 80, 50000, null)
|
||||
assert.equal(googleCidr.service, "Google")
|
||||
assert.equal(googleCidr.category, "Веб")
|
||||
|
||||
@@ -115,12 +115,90 @@ const googleCloud = classifyFlowDst("203.0.113.43", 6, 443, 1, {
|
||||
ok: true,
|
||||
fetchedAt: Date.now(),
|
||||
})
|
||||
assert.equal(googleCloud.service, "Google")
|
||||
assert.equal(googleCloud.category, "Веб")
|
||||
assert.equal(googleCloud.service, "YouTube")
|
||||
assert.equal(googleCloud.category, "Видео / стриминг")
|
||||
|
||||
const gre = classifyFlowDst("198.51.100.1", 47, 0, 0, null)
|
||||
assert.equal(gre.service, "GRE")
|
||||
assert.equal(gre.category, "Туннель")
|
||||
const greIgnore = classifyFlowDst("8.8.8.8", 47, 0, 0, {
|
||||
prefix: "8.8.8.0/24",
|
||||
asn: 15169,
|
||||
country: "US",
|
||||
lat: null,
|
||||
lng: null,
|
||||
holder: "GOOGLE",
|
||||
ok: true,
|
||||
fetchedAt: Date.now(),
|
||||
}, { ignoreTunnelProto: true })
|
||||
assert.equal(greIgnore.service, "Google")
|
||||
const dnsGoogle = classifyFlowDst("8.8.8.8", 17, 53, 53000, {
|
||||
prefix: "8.8.8.0/24",
|
||||
asn: 15169,
|
||||
country: "US",
|
||||
lat: null,
|
||||
lng: null,
|
||||
holder: "GOOGLE",
|
||||
ok: true,
|
||||
fetchedAt: Date.now(),
|
||||
})
|
||||
assert.equal(dnsGoogle.service, "Google")
|
||||
assert.notEqual(dnsGoogle.service, "Прочее")
|
||||
const ipv6Yt = classifyFlowDst("2001:4860:4860::8888", 17, 443, 50000, {
|
||||
prefix: "2001:4860:4860::8888/128",
|
||||
asn: 15169,
|
||||
country: "US",
|
||||
lat: null,
|
||||
lng: null,
|
||||
holder: "GOOGLE",
|
||||
ok: true,
|
||||
fetchedAt: Date.now(),
|
||||
})
|
||||
assert.equal(ipv6Yt.service, "YouTube")
|
||||
|
||||
const instagram = classifyFlowDst("157.240.12.52", 6, 443, 62598, {
|
||||
prefix: "157.240.0.0/16",
|
||||
asn: 32934,
|
||||
country: "US",
|
||||
lat: null,
|
||||
lng: null,
|
||||
holder: "FACEBOOK",
|
||||
ok: true,
|
||||
fetchedAt: Date.now(),
|
||||
})
|
||||
assert.equal(instagram.service, "Instagram")
|
||||
assert.equal(instagram.category, "Видео / стриминг")
|
||||
assert.notEqual(instagram.service, "Meta")
|
||||
|
||||
const metaHttp = classifyFlowDst("157.240.12.52", 6, 80, 50000, {
|
||||
prefix: "157.240.0.0/16",
|
||||
asn: 32934,
|
||||
country: "US",
|
||||
lat: null,
|
||||
lng: null,
|
||||
holder: "FACEBOOK",
|
||||
ok: true,
|
||||
fetchedAt: Date.now(),
|
||||
})
|
||||
assert.equal(metaHttp.service, "Meta")
|
||||
assert.equal(metaHttp.category, "CDN")
|
||||
|
||||
const igHolder = classifyFlowDst("203.0.113.80", 6, 443, 1, {
|
||||
prefix: "203.0.113.0/24",
|
||||
asn: 64503,
|
||||
country: "US",
|
||||
lat: null,
|
||||
lng: null,
|
||||
holder: "Instagram LLC",
|
||||
ok: true,
|
||||
fetchedAt: Date.now(),
|
||||
})
|
||||
assert.equal(igHolder.service, "Instagram")
|
||||
|
||||
const igCidr = classifyFlowDst("57.144.22.192", 17, 443, 50000, null)
|
||||
assert.equal(igCidr.service, "Instagram")
|
||||
assert.equal(igCidr.category, "Видео / стриминг")
|
||||
|
||||
const esp = classifyFlowDst("198.51.100.1", 50, 0, 0, null)
|
||||
assert.equal(esp.category, "Туннель")
|
||||
assert.equal(applicationName(17, 443, 50000), "QUIC")
|
||||
|
||||
@@ -48,7 +48,7 @@ export function seedFlowCatalogForTests(input: {
|
||||
export function categoryFromPurpose(purpose: string, proto: number, dstPort: number, srcPort: number): string {
|
||||
const p = purpose.toLowerCase()
|
||||
if (/gaming|steam|epic|riot|playstation|roblox|ubisoft/.test(p)) return "Игры"
|
||||
if (/streaming|youtube|netflix|twitch|video|spotify/.test(p)) return "Видео / стриминг"
|
||||
if (/streaming|youtube|netflix|twitch|video|spotify|instagram/.test(p)) return "Видео / стриминг"
|
||||
if (/cdn|cloudflare|akamai|fastly|hetzner|ovh|apple/.test(p)) return "CDN"
|
||||
if (/voip|discord|zoom/.test(p)) return "Голос"
|
||||
if (/openai|chatgpt|\bai\b/.test(p)) return "ИИ"
|
||||
@@ -73,9 +73,12 @@ export function classifyFlowDst(
|
||||
dstPort: number,
|
||||
srcPort: number,
|
||||
ripe: FlowIpMeta | null,
|
||||
opts?: { ignoreTunnelProto?: boolean },
|
||||
): FlowClassification {
|
||||
if (proto === 47) return { service: "GRE", category: "Туннель" }
|
||||
if (proto === 50) return { service: "ESP", category: "Туннель" }
|
||||
if (!opts?.ignoreTunnelProto) {
|
||||
if (proto === 47) return { service: "GRE", category: "Туннель" }
|
||||
if (proto === 50) return { service: "ESP", category: "Туннель" }
|
||||
}
|
||||
const app = applicationName(proto, dstPort, srcPort)
|
||||
if (app === "WireGuard") return { service: "WireGuard", category: "Туннель" }
|
||||
const hit = matchCidr(dst)
|
||||
|
||||
@@ -1,5 +1,10 @@
|
||||
import assert from "node:assert/strict"
|
||||
import { dedupFlowRowsMaxBytes, flowTupleKey } from "./traffic-flow-dedup.js"
|
||||
import {
|
||||
dedupFlowRowsAcrossExporters,
|
||||
dedupFlowRowsMaxBytes,
|
||||
flowConversationKey,
|
||||
flowTupleKey,
|
||||
} from "./traffic-flow-dedup.js"
|
||||
|
||||
const a = {
|
||||
serverId: 7,
|
||||
@@ -22,4 +27,13 @@ assert.equal(flowTupleKey(a), flowTupleKey(b))
|
||||
const sameIface = dedupFlowRowsMaxBytes([a, { ...a, bytes: 3_000, packets: 2 }])
|
||||
assert.equal(sameIface[0]?.bytes, 15_000)
|
||||
|
||||
const jh = { ...a, serverId: 7, bytes: 9_000 }
|
||||
const en = { ...a, serverId: 9, bytes: 11_000, inIface: "1" }
|
||||
assert.equal(flowConversationKey(jh), flowConversationKey(en))
|
||||
assert.notEqual(flowTupleKey(jh), flowTupleKey(en))
|
||||
const across = dedupFlowRowsAcrossExporters([en, jh], (x, y) => (x.serverId === 7 ? x : y))
|
||||
assert.equal(across.length, 1)
|
||||
assert.equal(across[0]?.serverId, 7)
|
||||
assert.equal(across[0]?.bytes, 9_000)
|
||||
|
||||
console.log("traffic-flow-dedup.test.ts: ok")
|
||||
|
||||
@@ -14,6 +14,32 @@ export function flowTupleKey(r: Pick<FlowTupleRow, "serverId" | "src" | "dst" |
|
||||
return `${r.serverId}|${r.src}|${r.dst}|${r.proto}|${r.srcPort}|${r.dstPort}`
|
||||
}
|
||||
|
||||
/** Один разговор на всех экспортёрах (JH+EN), без serverId. */
|
||||
export function flowConversationKey(r: Pick<FlowTupleRow, "src" | "dst" | "proto" | "srcPort" | "dstPort">): string {
|
||||
return `${r.src}|${r.dst}|${r.proto}|${r.srcPort}|${r.dstPort}`
|
||||
}
|
||||
|
||||
/**
|
||||
* Схлопнуть копии одного 5-tuple с разных серверов.
|
||||
* `prefer` выбирает ряд (клиент на JH важнее голого EN).
|
||||
*/
|
||||
export function dedupFlowRowsAcrossExporters<T extends FlowTupleRow>(
|
||||
rows: T[],
|
||||
prefer: (a: T, b: T) => T,
|
||||
): T[] {
|
||||
const byConv = new Map<string, T>()
|
||||
for (const row of rows) {
|
||||
const key = flowConversationKey(row)
|
||||
const prev = byConv.get(key)
|
||||
if (!prev) {
|
||||
byConv.set(key, row)
|
||||
continue
|
||||
}
|
||||
byConv.set(key, prefer(prev, row))
|
||||
}
|
||||
return [...byConv.values()]
|
||||
}
|
||||
|
||||
function ifaceKey(r: FlowTupleRow): string {
|
||||
return `${flowTupleKey(r)}|${r.inIface}`
|
||||
}
|
||||
|
||||
@@ -0,0 +1,256 @@
|
||||
import assert from "node:assert/strict"
|
||||
import {
|
||||
ingestParsedFlowsForServerForTests,
|
||||
resetEngineForTests,
|
||||
} from "./traffic-flow-engine.js"
|
||||
import { factsSnapshotForTests } from "./traffic-flow-facts.js"
|
||||
import { classifyInternetBrand, mapInternetBrand, resolveInternetDest } from "./traffic-flow-dest.js"
|
||||
import { disableCatalogFetchForTests, resetFlowCatalogForTests } from "./traffic-flow-classify.js"
|
||||
import {
|
||||
disableRipeEnqueueForTests,
|
||||
disableRipePersistForTests,
|
||||
resetRipeCacheForTests,
|
||||
seedRipeCacheForTests,
|
||||
} from "./traffic-flow-ripe.js"
|
||||
import { seedFlowTopologyForTests, type FlowTopology } from "./traffic-flow-topology.js"
|
||||
import { rememberServerIfaces, resetIfaceCacheForTests } from "./traffic-flow-ifindex.js"
|
||||
|
||||
disableCatalogFetchForTests()
|
||||
resetFlowCatalogForTests()
|
||||
disableRipePersistForTests()
|
||||
disableRipeEnqueueForTests()
|
||||
resetRipeCacheForTests()
|
||||
resetEngineForTests()
|
||||
resetIfaceCacheForTests()
|
||||
|
||||
seedRipeCacheForTests({
|
||||
prefix: "8.8.8.0/24",
|
||||
asn: 15169,
|
||||
country: "US",
|
||||
lat: null,
|
||||
lng: null,
|
||||
holder: "GOOGLE",
|
||||
ok: true,
|
||||
fetchedAt: Date.now(),
|
||||
})
|
||||
seedRipeCacheForTests({
|
||||
prefix: "95.167.0.0/16",
|
||||
asn: 12389,
|
||||
country: "RU",
|
||||
lat: null,
|
||||
lng: null,
|
||||
holder: "ROSTELECOM-AS",
|
||||
ok: true,
|
||||
fetchedAt: Date.now(),
|
||||
})
|
||||
|
||||
const topo: FlowTopology = {
|
||||
clientIfaces: new Map([[1, new Set(["gre-client"])]]),
|
||||
clientByIface: new Map([["1|gre-client", {
|
||||
userId: "u-rost",
|
||||
login: "alice",
|
||||
name: "Alice",
|
||||
serverId: 1,
|
||||
interfaceName: "gre-client",
|
||||
}]]),
|
||||
enNodes: [{ id: 2, name: "en", hosts: ["198.51.100.1"] }],
|
||||
enHosts: new Set(["198.51.100.1"]),
|
||||
jhHosts: new Set(["203.0.113.10"]),
|
||||
wanIfaces: new Map([[1, new Set(["ether1"])]]),
|
||||
plane: {
|
||||
clientIfaceNames: new Set(["gre-client"]),
|
||||
enHosts: new Set(["198.51.100.1"]),
|
||||
jhHosts: new Set(["203.0.113.10"]),
|
||||
},
|
||||
}
|
||||
seedFlowTopologyForTests(topo)
|
||||
rememberServerIfaces(1, [{ name: "gre-client", ifindex: "2" }])
|
||||
|
||||
ingestParsedFlowsForServerForTests(1, [
|
||||
{
|
||||
src: "95.167.1.10",
|
||||
dst: "10.200.100.53",
|
||||
proto: 6,
|
||||
srcPort: 51234,
|
||||
dstPort: 443,
|
||||
bytes: 100,
|
||||
packets: 2,
|
||||
inIface: "gre-client",
|
||||
outIface: "ether1",
|
||||
},
|
||||
{
|
||||
src: "95.167.1.10",
|
||||
dst: "8.8.8.8",
|
||||
proto: 6,
|
||||
srcPort: 51234,
|
||||
dstPort: 443,
|
||||
bytes: 50,
|
||||
packets: 1,
|
||||
inIface: "gre-client",
|
||||
outIface: "ether1",
|
||||
},
|
||||
{
|
||||
src: "203.0.113.10",
|
||||
dst: "198.51.100.1",
|
||||
proto: 47,
|
||||
srcPort: 0,
|
||||
dstPort: 0,
|
||||
bytes: 9_000,
|
||||
packets: 90,
|
||||
inIface: "NSK-SERVHOST-RTK",
|
||||
outIface: "NSK-SERVHOST-RTK",
|
||||
},
|
||||
{
|
||||
src: "10.200.100.53",
|
||||
dst: "10.200.100.1",
|
||||
proto: 6,
|
||||
srcPort: 53880,
|
||||
dstPort: 443,
|
||||
bytes: 70,
|
||||
packets: 1,
|
||||
inIface: "gre-client",
|
||||
outIface: "ether1",
|
||||
natDst: "8.8.8.8",
|
||||
natDstPort: 443,
|
||||
},
|
||||
])
|
||||
|
||||
const facts = factsSnapshotForTests()
|
||||
const total = facts.reduce((s, r) => s + r.bytes, 0)
|
||||
assert.equal(total, 120, "unique = Google payload + NAT, без overlay/пустого dest")
|
||||
assert.equal(facts.some((r) => r.asn === 12389), false, "ASN клиента не в кубе")
|
||||
assert.equal(facts.some((r) => r.service === "GRE"), false, "GRE не сервис unique")
|
||||
const google = facts.find((r) => r.asn === 15169)
|
||||
assert.ok(google)
|
||||
assert.equal(google.bytes, 120)
|
||||
assert.equal(facts.filter((r) => r.asn === 0).reduce((s, r) => s + r.bytes, 0), 0)
|
||||
|
||||
assert.equal(classifyInternetBrand("8.8.8.8", 47, 0, 0, null), null, "GRE не бренд")
|
||||
assert.equal(classifyInternetBrand("8.8.8.8", 6, 443, 51234, {
|
||||
prefix: "8.8.8.0/24",
|
||||
asn: 15169,
|
||||
country: "US",
|
||||
lat: null,
|
||||
lng: null,
|
||||
holder: "GOOGLE",
|
||||
ok: true,
|
||||
fetchedAt: Date.now(),
|
||||
})?.service, "Google")
|
||||
assert.equal(mapInternetBrand("203.0.113.50", 6, 443, 51234, null).service, "Прочее")
|
||||
assert.equal(mapInternetBrand("8.8.8.8", 47, 0, 0, null).service, "Прочее")
|
||||
assert.equal(mapInternetBrand("8.8.8.8", 6, 443, 51234, {
|
||||
prefix: "8.8.8.0/24",
|
||||
asn: 15169,
|
||||
country: "US",
|
||||
lat: null,
|
||||
lng: null,
|
||||
holder: "GOOGLE",
|
||||
ok: true,
|
||||
fetchedAt: Date.now(),
|
||||
}).service, "Google")
|
||||
|
||||
assert.equal(classifyInternetBrand("8.8.8.8", 17, 53, 53000, {
|
||||
prefix: "8.8.8.0/24",
|
||||
asn: 15169,
|
||||
country: "US",
|
||||
lat: null,
|
||||
lng: null,
|
||||
holder: "GOOGLE",
|
||||
ok: true,
|
||||
fetchedAt: Date.now(),
|
||||
})?.service, "Google")
|
||||
assert.equal(mapInternetBrand("8.8.8.8", 17, 53, 53000, {
|
||||
prefix: "8.8.8.0/24",
|
||||
asn: 15169,
|
||||
country: "US",
|
||||
lat: null,
|
||||
lng: null,
|
||||
holder: "GOOGLE",
|
||||
ok: true,
|
||||
fetchedAt: Date.now(),
|
||||
}).service, "Google")
|
||||
assert.equal(mapInternetBrand("2001:4860:4860::8888", 17, 443, 50000, {
|
||||
prefix: "2001:4860:4860::8888/128",
|
||||
asn: 15169,
|
||||
country: "US",
|
||||
lat: null,
|
||||
lng: null,
|
||||
holder: "GOOGLE",
|
||||
ok: true,
|
||||
fetchedAt: Date.now(),
|
||||
}).service, "YouTube")
|
||||
assert.equal(mapInternetBrand("64.233.161.1", 17, 443, 50000, null).service, "YouTube")
|
||||
assert.equal(mapInternetBrand("142.250.1.10", 6, 443, 1, null).service, "YouTube")
|
||||
|
||||
{
|
||||
const meta = resolveInternetDest({
|
||||
src: "74.125.104.196/32",
|
||||
dst: "10.200.100.53/32",
|
||||
proto: 17,
|
||||
srcPort: 443,
|
||||
dstPort: 62598,
|
||||
serverId: 1,
|
||||
inIface: "gre-client",
|
||||
topo,
|
||||
})
|
||||
assert.equal(meta.dest, "74.125.104.196")
|
||||
assert.equal(meta.classified.service, "YouTube")
|
||||
assert.notEqual(meta.classified.service, "Прочее")
|
||||
assert.equal(meta.endpoints.direction, "to_client")
|
||||
assert.equal(meta.endpoints.clientIp, "10.200.100.53")
|
||||
assert.equal(meta.asn, 0)
|
||||
}
|
||||
|
||||
{
|
||||
seedRipeCacheForTests({
|
||||
prefix: "146.75.0.0/16",
|
||||
asn: 54113,
|
||||
country: "US",
|
||||
lat: null,
|
||||
lng: null,
|
||||
holder: "FASTLY",
|
||||
ok: true,
|
||||
fetchedAt: Date.now(),
|
||||
})
|
||||
seedRipeCacheForTests({
|
||||
prefix: "3.174.0.0/16",
|
||||
asn: 16509,
|
||||
country: "US",
|
||||
lat: null,
|
||||
lng: null,
|
||||
holder: "AMAZON-AES",
|
||||
ok: true,
|
||||
fetchedAt: Date.now(),
|
||||
})
|
||||
const fastly = resolveInternetDest({
|
||||
src: "146.75.118.132/32",
|
||||
dst: "10.200.100.53/32",
|
||||
proto: 6,
|
||||
srcPort: 80,
|
||||
dstPort: 35026,
|
||||
serverId: 1,
|
||||
inIface: "gre-client",
|
||||
topo,
|
||||
})
|
||||
assert.equal(fastly.classified.service, "Fastly")
|
||||
assert.equal(fastly.asn, 54113)
|
||||
assert.equal(fastly.country, "US")
|
||||
const aws = resolveInternetDest({
|
||||
src: "3.174.2.35/32",
|
||||
dst: "10.200.100.53/32",
|
||||
proto: 6,
|
||||
srcPort: 443,
|
||||
dstPort: 43726,
|
||||
serverId: 1,
|
||||
inIface: "gre-client",
|
||||
topo,
|
||||
})
|
||||
assert.equal(aws.classified.service, "AWS")
|
||||
assert.equal(aws.asn, 16509)
|
||||
}
|
||||
|
||||
resetEngineForTests()
|
||||
seedFlowTopologyForTests(null)
|
||||
resetRipeCacheForTests()
|
||||
resetIfaceCacheForTests()
|
||||
console.log("traffic-flow-dest.test.ts: ok")
|
||||
@@ -0,0 +1,131 @@
|
||||
import { applicationName } from "./traffic-flow-apps.js"
|
||||
import { isIsoCountry, isNamedInternetService, OTHER_SERVICE, resolveFlowBrand } from "./traffic-flow-brands.js"
|
||||
import { classifyFlowDst, type FlowClassification } from "./traffic-flow-classify.js"
|
||||
import { resolveFlowIp } from "./traffic-flow-geoip.js"
|
||||
import { canonicalFactIface } from "./traffic-flow-ifindex.js"
|
||||
import {
|
||||
resolveFlowEndpoints,
|
||||
type FlowEndpoints,
|
||||
type InternetDestCtx,
|
||||
} from "./traffic-flow-ip.js"
|
||||
import type { FlowIpMeta } from "./traffic-flow-ripe.js"
|
||||
import {
|
||||
flowOursHosts,
|
||||
resolveClient,
|
||||
type FlowTopology,
|
||||
} from "./traffic-flow-topology.js"
|
||||
|
||||
export interface InternetDestMeta {
|
||||
dest: string
|
||||
ripe: FlowIpMeta | null
|
||||
classified: FlowClassification
|
||||
country: string
|
||||
asn: number
|
||||
endpoints: FlowEndpoints
|
||||
}
|
||||
|
||||
export function destCtxForIface(
|
||||
topo: FlowTopology | null | undefined,
|
||||
serverId: number,
|
||||
inIface: string,
|
||||
nat?: Pick<InternetDestCtx, "natSrc" | "natDst" | "natSrcPort" | "natDstPort">,
|
||||
): InternetDestCtx {
|
||||
const name = canonicalFactIface(serverId, inIface) || String(inIface ?? "").trim()
|
||||
return {
|
||||
ours: flowOursHosts(topo),
|
||||
boundClient: Boolean(
|
||||
topo && name && (
|
||||
resolveClient(topo, serverId, name)
|
||||
|| topo.clientIfaces.get(serverId)?.has(name)
|
||||
),
|
||||
),
|
||||
natSrc: nat?.natSrc,
|
||||
natDst: nat?.natDst,
|
||||
natSrcPort: nat?.natSrcPort,
|
||||
natDstPort: nat?.natDstPort,
|
||||
}
|
||||
}
|
||||
|
||||
const OTHER_BRAND: FlowClassification = { service: OTHER_SERVICE, category: OTHER_SERVICE }
|
||||
|
||||
function isTunnelProto(proto: number, dstPort: number, srcPort: number): boolean {
|
||||
if (proto === 47 || proto === 50) return true
|
||||
return applicationName(proto, dstPort, srcPort) === "WireGuard"
|
||||
}
|
||||
|
||||
/** Бренд интернет-dest: ASN/CIDR до skip DNS. GRE/ESP/WG — не сервис. */
|
||||
export function classifyInternetBrand(
|
||||
dst: string,
|
||||
proto: number,
|
||||
dstPort: number,
|
||||
srcPort: number,
|
||||
ripe: FlowIpMeta | null,
|
||||
): FlowClassification | null {
|
||||
if (isTunnelProto(proto, dstPort, srcPort)) return null
|
||||
const brand = resolveFlowBrand(dst, ripe?.asn ?? 0, ripe?.holder ?? "", proto, dstPort, srcPort)
|
||||
if (brand && isNamedInternetService(brand.service, brand.category)) return brand
|
||||
const app = applicationName(proto, dstPort, srcPort)
|
||||
if (app === "DNS" || app === "SSH" || app === "BGP") return null
|
||||
return null
|
||||
}
|
||||
|
||||
/** Тот же классификатор, что аналитика (GeoLite2 ASN + catalog). Туннель → Прочее. */
|
||||
export function mapInternetBrand(
|
||||
dst: string,
|
||||
proto: number,
|
||||
dstPort: number,
|
||||
srcPort: number,
|
||||
ripe: FlowIpMeta | null,
|
||||
): FlowClassification {
|
||||
if (isTunnelProto(proto, dstPort, srcPort)) return OTHER_BRAND
|
||||
const classified = classifyFlowDst(dst, proto, dstPort, srcPort, ripe, { ignoreTunnelProto: true })
|
||||
if (isNamedInternetService(classified.service, classified.category)) return classified
|
||||
return OTHER_BRAND
|
||||
}
|
||||
|
||||
export function resolveInternetDest(opts: {
|
||||
src: string
|
||||
dst: string
|
||||
proto: number
|
||||
srcPort: number
|
||||
dstPort: number
|
||||
serverId: number
|
||||
inIface: string
|
||||
topo?: FlowTopology | null
|
||||
natSrc?: string
|
||||
natDst?: string
|
||||
natSrcPort?: number
|
||||
natDstPort?: number
|
||||
}): InternetDestMeta {
|
||||
const ctx = destCtxForIface(opts.topo, opts.serverId, opts.inIface, {
|
||||
natSrc: opts.natSrc,
|
||||
natDst: opts.natDst,
|
||||
natSrcPort: opts.natSrcPort,
|
||||
natDstPort: opts.natDstPort,
|
||||
})
|
||||
const endpoints = resolveFlowEndpoints({
|
||||
src: opts.src,
|
||||
dst: opts.dst,
|
||||
srcPort: opts.srcPort,
|
||||
dstPort: opts.dstPort,
|
||||
ctx,
|
||||
})
|
||||
const dest = endpoints.internetPeer
|
||||
if (!dest) {
|
||||
return { dest: "", ripe: null, classified: OTHER_BRAND, country: "", asn: 0, endpoints }
|
||||
}
|
||||
const ripe = resolveFlowIp(dest)
|
||||
const classified = classifyFlowDst(
|
||||
dest,
|
||||
opts.proto,
|
||||
endpoints.peerPort,
|
||||
endpoints.otherPort,
|
||||
ripe,
|
||||
{ ignoreTunnelProto: true },
|
||||
)
|
||||
const country = ripe?.ok && isIsoCountry(ripe.country)
|
||||
? ripe.country
|
||||
: (ripe?.ok ? "" : "unknown")
|
||||
const asn = ripe?.ok && ripe.asn ? ripe.asn : 0
|
||||
return { dest, ripe, classified, country, asn, endpoints }
|
||||
}
|
||||
@@ -4,13 +4,26 @@ import { normalizeParsedFlow, parseFlowPacket, protoName, type ParsedFlow, type
|
||||
import { classifyFlowPlaneLite } from "./traffic-flow-planes.js"
|
||||
import { pickServerIdForExporter, type OverlayPeerRef } from "./traffic-flow-map-exporter.js"
|
||||
import { applicationName } from "./traffic-flow-apps.js"
|
||||
import { classifyFlowDst } from "./traffic-flow-classify.js"
|
||||
import { enqueueRipeMisses, pruneRipeSqlite } from "./traffic-flow-ripe.js"
|
||||
import { resolveFlowIp } from "./traffic-flow-geoip.js"
|
||||
import { invalidateTrafficFlowSettingsCache } from "./traffic-flow-settings.js"
|
||||
import { isIsoCountry } from "./traffic-flow-brands.js"
|
||||
import { maybeRefreshIfaces } from "./traffic-flow-ifaces.js"
|
||||
import { pickInternetPeer } from "./traffic-flow-ip.js"
|
||||
import { canonicalFactIface } from "./traffic-flow-ifindex.js"
|
||||
import { shouldWriteFlowFact } from "./traffic-flow-facts-filter.js"
|
||||
import { canonicalIp } from "./traffic-flow-ip.js"
|
||||
import { resolveInternetDest } from "./traffic-flow-dest.js"
|
||||
import {
|
||||
getServerCatalog,
|
||||
loadFlowTopology,
|
||||
peekFlowTopology,
|
||||
peekServerCatalog,
|
||||
} from "./traffic-flow-topology.js"
|
||||
import {
|
||||
bumpFlowFact,
|
||||
factsPendingSize,
|
||||
flushFlowFacts,
|
||||
hourBucketIso,
|
||||
resetFactsForTests,
|
||||
} from "./traffic-flow-facts.js"
|
||||
|
||||
export const TICK_MS = 2_000
|
||||
export const PERSIST_MS = 10_000
|
||||
@@ -49,15 +62,19 @@ export interface PendingFlowRow {
|
||||
nextHop: string
|
||||
flowStartMs: number
|
||||
flowEndMs: number
|
||||
natSrc: string
|
||||
natDst: string
|
||||
natSrcPort: number
|
||||
natDstPort: number
|
||||
}
|
||||
|
||||
function inetOrNull(value: string | null | undefined): string | null {
|
||||
const s = String(value ?? "").trim()
|
||||
const s = canonicalIp(value)
|
||||
return s.length > 0 ? s : null
|
||||
}
|
||||
|
||||
export function isValidFlowInet(value: string): boolean {
|
||||
const s = value.trim()
|
||||
const s = canonicalIp(value)
|
||||
if (!s) return false
|
||||
const v4 = /^(\d{1,3})\.(\d{1,3})\.(\d{1,3})\.(\d{1,3})$/.exec(s)
|
||||
if (v4) {
|
||||
@@ -78,17 +95,32 @@ function clampProto(n: number): number {
|
||||
return Math.max(0, Math.min(255, Math.trunc(n)))
|
||||
}
|
||||
|
||||
function clampPort(n: number): number {
|
||||
if (!Number.isFinite(n)) return 0
|
||||
return Math.max(0, Math.min(65535, Math.trunc(n)))
|
||||
}
|
||||
|
||||
function sanitizeNatIp(value: string | null | undefined): string {
|
||||
const s = canonicalIp(value)
|
||||
if (!s || s === "0.0.0.0") return ""
|
||||
return isValidFlowInet(s) ? s : ""
|
||||
}
|
||||
|
||||
function sanitizeFlowRow(r: PendingFlowRow): PendingFlowRow | null {
|
||||
const src = (r.src || "").trim() || "0.0.0.0"
|
||||
const dst = (r.dst || "").trim() || "0.0.0.0"
|
||||
const src = canonicalIp(r.src) || "0.0.0.0"
|
||||
const dst = canonicalIp(r.dst) || "0.0.0.0"
|
||||
if (!isValidFlowInet(src) || !isValidFlowInet(dst)) return null
|
||||
const next = inetOrNull(r.nextHop)
|
||||
const next = inetOrNull(canonicalIp(r.nextHop))
|
||||
return {
|
||||
...r,
|
||||
src,
|
||||
dst,
|
||||
nextHop: next && isValidFlowInet(next) ? next : "",
|
||||
proto: clampProto(r.proto),
|
||||
natSrc: sanitizeNatIp(r.natSrc),
|
||||
natDst: sanitizeNatIp(r.natDst),
|
||||
natSrcPort: clampPort(r.natSrcPort),
|
||||
natDstPort: clampPort(r.natDstPort),
|
||||
}
|
||||
}
|
||||
|
||||
@@ -108,6 +140,10 @@ function flowUpsertParams(r: PendingFlowRow) {
|
||||
nextHop: inetOrNull(r.nextHop),
|
||||
flowStartMs: r.flowStartMs,
|
||||
flowEndMs: r.flowEndMs,
|
||||
natSrc: inetOrNull(r.natSrc),
|
||||
natDst: inetOrNull(r.natDst),
|
||||
natSrcPort: r.natSrcPort,
|
||||
natDstPort: r.natDstPort,
|
||||
}
|
||||
}
|
||||
|
||||
@@ -328,20 +364,37 @@ export function getEngineStats(): EngineStats {
|
||||
export function queueParsedFlows(serverId: number, flows: ParsedFlowInput[]): void {
|
||||
if (flows.length) bumpDataEpoch()
|
||||
const bucketAt = minuteBucketIso()
|
||||
const hourAt = hourBucketIso()
|
||||
const ripeMisses: string[] = []
|
||||
const topo = peekFlowTopology()
|
||||
const catalog = peekServerCatalog()
|
||||
if (!topo) void loadFlowTopology().catch(() => {})
|
||||
if (!catalog) void getServerCatalog().catch(() => {})
|
||||
const serverType = catalog?.byId.get(serverId)?.type
|
||||
for (const raw of flows) {
|
||||
const flow = normalizeParsedFlow(raw)
|
||||
addToTick(serverId, flow, flow.bytes)
|
||||
bumpRollup(serverId, bucketAt, flow, flow.bytes, flow.packets)
|
||||
const peer = pickInternetPeer(flow.src, flow.dst, flow.srcPort, flow.dstPort)
|
||||
const ripe = resolveFlowIp(peer)
|
||||
if (peer && !ripe) ripeMisses.push(peer)
|
||||
const classified = classifyFlowDst(peer, flow.proto, flow.dstPort, flow.srcPort, ripe)
|
||||
const destMeta = resolveInternetDest({
|
||||
src: flow.src,
|
||||
dst: flow.dst,
|
||||
proto: flow.proto,
|
||||
srcPort: flow.srcPort,
|
||||
dstPort: flow.dstPort,
|
||||
serverId,
|
||||
inIface: flow.inIface,
|
||||
topo,
|
||||
natSrc: flow.natSrc,
|
||||
natDst: flow.natDst,
|
||||
natSrcPort: flow.natSrcPort,
|
||||
natDstPort: flow.natDstPort,
|
||||
})
|
||||
const ripe = destMeta.ripe
|
||||
if (destMeta.dest && !ripe) ripeMisses.push(destMeta.dest)
|
||||
const classified = destMeta.classified
|
||||
const app = applicationName(flow.proto, flow.dstPort, flow.srcPort)
|
||||
const country = ripe?.ok && isIsoCountry(ripe.country)
|
||||
? ripe.country
|
||||
: (ripe?.ok ? "" : "unknown")
|
||||
const asnKey = ripe?.ok && ripe.asn ? String(ripe.asn) : "unknown"
|
||||
const country = destMeta.country
|
||||
const asnKey = destMeta.asn ? String(destMeta.asn) : "unknown"
|
||||
bumpDim(serverId, bucketAt, "proto", protoName(flow.proto), flow.bytes, flow.packets)
|
||||
bumpDim(serverId, bucketAt, "app", app, flow.bytes, flow.packets)
|
||||
bumpDim(serverId, bucketAt, "iface", flow.inIface || "__unknown__", flow.bytes, flow.packets)
|
||||
@@ -349,6 +402,34 @@ export function queueParsedFlows(serverId: number, flows: ParsedFlowInput[]): vo
|
||||
bumpDim(serverId, bucketAt, "service", classified.service, flow.bytes, flow.packets)
|
||||
if (country) bumpDim(serverId, bucketAt, "country", country, flow.bytes, flow.packets)
|
||||
bumpDim(serverId, bucketAt, "asn", asnKey, flow.bytes, flow.packets)
|
||||
if (shouldWriteFlowFact({
|
||||
serverId,
|
||||
serverType,
|
||||
inIface: flow.inIface,
|
||||
outIface: flow.outIface,
|
||||
proto: flow.proto,
|
||||
srcPort: flow.srcPort,
|
||||
dstPort: flow.dstPort,
|
||||
src: flow.src,
|
||||
dst: flow.dst,
|
||||
topo,
|
||||
dest: destMeta.dest,
|
||||
natSrc: flow.natSrc,
|
||||
natDst: flow.natDst,
|
||||
natSrcPort: flow.natSrcPort,
|
||||
natDstPort: flow.natDstPort,
|
||||
})) {
|
||||
bumpFlowFact({
|
||||
serverId,
|
||||
bucketAt: hourAt,
|
||||
iface: canonicalFactIface(serverId, flow.inIface),
|
||||
country: country || "XX",
|
||||
service: classified.service,
|
||||
asn: destMeta.asn,
|
||||
bytes: flow.bytes,
|
||||
packets: flow.packets,
|
||||
})
|
||||
}
|
||||
|
||||
const key = pendingKey(serverId, bucketAt, flow)
|
||||
const prev = pending.get(key)
|
||||
@@ -357,6 +438,10 @@ export function queueParsedFlows(serverId: number, flows: ParsedFlowInput[]): vo
|
||||
prev.packets += flow.packets
|
||||
if (flow.outIface && !prev.flow.outIface) prev.flow.outIface = flow.outIface
|
||||
if (flow.nextHop && !prev.flow.nextHop) prev.flow.nextHop = flow.nextHop
|
||||
if (flow.natSrc && !prev.flow.natSrc) prev.flow.natSrc = flow.natSrc
|
||||
if (flow.natDst && !prev.flow.natDst) prev.flow.natDst = flow.natDst
|
||||
if (flow.natSrcPort && !prev.flow.natSrcPort) prev.flow.natSrcPort = flow.natSrcPort
|
||||
if (flow.natDstPort && !prev.flow.natDstPort) prev.flow.natDstPort = flow.natDstPort
|
||||
if (flow.flowStartMs && (!prev.flow.flowStartMs || flow.flowStartMs < prev.flow.flowStartMs)) {
|
||||
prev.flow.flowStartMs = flow.flowStartMs
|
||||
}
|
||||
@@ -412,6 +497,10 @@ function toPendingRow(row: PendingEntry): PendingFlowRow {
|
||||
nextHop: flow.nextHop,
|
||||
flowStartMs: flow.flowStartMs,
|
||||
flowEndMs: flow.flowEndMs,
|
||||
natSrc: flow.natSrc,
|
||||
natDst: flow.natDst,
|
||||
natSrcPort: flow.natSrcPort,
|
||||
natDstPort: flow.natDstPort,
|
||||
}
|
||||
}
|
||||
|
||||
@@ -423,6 +512,10 @@ function mergeInto(map: Map<string, PendingFlowRow>, row: PendingFlowRow): void
|
||||
prev.packets += row.packets
|
||||
if (row.outIface && !prev.outIface) prev.outIface = row.outIface
|
||||
if (row.nextHop && !prev.nextHop) prev.nextHop = row.nextHop
|
||||
if (row.natSrc && !prev.natSrc) prev.natSrc = row.natSrc
|
||||
if (row.natDst && !prev.natDst) prev.natDst = row.natDst
|
||||
if (row.natSrcPort && !prev.natSrcPort) prev.natSrcPort = row.natSrcPort
|
||||
if (row.natDstPort && !prev.natDstPort) prev.natDstPort = row.natDstPort
|
||||
if (row.flowStartMs && (!prev.flowStartMs || row.flowStartMs < prev.flowStartMs)) prev.flowStartMs = row.flowStartMs
|
||||
if (row.flowEndMs > (prev.flowEndMs ?? 0)) prev.flowEndMs = row.flowEndMs
|
||||
return
|
||||
@@ -731,7 +824,7 @@ async function upsertFlowBucketsBatch(rows: PendingFlowRow[]): Promise<void> {
|
||||
await pool.query({
|
||||
text: `
|
||||
INSERT INTO flow_buckets (
|
||||
server_id, bucket_at, src, dst, proto, src_port, dst_port, bytes, packets, in_iface, out_iface, next_hop, flow_start_ms, flow_end_ms
|
||||
server_id, bucket_at, src, dst, proto, src_port, dst_port, bytes, packets, in_iface, out_iface, next_hop, flow_start_ms, flow_end_ms, nat_src, nat_dst, nat_src_port, nat_dst_port
|
||||
)
|
||||
SELECT *
|
||||
FROM UNNEST(
|
||||
@@ -748,8 +841,12 @@ async function upsertFlowBucketsBatch(rows: PendingFlowRow[]): Promise<void> {
|
||||
$11::text[],
|
||||
$12::inet[],
|
||||
$13::bigint[],
|
||||
$14::bigint[]
|
||||
) AS t(server_id, bucket_at, src, dst, proto, src_port, dst_port, bytes, packets, in_iface, out_iface, next_hop, flow_start_ms, flow_end_ms)
|
||||
$14::bigint[],
|
||||
$15::inet[],
|
||||
$16::inet[],
|
||||
$17::int[],
|
||||
$18::int[]
|
||||
) AS t(server_id, bucket_at, src, dst, proto, src_port, dst_port, bytes, packets, in_iface, out_iface, next_hop, flow_start_ms, flow_end_ms, nat_src, nat_dst, nat_src_port, nat_dst_port)
|
||||
ON CONFLICT (server_id, bucket_at, src, dst, proto, src_port, dst_port, in_iface)
|
||||
DO UPDATE SET
|
||||
bytes = flow_buckets.bytes + excluded.bytes,
|
||||
@@ -759,7 +856,11 @@ async function upsertFlowBucketsBatch(rows: PendingFlowRow[]): Promise<void> {
|
||||
flow_start_ms = CASE
|
||||
WHEN excluded.flow_start_ms > 0 AND (flow_buckets.flow_start_ms = 0 OR excluded.flow_start_ms < flow_buckets.flow_start_ms)
|
||||
THEN excluded.flow_start_ms ELSE flow_buckets.flow_start_ms END,
|
||||
flow_end_ms = GREATEST(flow_buckets.flow_end_ms, excluded.flow_end_ms)
|
||||
flow_end_ms = GREATEST(flow_buckets.flow_end_ms, excluded.flow_end_ms),
|
||||
nat_src = COALESCE(excluded.nat_src, flow_buckets.nat_src),
|
||||
nat_dst = COALESCE(excluded.nat_dst, flow_buckets.nat_dst),
|
||||
nat_src_port = CASE WHEN excluded.nat_src_port > 0 THEN excluded.nat_src_port ELSE flow_buckets.nat_src_port END,
|
||||
nat_dst_port = CASE WHEN excluded.nat_dst_port > 0 THEN excluded.nat_dst_port ELSE flow_buckets.nat_dst_port END
|
||||
`,
|
||||
values: [
|
||||
rows.map((r) => r.serverId),
|
||||
@@ -776,15 +877,19 @@ async function upsertFlowBucketsBatch(rows: PendingFlowRow[]): Promise<void> {
|
||||
rows.map((r) => inetOrNull(r.nextHop)),
|
||||
rows.map((r) => r.flowStartMs),
|
||||
rows.map((r) => r.flowEndMs),
|
||||
rows.map((r) => inetOrNull(r.natSrc)),
|
||||
rows.map((r) => inetOrNull(r.natDst)),
|
||||
rows.map((r) => r.natSrcPort),
|
||||
rows.map((r) => r.natDstPort),
|
||||
],
|
||||
})
|
||||
}
|
||||
|
||||
const FLOW_UPSERT_SQL = `
|
||||
INSERT INTO flow_buckets (
|
||||
server_id, bucket_at, src, dst, proto, src_port, dst_port, bytes, packets, in_iface, out_iface, next_hop, flow_start_ms, flow_end_ms
|
||||
server_id, bucket_at, src, dst, proto, src_port, dst_port, bytes, packets, in_iface, out_iface, next_hop, flow_start_ms, flow_end_ms, nat_src, nat_dst, nat_src_port, nat_dst_port
|
||||
) VALUES (
|
||||
@serverId, @bucketAt, @src, @dst, @proto, @srcPort, @dstPort, @bytes, @packets, @inIface, @outIface, @nextHop, @flowStartMs, @flowEndMs
|
||||
@serverId, @bucketAt, @src, @dst, @proto, @srcPort, @dstPort, @bytes, @packets, @inIface, @outIface, @nextHop, @flowStartMs, @flowEndMs, @natSrc, @natDst, @natSrcPort, @natDstPort
|
||||
)
|
||||
ON CONFLICT(server_id, bucket_at, src, dst, proto, src_port, dst_port, in_iface)
|
||||
DO UPDATE SET
|
||||
@@ -795,7 +900,11 @@ const FLOW_UPSERT_SQL = `
|
||||
flow_start_ms = CASE
|
||||
WHEN excluded.flow_start_ms > 0 AND (flow_buckets.flow_start_ms = 0 OR excluded.flow_start_ms < flow_buckets.flow_start_ms)
|
||||
THEN excluded.flow_start_ms ELSE flow_buckets.flow_start_ms END,
|
||||
flow_end_ms = GREATEST(flow_buckets.flow_end_ms, excluded.flow_end_ms)
|
||||
flow_end_ms = GREATEST(flow_buckets.flow_end_ms, excluded.flow_end_ms),
|
||||
nat_src = COALESCE(excluded.nat_src, flow_buckets.nat_src),
|
||||
nat_dst = COALESCE(excluded.nat_dst, flow_buckets.nat_dst),
|
||||
nat_src_port = CASE WHEN excluded.nat_src_port > 0 THEN excluded.nat_src_port ELSE flow_buckets.nat_src_port END,
|
||||
nat_dst_port = CASE WHEN excluded.nat_dst_port > 0 THEN excluded.nat_dst_port ELSE flow_buckets.nat_dst_port END
|
||||
`
|
||||
|
||||
async function upsertFlowBuckets(rows: PendingFlowRow[]): Promise<number> {
|
||||
@@ -823,11 +932,12 @@ async function upsertFlowBuckets(rows: PendingFlowRow[]): Promise<number> {
|
||||
}
|
||||
}
|
||||
|
||||
export async function flushPending(opts?: { force?: boolean }): Promise<void> {
|
||||
export async function flushPending(opts?: { force?: boolean; prune?: boolean }): Promise<void> {
|
||||
pruneRecent()
|
||||
rollFlowRings()
|
||||
const force = Boolean(opts?.force)
|
||||
const hasWork = pending.size > 0 || minuteRollup.size > 0 || minuteDims.size > 0
|
||||
const doPrune = opts?.prune !== false
|
||||
const hasWork = pending.size > 0 || minuteRollup.size > 0 || minuteDims.size > 0 || factsPendingSize() > 0
|
||||
const due = persistDue(force, hasWork)
|
||||
try {
|
||||
await persistListenerStats(force)
|
||||
@@ -836,7 +946,7 @@ export async function flushPending(opts?: { force?: boolean }): Promise<void> {
|
||||
}
|
||||
|
||||
if (!hasWork) {
|
||||
if (force) {
|
||||
if (force && doPrune) {
|
||||
try {
|
||||
await pruneStored()
|
||||
} catch {
|
||||
@@ -886,9 +996,16 @@ export async function flushPending(opts?: { force?: boolean }): Promise<void> {
|
||||
/* rollup best-effort */
|
||||
}
|
||||
try {
|
||||
await pruneStored()
|
||||
await flushFlowFacts()
|
||||
} catch {
|
||||
/* prune best-effort */
|
||||
/* statistics cube best-effort */
|
||||
}
|
||||
if (doPrune) {
|
||||
try {
|
||||
await pruneStored()
|
||||
} catch {
|
||||
/* prune best-effort */
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -896,8 +1013,12 @@ export function lastFlushUsedTransactionForTests(): boolean {
|
||||
return lastFlushUsedTransaction
|
||||
}
|
||||
|
||||
export async function flushEngineNow(opts?: { prune?: boolean }): Promise<void> {
|
||||
await flushPending({ force: true, prune: opts?.prune })
|
||||
}
|
||||
|
||||
export async function flushPendingForTests(): Promise<void> {
|
||||
await flushPending({ force: true })
|
||||
await flushEngineNow()
|
||||
}
|
||||
|
||||
export function onEngineTick(): void {
|
||||
@@ -916,6 +1037,7 @@ export function resetEngineForTests(): void {
|
||||
rings.clear()
|
||||
minuteRollup.clear()
|
||||
minuteDims.clear()
|
||||
resetFactsForTests()
|
||||
packetsReceived = 0
|
||||
lastExporterIp = null
|
||||
lastError = ""
|
||||
|
||||
@@ -0,0 +1,207 @@
|
||||
import assert from "node:assert/strict"
|
||||
import {
|
||||
isJunkFactIface,
|
||||
isOverlayGreIface,
|
||||
isOverlayTunnelIface,
|
||||
isWanFactIface,
|
||||
shouldWriteFlowFact,
|
||||
} from "./traffic-flow-facts-filter.js"
|
||||
import { seedFlowTopologyForTests, type FlowTopology } from "./traffic-flow-topology.js"
|
||||
import { rememberServerIfaces, resetIfaceCacheForTests } from "./traffic-flow-ifindex.js"
|
||||
|
||||
function topo(partial: Partial<FlowTopology> = {}): FlowTopology {
|
||||
const wanIfaces = partial.wanIfaces ?? new Map([[1, new Set(["ether1"])]])
|
||||
const clientIfaces = partial.clientIfaces ?? new Map([[1, new Set(["gre-client"])]])
|
||||
const clientByIface = partial.clientByIface ?? new Map()
|
||||
const enHosts = partial.enHosts ?? new Set(["198.51.100.1"])
|
||||
const jhHosts = partial.jhHosts ?? new Set(["203.0.113.10"])
|
||||
return {
|
||||
clientIfaces,
|
||||
clientByIface,
|
||||
enNodes: partial.enNodes ?? [{ id: 2, name: "en", hosts: ["198.51.100.1"] }],
|
||||
enHosts,
|
||||
jhHosts,
|
||||
wanIfaces,
|
||||
tunnelIfaces: partial.tunnelIfaces,
|
||||
plane: partial.plane ?? {
|
||||
clientIfaceNames: new Set(["gre-client"]),
|
||||
enHosts,
|
||||
jhHosts,
|
||||
},
|
||||
}
|
||||
}
|
||||
|
||||
resetIfaceCacheForTests()
|
||||
rememberServerIfaces(1, [{ name: "ether1", ifindex: "2" }])
|
||||
seedFlowTopologyForTests(topo())
|
||||
|
||||
assert.equal(isJunkFactIface("0"), true)
|
||||
assert.equal(isJunkFactIface(""), true)
|
||||
assert.equal(isJunkFactIface("wg-flow"), true)
|
||||
assert.equal(isJunkFactIface("ether1"), false)
|
||||
assert.equal(isWanFactIface(topo(), 1, "ether1"), true)
|
||||
assert.equal(isOverlayGreIface(topo(), 1, "gre-en"), true)
|
||||
assert.equal(isOverlayGreIface(topo(), 1, "gre-client"), false)
|
||||
assert.equal(isOverlayGreIface(topo(), 1, "ether1"), false)
|
||||
|
||||
const typed = topo({
|
||||
clientIfaces: new Map([[1, new Set(["gre-client", "wg-server"])]]),
|
||||
tunnelIfaces: new Map([[1, new Set(["gre-en", "NSK-SERVHOST-RTK", "wg-jh-en", "wg-server"])]]),
|
||||
plane: {
|
||||
clientIfaceNames: new Set(["gre-client", "wg-server"]),
|
||||
enHosts: new Set(["198.51.100.1"]),
|
||||
jhHosts: new Set(["203.0.113.10"]),
|
||||
},
|
||||
})
|
||||
assert.equal(isOverlayTunnelIface(typed, 1, "NSK-SERVHOST-RTK"), true, "кастомное GRE overlay по type")
|
||||
assert.equal(isOverlayTunnelIface(typed, 1, "wg-jh-en"), true, "WG overlay по type")
|
||||
assert.equal(isOverlayTunnelIface(typed, 1, "wg-server"), false, "клиентский WG с binding")
|
||||
assert.equal(isOverlayTunnelIface(typed, 1, "wg-flow"), false, "wg-flow не overlay")
|
||||
assert.equal(isOverlayTunnelIface(topo(), 1, "NSK-SERVHOST-RTK"), false, "без type в снимке — не overlay")
|
||||
|
||||
const overlayOuter = shouldWriteFlowFact({
|
||||
serverId: 1,
|
||||
serverType: "jump-host",
|
||||
inIface: "ether1",
|
||||
outIface: "gre-en",
|
||||
proto: 47,
|
||||
srcPort: 0,
|
||||
dstPort: 0,
|
||||
src: "203.0.113.10",
|
||||
dst: "198.51.100.1",
|
||||
topo: topo(),
|
||||
})
|
||||
assert.equal(overlayOuter, false, "overlay proto 47 на ether1 не в facts")
|
||||
|
||||
const payloadGre = shouldWriteFlowFact({
|
||||
serverId: 1,
|
||||
serverType: "jump-host",
|
||||
inIface: "gre-client",
|
||||
outIface: "gre-en",
|
||||
proto: 6,
|
||||
srcPort: 51234,
|
||||
dstPort: 443,
|
||||
src: "10.100.1.17",
|
||||
dst: "8.8.8.8",
|
||||
topo: topo(),
|
||||
})
|
||||
assert.equal(payloadGre, true, "payload на GRE — да")
|
||||
|
||||
const payloadWan = shouldWriteFlowFact({
|
||||
serverId: 1,
|
||||
serverType: "jump-host",
|
||||
inIface: "ether1",
|
||||
outIface: "gre-client",
|
||||
proto: 6,
|
||||
srcPort: 443,
|
||||
dstPort: 51234,
|
||||
src: "8.8.8.8",
|
||||
dst: "10.100.1.17",
|
||||
topo: topo(),
|
||||
})
|
||||
assert.equal(payloadWan, true, "payload на ether1 WAN — да")
|
||||
|
||||
const junkZero = shouldWriteFlowFact({
|
||||
serverId: 1,
|
||||
serverType: "jump-host",
|
||||
inIface: "0",
|
||||
proto: 6,
|
||||
srcPort: 443,
|
||||
dstPort: 80,
|
||||
src: "1.1.1.1",
|
||||
dst: "8.8.8.8",
|
||||
topo: topo(),
|
||||
})
|
||||
assert.equal(junkZero, false)
|
||||
|
||||
const enTopo = topo({
|
||||
clientIfaces: new Map([[2, new Set()]]),
|
||||
wanIfaces: new Map([[2, new Set(["ether1"])]]),
|
||||
})
|
||||
const enTransit = shouldWriteFlowFact({
|
||||
serverId: 2,
|
||||
serverType: "exit-node",
|
||||
inIface: "gre-jh",
|
||||
outIface: "ether1",
|
||||
proto: 6,
|
||||
srcPort: 51234,
|
||||
dstPort: 443,
|
||||
src: "10.100.1.17",
|
||||
dst: "8.8.8.8",
|
||||
topo: enTopo,
|
||||
})
|
||||
assert.equal(enTransit, false, "EN-транзит без клиента — нет")
|
||||
|
||||
const enWan = shouldWriteFlowFact({
|
||||
serverId: 2,
|
||||
serverType: "exit-node",
|
||||
inIface: "ether1",
|
||||
proto: 6,
|
||||
srcPort: 443,
|
||||
dstPort: 80,
|
||||
src: "8.8.8.8",
|
||||
dst: "198.51.100.1",
|
||||
topo: enTopo,
|
||||
})
|
||||
assert.equal(enWan, true, "WAN payload на EN — да")
|
||||
|
||||
const emptyDest = shouldWriteFlowFact({
|
||||
serverId: 1,
|
||||
serverType: "jump-host",
|
||||
inIface: "gre-client",
|
||||
outIface: "ether1",
|
||||
proto: 6,
|
||||
srcPort: 51234,
|
||||
dstPort: 443,
|
||||
src: "95.167.1.10",
|
||||
dst: "10.200.100.53",
|
||||
topo: topo(),
|
||||
})
|
||||
assert.equal(emptyDest, false, "пустой интернет-dest не в facts")
|
||||
|
||||
const jhToEnHosts = shouldWriteFlowFact({
|
||||
serverId: 1,
|
||||
serverType: "jump-host",
|
||||
inIface: "ether1",
|
||||
proto: 6,
|
||||
srcPort: 0,
|
||||
dstPort: 0,
|
||||
src: "203.0.113.10",
|
||||
dst: "198.51.100.1",
|
||||
topo: topo(),
|
||||
})
|
||||
assert.equal(jhToEnHosts, false, "JH↔EN hosts не dest")
|
||||
|
||||
const overlayNamed = shouldWriteFlowFact({
|
||||
serverId: 1,
|
||||
serverType: "jump-host",
|
||||
inIface: "NSK-SERVHOST-RTK",
|
||||
outIface: "NSK-SERVHOST-RTK",
|
||||
proto: 47,
|
||||
srcPort: 0,
|
||||
dstPort: 0,
|
||||
src: "203.0.113.10",
|
||||
dst: "198.51.100.1",
|
||||
topo: typed,
|
||||
})
|
||||
assert.equal(overlayNamed, false, "overlay proto 47 на NSK-SERVHOST-RTK не в facts")
|
||||
|
||||
const natPayload = shouldWriteFlowFact({
|
||||
serverId: 1,
|
||||
serverType: "jump-host",
|
||||
inIface: "gre-client",
|
||||
outIface: "ether1",
|
||||
proto: 6,
|
||||
srcPort: 53880,
|
||||
dstPort: 443,
|
||||
src: "10.200.100.53",
|
||||
dst: "10.200.100.1",
|
||||
natDst: "8.8.8.8",
|
||||
natDstPort: 443,
|
||||
topo: topo(),
|
||||
})
|
||||
assert.equal(natPayload, true, "NAT Google на client GRE — да")
|
||||
|
||||
seedFlowTopologyForTests(null)
|
||||
resetIfaceCacheForTests()
|
||||
console.log("traffic-flow-facts-filter.test.ts: ok")
|
||||
@@ -0,0 +1,144 @@
|
||||
import { mapRosInterfaceType } from "../modules/users/iface-type.js"
|
||||
import { STATISTICS_DUP_MARK, STATISTICS_WAN_MARK } from "@mmapp/contracts/statistics"
|
||||
import { destCtxForIface } from "./traffic-flow-dest.js"
|
||||
import { canonicalFactIface } from "./traffic-flow-ifindex.js"
|
||||
import { pickInternetDest, isLocalIp } from "./traffic-flow-ip.js"
|
||||
import { classifyFlowPlane, isTunnelProto } from "./traffic-flow-planes.js"
|
||||
import { flowOursHosts, resolveClient, type FlowTopology } from "./traffic-flow-topology.js"
|
||||
|
||||
const JUNK_IFACE = new Set(["", "0", "—", "__unknown__", "wg-flow"])
|
||||
|
||||
export { STATISTICS_WAN_MARK, STATISTICS_DUP_MARK }
|
||||
|
||||
export function isJunkFactIface(iface: string | null | undefined): boolean {
|
||||
const n = String(iface ?? "").trim()
|
||||
if (JUNK_IFACE.has(n)) return true
|
||||
return /^#?0$/.test(n)
|
||||
}
|
||||
|
||||
export function isDashDisplayIface(iface: string): boolean {
|
||||
return String(iface ?? "").trim() === "—"
|
||||
}
|
||||
|
||||
export function isMgmtIface(name: string): boolean {
|
||||
const n = String(name ?? "").trim().toLowerCase()
|
||||
return n === "wg-flow" || n.endsWith("/wg-flow") || n.includes("wg-flow")
|
||||
}
|
||||
|
||||
/** GRE или WG по снимку RouterOS, иначе по имени. */
|
||||
export function isTunnelIfaceName(
|
||||
topo: FlowTopology | null | undefined,
|
||||
serverId: number,
|
||||
iface: string,
|
||||
): boolean {
|
||||
const name = String(iface ?? "").trim()
|
||||
if (!name || isJunkFactIface(name) || isMgmtIface(name)) return false
|
||||
const typed = topo?.tunnelIfaces?.get(serverId)
|
||||
if (typed && typed.size > 0) return typed.has(name)
|
||||
const t = mapRosInterfaceType("", name)
|
||||
return t === "gre" || t === "wg"
|
||||
}
|
||||
|
||||
/** WAN uplink: `wanIfaces` топологии, иначе ether1 у JH/EN без wan_uplinks. */
|
||||
export function isWanFactIface(
|
||||
topo: FlowTopology | null | undefined,
|
||||
serverId: number,
|
||||
iface: string,
|
||||
): boolean {
|
||||
const name = String(iface ?? "").trim()
|
||||
if (!name || isJunkFactIface(name) || isDashDisplayIface(name)) return false
|
||||
const wan = topo?.wanIfaces.get(serverId)
|
||||
if (wan && wan.size > 0) return wan.has(name)
|
||||
return /^ether1$/i.test(name)
|
||||
}
|
||||
|
||||
/** Overlay JH↔EN: GRE/WG не клиент, не WAN, не wg-flow. */
|
||||
export function isOverlayTunnelIface(
|
||||
topo: FlowTopology | null | undefined,
|
||||
serverId: number,
|
||||
iface: string,
|
||||
): boolean {
|
||||
const name = String(iface ?? "").trim()
|
||||
if (!name || isWanFactIface(topo, serverId, name) || isMgmtIface(name)) return false
|
||||
if (topo?.clientIfaces.get(serverId)?.has(name)) return false
|
||||
return isTunnelIfaceName(topo, serverId, name)
|
||||
}
|
||||
|
||||
/** @deprecated используйте isOverlayTunnelIface (GRE и WG). */
|
||||
export function isOverlayGreIface(
|
||||
topo: FlowTopology | null | undefined,
|
||||
serverId: number,
|
||||
iface: string,
|
||||
): boolean {
|
||||
return isOverlayTunnelIface(topo, serverId, iface)
|
||||
}
|
||||
|
||||
export function shouldWriteFlowFact(opts: {
|
||||
serverId: number
|
||||
serverType?: string
|
||||
inIface: string
|
||||
outIface?: string
|
||||
proto: number
|
||||
srcPort: number
|
||||
dstPort: number
|
||||
src: string
|
||||
dst: string
|
||||
topo?: FlowTopology | null
|
||||
dest?: string
|
||||
natSrc?: string
|
||||
natDst?: string
|
||||
natSrcPort?: number
|
||||
natDstPort?: number
|
||||
}): boolean {
|
||||
const inName = canonicalFactIface(opts.serverId, opts.inIface) || String(opts.inIface ?? "").trim()
|
||||
if (isJunkFactIface(inName) || isJunkFactIface(opts.inIface)) return false
|
||||
const outRaw = String(opts.outIface ?? "").trim()
|
||||
const outName = outRaw ? (canonicalFactIface(opts.serverId, outRaw) || outRaw) : ""
|
||||
const plane = classifyFlowPlane({
|
||||
src: opts.src,
|
||||
dst: opts.dst,
|
||||
proto: opts.proto,
|
||||
srcPort: opts.srcPort,
|
||||
dstPort: opts.dstPort,
|
||||
inIface: inName,
|
||||
outIface: outName || undefined,
|
||||
}, opts.topo?.plane)
|
||||
if (plane === "mgmt") return false
|
||||
if (plane === "overlay" || isTunnelProto(opts.proto, opts.srcPort, opts.dstPort)) return false
|
||||
const dest = opts.dest !== undefined
|
||||
? opts.dest
|
||||
: pickInternetDest(
|
||||
opts.src,
|
||||
opts.dst,
|
||||
opts.srcPort,
|
||||
opts.dstPort,
|
||||
destCtxForIface(opts.topo, opts.serverId, inName, {
|
||||
natSrc: opts.natSrc,
|
||||
natDst: opts.natDst,
|
||||
natSrcPort: opts.natSrcPort,
|
||||
natDstPort: opts.natDstPort,
|
||||
}),
|
||||
)
|
||||
if (!dest) return false
|
||||
const ours = flowOursHosts(opts.topo)
|
||||
if (isLocalIp(dest, ours) || ours.has(dest)) return false
|
||||
if (opts.serverType === "exit-node" && opts.topo) {
|
||||
const client =
|
||||
resolveClient(opts.topo, opts.serverId, inName)
|
||||
?? (outName ? resolveClient(opts.topo, opts.serverId, outName) : null)
|
||||
if (!client && isOverlayTunnelIface(opts.topo, opts.serverId, inName)) return false
|
||||
}
|
||||
return true
|
||||
}
|
||||
|
||||
export function wanIfaceLabel(serverName: string, iface: string): string {
|
||||
return `${serverName} · ${iface} · ${STATISTICS_WAN_MARK}`
|
||||
}
|
||||
|
||||
export function overlayDupLabel(serverName: string, iface: string): string {
|
||||
return `${serverName} · ${iface} · ${STATISTICS_DUP_MARK}`
|
||||
}
|
||||
|
||||
export function isNonUniqueShareLabel(label: string): boolean {
|
||||
return label.includes(STATISTICS_WAN_MARK) || label.includes(`· ${STATISTICS_DUP_MARK}`)
|
||||
}
|
||||
@@ -0,0 +1,141 @@
|
||||
import assert from "node:assert/strict"
|
||||
import { dbQuery } from "../db/index.js"
|
||||
import { withPgOrSkip } from "../test/pg.js"
|
||||
import { ensurePartitionFor } from "../db/partitions.js"
|
||||
import { pool } from "../db/index.js"
|
||||
import { applySqlMigrations } from "../db/migrate.js"
|
||||
import { invalidateFlowCatalogCache } from "./traffic-flow-topology.js"
|
||||
import {
|
||||
disableRipeEnqueueForTests,
|
||||
disableRipePersistForTests,
|
||||
resetRipeCacheForTests,
|
||||
seedRipeCacheForTests,
|
||||
} from "./traffic-flow-ripe.js"
|
||||
import { rememberServerIfaces, resetIfaceCacheForTests } from "./traffic-flow-ifindex.js"
|
||||
import { resetEngineForTests } from "./traffic-flow-engine.js"
|
||||
import { disableCatalogFetchForTests, resetFlowCatalogForTests } from "./traffic-flow-classify.js"
|
||||
|
||||
if (!(await withPgOrSkip())) {
|
||||
console.log("traffic-flow-facts-rebuild.test.ts: skip")
|
||||
process.exit(0)
|
||||
}
|
||||
|
||||
await applySqlMigrations(pool)
|
||||
|
||||
const nServers = (await dbQuery<{ n: number }>(`SELECT COUNT(*)::int AS n FROM servers`)).rows[0]?.n ?? 0
|
||||
if (nServers > 10) {
|
||||
console.warn("traffic-flow-facts-rebuild.test.ts: skip (не пустая БД)")
|
||||
process.exit(0)
|
||||
}
|
||||
|
||||
disableCatalogFetchForTests()
|
||||
resetFlowCatalogForTests()
|
||||
disableRipePersistForTests()
|
||||
disableRipeEnqueueForTests()
|
||||
resetRipeCacheForTests()
|
||||
resetEngineForTests()
|
||||
resetIfaceCacheForTests()
|
||||
|
||||
seedRipeCacheForTests({
|
||||
prefix: "8.8.8.0/24",
|
||||
asn: 15169,
|
||||
country: "US",
|
||||
lat: null,
|
||||
lng: null,
|
||||
holder: "GOOGLE",
|
||||
ok: true,
|
||||
fetchedAt: Date.now(),
|
||||
})
|
||||
seedRipeCacheForTests({
|
||||
prefix: "95.167.0.0/16",
|
||||
asn: 12389,
|
||||
country: "RU",
|
||||
lat: null,
|
||||
lng: null,
|
||||
holder: "ROSTELECOM-AS",
|
||||
ok: true,
|
||||
fetchedAt: Date.now(),
|
||||
})
|
||||
|
||||
const inserted = await dbQuery<{ id: number }>(`
|
||||
INSERT INTO servers (name, host, type, wan_uplinks)
|
||||
VALUES ('rebuild-facts-jh', '203.0.113.10', 'jump-host', '[{"iface":"ether1"}]'::jsonb)
|
||||
RETURNING id
|
||||
`)
|
||||
const serverId = inserted.rows[0]?.id
|
||||
if (serverId == null) throw new Error("no server")
|
||||
|
||||
const ts = new Date()
|
||||
await ensurePartitionFor(pool, "flow_buckets", "day", ts)
|
||||
await ensurePartitionFor(pool, "flow_hour_facts", "day", ts)
|
||||
await ensurePartitionFor(pool, "flow_daily_facts", "month", ts)
|
||||
|
||||
await dbQuery(`DELETE FROM flow_buckets WHERE server_id = $1`, [serverId])
|
||||
await dbQuery(`DELETE FROM flow_hour_facts WHERE server_id = $1`, [serverId])
|
||||
await dbQuery(`DELETE FROM flow_daily_facts WHERE server_id = $1`, [serverId])
|
||||
await dbQuery(`DELETE FROM user_interface_bindings WHERE server_id = $1`, [serverId])
|
||||
await dbQuery(`DELETE FROM app_users WHERE id = 'u-rebuild-1'`)
|
||||
|
||||
await dbQuery(`
|
||||
INSERT INTO app_users (id, name, login, role, active)
|
||||
VALUES ('u-rebuild-1', 'Клиент', 'rebuild-user', 'viewer', TRUE)
|
||||
ON CONFLICT (id) DO NOTHING
|
||||
`)
|
||||
await dbQuery(`
|
||||
INSERT INTO user_interface_bindings (id, user_id, server_id, interface_name, interface_type)
|
||||
VALUES ('bind-rebuild-1', 'u-rebuild-1', $1, 'gre-client', 'gre')
|
||||
`, [serverId])
|
||||
await dbQuery(`
|
||||
INSERT INTO server_snapshots (server_id, polled_at, status, raw_interfaces)
|
||||
VALUES ($1, now(), 'online', $2::jsonb)
|
||||
`, [serverId, JSON.stringify([{ name: "gre-client", type: "gre-tunnel" }, { name: "ether1", type: "ether" }])])
|
||||
|
||||
rememberServerIfaces(serverId, [{ name: "gre-client", ifindex: "2" }])
|
||||
invalidateFlowCatalogCache()
|
||||
|
||||
const bucketAt = new Date(Date.UTC(
|
||||
ts.getUTCFullYear(),
|
||||
ts.getUTCMonth(),
|
||||
ts.getUTCDate(),
|
||||
ts.getUTCHours(),
|
||||
0, 0, 0,
|
||||
)).toISOString()
|
||||
|
||||
await dbQuery(`
|
||||
INSERT INTO flow_buckets (server_id, bucket_at, src, dst, proto, src_port, dst_port, bytes, packets, in_iface, out_iface)
|
||||
VALUES
|
||||
($1, $2, '95.167.1.10', '10.200.100.53', 6, 51234, 443, 100, 2, 'gre-client', 'ether1'),
|
||||
($1, $2, '95.167.1.10', '8.8.8.8', 6, 51234, 443, 50, 1, 'gre-client', 'ether1')
|
||||
`, [serverId, bucketAt])
|
||||
|
||||
try {
|
||||
const { rebuildFlowFactsFromBuckets } = await import("./traffic-flow-facts-rebuild.js")
|
||||
const result = await rebuildFlowFactsFromBuckets()
|
||||
assert.equal(result.ok, true)
|
||||
assert.ok(result.buckets >= 2)
|
||||
|
||||
const rows = await dbQuery<{ asn: number; bytes: number }>(`
|
||||
SELECT asn, SUM(bytes)::bigint AS bytes
|
||||
FROM flow_hour_facts
|
||||
WHERE server_id = $1
|
||||
GROUP BY asn
|
||||
`, [serverId])
|
||||
const byAsn = new Map(rows.rows.map((r) => [Number(r.asn), Number(r.bytes)]))
|
||||
const total = [...byAsn.values()].reduce((s, n) => s + n, 0)
|
||||
assert.equal(total, 50)
|
||||
assert.equal(byAsn.get(12389), undefined, "ASN клиента не в hour facts")
|
||||
assert.equal(byAsn.get(15169), 50)
|
||||
assert.equal(byAsn.get(0), undefined)
|
||||
} finally {
|
||||
await dbQuery(`DELETE FROM flow_hour_facts WHERE server_id = $1`, [serverId])
|
||||
await dbQuery(`DELETE FROM flow_daily_facts WHERE server_id = $1`, [serverId])
|
||||
await dbQuery(`DELETE FROM flow_buckets WHERE server_id = $1`, [serverId])
|
||||
await dbQuery(`DELETE FROM user_interface_bindings WHERE server_id = $1`, [serverId])
|
||||
await dbQuery(`DELETE FROM server_snapshots WHERE server_id = $1`, [serverId])
|
||||
await dbQuery(`DELETE FROM app_users WHERE id = 'u-rebuild-1'`)
|
||||
await dbQuery(`DELETE FROM servers WHERE id = $1`, [serverId])
|
||||
resetEngineForTests()
|
||||
invalidateFlowCatalogCache()
|
||||
}
|
||||
|
||||
console.log("traffic-flow-facts-rebuild.test.ts: ok")
|
||||
@@ -0,0 +1,145 @@
|
||||
import { dbAll, dbGet, dbQuery, withAdvisoryLock } from "../db/index.js"
|
||||
import { flushEngineNow } from "./traffic-flow-engine.js"
|
||||
import { resolveInternetDest } from "./traffic-flow-dest.js"
|
||||
import {
|
||||
bumpFlowFact,
|
||||
discardPendingFacts,
|
||||
flushFlowFacts,
|
||||
hourBucketIso,
|
||||
} from "./traffic-flow-facts.js"
|
||||
import { shouldWriteFlowFact } from "./traffic-flow-facts-filter.js"
|
||||
import { canonicalFactIface } from "./traffic-flow-ifindex.js"
|
||||
import { getServerCatalog, loadFlowTopology } from "./traffic-flow-topology.js"
|
||||
|
||||
export const FACT_REBUILD_LOCK_KEY = 8_723_104
|
||||
const BATCH = 4_000
|
||||
|
||||
export interface FlowFactsRebuildResult {
|
||||
ok: true
|
||||
buckets: number
|
||||
facts: number
|
||||
days: string[]
|
||||
}
|
||||
|
||||
function hourFromBucket(raw: Date | string): string {
|
||||
const iso = raw instanceof Date ? raw.toISOString() : String(raw)
|
||||
const ms = new Date(iso).getTime()
|
||||
return hourBucketIso(Number.isFinite(ms) ? ms : Date.now())
|
||||
}
|
||||
|
||||
export async function rebuildFlowFactsFromBuckets(): Promise<FlowFactsRebuildResult> {
|
||||
return await withAdvisoryLock(FACT_REBUILD_LOCK_KEY, async () => {
|
||||
await flushEngineNow({ prune: false })
|
||||
discardPendingFacts()
|
||||
|
||||
const days = await dbAll<{ day: string }>(`
|
||||
SELECT DISTINCT (bucket_at AT TIME ZONE 'UTC')::date::text AS day
|
||||
FROM flow_buckets
|
||||
ORDER BY 1
|
||||
`)
|
||||
const dayList = days.map((r) => r.day).filter(Boolean)
|
||||
if (dayList.length === 0) {
|
||||
return { ok: true as const, buckets: 0, facts: 0, days: [] }
|
||||
}
|
||||
|
||||
await dbQuery(
|
||||
`DELETE FROM flow_hour_facts WHERE (bucket_at AT TIME ZONE 'UTC')::date = ANY(?::date[])`,
|
||||
[dayList],
|
||||
)
|
||||
await dbQuery(
|
||||
`DELETE FROM flow_daily_facts WHERE day = ANY(?::date[])`,
|
||||
[dayList],
|
||||
)
|
||||
|
||||
const topo = await loadFlowTopology()
|
||||
const catalog = await getServerCatalog()
|
||||
let offset = 0
|
||||
let buckets = 0
|
||||
|
||||
for (;;) {
|
||||
const rows = await dbAll<{
|
||||
serverId: number
|
||||
bucketAt: Date | string
|
||||
src: string
|
||||
dst: string
|
||||
proto: number
|
||||
srcPort: number
|
||||
dstPort: number
|
||||
bytes: number
|
||||
packets: number
|
||||
inIface: string
|
||||
outIface: string
|
||||
natSrc: string
|
||||
natDst: string
|
||||
natSrcPort: number
|
||||
natDstPort: number
|
||||
}>(`
|
||||
SELECT server_id AS "serverId", bucket_at AS "bucketAt",
|
||||
host(src) AS src, host(dst) AS dst, proto, src_port AS "srcPort", dst_port AS "dstPort",
|
||||
bytes, packets, in_iface AS "inIface", COALESCE(out_iface, '') AS "outIface",
|
||||
COALESCE(host(nat_src), '') AS "natSrc", COALESCE(host(nat_dst), '') AS "natDst",
|
||||
COALESCE(nat_src_port, 0) AS "natSrcPort", COALESCE(nat_dst_port, 0) AS "natDstPort"
|
||||
FROM flow_buckets
|
||||
ORDER BY bucket_at, server_id
|
||||
LIMIT ? OFFSET ?
|
||||
`, [BATCH, offset])
|
||||
if (rows.length === 0) break
|
||||
for (const row of rows) {
|
||||
buckets += 1
|
||||
const serverType = catalog.byId.get(row.serverId)?.type
|
||||
const destMeta = resolveInternetDest({
|
||||
src: row.src,
|
||||
dst: row.dst,
|
||||
proto: Number(row.proto) || 0,
|
||||
srcPort: Number(row.srcPort) || 0,
|
||||
dstPort: Number(row.dstPort) || 0,
|
||||
serverId: row.serverId,
|
||||
inIface: row.inIface,
|
||||
topo,
|
||||
natSrc: row.natSrc,
|
||||
natDst: row.natDst,
|
||||
natSrcPort: Number(row.natSrcPort) || 0,
|
||||
natDstPort: Number(row.natDstPort) || 0,
|
||||
})
|
||||
if (!shouldWriteFlowFact({
|
||||
serverId: row.serverId,
|
||||
serverType,
|
||||
inIface: row.inIface,
|
||||
outIface: row.outIface,
|
||||
proto: Number(row.proto) || 0,
|
||||
srcPort: Number(row.srcPort) || 0,
|
||||
dstPort: Number(row.dstPort) || 0,
|
||||
src: row.src,
|
||||
dst: row.dst,
|
||||
topo,
|
||||
dest: destMeta.dest,
|
||||
natSrc: row.natSrc,
|
||||
natDst: row.natDst,
|
||||
natSrcPort: Number(row.natSrcPort) || 0,
|
||||
natDstPort: Number(row.natDstPort) || 0,
|
||||
})) continue
|
||||
bumpFlowFact({
|
||||
serverId: row.serverId,
|
||||
bucketAt: hourFromBucket(row.bucketAt),
|
||||
iface: canonicalFactIface(row.serverId, row.inIface),
|
||||
country: destMeta.country || "XX",
|
||||
service: destMeta.classified.service,
|
||||
asn: destMeta.asn,
|
||||
bytes: Number(row.bytes) || 0,
|
||||
packets: Number(row.packets) || 0,
|
||||
})
|
||||
}
|
||||
offset += rows.length
|
||||
if (rows.length < BATCH) break
|
||||
}
|
||||
|
||||
const facts = await flushFlowFacts()
|
||||
const range = await dbGet<{ n: number }>(`SELECT COUNT(*)::int AS n FROM flow_buckets`)
|
||||
return {
|
||||
ok: true as const,
|
||||
buckets: Number(range?.n) || buckets,
|
||||
facts,
|
||||
days: dayList,
|
||||
}
|
||||
})
|
||||
}
|
||||
@@ -0,0 +1,71 @@
|
||||
import assert from "node:assert/strict"
|
||||
import {
|
||||
bumpFlowFact,
|
||||
collectCappedFacts,
|
||||
FACT_ASN_TOP,
|
||||
FACT_TUPLE_CAP,
|
||||
resetFactsForTests,
|
||||
} from "./traffic-flow-facts.js"
|
||||
|
||||
resetFactsForTests()
|
||||
bumpFlowFact({
|
||||
serverId: 1,
|
||||
bucketAt: "2026-09-10T10:00:00.000Z",
|
||||
iface: "ether1",
|
||||
country: "US",
|
||||
service: "https",
|
||||
asn: 15169,
|
||||
bytes: 100,
|
||||
packets: 2,
|
||||
})
|
||||
bumpFlowFact({
|
||||
serverId: 1,
|
||||
bucketAt: "2026-09-10T10:00:00.000Z",
|
||||
iface: "ether1",
|
||||
country: "us",
|
||||
service: "https",
|
||||
asn: 15169,
|
||||
bytes: 50,
|
||||
packets: 1,
|
||||
})
|
||||
const merged = collectCappedFacts()
|
||||
assert.equal(merged.length, 1)
|
||||
assert.equal(merged[0]?.bytes, 150)
|
||||
assert.equal(merged[0]?.country, "US")
|
||||
assert.equal(merged[0]?.asn, 15169)
|
||||
|
||||
resetFactsForTests()
|
||||
for (let i = 1; i <= FACT_ASN_TOP + 20; i++) {
|
||||
bumpFlowFact({
|
||||
serverId: 2,
|
||||
bucketAt: "2026-09-10T11:00:00.000Z",
|
||||
iface: "ether1",
|
||||
country: "DE",
|
||||
service: "https",
|
||||
asn: i,
|
||||
bytes: FACT_ASN_TOP + 21 - i,
|
||||
packets: 1,
|
||||
})
|
||||
}
|
||||
const cappedAsn = collectCappedFacts()
|
||||
const asns = new Set(cappedAsn.map((r) => r.asn))
|
||||
assert.ok(asns.has(0))
|
||||
assert.ok(asns.size <= FACT_ASN_TOP + 1)
|
||||
|
||||
resetFactsForTests()
|
||||
for (let i = 0; i < FACT_TUPLE_CAP + 30; i++) {
|
||||
bumpFlowFact({
|
||||
serverId: 3,
|
||||
bucketAt: "2026-09-10T12:00:00.000Z",
|
||||
iface: `ether${i % 3}`,
|
||||
country: "NL",
|
||||
service: `svc-${i}`,
|
||||
asn: 1,
|
||||
bytes: 10,
|
||||
packets: 1,
|
||||
})
|
||||
}
|
||||
const cappedTuples = collectCappedFacts()
|
||||
assert.ok(cappedTuples.length <= FACT_TUPLE_CAP + 3)
|
||||
|
||||
console.log("traffic-flow-facts.test.ts: ok")
|
||||
@@ -0,0 +1,289 @@
|
||||
import { pool } from "../db/index.js"
|
||||
import { ensurePartitionFor, specForParent } from "../db/partitions.js"
|
||||
|
||||
export const FACT_ASN_TOP = 200
|
||||
export const FACT_TUPLE_CAP = 8000
|
||||
export const FACT_SERVICE_MAX_LEN = 64
|
||||
export const UNKNOWN_COUNTRY = "XX"
|
||||
export const OTHER_SERVICE = "other"
|
||||
export const UNKNOWN_IFACE = "__unknown__"
|
||||
|
||||
export interface FactAcc {
|
||||
bytes: number
|
||||
packets: number
|
||||
}
|
||||
|
||||
export interface FactRow {
|
||||
serverId: number
|
||||
bucketAt: string
|
||||
iface: string
|
||||
country: string
|
||||
service: string
|
||||
asn: number
|
||||
bytes: number
|
||||
packets: number
|
||||
}
|
||||
|
||||
const hourFacts = new Map<string, FactAcc>()
|
||||
const ensuredParts = new Set<string>()
|
||||
|
||||
export function hourBucketIso(at = Date.now()): string {
|
||||
const d = new Date(at)
|
||||
d.setMinutes(0, 0, 0)
|
||||
return d.toISOString()
|
||||
}
|
||||
|
||||
export function normalizeFactCountry(raw: string): string {
|
||||
const iso = raw.trim().toUpperCase()
|
||||
if (/^[A-Z]{2}$/.test(iso)) return iso
|
||||
return UNKNOWN_COUNTRY
|
||||
}
|
||||
|
||||
export function normalizeFactService(raw: string): string {
|
||||
const s = raw.trim().slice(0, FACT_SERVICE_MAX_LEN)
|
||||
return s || OTHER_SERVICE
|
||||
}
|
||||
|
||||
export function normalizeFactIface(raw: string): string {
|
||||
return raw.trim() || UNKNOWN_IFACE
|
||||
}
|
||||
|
||||
export function normalizeFactAsn(raw: number): number {
|
||||
if (!Number.isFinite(raw) || raw <= 0) return 0
|
||||
return Math.trunc(raw)
|
||||
}
|
||||
|
||||
function factKey(
|
||||
serverId: number,
|
||||
bucketAt: string,
|
||||
iface: string,
|
||||
country: string,
|
||||
service: string,
|
||||
asn: number,
|
||||
): string {
|
||||
return `${serverId}\0${bucketAt}\0${iface}\0${country}\0${service}\0${asn}`
|
||||
}
|
||||
|
||||
function parseFactKey(k: string, acc: FactAcc): FactRow | null {
|
||||
const parts = k.split("\0")
|
||||
if (parts.length !== 6) return null
|
||||
const serverId = Number(parts[0])
|
||||
const asn = Number(parts[5])
|
||||
if (!Number.isFinite(serverId) || !Number.isFinite(asn)) return null
|
||||
return {
|
||||
serverId,
|
||||
bucketAt: parts[1] ?? "",
|
||||
iface: parts[2] ?? UNKNOWN_IFACE,
|
||||
country: parts[3] ?? UNKNOWN_COUNTRY,
|
||||
service: parts[4] ?? OTHER_SERVICE,
|
||||
asn,
|
||||
bytes: acc.bytes,
|
||||
packets: acc.packets,
|
||||
}
|
||||
}
|
||||
|
||||
export function bumpFlowFact(row: {
|
||||
serverId: number
|
||||
bucketAt: string
|
||||
iface: string
|
||||
country: string
|
||||
service: string
|
||||
asn: number
|
||||
bytes: number
|
||||
packets: number
|
||||
}): void {
|
||||
const iface = normalizeFactIface(row.iface)
|
||||
const country = normalizeFactCountry(row.country)
|
||||
const service = normalizeFactService(row.service)
|
||||
const asn = normalizeFactAsn(row.asn)
|
||||
const k = factKey(row.serverId, row.bucketAt, iface, country, service, asn)
|
||||
const prev = hourFacts.get(k)
|
||||
if (prev) {
|
||||
prev.bytes += row.bytes
|
||||
prev.packets += row.packets
|
||||
return
|
||||
}
|
||||
hourFacts.set(k, { bytes: row.bytes, packets: row.packets })
|
||||
}
|
||||
|
||||
function groupKey(row: FactRow): string {
|
||||
return `${row.serverId}\0${row.bucketAt}`
|
||||
}
|
||||
|
||||
function mergeRow(map: Map<string, FactRow>, row: FactRow): void {
|
||||
const k = factKey(row.serverId, row.bucketAt, row.iface, row.country, row.service, row.asn)
|
||||
const prev = map.get(k)
|
||||
if (prev) {
|
||||
prev.bytes += row.bytes
|
||||
prev.packets += row.packets
|
||||
return
|
||||
}
|
||||
map.set(k, { ...row })
|
||||
}
|
||||
|
||||
/** Cap ASN tail and tuple count per server×hour before persist. */
|
||||
export function collectCappedFacts(): FactRow[] {
|
||||
const parsed: FactRow[] = []
|
||||
for (const [k, acc] of hourFacts) {
|
||||
const row = parseFactKey(k, acc)
|
||||
if (row) parsed.push(row)
|
||||
}
|
||||
hourFacts.clear()
|
||||
|
||||
const groups = new Map<string, FactRow[]>()
|
||||
for (const row of parsed) {
|
||||
const g = groupKey(row)
|
||||
const list = groups.get(g) ?? []
|
||||
list.push(row)
|
||||
groups.set(g, list)
|
||||
}
|
||||
|
||||
const out = new Map<string, FactRow>()
|
||||
for (const list of groups.values()) {
|
||||
const byAsn = new Map<number, number>()
|
||||
for (const row of list) {
|
||||
byAsn.set(row.asn, (byAsn.get(row.asn) ?? 0) + row.bytes)
|
||||
}
|
||||
const asnKeep = new Set(
|
||||
[...byAsn.entries()]
|
||||
.sort((a, b) => b[1] - a[1])
|
||||
.slice(0, FACT_ASN_TOP)
|
||||
.map(([asn]) => asn),
|
||||
)
|
||||
const afterAsn: FactRow[] = []
|
||||
for (const row of list) {
|
||||
if (asnKeep.has(row.asn) || row.asn === 0) {
|
||||
afterAsn.push(row)
|
||||
continue
|
||||
}
|
||||
afterAsn.push({ ...row, asn: 0 })
|
||||
}
|
||||
const collapsed = new Map<string, FactRow>()
|
||||
for (const row of afterAsn) mergeRow(collapsed, row)
|
||||
const tuples = [...collapsed.values()].sort((a, b) => b.bytes - a.bytes)
|
||||
const keep = tuples.slice(0, FACT_TUPLE_CAP)
|
||||
const tail = tuples.slice(FACT_TUPLE_CAP)
|
||||
for (const row of keep) mergeRow(out, row)
|
||||
for (const row of tail) {
|
||||
mergeRow(out, {
|
||||
...row,
|
||||
country: UNKNOWN_COUNTRY,
|
||||
service: OTHER_SERVICE,
|
||||
asn: 0,
|
||||
})
|
||||
}
|
||||
}
|
||||
return [...out.values()]
|
||||
}
|
||||
|
||||
async function ensureParentPartition(parent: string, ts: string): Promise<void> {
|
||||
const spec = specForParent(parent)
|
||||
if (!spec) return
|
||||
const iso = ts.length === 10 ? `${ts}T00:00:00Z` : ts
|
||||
const key = `${parent}:${iso.slice(0, 10)}`
|
||||
if (ensuredParts.has(key)) return
|
||||
await ensurePartitionFor(pool, parent, spec.kind, new Date(iso))
|
||||
ensuredParts.add(key)
|
||||
}
|
||||
|
||||
function dayKey(bucketAt: string): string {
|
||||
return bucketAt.slice(0, 10)
|
||||
}
|
||||
|
||||
export async function flushFlowFacts(): Promise<number> {
|
||||
const rows = collectCappedFacts()
|
||||
if (rows.length === 0) return 0
|
||||
const hours = new Set(rows.map((r) => r.bucketAt))
|
||||
const days = new Set(rows.map((r) => dayKey(r.bucketAt)))
|
||||
for (const h of hours) await ensureParentPartition("flow_hour_facts", h)
|
||||
for (const d of days) await ensureParentPartition("flow_daily_facts", d)
|
||||
|
||||
await pool.query({
|
||||
text: `
|
||||
INSERT INTO flow_hour_facts (
|
||||
server_id, bucket_at, iface, country, service, asn, bytes, packets
|
||||
)
|
||||
SELECT *
|
||||
FROM UNNEST(
|
||||
$1::bigint[],
|
||||
$2::timestamptz[],
|
||||
$3::text[],
|
||||
$4::char(2)[],
|
||||
$5::text[],
|
||||
$6::int[],
|
||||
$7::bigint[],
|
||||
$8::bigint[]
|
||||
) AS t(server_id, bucket_at, iface, country, service, asn, bytes, packets)
|
||||
ON CONFLICT (server_id, bucket_at, iface, country, service, asn)
|
||||
DO UPDATE SET
|
||||
bytes = flow_hour_facts.bytes + excluded.bytes,
|
||||
packets = flow_hour_facts.packets + excluded.packets
|
||||
`,
|
||||
values: [
|
||||
rows.map((r) => r.serverId),
|
||||
rows.map((r) => r.bucketAt),
|
||||
rows.map((r) => r.iface),
|
||||
rows.map((r) => r.country),
|
||||
rows.map((r) => r.service),
|
||||
rows.map((r) => r.asn),
|
||||
rows.map((r) => r.bytes),
|
||||
rows.map((r) => r.packets),
|
||||
],
|
||||
})
|
||||
|
||||
await pool.query({
|
||||
text: `
|
||||
INSERT INTO flow_daily_facts (
|
||||
server_id, day, iface, country, service, asn, bytes, packets
|
||||
)
|
||||
SELECT *
|
||||
FROM UNNEST(
|
||||
$1::bigint[],
|
||||
$2::date[],
|
||||
$3::text[],
|
||||
$4::char(2)[],
|
||||
$5::text[],
|
||||
$6::int[],
|
||||
$7::bigint[],
|
||||
$8::bigint[]
|
||||
) AS t(server_id, day, iface, country, service, asn, bytes, packets)
|
||||
ON CONFLICT (server_id, day, iface, country, service, asn)
|
||||
DO UPDATE SET
|
||||
bytes = flow_daily_facts.bytes + excluded.bytes,
|
||||
packets = flow_daily_facts.packets + excluded.packets
|
||||
`,
|
||||
values: [
|
||||
rows.map((r) => r.serverId),
|
||||
rows.map((r) => dayKey(r.bucketAt)),
|
||||
rows.map((r) => r.iface),
|
||||
rows.map((r) => r.country),
|
||||
rows.map((r) => r.service),
|
||||
rows.map((r) => r.asn),
|
||||
rows.map((r) => r.bytes),
|
||||
rows.map((r) => r.packets),
|
||||
],
|
||||
})
|
||||
return rows.length
|
||||
}
|
||||
|
||||
export function factsPendingSize(): number {
|
||||
return hourFacts.size
|
||||
}
|
||||
|
||||
export function resetFactsForTests(): void {
|
||||
hourFacts.clear()
|
||||
ensuredParts.clear()
|
||||
}
|
||||
|
||||
export function discardPendingFacts(): void {
|
||||
hourFacts.clear()
|
||||
}
|
||||
|
||||
export function factsSnapshotForTests(): FactRow[] {
|
||||
const parsed: FactRow[] = []
|
||||
for (const [k, acc] of hourFacts) {
|
||||
const row = parseFactKey(k, acc)
|
||||
if (row) parsed.push(row)
|
||||
}
|
||||
return parsed
|
||||
}
|
||||
@@ -21,6 +21,7 @@ import {
|
||||
minuteDimsSnapshotForTests,
|
||||
} from "./traffic-flow-engine.js"
|
||||
import { classifyFlowDst } from "./traffic-flow-classify.js"
|
||||
import { mapInternetBrand } from "./traffic-flow-dest.js"
|
||||
import { disableGeoipDbForTests } from "./geoip-settings.js"
|
||||
import {
|
||||
collectGeoipUpdateOnce,
|
||||
@@ -89,6 +90,7 @@ setGeoipReadersForTests({
|
||||
const hit = resolveFlowIp("8.8.8.8")
|
||||
assert.equal(hit?.country, "US")
|
||||
assert.equal(hit?.asn, 15169)
|
||||
assert.equal(resolveFlowIp("8.8.8.8/32")?.asn, 15169, "GeoIP по inet::text /32")
|
||||
assert.equal(hit?.holder, "GOOGLE")
|
||||
assert.equal(hit?.ok, true)
|
||||
|
||||
@@ -102,6 +104,21 @@ assert.equal(lookupGeoip("6.6.6.6")?.country, "US")
|
||||
const classified = classifyFlowDst("8.8.8.8", 6, 443, 51504, hit)
|
||||
assert.equal(classified.service, "Google")
|
||||
|
||||
setGeoipReadersForTests({
|
||||
country: fakeCountryReader({ "8.8.8.8": "US", "2001:4860:4860::8888": "US" }),
|
||||
asn: fakeAsnReader({
|
||||
"8.8.8.8": { asn: 15169, org: "GOOGLE" },
|
||||
"2001:4860:4860::8888": { asn: 15169, org: "GOOGLE" },
|
||||
"64.233.161.1": { asn: 15169, org: "GOOGLE" },
|
||||
}),
|
||||
})
|
||||
const v6meta = resolveFlowIp("2001:4860:4860::8888")
|
||||
assert.equal(v6meta?.asn, 15169)
|
||||
assert.equal(mapInternetBrand("2001:4860:4860::8888", 17, 443, 50000, v6meta).service, "YouTube")
|
||||
assert.notEqual(mapInternetBrand("2001:4860:4860::8888", 17, 443, 50000, v6meta).service, "Прочее")
|
||||
const cidrYt = mapInternetBrand("64.233.161.1", 17, 443, 50000, resolveFlowIp("64.233.161.1"))
|
||||
assert.equal(cidrYt.service, "YouTube")
|
||||
|
||||
// ── движок: dims country/asn наполняются из geoip-ридеров ────────────────────
|
||||
resetEngineForTests()
|
||||
ingestParsedFlowsForServerForTests(1, [{
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
import { existsSync } from "node:fs"
|
||||
import path from "node:path"
|
||||
import { open, type AsnResponse, type CountryResponse, type Reader } from "maxmind"
|
||||
import { isNonPublicIp } from "./traffic-flow-ip.js"
|
||||
import { canonicalIp, isNonPublicIp } from "./traffic-flow-ip.js"
|
||||
import { isIsoCountry, resolveRipeCountry } from "./traffic-flow-brands.js"
|
||||
import { lookupRipeCached, type FlowIpMeta } from "./traffic-flow-ripe.js"
|
||||
|
||||
@@ -117,7 +117,7 @@ function safeAsn(reader: Reader<AsnResponse>, ip: string): { asn: number; holder
|
||||
* (ok=true когда есть страна или ASN; null — данных нет, пусть пробует RIPE).
|
||||
*/
|
||||
export function lookupGeoip(ip: string): FlowIpMeta | null {
|
||||
const trimmed = String(ip ?? "").trim()
|
||||
const trimmed = canonicalIp(ip)
|
||||
if (!trimmed) return null
|
||||
if (isNonPublicIp(trimmed)) return negativeMeta(trimmed)
|
||||
const { country: countryReader, asn: asnReader } = readers
|
||||
|
||||
@@ -22,8 +22,9 @@ rememberServerIfaces(7, [
|
||||
{ ".id": "*A", name: "wg-flow" },
|
||||
{ ".id": "*D", name: "bridge" },
|
||||
])
|
||||
assert.equal(resolveIfaceName(7, "2").name, "ether1")
|
||||
assert.equal(resolveIfaceName(7, "10").name, "wg-flow")
|
||||
assert.equal(resolveIfaceName(7, "2").name, "ether1")
|
||||
assert.equal(resolveIfaceName(7, "#2").name, "ether1")
|
||||
assert.equal(resolveIfaceName(7, "10").name, "wg-flow")
|
||||
assert.equal(resolveIfaceName(7, "13").name, "bridge")
|
||||
assert.equal(resolveIfaceName(7, "0").name, "—")
|
||||
assert.equal(resolveIfaceName(7, "ether1").name, "ether1")
|
||||
|
||||
@@ -0,0 +1,58 @@
|
||||
import assert from "node:assert/strict"
|
||||
import {
|
||||
bindingIfaceAliases,
|
||||
bindingIfaceAliasesAllServers,
|
||||
canonicalFactIface,
|
||||
collapseServerIfaceRows,
|
||||
displayFactIface,
|
||||
expandBindingIfaces,
|
||||
factIfaceAliases,
|
||||
rememberServerIfaces,
|
||||
resetIfaceCacheForTests,
|
||||
resolveIfaceName,
|
||||
} from "./traffic-flow-ifindex.js"
|
||||
|
||||
resetIfaceCacheForTests()
|
||||
assert.equal(canonicalFactIface(1, "2"), "2")
|
||||
assert.deepEqual(bindingIfaceAliases(1, "gre-client"), ["gre-client"])
|
||||
|
||||
rememberServerIfaces(1, [{ name: "gre-client", ifindex: "2" }])
|
||||
assert.equal(canonicalFactIface(1, "2"), "gre-client")
|
||||
assert.equal(canonicalFactIface(1, "gre-client"), "gre-client")
|
||||
assert.equal(canonicalFactIface(1, "9"), "9")
|
||||
assert.equal(resolveIfaceName(1, "9").name, "#9")
|
||||
assert.equal(resolveIfaceName(1, "2").name, "gre-client")
|
||||
assert.equal(resolveIfaceName(1, "#2").name, "gre-client")
|
||||
assert.equal(displayFactIface(1, "2"), "gre-client")
|
||||
|
||||
const aliases = bindingIfaceAliases(1, "gre-client")
|
||||
assert.ok(aliases.includes("gre-client"))
|
||||
assert.ok(aliases.includes("2"))
|
||||
assert.ok(aliases.includes("#2"))
|
||||
|
||||
const fromIndex = factIfaceAliases("2", 1)
|
||||
assert.ok(fromIndex.includes("gre-client"))
|
||||
assert.ok(fromIndex.includes("2"))
|
||||
assert.ok(fromIndex.includes("#2"))
|
||||
|
||||
const all = bindingIfaceAliasesAllServers("gre-client")
|
||||
assert.ok(all.includes("2"))
|
||||
|
||||
const expanded = expandBindingIfaces([{ serverId: 1, iface: "gre-client" }])
|
||||
assert.ok(expanded.some((x) => x.iface === "2"))
|
||||
assert.ok(expanded.some((x) => x.iface === "gre-client"))
|
||||
|
||||
const collapsed = collapseServerIfaceRows([
|
||||
{ serverId: 1, iface: "2", bytes: 10, packets: 1 },
|
||||
{ serverId: 1, iface: "gre-client", bytes: 5, packets: 2 },
|
||||
{ serverId: 1, iface: "wan1", bytes: 3, packets: 1 },
|
||||
])
|
||||
assert.equal(collapsed.length, 2)
|
||||
const gre = collapsed.find((r) => r.iface === "gre-client")
|
||||
assert.ok(gre)
|
||||
assert.equal(gre.bytes, 15)
|
||||
assert.equal(gre.packets, 3)
|
||||
assert.ok(collapsed.some((r) => r.iface === "wan1"))
|
||||
|
||||
resetIfaceCacheForTests()
|
||||
console.log("traffic-flow-ifindex.test.ts: ok")
|
||||
@@ -36,12 +36,135 @@ export function rememberServerIfaces(serverId: number, rows: RosIfaceIndexRow[])
|
||||
|
||||
export function resolveIfaceName(serverId: number, indexOrName: string): { name: string; index: string } {
|
||||
const trimmed = String(indexOrName ?? "").trim()
|
||||
if (!trimmed || trimmed === "0") return { name: "—", index: trimmed }
|
||||
if (!/^\d+$/.test(trimmed)) return { name: trimmed, index: "" }
|
||||
const idx = Number(trimmed)
|
||||
const name = cache.get(serverId)?.get(idx)
|
||||
if (name) return { name, index: trimmed }
|
||||
return { name: `#${trimmed}`, index: trimmed }
|
||||
const asIndex = trimmed.startsWith("#") && /^\d+$/.test(trimmed.slice(1)) ? trimmed.slice(1) : trimmed
|
||||
if (!asIndex || asIndex === "0") return { name: "—", index: asIndex }
|
||||
if (!/^\d+$/.test(asIndex)) return { name: trimmed, index: "" }
|
||||
const name = cache.get(serverId)?.get(Number(asIndex))
|
||||
if (name) return { name, index: asIndex }
|
||||
return { name: `#${asIndex}`, index: asIndex }
|
||||
}
|
||||
|
||||
/** Имя iface для факта куба: ifIndex→имя, без `#13` при пустом кэше. */
|
||||
export function canonicalFactIface(serverId: number, inIface: string): string {
|
||||
const trimmed = String(inIface ?? "").trim()
|
||||
if (!trimmed) return trimmed
|
||||
if (!/^\d+$/.test(trimmed)) return trimmed
|
||||
const name = cache.get(serverId)?.get(Number(trimmed))
|
||||
return name || trimmed
|
||||
}
|
||||
|
||||
function numericIfaceIndex(iface: string): string | null {
|
||||
const raw = String(iface ?? "").trim()
|
||||
if (/^\d+$/.test(raw)) return raw
|
||||
if (raw.startsWith("#") && /^\d+$/.test(raw.slice(1))) return raw.slice(1)
|
||||
return null
|
||||
}
|
||||
|
||||
/** Имя для UI: ifIndex → RouterOS name; `0` → «—»; miss → `#n`. */
|
||||
export function displayFactIface(serverId: number, iface: string): string {
|
||||
return resolveIfaceName(serverId, iface).name
|
||||
}
|
||||
|
||||
/** Склеить факты `2` + `ether1` в одну строку после резолва ifIndex. */
|
||||
export function collapseServerIfaceRows(
|
||||
rows: Array<{ serverId: number; iface: string; bytes: number; packets: number }>,
|
||||
): Array<{ serverId: number; iface: string; bytes: number; packets: number }> {
|
||||
const acc = new Map<string, { serverId: number; iface: string; bytes: number; packets: number }>()
|
||||
for (const r of rows) {
|
||||
const name = displayFactIface(r.serverId, r.iface)
|
||||
const k = `${r.serverId}\0${name}`
|
||||
const prev = acc.get(k)
|
||||
const bytes = Number(r.bytes) || 0
|
||||
const packets = Number(r.packets) || 0
|
||||
if (prev) {
|
||||
prev.bytes += bytes
|
||||
prev.packets += packets
|
||||
} else {
|
||||
acc.set(k, { serverId: r.serverId, iface: name, bytes, packets })
|
||||
}
|
||||
}
|
||||
return [...acc.values()]
|
||||
}
|
||||
|
||||
/** Ключи факта для фильтра: имя, ifIndex и `#n`. */
|
||||
export function factIfaceAliases(iface: string, serverId?: number): string[] {
|
||||
const raw = String(iface ?? "").trim()
|
||||
if (!raw) return []
|
||||
const out = new Set<string>([raw])
|
||||
const idx = numericIfaceIndex(raw)
|
||||
if (idx) {
|
||||
out.add(idx)
|
||||
out.add(`#${idx}`)
|
||||
const n = Number(idx)
|
||||
if (serverId != null) {
|
||||
const name = cache.get(serverId)?.get(n)
|
||||
if (name) out.add(name)
|
||||
} else {
|
||||
for (const map of cache.values()) {
|
||||
const name = map.get(n)
|
||||
if (name) out.add(name)
|
||||
}
|
||||
}
|
||||
}
|
||||
if (serverId != null) {
|
||||
for (const a of bindingIfaceAliases(serverId, raw)) out.add(a)
|
||||
} else {
|
||||
for (const a of bindingIfaceAliasesAllServers(raw)) out.add(a)
|
||||
}
|
||||
return [...out]
|
||||
}
|
||||
|
||||
/** Имя + ifIndex + `#n` — тот же матч, что карта `/traffic`. */
|
||||
export function bindingIfaceAliases(serverId: number, interfaceName: string): string[] {
|
||||
const name = String(interfaceName ?? "").trim()
|
||||
if (!name) return []
|
||||
const out = new Set<string>([name])
|
||||
const map = cache.get(serverId)
|
||||
const idx = numericIfaceIndex(name)
|
||||
const canonical = (idx && map?.get(Number(idx))) || name
|
||||
out.add(canonical)
|
||||
if (idx) {
|
||||
out.add(idx)
|
||||
out.add(`#${idx}`)
|
||||
}
|
||||
if (!map) return [...out]
|
||||
for (const [i, n] of map) {
|
||||
if (n !== canonical && n !== name) continue
|
||||
out.add(String(i))
|
||||
out.add(`#${i}`)
|
||||
}
|
||||
return [...out]
|
||||
}
|
||||
|
||||
export function bindingIfaceAliasesAllServers(interfaceName: string): string[] {
|
||||
const name = String(interfaceName ?? "").trim()
|
||||
const out = new Set<string>(name ? [name] : [])
|
||||
for (const serverId of cache.keys()) {
|
||||
for (const alias of bindingIfaceAliases(serverId, name)) out.add(alias)
|
||||
}
|
||||
return [...out]
|
||||
}
|
||||
|
||||
export function expandBindingIfaces(
|
||||
binds: Array<{ serverId: number; iface: string }>,
|
||||
): Array<{ serverId: number; iface: string }> {
|
||||
const seen = new Set<string>()
|
||||
const out: Array<{ serverId: number; iface: string }> = []
|
||||
for (const b of binds) {
|
||||
for (const iface of bindingIfaceAliases(b.serverId, b.iface)) {
|
||||
const k = `${b.serverId}\0${iface}`
|
||||
if (seen.has(k)) continue
|
||||
seen.add(k)
|
||||
out.push({ serverId: b.serverId, iface })
|
||||
}
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
export function listCachedIfaceNames(serverId: number): string[] {
|
||||
const map = cache.get(serverId)
|
||||
if (!map) return []
|
||||
return [...new Set(map.values())]
|
||||
}
|
||||
|
||||
export function ifaceCacheHas(serverId: number): boolean {
|
||||
|
||||
@@ -15,6 +15,7 @@ import {
|
||||
lastFlushUsedTransactionForTests,
|
||||
maybeRefreshIfaces,
|
||||
peekPendingFlows,
|
||||
capFlowRowsPerServerBucket,
|
||||
resetFlowRingsForTests,
|
||||
setPendingCapForTests,
|
||||
setRefreshIfacesForTests,
|
||||
@@ -71,6 +72,7 @@ setPendingCapForTests(null)
|
||||
|
||||
assert.equal(isValidFlowInet("10.0.0.1"), true)
|
||||
assert.equal(isValidFlowInet("8.8.8.8"), true)
|
||||
assert.equal(isValidFlowInet("8.8.8.8/32"), true)
|
||||
assert.equal(isValidFlowInet("0:0:0:0:0:0:0:1"), true)
|
||||
assert.equal(isValidFlowInet("not-an-ip"), false)
|
||||
assert.equal(isValidFlowInet("999.1.1.1"), false)
|
||||
@@ -146,4 +148,20 @@ resetFlowRingsForTests()
|
||||
resetIfaceCacheForTests()
|
||||
setRefreshIfacesForTests(null)
|
||||
|
||||
{
|
||||
const minute0 = "2026-01-01T00:00:00.000Z"
|
||||
const minute1 = "2026-01-01T00:01:00.000Z"
|
||||
const rows: Array<{ serverId: number; bucketAt: string; bytes: number; id: string }> = []
|
||||
for (let i = 1; i <= 25; i++) {
|
||||
rows.push({ serverId: 1, bucketAt: minute0, bytes: i, id: `a${i}` })
|
||||
rows.push({ serverId: 2, bucketAt: minute0, bytes: i, id: `b${i}` })
|
||||
}
|
||||
rows.push({ serverId: 1, bucketAt: minute1, bytes: 1, id: "a-min-other-minute" })
|
||||
const capped = capFlowRowsPerServerBucket(rows, 20)
|
||||
assert.equal(capped.filter((r) => r.serverId === 1 && r.bucketAt === minute0).length, 20)
|
||||
assert.equal(capped.filter((r) => r.serverId === 2).length, 20)
|
||||
assert.ok(capped.some((r) => r.id === "a-min-other-minute"), "другая минута не режется глобальным top-N")
|
||||
assert.ok(!capped.some((r) => r.id === "a1" || r.id === "b1"), "мелкие 5-tuple сервера выпадают только в своём bucket")
|
||||
}
|
||||
|
||||
console.log("traffic-flow-ingest.test.ts: ok")
|
||||
|
||||
@@ -1,8 +1,7 @@
|
||||
import { Worker } from "node:worker_threads"
|
||||
import { gte, sql } from "drizzle-orm"
|
||||
import { db, dbGet, dbQuery, pool, withAdvisoryLock } from "../db/index.js"
|
||||
import { db, dbAll, dbGet, dbQuery, pool, withAdvisoryLock } from "../db/index.js"
|
||||
import { dropExpiredPartitions } from "../db/partitions.js"
|
||||
import { flowBuckets, servers } from "../db/schema.js"
|
||||
import { servers } from "../db/schema.js"
|
||||
import type { FlowPurgeDto, FlowStatsDto, FlowTalkerDto } from "@mmapp/contracts/traffic-flow"
|
||||
import { protoName, type ParsedFlowInput } from "./traffic-flow-parse.js"
|
||||
import type { CollectorHeartbeat, ExporterMapPayload, MainToWorker, WorkerToMain } from "./traffic-flow-collector-ipc.js"
|
||||
@@ -30,6 +29,7 @@ import {
|
||||
} from "./traffic-flow-settings.js"
|
||||
import { applicationName } from "./traffic-flow-apps.js"
|
||||
import { resolveIfaceName } from "./traffic-flow-ifaces.js"
|
||||
import { canonicalIp } from "./traffic-flow-ip.js"
|
||||
import { getServerCatalog } from "./traffic-flow-topology.js"
|
||||
|
||||
export type { PendingFlowRow }
|
||||
@@ -288,6 +288,10 @@ function mergeInto(map: Map<string, PendingFlowRow>, row: PendingFlowRow): void
|
||||
prev.packets += row.packets
|
||||
if (row.outIface && !prev.outIface) prev.outIface = row.outIface
|
||||
if (row.nextHop && !prev.nextHop) prev.nextHop = row.nextHop
|
||||
if (row.natSrc && !prev.natSrc) prev.natSrc = row.natSrc
|
||||
if (row.natDst && !prev.natDst) prev.natDst = row.natDst
|
||||
if (row.natSrcPort && !prev.natSrcPort) prev.natSrcPort = row.natSrcPort
|
||||
if (row.natDstPort && !prev.natDstPort) prev.natDstPort = row.natDstPort
|
||||
if (row.flowStartMs && (!prev.flowStartMs || row.flowStartMs < prev.flowStartMs)) prev.flowStartMs = row.flowStartMs
|
||||
if (row.flowEndMs > (prev.flowEndMs ?? 0)) prev.flowEndMs = row.flowEndMs
|
||||
return
|
||||
@@ -295,6 +299,32 @@ function mergeInto(map: Map<string, PendingFlowRow>, row: PendingFlowRow): void
|
||||
map.set(key, { ...row })
|
||||
}
|
||||
|
||||
/** Top-N разговоров на (server_id, bucket_at), как prune persist — не глобальный ORDER BY bytes. */
|
||||
export function capFlowRowsPerServerBucket<T extends { serverId: number; bucketAt: string; bytes: number }>(
|
||||
rows: T[],
|
||||
keep: number,
|
||||
): T[] {
|
||||
const cap = Math.max(20, keep)
|
||||
const groups = new Map<string, T[]>()
|
||||
for (const row of rows) {
|
||||
const k = `${row.serverId}\0${row.bucketAt}`
|
||||
const list = groups.get(k)
|
||||
if (list) list.push(row)
|
||||
else groups.set(k, [row])
|
||||
}
|
||||
const out: T[] = []
|
||||
for (const list of groups.values()) {
|
||||
list.sort((a, b) => b.bytes - a.bytes)
|
||||
out.push(...list.slice(0, cap))
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
function isoBucketAt(v: unknown): string {
|
||||
if (v instanceof Date) return v.toISOString()
|
||||
return String(v ?? "")
|
||||
}
|
||||
|
||||
export async function listLiveFlowRows(sinceIso: string): Promise<PendingFlowRow[]> {
|
||||
if (worker && lastHeartbeat?.workerAlive) {
|
||||
return await listStoredFlowRows(sinceIso)
|
||||
@@ -302,30 +332,67 @@ export async function listLiveFlowRows(sinceIso: string): Promise<PendingFlowRow
|
||||
return engineListLive(sinceIso)
|
||||
}
|
||||
|
||||
interface StoredBucketRow {
|
||||
server_id: number
|
||||
bucket_at: string | Date
|
||||
src: string
|
||||
dst: string
|
||||
proto: number
|
||||
src_port: number
|
||||
dst_port: number
|
||||
bytes: number
|
||||
packets: number
|
||||
in_iface: string
|
||||
out_iface: string | null
|
||||
next_hop: string | null
|
||||
flow_start_ms: number | null
|
||||
flow_end_ms: number | null
|
||||
nat_src: string | null
|
||||
nat_dst: string | null
|
||||
nat_src_port: number | null
|
||||
nat_dst_port: number | null
|
||||
}
|
||||
|
||||
export async function listStoredFlowRows(sinceIso: string): Promise<PendingFlowRow[]> {
|
||||
const settings = await getTrafficFlowSettingsRow()
|
||||
const cap = Math.max(20, settings.topN) * 60
|
||||
const stored = await db.select().from(flowBuckets)
|
||||
.where(gte(flowBuckets.bucketAt, sinceIso))
|
||||
.orderBy(sql`${flowBuckets.bytes} DESC`)
|
||||
.limit(cap)
|
||||
const keep = Math.max(20, settings.topN)
|
||||
const stored = await dbAll<StoredBucketRow>(`
|
||||
SELECT
|
||||
server_id, bucket_at, COALESCE(host(src), '') AS src, COALESCE(host(dst), '') AS dst, proto,
|
||||
src_port, dst_port, bytes, packets, in_iface, out_iface,
|
||||
COALESCE(host(next_hop), '') AS next_hop, flow_start_ms, flow_end_ms,
|
||||
COALESCE(host(nat_src), '') AS nat_src, COALESCE(host(nat_dst), '') AS nat_dst, nat_src_port, nat_dst_port
|
||||
FROM (
|
||||
SELECT fb.*,
|
||||
ROW_NUMBER() OVER (
|
||||
PARTITION BY server_id, bucket_at ORDER BY bytes DESC
|
||||
) AS rn
|
||||
FROM flow_buckets fb
|
||||
WHERE bucket_at >= $1
|
||||
) ranked
|
||||
WHERE rn <= $2
|
||||
`, [sinceIso, keep])
|
||||
const merged = new Map<string, PendingFlowRow>()
|
||||
for (const r of stored) {
|
||||
mergeInto(merged, {
|
||||
serverId: r.serverId,
|
||||
bucketAt: r.bucketAt,
|
||||
src: r.src,
|
||||
dst: r.dst,
|
||||
proto: r.proto,
|
||||
srcPort: r.srcPort,
|
||||
dstPort: r.dstPort,
|
||||
bytes: r.bytes,
|
||||
packets: r.packets,
|
||||
inIface: r.inIface,
|
||||
outIface: r.outIface ?? "",
|
||||
nextHop: r.nextHop ?? "",
|
||||
flowStartMs: r.flowStartMs ?? 0,
|
||||
flowEndMs: r.flowEndMs ?? 0,
|
||||
serverId: Number(r.server_id),
|
||||
bucketAt: isoBucketAt(r.bucket_at),
|
||||
src: canonicalIp(r.src),
|
||||
dst: canonicalIp(r.dst),
|
||||
proto: Number(r.proto) || 0,
|
||||
srcPort: Number(r.src_port) || 0,
|
||||
dstPort: Number(r.dst_port) || 0,
|
||||
bytes: Number(r.bytes) || 0,
|
||||
packets: Number(r.packets) || 0,
|
||||
inIface: r.in_iface ?? "",
|
||||
outIface: r.out_iface ?? "",
|
||||
nextHop: canonicalIp(r.next_hop ?? ""),
|
||||
flowStartMs: Number(r.flow_start_ms) || 0,
|
||||
flowEndMs: Number(r.flow_end_ms) || 0,
|
||||
natSrc: canonicalIp(r.nat_src ?? ""),
|
||||
natDst: canonicalIp(r.nat_dst ?? ""),
|
||||
natSrcPort: Number(r.nat_src_port) || 0,
|
||||
natDstPort: Number(r.nat_dst_port) || 0,
|
||||
})
|
||||
}
|
||||
if (!worker) {
|
||||
@@ -334,7 +401,7 @@ export async function listStoredFlowRows(sinceIso: string): Promise<PendingFlowR
|
||||
mergeInto(merged, p)
|
||||
}
|
||||
}
|
||||
return [...merged.values()]
|
||||
return capFlowRowsPerServerBucket([...merged.values()], keep)
|
||||
}
|
||||
|
||||
export async function listFlowRowsForWindow(minutes: number): Promise<PendingFlowRow[]> {
|
||||
@@ -467,11 +534,15 @@ export async function purgeTrafficFlowStore(): Promise<FlowPurgeDto> {
|
||||
minuteStats: await tableCount("flow_minute_stats"),
|
||||
minuteDims: await tableCount("flow_minute_dims"),
|
||||
dailyDims: await tableCount("flow_daily_dims"),
|
||||
hourFacts: await tableCount("flow_hour_facts"),
|
||||
dailyFacts: await tableCount("flow_daily_facts"),
|
||||
}
|
||||
await dbQuery(`DELETE FROM flow_buckets`)
|
||||
await dbQuery(`DELETE FROM flow_minute_stats`)
|
||||
await dbQuery(`DELETE FROM flow_minute_dims`)
|
||||
await dbQuery(`DELETE FROM flow_daily_dims`)
|
||||
await dbQuery(`DELETE FROM flow_hour_facts`)
|
||||
await dbQuery(`DELETE FROM flow_daily_facts`)
|
||||
await resetFlowIngestCounters()
|
||||
await dropExpiredPartitions(pool)
|
||||
try {
|
||||
|
||||
@@ -1,14 +1,33 @@
|
||||
import assert from "node:assert/strict"
|
||||
import { isNonPublicIp, pickInternetPeer } from "./traffic-flow-ip.js"
|
||||
import {
|
||||
canonicalIp,
|
||||
isNonPublicIp,
|
||||
pickInternetDest,
|
||||
pickInternetPeer,
|
||||
pickMapInternetDest,
|
||||
resolveFlowEndpoints,
|
||||
} from "./traffic-flow-ip.js"
|
||||
|
||||
assert.equal(canonicalIp("74.125.104.196/32"), "74.125.104.196")
|
||||
assert.equal(canonicalIp("10.200.100.53/32"), "10.200.100.53")
|
||||
assert.equal(canonicalIp("::ffff:8.8.8.8"), "8.8.8.8")
|
||||
assert.equal(canonicalIp("2001:4860:4860::8888/128"), "2001:4860:4860::8888")
|
||||
|
||||
assert.equal(isNonPublicIp("10.200.100.53"), true)
|
||||
assert.equal(isNonPublicIp("10.200.100.53/32"), true)
|
||||
assert.equal(isNonPublicIp("173.194.151.65"), false)
|
||||
assert.equal(isNonPublicIp("74.125.104.196/32"), false, "PG inet::text не делает Google приватным")
|
||||
|
||||
assert.equal(
|
||||
pickInternetPeer("173.194.151.65", "10.200.100.53", 443, 57182),
|
||||
"173.194.151.65",
|
||||
"reverse IPFIX: Google:443 → RFC1918",
|
||||
)
|
||||
assert.equal(
|
||||
pickInternetPeer("74.125.104.196/32", "10.200.100.53/32", 443, 62598),
|
||||
"74.125.104.196",
|
||||
"inet::text /32 reverse Google",
|
||||
)
|
||||
assert.equal(
|
||||
pickInternetPeer("10.200.100.53", "104.18.35.51", 53880, 443),
|
||||
"104.18.35.51",
|
||||
@@ -22,4 +41,118 @@ assert.equal(
|
||||
)
|
||||
assert.equal(pickInternetPeer("10.1.1.1", "10.2.2.2", 443, 80), "10.2.2.2")
|
||||
|
||||
const rost = "95.167.1.10"
|
||||
const ours = new Set(["198.51.100.1", "203.0.113.10"])
|
||||
const client = { ours, boundClient: true }
|
||||
|
||||
assert.equal(
|
||||
pickInternetDest("10.200.100.53", "104.18.35.51", 53880, 443, client),
|
||||
"104.18.35.51",
|
||||
"RFC1918 → CF на client GRE",
|
||||
)
|
||||
assert.equal(
|
||||
pickInternetDest("173.194.151.65", "10.200.100.53", 443, 57182, client),
|
||||
"173.194.151.65",
|
||||
"Google:443 → RFC1918 на client GRE",
|
||||
)
|
||||
assert.equal(
|
||||
pickInternetDest(rost, "10.200.100.53", 51234, 443, client),
|
||||
"",
|
||||
"Rostelecom → overlay 10.x: не dest ASN клиента",
|
||||
)
|
||||
assert.equal(
|
||||
pickInternetDest(rost, "8.8.8.8", 51234, 443, client),
|
||||
"8.8.8.8",
|
||||
"Rostelecom → Google:443 на client GRE",
|
||||
)
|
||||
assert.equal(
|
||||
pickInternetDest(rost, "1.1.1.1", 51234, 40000, client),
|
||||
"1.1.1.1",
|
||||
"оба публичные без well-known на client GRE → dst",
|
||||
)
|
||||
assert.equal(
|
||||
pickInternetDest("8.8.8.8", "198.51.100.1", 443, 51234, { ours }),
|
||||
"8.8.8.8",
|
||||
"ours как dst: dest = публичный src",
|
||||
)
|
||||
assert.equal(
|
||||
pickInternetDest("203.0.113.10", "198.51.100.1", 0, 0, { ours }),
|
||||
"",
|
||||
"JH ours → EN ours: dest нет",
|
||||
)
|
||||
assert.equal(
|
||||
pickInternetDest("10.200.100.53", "10.200.100.1", 53880, 443, {
|
||||
...client,
|
||||
natDst: "8.8.8.8",
|
||||
natDstPort: 443,
|
||||
}),
|
||||
"8.8.8.8",
|
||||
"RFC1918 + NAT Google",
|
||||
)
|
||||
assert.equal(
|
||||
pickInternetDest("10.200.100.53", "10.200.100.1", 53880, 443, {
|
||||
...client,
|
||||
natDst: "0.0.0.0",
|
||||
}),
|
||||
"",
|
||||
"NAT 0.0.0.0 не dest",
|
||||
)
|
||||
assert.equal(
|
||||
pickMapInternetDest("10.200.100.53", "10.200.100.1", 53880, 443, {
|
||||
...client,
|
||||
natDst: "8.8.8.8",
|
||||
natDstPort: 443,
|
||||
}),
|
||||
"8.8.8.8",
|
||||
"карта: RFC1918 + NAT Google",
|
||||
)
|
||||
assert.equal(
|
||||
pickMapInternetDest("10.200.100.53", "10.200.100.1", 53880, 443, client),
|
||||
"",
|
||||
"карта: RFC1918 без NAT → Прочее",
|
||||
)
|
||||
assert.equal(
|
||||
pickInternetDest("173.194.151.65", "10.200.100.53", 12345, 57182, client),
|
||||
"173.194.151.65",
|
||||
"реверс googlevideo не :443 — публичный src",
|
||||
)
|
||||
assert.equal(
|
||||
pickMapInternetDest("173.194.151.65", "10.200.100.53", 12345, 57182, client),
|
||||
"173.194.151.65",
|
||||
"карта: реверс googlevideo не :443 — всё равно публичный src",
|
||||
)
|
||||
assert.equal(
|
||||
pickMapInternetDest("203.0.113.10", "198.51.100.1", 0, 0, { ours }),
|
||||
"",
|
||||
"карта: JH ours → EN ours всё ещё не dest",
|
||||
)
|
||||
|
||||
{
|
||||
const ep = resolveFlowEndpoints({
|
||||
src: "74.125.104.196/32",
|
||||
dst: "10.200.100.53/32",
|
||||
srcPort: 443,
|
||||
dstPort: 62598,
|
||||
})
|
||||
assert.equal(ep.internetPeer, "74.125.104.196")
|
||||
assert.equal(ep.peerPort, 443)
|
||||
assert.equal(ep.clientIp, "10.200.100.53")
|
||||
assert.equal(ep.direction, "to_client")
|
||||
assert.equal(ep.packetSrc, "74.125.104.196")
|
||||
assert.equal(ep.packetDst, "10.200.100.53")
|
||||
}
|
||||
|
||||
{
|
||||
const ep = resolveFlowEndpoints({
|
||||
src: "10.200.100.53",
|
||||
dst: "104.18.35.51",
|
||||
srcPort: 53880,
|
||||
dstPort: 443,
|
||||
})
|
||||
assert.equal(ep.internetPeer, "104.18.35.51")
|
||||
assert.equal(ep.peerPort, 443)
|
||||
assert.equal(ep.clientIp, "10.200.100.53")
|
||||
assert.equal(ep.direction, "from_client")
|
||||
}
|
||||
|
||||
console.log("traffic-flow-ip.test.ts: ok")
|
||||
|
||||
@@ -1,7 +1,25 @@
|
||||
/** IPv4 helpers for RIPEstat prefix cache and EvoBGP CIDR match. */
|
||||
|
||||
/**
|
||||
* Host-семантика PostgreSQL `host(inet)`: снимает `/32` `/128`, `::ffff:`.
|
||||
* IPFIX 5-tuple не меняем — только канонический вид адреса.
|
||||
*/
|
||||
export function canonicalIp(raw: string | undefined | null): string {
|
||||
let t = String(raw ?? "").trim()
|
||||
if (!t) return ""
|
||||
const zone = t.indexOf("%")
|
||||
if (zone >= 0) t = t.slice(0, zone)
|
||||
if (t.toLowerCase().startsWith("::ffff:")) t = t.slice(7)
|
||||
const slash = t.lastIndexOf("/")
|
||||
if (slash >= 0) {
|
||||
const plen = t.slice(slash + 1)
|
||||
if (/^\d+$/.test(plen)) t = t.slice(0, slash)
|
||||
}
|
||||
return t.trim()
|
||||
}
|
||||
|
||||
export function ipv4ToInt(ip: string): number | null {
|
||||
const parts = String(ip ?? "").trim().split(".")
|
||||
const parts = canonicalIp(ip).split(".")
|
||||
if (parts.length !== 4) return null
|
||||
let n = 0
|
||||
for (const p of parts) {
|
||||
@@ -26,12 +44,12 @@ export function parseCidrV4(cidr: string): { net: number; mask: number; prefixLe
|
||||
export function ipInCidrV4(ip: string, cidr: string): boolean {
|
||||
const addr = ipv4ToInt(ip)
|
||||
const parsed = parseCidrV4(cidr)
|
||||
if (addr == null || !parsed) return false
|
||||
if (addr == null || parsed == null) return false
|
||||
return ((addr & parsed.mask) >>> 0) === parsed.net
|
||||
}
|
||||
|
||||
export function isNonPublicIp(ip: string): boolean {
|
||||
const trimmed = String(ip ?? "").trim()
|
||||
const trimmed = canonicalIp(ip)
|
||||
if (!trimmed) return true
|
||||
if (trimmed.includes(":")) {
|
||||
const lower = trimmed.toLowerCase()
|
||||
@@ -55,20 +73,179 @@ export function isNonPublicIp(ip: string): boolean {
|
||||
|
||||
const PEER_WELL_KNOWN_PORTS = new Set([80, 443, 53, 853])
|
||||
|
||||
/**
|
||||
* Интернет-сторона потока: у IPFIX сервис часто в src (Google:443 → RFC1918:ephemeral).
|
||||
* Классифицировать этот IP, не слепой dst.
|
||||
*/
|
||||
export function pickInternetPeer(src: string, dst: string, srcPort: number, dstPort: number): string {
|
||||
const srcPub = !isNonPublicIp(src)
|
||||
const dstPub = !isNonPublicIp(dst)
|
||||
if (srcPub && !dstPub) return src
|
||||
if (dstPub && !srcPub) return dst
|
||||
if (srcPub && dstPub) {
|
||||
const srcWk = PEER_WELL_KNOWN_PORTS.has(srcPort)
|
||||
const dstWk = PEER_WELL_KNOWN_PORTS.has(dstPort)
|
||||
if (srcWk && !dstWk) return src
|
||||
if (dstWk && !srcWk) return dst
|
||||
export function isUnspecifiedIp(ip: string): boolean {
|
||||
const t = canonicalIp(ip)
|
||||
if (!t) return true
|
||||
const lower = t.toLowerCase()
|
||||
return t === "0.0.0.0" || lower === "::" || lower === "::0"
|
||||
}
|
||||
|
||||
function usableIp(ip: string | undefined): string {
|
||||
const t = canonicalIp(ip)
|
||||
return isUnspecifiedIp(t) ? "" : t
|
||||
}
|
||||
|
||||
export interface InternetDestCtx {
|
||||
/** WAN IP узлов сети (EN/JH) — не интернет-назначение. */
|
||||
ours?: ReadonlySet<string>
|
||||
/** Ingress с bound GRE/WG клиента: dest = нелокальный IP, не ASN клиента. */
|
||||
boundClient?: boolean
|
||||
/** IPFIX postNAT (IANA 225/226). */
|
||||
natSrc?: string
|
||||
natDst?: string
|
||||
/** IPFIX postNAPT ports (IANA 227/228). */
|
||||
natSrcPort?: number
|
||||
natDstPort?: number
|
||||
}
|
||||
|
||||
export function isLocalIp(ip: string, ours?: ReadonlySet<string>): boolean {
|
||||
const host = canonicalIp(ip)
|
||||
if (isUnspecifiedIp(host) || isNonPublicIp(host)) return true
|
||||
if (!ours || ours.size === 0) return false
|
||||
if (ours.has(host) || ours.has(String(ip ?? "").trim())) return true
|
||||
for (const o of ours) {
|
||||
if (canonicalIp(o) === host) return true
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
/**
|
||||
* Интернет-назначение потока для ASN/страны/сервиса.
|
||||
* Пустая строка — dest нет (не GeoIP IP клиента / GRE-пира).
|
||||
*
|
||||
* boundClient: download CDN→overlay берём публичный src даже без :80/:443
|
||||
* (googlevideo). Client-ISP → overlay:well-known — не dest (ASN клиента).
|
||||
*/
|
||||
export function pickInternetDest(
|
||||
srcRaw: string,
|
||||
dstRaw: string,
|
||||
srcPort: number,
|
||||
dstPort: number,
|
||||
ctx?: InternetDestCtx,
|
||||
): string {
|
||||
const ours = ctx?.ours
|
||||
const src = usableIp(srcRaw)
|
||||
const dst = usableIp(dstRaw)
|
||||
const natSrc = usableIp(ctx?.natSrc)
|
||||
const natDst = usableIp(ctx?.natDst)
|
||||
const internet = (ip: string) => Boolean(ip) && !isLocalIp(ip, ours)
|
||||
const dstIp = internet(dst) ? dst : (internet(natDst) ? natDst : "")
|
||||
const srcIp = internet(src) ? src : (internet(natSrc) ? natSrc : "")
|
||||
const dstPortEff = internet(dst) ? dstPort : (internet(natDst) ? (ctx?.natDstPort || dstPort) : dstPort)
|
||||
const srcPortEff = internet(src) ? srcPort : (internet(natSrc) ? (ctx?.natSrcPort || srcPort) : srcPort)
|
||||
|
||||
if (ctx?.boundClient) {
|
||||
if (dstIp) return dstIp
|
||||
if (srcIp) {
|
||||
const srcWk = PEER_WELL_KNOWN_PORTS.has(srcPortEff)
|
||||
const dstWk = PEER_WELL_KNOWN_PORTS.has(dstPortEff)
|
||||
if (!srcWk && dstWk) return ""
|
||||
return srcIp
|
||||
}
|
||||
return ""
|
||||
}
|
||||
|
||||
if (srcIp && !dstIp) return srcIp
|
||||
if (dstIp && !srcIp) return dstIp
|
||||
if (srcIp && dstIp) {
|
||||
const srcWk = PEER_WELL_KNOWN_PORTS.has(srcPortEff)
|
||||
const dstWk = PEER_WELL_KNOWN_PORTS.has(dstPortEff)
|
||||
if (srcWk && !dstWk) return srcIp
|
||||
if (dstWk && !srcWk) return dstIp
|
||||
return dstIp
|
||||
}
|
||||
|
||||
if (src && dst && ours?.has(src) && ours.has(dst)) return ""
|
||||
return dst
|
||||
}
|
||||
|
||||
/**
|
||||
* Dest для карты = тот же internet peer, что аналитика/факты.
|
||||
* (исторически отдельный fallback без well-known trap — теперь в pickInternetDest.)
|
||||
*/
|
||||
export function pickMapInternetDest(
|
||||
srcRaw: string,
|
||||
dstRaw: string,
|
||||
srcPort: number,
|
||||
dstPort: number,
|
||||
ctx?: InternetDestCtx,
|
||||
): string {
|
||||
return pickInternetDest(srcRaw, dstRaw, srcPort, dstPort, ctx)
|
||||
}
|
||||
|
||||
/**
|
||||
* Интернет-сторона потока без топологии: у IPFIX сервис часто в src (Google:443 → RFC1918).
|
||||
*/
|
||||
export function pickInternetPeer(src: string, dst: string, srcPort: number, dstPort: number): string {
|
||||
return pickInternetDest(src, dst, srcPort, dstPort) || ""
|
||||
}
|
||||
|
||||
export type FlowDirection = "to_client" | "from_client" | "transit"
|
||||
|
||||
export interface FlowEndpoints {
|
||||
packetSrc: string
|
||||
packetDst: string
|
||||
internetPeer: string
|
||||
peerPort: number
|
||||
otherPort: number
|
||||
clientIp: string
|
||||
direction: FlowDirection
|
||||
}
|
||||
|
||||
function sameHost(a: string, b: string | undefined): boolean {
|
||||
const x = canonicalIp(a)
|
||||
const y = canonicalIp(b)
|
||||
return Boolean(x) && x === y
|
||||
}
|
||||
|
||||
/** Роли концов IPFIX-пакета. 5-tuple не переворачивается. */
|
||||
export function resolveFlowEndpoints(opts: {
|
||||
src: string
|
||||
dst: string
|
||||
srcPort: number
|
||||
dstPort: number
|
||||
ctx?: InternetDestCtx
|
||||
}): FlowEndpoints {
|
||||
const packetSrc = usableIp(opts.src)
|
||||
const packetDst = usableIp(opts.dst)
|
||||
const internetPeer = pickInternetDest(opts.src, opts.dst, opts.srcPort, opts.dstPort, opts.ctx)
|
||||
const ours = opts.ctx?.ours
|
||||
const srcLocal = Boolean(packetSrc) && isLocalIp(packetSrc, ours)
|
||||
const dstLocal = Boolean(packetDst) && isLocalIp(packetDst, ours)
|
||||
|
||||
let peerPort = 0
|
||||
let otherPort = 0
|
||||
if (internetPeer) {
|
||||
if (sameHost(internetPeer, packetSrc) || sameHost(internetPeer, opts.ctx?.natSrc)) {
|
||||
peerPort = opts.srcPort
|
||||
otherPort = opts.dstPort
|
||||
} else if (sameHost(internetPeer, packetDst) || sameHost(internetPeer, opts.ctx?.natDst)) {
|
||||
peerPort = opts.dstPort
|
||||
otherPort = opts.srcPort
|
||||
} else {
|
||||
peerPort = opts.ctx?.natDstPort || opts.dstPort
|
||||
otherPort = opts.srcPort
|
||||
}
|
||||
}
|
||||
|
||||
let clientIp = ""
|
||||
if (srcLocal && !dstLocal) clientIp = packetSrc
|
||||
else if (dstLocal && !srcLocal) clientIp = packetDst
|
||||
else if (srcLocal) clientIp = packetSrc
|
||||
else if (dstLocal) clientIp = packetDst
|
||||
|
||||
let direction: FlowDirection = "transit"
|
||||
if (internetPeer && clientIp) {
|
||||
direction = sameHost(internetPeer, packetSrc) ? "to_client" : "from_client"
|
||||
}
|
||||
|
||||
return {
|
||||
packetSrc,
|
||||
packetDst,
|
||||
internetPeer,
|
||||
peerPort,
|
||||
otherPort,
|
||||
clientIp,
|
||||
direction,
|
||||
}
|
||||
}
|
||||
|
||||
@@ -6,6 +6,8 @@ import {
|
||||
} from "./traffic-flow-ingest.js"
|
||||
import {
|
||||
buildFlowMapHops,
|
||||
MAP_COUNTRY_SERVICE_MIN_NODES,
|
||||
MAP_COUNTRY_SERVICE_NODE_CAP,
|
||||
MAP_SERVICE_MIN_NODES,
|
||||
MAP_SERVICE_NODE_CAP,
|
||||
pickMapServices,
|
||||
@@ -70,6 +72,21 @@ import {
|
||||
console.log("traffic-flow-map-hops.test.ts: pickMapServices ok")
|
||||
}
|
||||
|
||||
{
|
||||
const leak = pickMapServices(
|
||||
[
|
||||
{ id: "svc:other", label: "Прочее", category: "Прочее", bytes: 19_000, bps: 0, share: 19 / 30 },
|
||||
{ id: "svc:google", label: "Google", category: "Веб", bytes: 11_000, bps: 0, share: 11 / 30 },
|
||||
],
|
||||
5,
|
||||
)
|
||||
assert.equal(leak.reduce((n, s) => n + s.bytes, 0), 30_000)
|
||||
const google = leak.find((s) => s.id === "svc:google")
|
||||
assert.ok(google)
|
||||
assert.ok(google.share < 0.4, "доля от окна хопа, не от named-only")
|
||||
assert.ok(leak.some((s) => s.id === "svc:other"), "дыра видна как Прочее")
|
||||
}
|
||||
|
||||
if (!(await withPgOrSkip())) {
|
||||
console.log("traffic-flow-map-hops.test.ts: skip")
|
||||
process.exit(0)
|
||||
@@ -279,12 +296,27 @@ try {
|
||||
resetFlowMapHopsCacheForTests()
|
||||
const six = await buildFlowMapHops({ minutes: 5, minSharePct: 5 })
|
||||
assert.equal(six.totalBytes, 10_000)
|
||||
assert.equal(six.namedBytes, 600)
|
||||
assert.equal(six.unclassifiedBytes, 9400)
|
||||
const google = six.services?.find((s) => s.id === "svc:google")
|
||||
const otherSix = six.services?.find((s) => s.id === "svc:other")
|
||||
assert.ok(google, "Google ≥ 5%")
|
||||
assert.ok(google.share >= 0.05)
|
||||
assert.ok(otherSix, "остаток — Прочее")
|
||||
assert.ok(google.share >= 0.05 && google.share < 0.1)
|
||||
assert.ok(otherSix.share >= 0.9)
|
||||
const googleEdge = six.serviceEdges?.find((e) => e.toId === "svc:google" && e.fromId === "9")
|
||||
assert.ok(googleEdge)
|
||||
assert.equal(googleEdge.clientName, "Alice")
|
||||
assert.equal((six.serviceEdges ?? []).reduce((n, e) => n + e.bytes, 0), 10_000)
|
||||
const us = six.countries?.find((s) => s.id === "cc:us")
|
||||
assert.ok(us, "Google ripe country US")
|
||||
assert.equal(us.label, "US")
|
||||
const usEdge = six.countryEdges?.find((e) => e.toId === "cc:us" && e.fromId === "9")
|
||||
assert.ok(usEdge)
|
||||
assert.ok(!(six.countryEdges ?? []).some((e) => e.toId.startsWith("svc:")), "страны не смешиваются с svc:*")
|
||||
const usPath = six.countryPaths?.find((p) => p.serviceId === "cc:us" && p.enId === "9")
|
||||
assert.ok(usPath)
|
||||
assert.equal(usPath.clientName, "Alice")
|
||||
} finally {
|
||||
resetFlowRingsForTests()
|
||||
resetIfaceCacheForTests()
|
||||
@@ -309,9 +341,14 @@ try {
|
||||
resetFlowMapHopsCacheForTests()
|
||||
const four = await buildFlowMapHops({ minutes: 5, minSharePct: 5 })
|
||||
assert.equal(four.totalBytes, 10_000)
|
||||
assert.equal(four.namedBytes, 400)
|
||||
assert.equal(four.unclassifiedBytes, 9600)
|
||||
const googleFour = four.services?.find((s) => s.id === "svc:google")
|
||||
assert.ok(googleFour, "единственный бренд виден при 4% от окна")
|
||||
assert.ok(googleFour.share >= 0.99, "доля среди брендов ≈ 1")
|
||||
const otherFour = four.services?.find((s) => s.id === "svc:other")
|
||||
assert.ok(googleFour, "бренд виден при 4% от окна (MIN_NODES)")
|
||||
assert.ok(otherFour, "Прочее держит остаток окна")
|
||||
assert.ok(googleFour.share < 0.1, "доля от totalBytes, не от named")
|
||||
assert.ok(otherFour.share > 0.9)
|
||||
resetFlowMapHopsCacheForTests()
|
||||
const off = await buildFlowMapHops({ minutes: 5, minSharePct: 0 })
|
||||
assert.ok(off.services?.some((s) => s.id === "svc:google"), "порог 0 показывает Google")
|
||||
@@ -340,10 +377,13 @@ try {
|
||||
const two = await buildFlowMapHops({ minutes: 5, minSharePct: 5 })
|
||||
const googleTwo = two.services?.find((s) => s.id === "svc:google")
|
||||
const cfTwo = two.services?.find((s) => s.id === "svc:cloudflare")
|
||||
const otherTwo = two.services?.find((s) => s.id === "svc:other")
|
||||
assert.ok(googleTwo, "Google среди брендов")
|
||||
assert.ok(cfTwo, "Cloudflare среди брендов")
|
||||
assert.ok(googleTwo.share >= 0.05)
|
||||
assert.ok(cfTwo.share >= 0.05)
|
||||
assert.ok(otherTwo, "Прочее")
|
||||
assert.ok(googleTwo.share > 0.03 && googleTwo.share < 0.05)
|
||||
assert.ok(cfTwo.share > 0.03 && cfTwo.share < 0.05)
|
||||
assert.ok(otherTwo.share > 0.9)
|
||||
} finally {
|
||||
resetFlowRingsForTests()
|
||||
resetIfaceCacheForTests()
|
||||
@@ -373,6 +413,43 @@ try {
|
||||
resetRipeCacheForTests()
|
||||
}
|
||||
|
||||
resetFlowRingsForTests()
|
||||
resetIfaceCacheForTests()
|
||||
resetRipeCacheForTests()
|
||||
disableRipeEnqueueForTests()
|
||||
seedFlowTopologyForTests(topo)
|
||||
rememberServerIfaces(7, [
|
||||
{ ".id": "*2", name: "gre-client" },
|
||||
{ ".id": "*3", name: "gre-jh-en" },
|
||||
])
|
||||
ingestParsedFlowsForServerForTests(7, [
|
||||
payloadFlow("64.233.161.1", 10_000),
|
||||
payloadFlow("203.0.113.50", 20_000),
|
||||
])
|
||||
try {
|
||||
resetFlowMapHopsCacheForTests()
|
||||
const leak = await buildFlowMapHops({ minutes: 5, minSharePct: 0 })
|
||||
const hop = leak.hops.find((h) => h.kind === "gre" && h.fromId === "7" && h.toId === "9")
|
||||
assert.ok(hop, "GRE hop JH→EN")
|
||||
assert.equal(hop.bytes, 30_000)
|
||||
assert.equal(leak.totalBytes, 30_000)
|
||||
assert.equal(leak.namedBytes, 10_000)
|
||||
assert.equal(leak.unclassifiedBytes, 20_000)
|
||||
const yt = leak.services?.find((s) => s.id === "svc:youtube")
|
||||
const other = leak.services?.find((s) => s.id === "svc:other")
|
||||
assert.ok(yt, "64.233:443 без RIPE → YouTube")
|
||||
assert.ok(other, "dest без бренда → Прочее")
|
||||
assert.equal(yt.bytes, 10_000)
|
||||
assert.equal(other.bytes, 20_000)
|
||||
assert.ok(Math.abs(yt.share - 10 / 30) < 0.01)
|
||||
assert.ok(Math.abs(other.share - 20 / 30) < 0.01)
|
||||
assert.equal((leak.serviceEdges ?? []).reduce((n, e) => n + e.bytes, 0), hop.bytes)
|
||||
} finally {
|
||||
resetFlowRingsForTests()
|
||||
resetIfaceCacheForTests()
|
||||
resetRipeCacheForTests()
|
||||
}
|
||||
|
||||
const smallBrands: Array<{ ip: string; asn: number; holder: string; bytes: number; id: string }> = [
|
||||
{ ip: "203.0.113.1", asn: 714, holder: "APPLE-ENGINEERING", bytes: 400, id: "svc:apple" },
|
||||
{ ip: "203.0.113.2", asn: 36459, holder: "GITHUB", bytes: 390, id: "svc:github" },
|
||||
@@ -450,6 +527,72 @@ try {
|
||||
resetFlowCatalogForTests()
|
||||
}
|
||||
|
||||
resetFlowRingsForTests()
|
||||
resetIfaceCacheForTests()
|
||||
resetRipeCacheForTests()
|
||||
disableRipeEnqueueForTests()
|
||||
seedFlowTopologyForTests(topo)
|
||||
rememberServerIfaces(7, [
|
||||
{ ".id": "*2", name: "gre-client" },
|
||||
{ ".id": "*3", name: "gre-jh-en" },
|
||||
])
|
||||
googleRipe()
|
||||
ingestParsedFlowsForServerForTests(7, [
|
||||
{
|
||||
src: "10.100.1.17",
|
||||
dst: "8.8.8.8",
|
||||
proto: 6,
|
||||
srcPort: 51234,
|
||||
dstPort: 443,
|
||||
bytes: 4_000,
|
||||
packets: 10,
|
||||
inIface: "2",
|
||||
outIface: "3",
|
||||
nextHop: "198.51.100.1",
|
||||
},
|
||||
{
|
||||
src: "203.0.113.10",
|
||||
dst: "198.51.100.1",
|
||||
proto: 47,
|
||||
srcPort: 0,
|
||||
dstPort: 0,
|
||||
bytes: 2_000_000,
|
||||
packets: 400,
|
||||
inIface: "3",
|
||||
outIface: "3",
|
||||
},
|
||||
{
|
||||
src: "10.200.100.53",
|
||||
dst: "10.200.100.1",
|
||||
proto: 6,
|
||||
srcPort: 53880,
|
||||
dstPort: 443,
|
||||
bytes: 3_000,
|
||||
packets: 8,
|
||||
inIface: "2",
|
||||
outIface: "3",
|
||||
nextHop: "198.51.100.1",
|
||||
natDst: "8.8.8.8",
|
||||
natDstPort: 443,
|
||||
},
|
||||
])
|
||||
try {
|
||||
resetFlowMapHopsCacheForTests()
|
||||
const path = await buildFlowMapHops({ minutes: 5, excludeOverlay: false, excludeMesh: false, minSharePct: 0 })
|
||||
const hop = path.hops.find((h) => h.kind === "gre" && h.fromId === "7" && h.toId === "9")
|
||||
assert.ok(hop, "hop JH→EN")
|
||||
const google = path.services?.find((s) => s.id === "svc:google")
|
||||
assert.ok(google, "сервис Google")
|
||||
assert.equal(google.bytes, 7_000)
|
||||
assert.ok(!(path.services ?? []).some((s) => s.label === "GRE"), "GRE не dest")
|
||||
} finally {
|
||||
seedFlowTopologyForTests(null)
|
||||
resetFlowRingsForTests()
|
||||
resetIfaceCacheForTests()
|
||||
resetRipeCacheForTests()
|
||||
resetFlowCatalogForTests()
|
||||
}
|
||||
|
||||
resetFlowRingsForTests()
|
||||
resetIfaceCacheForTests()
|
||||
resetRipeCacheForTests()
|
||||
@@ -488,9 +631,9 @@ ingestParsedFlowsForServerForTests(7, [
|
||||
try {
|
||||
resetFlowMapHopsCacheForTests()
|
||||
const rev = await buildFlowMapHops({ minutes: 5, minSharePct: 0 })
|
||||
assert.ok(rev.services?.some((s) => s.id === "svc:google"), "реверс Google:443 → 10.x")
|
||||
assert.ok(rev.services?.some((s) => s.id === "svc:youtube"), "реверс googlevideo:443 → YouTube")
|
||||
assert.ok(rev.services?.some((s) => s.id === "svc:cloudflare"), "реверс Cloudflare:443 → 10.x")
|
||||
assert.ok(rev.serviceEdges?.some((e) => e.toId === "svc:google" && e.fromId === "9"))
|
||||
assert.ok(rev.serviceEdges?.some((e) => e.toId === "svc:youtube" && e.fromId === "9"))
|
||||
} finally {
|
||||
seedFlowTopologyForTests(null)
|
||||
resetFlowRingsForTests()
|
||||
@@ -566,13 +709,13 @@ ingestParsedFlowsForServerForTests(7, [
|
||||
try {
|
||||
resetFlowMapHopsCacheForTests()
|
||||
const wanOnly = await buildFlowMapHops({ minutes: 5, minSharePct: 0 })
|
||||
const googleEdge = wanOnly.serviceEdges?.find((e) => e.toId === "svc:google")
|
||||
assert.ok(googleEdge, "Google WAN без GRE payload")
|
||||
const googleEdge = wanOnly.serviceEdges?.find((e) => e.toId === "svc:youtube")
|
||||
assert.ok(googleEdge, "YouTube WAN без GRE payload")
|
||||
assert.equal(googleEdge.fromId, "9", "единственный EN, даже без nextHop")
|
||||
assert.ok(googleEdge.bps > 0, "скорость на hop EN→сервис")
|
||||
assert.ok(!(wanOnly.serviceEdges ?? []).some((e) => e.fromId === "7"), "нет пунктира с JH")
|
||||
const googlePath = wanOnly.servicePaths?.find((p) => p.serviceId === "svc:google")
|
||||
assert.ok(googlePath, "путь WAN Google")
|
||||
const googlePath = wanOnly.servicePaths?.find((p) => p.serviceId === "svc:youtube")
|
||||
assert.ok(googlePath, "путь WAN YouTube")
|
||||
assert.equal(googlePath.viaId, "7", "via = JH exporter")
|
||||
assert.equal(googlePath.enId, "9", "якорь EN")
|
||||
assert.ok(googlePath.bps > 0, "скорость на пути клиента")
|
||||
@@ -584,4 +727,274 @@ try {
|
||||
resetFlowCatalogForTests()
|
||||
}
|
||||
|
||||
resetFlowRingsForTests()
|
||||
resetIfaceCacheForTests()
|
||||
resetRipeCacheForTests()
|
||||
disableRipeEnqueueForTests()
|
||||
seedFlowTopologyForTests(topo)
|
||||
rememberServerIfaces(7, [
|
||||
{ ".id": "*2", name: "gre-client" },
|
||||
{ ".id": "*3", name: "gre-jh-en" },
|
||||
])
|
||||
rememberServerIfaces(9, [
|
||||
{ ".id": "*1", name: "ether1" },
|
||||
])
|
||||
googleRipe()
|
||||
ingestParsedFlowsForServerForTests(7, [payloadFlow("8.8.8.8", 12_000)])
|
||||
ingestParsedFlowsForServerForTests(9, [{
|
||||
src: "10.100.1.17",
|
||||
dst: "8.8.8.8",
|
||||
proto: 6,
|
||||
srcPort: 51234,
|
||||
dstPort: 443,
|
||||
bytes: 12_000,
|
||||
packets: 10,
|
||||
inIface: "1",
|
||||
outIface: "1",
|
||||
nextHop: "",
|
||||
}])
|
||||
try {
|
||||
resetFlowMapHopsCacheForTests()
|
||||
const dual = await buildFlowMapHops({ minutes: 5, minSharePct: 0 })
|
||||
const gre = dual.hops.find((h) => h.kind === "gre" && h.fromId === "7" && h.toId === "9")
|
||||
assert.ok(gre, "GRE JH→EN сохранён")
|
||||
assert.equal(gre.bytes, 12_000)
|
||||
const googlePaths = (dual.servicePaths ?? []).filter((p) => p.serviceId === "svc:google")
|
||||
assert.equal(googlePaths.length, 1, "один путь без копии EN")
|
||||
assert.equal(googlePaths[0]?.clientId, "u1")
|
||||
assert.equal(googlePaths[0]?.viaId, "7")
|
||||
const googleEdge = dual.serviceEdges?.find((e) => e.toId === "svc:google" && e.fromId === "9")
|
||||
assert.ok(googleEdge)
|
||||
assert.equal(googleEdge.bytes, 12_000)
|
||||
assert.equal(googleEdge.bps, googlePaths[0]?.bps)
|
||||
} finally {
|
||||
seedFlowTopologyForTests(null)
|
||||
resetFlowRingsForTests()
|
||||
resetIfaceCacheForTests()
|
||||
resetRipeCacheForTests()
|
||||
resetFlowCatalogForTests()
|
||||
}
|
||||
|
||||
resetFlowRingsForTests()
|
||||
resetIfaceCacheForTests()
|
||||
resetRipeCacheForTests()
|
||||
disableRipeEnqueueForTests()
|
||||
seedFlowTopologyForTests(topo)
|
||||
rememberServerIfaces(7, [
|
||||
{ ".id": "*2", name: "gre-client" },
|
||||
{ ".id": "*3", name: "gre-jh-en" },
|
||||
])
|
||||
googleRipe()
|
||||
ingestParsedFlowsForServerForTests(7, [
|
||||
payloadFlow("8.8.8.8", 8_000),
|
||||
{
|
||||
src: "8.8.8.8",
|
||||
dst: "10.100.1.17",
|
||||
proto: 6,
|
||||
srcPort: 443,
|
||||
dstPort: 51234,
|
||||
bytes: 4_000,
|
||||
packets: 8,
|
||||
inIface: "3",
|
||||
outIface: "2",
|
||||
nextHop: "",
|
||||
},
|
||||
])
|
||||
try {
|
||||
resetFlowMapHopsCacheForTests()
|
||||
const bothDir = await buildFlowMapHops({ minutes: 5, minSharePct: 0 })
|
||||
const gre = bothDir.hops.find((h) => h.kind === "gre" && h.fromId === "7" && h.toId === "9")
|
||||
assert.ok(gre, "GRE-hop при fwd/rev")
|
||||
const googlePaths = (bothDir.servicePaths ?? []).filter((p) => p.serviceId === "svc:google")
|
||||
assert.equal(googlePaths.length, 1, "fwd+rev — один клиент")
|
||||
assert.equal(googlePaths[0]?.clientId, "u1")
|
||||
assert.ok(!(bothDir.servicePaths ?? []).some((p) => p.serviceId === "svc:google" && p.clientId === "—"))
|
||||
const googleEdge = bothDir.serviceEdges?.find((e) => e.toId === "svc:google" && e.fromId === "9")
|
||||
assert.ok(googleEdge)
|
||||
assert.equal(googleEdge.bytes, 12_000)
|
||||
assert.equal(googleEdge.bps, googlePaths[0]?.bps)
|
||||
} finally {
|
||||
seedFlowTopologyForTests(null)
|
||||
resetFlowRingsForTests()
|
||||
resetIfaceCacheForTests()
|
||||
resetRipeCacheForTests()
|
||||
resetFlowCatalogForTests()
|
||||
}
|
||||
|
||||
resetFlowRingsForTests()
|
||||
resetIfaceCacheForTests()
|
||||
resetRipeCacheForTests()
|
||||
disableRipeEnqueueForTests()
|
||||
seedFlowTopologyForTests(topo)
|
||||
rememberServerIfaces(7, [
|
||||
{ ".id": "*2", name: "gre-client" },
|
||||
{ ".id": "*3", name: "gre-jh-en" },
|
||||
])
|
||||
googleRipe()
|
||||
seedRipeCacheForTests({
|
||||
prefix: "185.45.12.0/24",
|
||||
asn: 13335,
|
||||
country: "NL",
|
||||
lat: 52.3,
|
||||
lng: 4.9,
|
||||
holder: "CLOUDFLARENET, NL",
|
||||
ok: true,
|
||||
fetchedAt: Date.now(),
|
||||
})
|
||||
ingestParsedFlowsForServerForTests(7, [
|
||||
payloadFlow("8.8.8.8", 5000),
|
||||
payloadFlow("185.45.12.10", 5000),
|
||||
])
|
||||
try {
|
||||
resetFlowMapHopsCacheForTests()
|
||||
const split = await buildFlowMapHops({ minutes: 5, minSharePct: 5 })
|
||||
const us = split.countries?.find((s) => s.id === "cc:us")
|
||||
const nl = split.countries?.find((s) => s.id === "cc:nl")
|
||||
assert.ok(us, "US из ripe Google")
|
||||
assert.ok(nl, "NL из ripe Cloudflare")
|
||||
assert.equal(us.bytes, 5000)
|
||||
assert.equal(nl.bytes, 5000)
|
||||
assert.ok(split.countryEdges?.some((e) => e.toId === "cc:us" && e.fromId === "9"))
|
||||
assert.ok(split.countryEdges?.some((e) => e.toId === "cc:nl" && e.fromId === "9"))
|
||||
assert.ok(!(split.countryEdges ?? []).some((e) => e.toId.startsWith("svc:")))
|
||||
assert.ok(split.serviceEdges?.some((e) => e.toId === "svc:google"))
|
||||
assert.ok(!(split.serviceEdges ?? []).some((e) => e.toId.startsWith("cc:")))
|
||||
assert.ok(split.countryPaths?.some((p) => p.serviceId === "cc:nl" && p.enId === "9"))
|
||||
} finally {
|
||||
seedFlowTopologyForTests(null)
|
||||
resetFlowRingsForTests()
|
||||
resetIfaceCacheForTests()
|
||||
resetRipeCacheForTests()
|
||||
resetFlowCatalogForTests()
|
||||
}
|
||||
|
||||
resetFlowRingsForTests()
|
||||
resetIfaceCacheForTests()
|
||||
resetRipeCacheForTests()
|
||||
disableRipeEnqueueForTests()
|
||||
seedFlowTopologyForTests(topo)
|
||||
rememberServerIfaces(7, [
|
||||
{ ".id": "*2", name: "gre-client" },
|
||||
{ ".id": "*3", name: "gre-jh-en" },
|
||||
])
|
||||
googleRipe()
|
||||
ingestParsedFlowsForServerForTests(7, [
|
||||
payloadFlow("8.8.8.8", 600),
|
||||
payloadFlow("203.0.113.50", 9400),
|
||||
])
|
||||
try {
|
||||
resetFlowMapHopsCacheForTests()
|
||||
const nested = await buildFlowMapHops({ minutes: 5, minSharePct: 5 })
|
||||
const usGroup = nested.countryServiceGroups?.find((g) => g.countryId === "cc:us")
|
||||
assert.ok(usGroup, "группа сервисов cc:us")
|
||||
const nestedGoogle = usGroup.services.find((s) => s.id === "cc:us|svc:google")
|
||||
assert.ok(nestedGoogle, "cc:us|svc:google в группе")
|
||||
assert.equal(nestedGoogle.label, "Google")
|
||||
assert.ok(Math.abs(nestedGoogle.share - 1) < 0.01, "доля от байтов страны (600/600), не окна")
|
||||
const nestedEdge = usGroup.edges.find((e) => e.toId === "cc:us|svc:google")
|
||||
assert.ok(nestedEdge)
|
||||
assert.equal(nestedEdge.fromId, "cc:us", "ребро вложенного слоя: страна → сервис")
|
||||
assert.equal(nestedEdge.bytes, 600)
|
||||
assert.ok(!usGroup.edges.some((e) => e.fromId.startsWith("svc:")), "fromId вложенных рёбер не svc:*")
|
||||
const nestedPath = usGroup.paths.find((p) => p.serviceId === "cc:us|svc:google")
|
||||
assert.ok(nestedPath)
|
||||
assert.equal(nestedPath.enId, "9", "путь держит реальный EN для подсветки HR→JH→EN")
|
||||
assert.equal(nestedPath.clientName, "Alice")
|
||||
const otherGroup = nested.countryServiceGroups?.find((g) => g.countryId === "cc:other")
|
||||
assert.ok(otherGroup, "группа cc:other (Прочее-страна)")
|
||||
assert.ok(otherGroup.services.some((s) => s.id === "cc:other|svc:other"))
|
||||
} finally {
|
||||
seedFlowTopologyForTests(null)
|
||||
resetFlowRingsForTests()
|
||||
resetIfaceCacheForTests()
|
||||
resetRipeCacheForTests()
|
||||
resetFlowCatalogForTests()
|
||||
}
|
||||
|
||||
resetFlowRingsForTests()
|
||||
resetIfaceCacheForTests()
|
||||
resetRipeCacheForTests()
|
||||
disableRipeEnqueueForTests()
|
||||
seedFlowTopologyForTests(topo)
|
||||
rememberServerIfaces(7, [
|
||||
{ ".id": "*2", name: "gre-client" },
|
||||
{ ".id": "*3", name: "gre-jh-en" },
|
||||
])
|
||||
googleRipe()
|
||||
seedRipeCacheForTests({
|
||||
prefix: "185.45.12.0/24",
|
||||
asn: 13335,
|
||||
country: "NL",
|
||||
lat: 52.3,
|
||||
lng: 4.9,
|
||||
holder: "CLOUDFLARENET, NL",
|
||||
ok: true,
|
||||
fetchedAt: Date.now(),
|
||||
})
|
||||
ingestParsedFlowsForServerForTests(7, [
|
||||
payloadFlow("8.8.8.8", 5000),
|
||||
payloadFlow("185.45.12.10", 5000),
|
||||
])
|
||||
try {
|
||||
resetFlowMapHopsCacheForTests()
|
||||
const nestedSplit = await buildFlowMapHops({ minutes: 5, minSharePct: 5 })
|
||||
const usNested = nestedSplit.countryServiceGroups?.find((g) => g.countryId === "cc:us")
|
||||
const nlNested = nestedSplit.countryServiceGroups?.find((g) => g.countryId === "cc:nl")
|
||||
assert.ok(usNested && nlNested, "группы у обеих стран")
|
||||
assert.ok(usNested.services.every((s) => s.id.startsWith("cc:us|")), "сервисы US только cc:us|*")
|
||||
assert.ok(nlNested.services.every((s) => s.id.startsWith("cc:nl|")), "сервисы NL только cc:nl|*")
|
||||
assert.ok(usNested.services.some((s) => s.id === "cc:us|svc:google"), "Google в US")
|
||||
assert.ok(nlNested.services.some((s) => s.id === "cc:nl|svc:cloudflare"), "Cloudflare в NL")
|
||||
assert.ok(!nlNested.services.some((s) => s.id === "cc:us|svc:google"), "Google US не утек в NL")
|
||||
assert.ok(usNested.edges.every((e) => e.fromId === "cc:us"))
|
||||
assert.ok(nlNested.edges.every((e) => e.fromId === "cc:nl"))
|
||||
} finally {
|
||||
seedFlowTopologyForTests(null)
|
||||
resetFlowRingsForTests()
|
||||
resetIfaceCacheForTests()
|
||||
resetRipeCacheForTests()
|
||||
resetFlowCatalogForTests()
|
||||
}
|
||||
|
||||
resetFlowRingsForTests()
|
||||
resetIfaceCacheForTests()
|
||||
resetRipeCacheForTests()
|
||||
disableRipeEnqueueForTests()
|
||||
seedFlowTopologyForTests(topo)
|
||||
rememberServerIfaces(7, [
|
||||
{ ".id": "*2", name: "gre-client" },
|
||||
{ ".id": "*3", name: "gre-jh-en" },
|
||||
])
|
||||
googleRipe()
|
||||
for (const b of smallBrands) seedRipeAsn(b.ip, b.asn, b.holder)
|
||||
ingestParsedFlowsForServerForTests(7, [
|
||||
payloadFlow("8.8.8.8", 5000),
|
||||
...smallBrands.map((b) => payloadFlow(b.ip, b.bytes)),
|
||||
])
|
||||
try {
|
||||
resetFlowMapHopsCacheForTests()
|
||||
const nestedTop = await buildFlowMapHops({ minutes: 5, minSharePct: 5 })
|
||||
const usTop = nestedTop.countryServiceGroups?.find((g) => g.countryId === "cc:us")
|
||||
assert.ok(usTop)
|
||||
assert.equal(
|
||||
usTop.services.length,
|
||||
MAP_COUNTRY_SERVICE_MIN_NODES,
|
||||
"мелкий хвост держится минимумом узлов внутри страны",
|
||||
)
|
||||
assert.ok(usTop.services.some((s) => s.id === "cc:us|svc:google"))
|
||||
resetFlowMapHopsCacheForTests()
|
||||
const nestedAll = await buildFlowMapHops({ minutes: 5, minSharePct: 0 })
|
||||
const usAll = nestedAll.countryServiceGroups?.find((g) => g.countryId === "cc:us")
|
||||
assert.ok(usAll)
|
||||
assert.equal(usAll.services.length, MAP_COUNTRY_SERVICE_NODE_CAP, "cap вложенного слоя = 8")
|
||||
assert.ok(!usAll.services.some((s) => s.id === "cc:us|svc:epic"), "ранг 9+ скрыт")
|
||||
assert.ok(!usAll.services.some((s) => s.id === "cc:us|svc:riot"))
|
||||
} finally {
|
||||
seedFlowTopologyForTests(null)
|
||||
resetFlowRingsForTests()
|
||||
resetIfaceCacheForTests()
|
||||
resetRipeCacheForTests()
|
||||
resetFlowCatalogForTests()
|
||||
}
|
||||
|
||||
console.log("traffic-flow-map-hops.test.ts: ok")
|
||||
|
||||
@@ -1,28 +1,31 @@
|
||||
import { eq } from "drizzle-orm"
|
||||
import type { FlowMapHop, FlowMapHopsDto, FlowMapService, FlowMapServiceEdge, FlowMapServicePath } from "@mmapp/contracts/traffic-flow"
|
||||
import type { FlowMapCountryServiceGroup, FlowMapHop, FlowMapHopsDto, FlowMapService, FlowMapServiceEdge, FlowMapServicePath } from "@mmapp/contracts/traffic-flow"
|
||||
import { db } from "../db/index.js"
|
||||
import { userInterfaceBindings } from "../db/schema.js"
|
||||
import { applicationName, flowRowMatchesFilter } from "./traffic-flow-apps.js"
|
||||
import {
|
||||
isNamedInternetService,
|
||||
mapServiceNodeId,
|
||||
resolveFlowBrand,
|
||||
} from "./traffic-flow-brands.js"
|
||||
import { dedupFlowRowsMaxBytes } from "./traffic-flow-dedup.js"
|
||||
import { flowRowMatchesFilter } from "./traffic-flow-apps.js"
|
||||
import { OTHER_SERVICE, isNamedInternetService, mapCountryNodeId, mapCountryServiceNodeId, mapServiceNodeId, resolveRipeCountry } from "./traffic-flow-brands.js"
|
||||
import type { FlowIpMeta } from "./traffic-flow-ripe.js"
|
||||
import { refreshFlowCatalogInBackground } from "./traffic-flow-classify.js"
|
||||
import { dedupFlowRowsAcrossExporters, dedupFlowRowsMaxBytes } from "./traffic-flow-dedup.js"
|
||||
import { getFlowListenerState, listFlowRowsForWindow } from "./traffic-flow-ingest.js"
|
||||
import { resolveIfaceName } from "./traffic-flow-ifaces.js"
|
||||
import { classifyFlowPlane, shouldKeepPlane } from "./traffic-flow-planes.js"
|
||||
import { pickInternetPeer } from "./traffic-flow-ip.js"
|
||||
import { type FlowIpMeta } from "./traffic-flow-ripe.js"
|
||||
import { resolveFlowIp } from "./traffic-flow-geoip.js"
|
||||
import { destCtxForIface, mapInternetBrand } from "./traffic-flow-dest.js"
|
||||
import { resolveFlowEndpoints } from "./traffic-flow-ip.js"
|
||||
import { geoipReadersStatus, resolveFlowIp } from "./traffic-flow-geoip.js"
|
||||
import { getTrafficFlowSettingsRow } from "./traffic-flow-settings.js"
|
||||
import { loadFlowTopology, resolveClient, resolveEn, getServerCatalog } from "./traffic-flow-topology.js"
|
||||
import { loadFlowTopology, resolveClient, resolveEn, getServerCatalog, type FlowTopology } from "./traffic-flow-topology.js"
|
||||
import { flowDataEpoch } from "./traffic-flow-engine.js"
|
||||
|
||||
export const DEFAULT_MAP_SERVICE_MIN_SHARE_PCT = 5
|
||||
export const MAP_SERVICE_NODE_CAP = 20
|
||||
/** Минимум узлов-брендов на карте, даже если доля ниже порога. */
|
||||
export const MAP_SERVICE_MIN_NODES = 8
|
||||
/** Cap сервисов внутри раскрытой страны (база доли — байты страны, не окна). */
|
||||
export const MAP_COUNTRY_SERVICE_NODE_CAP = 8
|
||||
/** Минимум узлов-сервисов внутри страны, даже если доля ниже порога. */
|
||||
export const MAP_COUNTRY_SERVICE_MIN_NODES = 4
|
||||
export const MAP_COUNTRY_CATEGORY = "Страна"
|
||||
const HOPS_CACHE_TTL_MS = 2000
|
||||
|
||||
export interface FlowMapHopsQuery {
|
||||
@@ -67,6 +70,32 @@ interface DstAcc {
|
||||
fromBytes: Map<string, FromAcc>
|
||||
}
|
||||
|
||||
interface DestTotal {
|
||||
label: string
|
||||
category: string
|
||||
bytes: number
|
||||
}
|
||||
|
||||
interface DestEdgeAcc {
|
||||
fromId: string
|
||||
toId: string
|
||||
bytes: number
|
||||
bytesFwd: number
|
||||
bytesRev: number
|
||||
clients: Map<string, string>
|
||||
}
|
||||
|
||||
interface DestPathAcc {
|
||||
clientId: string
|
||||
clientName: string
|
||||
viaId: string
|
||||
viaName: string
|
||||
enId: string
|
||||
enName: string
|
||||
serviceId: string
|
||||
bytes: number
|
||||
}
|
||||
|
||||
function bumpClient(clients: Map<string, ClientAcc>, bytes: number, client: { userId: string; name: string } | null): void {
|
||||
const id = client?.userId || "—"
|
||||
const name = client?.name || "—"
|
||||
@@ -102,13 +131,195 @@ export function clampMapServiceMinSharePct(n: unknown): number {
|
||||
return Math.min(100, Math.max(0, v))
|
||||
}
|
||||
|
||||
/** Доля среди именованных брендов; порог ИЛИ топ-N, затем cap. */
|
||||
export function pickMapServices(ranked: FlowMapService[], minSharePct: number): FlowMapService[] {
|
||||
if (minSharePct <= 0) return ranked.slice(0, MAP_SERVICE_NODE_CAP)
|
||||
/** Доля от payload окна; порог ИЛИ топ-N, затем cap. */
|
||||
export function pickMapServices(
|
||||
ranked: FlowMapService[],
|
||||
minSharePct: number,
|
||||
cap: number = MAP_SERVICE_NODE_CAP,
|
||||
minNodes: number = MAP_SERVICE_MIN_NODES,
|
||||
): FlowMapService[] {
|
||||
if (minSharePct <= 0) return ranked.slice(0, cap)
|
||||
const minShare = minSharePct / 100
|
||||
return ranked
|
||||
.filter((s, i) => s.share >= minShare || i < MAP_SERVICE_MIN_NODES)
|
||||
.slice(0, MAP_SERVICE_NODE_CAP)
|
||||
.filter((s, i) => s.share >= minShare || i < minNodes)
|
||||
.slice(0, cap)
|
||||
}
|
||||
|
||||
function bumpDestTotal(totals: Map<string, DestTotal>, id: string, label: string, category: string, bytes: number): void {
|
||||
const prev = totals.get(id)
|
||||
if (prev) {
|
||||
prev.bytes += bytes
|
||||
return
|
||||
}
|
||||
totals.set(id, { label, category, bytes })
|
||||
}
|
||||
|
||||
function bumpDestFrom(
|
||||
edges: Map<string, DestEdgeAcc>,
|
||||
paths: Map<string, DestPathAcc>,
|
||||
toId: string,
|
||||
from: FromAcc,
|
||||
exporterId: string,
|
||||
fromId: string,
|
||||
enName: string,
|
||||
viaName: string,
|
||||
/** fromId ребра, если отличается от EN (вложенный слой: страна → сервис). */
|
||||
edgeFromId: string = fromId,
|
||||
): void {
|
||||
const edgeKey = `${edgeFromId}|${toId}`
|
||||
const prevEdge = edges.get(edgeKey)
|
||||
const namedClients = new Map<string, string>()
|
||||
for (const [id, c] of from.clients) {
|
||||
if (id !== "—") namedClients.set(id, c.name)
|
||||
}
|
||||
if (prevEdge) {
|
||||
prevEdge.bytes += from.bytes
|
||||
prevEdge.bytesFwd += from.bytes
|
||||
for (const [id, name] of namedClients) prevEdge.clients.set(id, name)
|
||||
} else {
|
||||
edges.set(edgeKey, {
|
||||
fromId: edgeFromId,
|
||||
toId,
|
||||
bytes: from.bytes,
|
||||
bytesFwd: from.bytes,
|
||||
bytesRev: 0,
|
||||
clients: namedClients,
|
||||
})
|
||||
}
|
||||
for (const [clientId, c] of from.clients) {
|
||||
const pathKey = `${clientId}|${fromId}|${toId}`
|
||||
const prevPath = paths.get(pathKey)
|
||||
if (prevPath) {
|
||||
prevPath.bytes += c.bytes
|
||||
if (exporterId !== fromId && prevPath.viaId === fromId) {
|
||||
prevPath.viaId = exporterId
|
||||
prevPath.viaName = viaName
|
||||
}
|
||||
if (prevPath.clientName === "—" && c.name !== "—") prevPath.clientName = c.name
|
||||
} else {
|
||||
paths.set(pathKey, {
|
||||
clientId,
|
||||
clientName: c.name,
|
||||
viaId: exporterId,
|
||||
viaName,
|
||||
enId: fromId,
|
||||
enName,
|
||||
serviceId: toId,
|
||||
bytes: c.bytes,
|
||||
})
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
function finalizeDestLayer(
|
||||
totals: Map<string, DestTotal>,
|
||||
edges: Map<string, DestEdgeAcc>,
|
||||
paths: Map<string, DestPathAcc>,
|
||||
windowSec: number,
|
||||
minSharePct: number,
|
||||
shareBase: number,
|
||||
cap: number = MAP_SERVICE_NODE_CAP,
|
||||
minNodes: number = MAP_SERVICE_MIN_NODES,
|
||||
): { nodes: FlowMapService[]; edges: FlowMapServiceEdge[]; paths: FlowMapServicePath[] } {
|
||||
const nodes = pickMapServices(
|
||||
[...totals.entries()]
|
||||
.map(([id, s]) => ({
|
||||
id,
|
||||
label: s.label,
|
||||
category: s.category,
|
||||
bytes: s.bytes,
|
||||
bps: (s.bytes * 8) / windowSec,
|
||||
share: shareBase > 0 ? s.bytes / shareBase : 0,
|
||||
}))
|
||||
.sort((a, b) => b.bytes - a.bytes),
|
||||
minSharePct,
|
||||
cap,
|
||||
minNodes,
|
||||
)
|
||||
const keep = new Set(nodes.map((s) => s.id))
|
||||
const outEdges: FlowMapServiceEdge[] = [...edges.values()]
|
||||
.filter((e) => keep.has(e.toId))
|
||||
.map((e) => {
|
||||
const clients = [...e.clients.entries()].map(([id, name]) => ({ id, name }))
|
||||
const first = clients[0]
|
||||
return {
|
||||
fromId: e.fromId,
|
||||
toId: e.toId,
|
||||
bytes: e.bytes,
|
||||
bps: (e.bytes * 8) / windowSec,
|
||||
bpsFwd: (e.bytesFwd * 8) / windowSec,
|
||||
bpsRev: (e.bytesRev * 8) / windowSec,
|
||||
...(first ? { clientId: first.id, clientName: first.name } : {}),
|
||||
...(clients.length ? { clients } : {}),
|
||||
}
|
||||
})
|
||||
.sort((a, b) => b.bytes - a.bytes)
|
||||
const outPaths: FlowMapServicePath[] = [...paths.values()]
|
||||
.filter((p) => keep.has(p.serviceId))
|
||||
.map((p) => ({
|
||||
clientId: p.clientId,
|
||||
clientName: p.clientName,
|
||||
viaId: p.viaId,
|
||||
viaName: p.viaName,
|
||||
enId: p.enId,
|
||||
enName: p.enName,
|
||||
serviceId: p.serviceId,
|
||||
bytes: p.bytes,
|
||||
bps: (p.bytes * 8) / windowSec,
|
||||
}))
|
||||
.sort((a, b) => b.bps - a.bps)
|
||||
return { nodes, edges: outEdges, paths: outPaths }
|
||||
}
|
||||
|
||||
/** Группы сервисов по странам: только страны, прошедшие отбор слоя стран; доля — от байтов страны, не окна. */
|
||||
function buildCountryServiceGroups(
|
||||
nestedTotals: Map<string, DestTotal>,
|
||||
nestedEdges: Map<string, DestEdgeAcc>,
|
||||
nestedPaths: Map<string, DestPathAcc>,
|
||||
countryNodes: FlowMapService[],
|
||||
windowSec: number,
|
||||
minSharePct: number,
|
||||
): FlowMapCountryServiceGroup[] {
|
||||
const groups: FlowMapCountryServiceGroup[] = []
|
||||
for (const country of countryNodes) {
|
||||
const prefix = `${country.id}|`
|
||||
const totals = new Map<string, DestTotal>()
|
||||
const edges = new Map<string, DestEdgeAcc>()
|
||||
const paths = new Map<string, DestPathAcc>()
|
||||
for (const [id, t] of nestedTotals) {
|
||||
if (id.startsWith(prefix)) totals.set(id, t)
|
||||
}
|
||||
if (totals.size === 0) continue
|
||||
for (const [key, e] of nestedEdges) {
|
||||
if (e.toId.startsWith(prefix)) edges.set(key, e)
|
||||
}
|
||||
for (const [key, p] of nestedPaths) {
|
||||
if (p.serviceId.startsWith(prefix)) paths.set(key, p)
|
||||
}
|
||||
const out = finalizeDestLayer(
|
||||
totals,
|
||||
edges,
|
||||
paths,
|
||||
windowSec,
|
||||
minSharePct,
|
||||
country.bytes,
|
||||
MAP_COUNTRY_SERVICE_NODE_CAP,
|
||||
MAP_COUNTRY_SERVICE_MIN_NODES,
|
||||
)
|
||||
groups.push({ countryId: country.id, services: out.nodes, edges: out.edges, paths: out.paths })
|
||||
}
|
||||
return groups
|
||||
}
|
||||
|
||||
function countryDestFromRipe(ripe: FlowIpMeta | null, destKey: string): { id: string; label: string; category: string } {
|
||||
if (!destKey || destKey === "__other__" || !ripe?.ok) {
|
||||
return { id: mapCountryNodeId(""), label: OTHER_SERVICE, category: MAP_COUNTRY_CATEGORY }
|
||||
}
|
||||
const iso = resolveRipeCountry(ripe.country, ripe.asn, ripe.holder)
|
||||
if (!iso) {
|
||||
return { id: mapCountryNodeId(""), label: OTHER_SERVICE, category: MAP_COUNTRY_CATEGORY }
|
||||
}
|
||||
return { id: mapCountryNodeId(iso), label: iso, category: MAP_COUNTRY_CATEGORY }
|
||||
}
|
||||
|
||||
function hopsQueryKey(q: FlowMapHopsQuery, minSharePct: number): string {
|
||||
@@ -141,6 +352,16 @@ function ifaceUsable(name: string): boolean {
|
||||
return Boolean(name) && name !== "—"
|
||||
}
|
||||
|
||||
function resolveMapClient(
|
||||
topo: FlowTopology,
|
||||
serverId: number,
|
||||
inName: string,
|
||||
outName: string,
|
||||
) {
|
||||
return resolveClient(topo, serverId, inName)
|
||||
?? (ifaceUsable(outName) ? resolveClient(topo, serverId, outName) : null)
|
||||
}
|
||||
|
||||
function bump(acc: Map<string, HopAcc>, key: string, seed: Omit<HopAcc, "bytes" | "bytesFwd" | "bytesRev">, bytes: number, dir: "fwd" | "rev" | "both"): void {
|
||||
const prev = acc.get(key)
|
||||
const addFwd = dir === "fwd" || dir === "both" ? bytes : 0
|
||||
@@ -175,22 +396,6 @@ function toHop(a: HopAcc, windowSec: number): FlowMapHop {
|
||||
}
|
||||
}
|
||||
|
||||
/** Имя бренда без каталога EvoBGP — только ASN/CIDR кэш + proto. */
|
||||
function classifyMapDstLite(
|
||||
dst: string,
|
||||
proto: number,
|
||||
dstPort: number,
|
||||
srcPort: number,
|
||||
ripe: FlowIpMeta | null,
|
||||
): { service: string; category: string } | null {
|
||||
if (proto === 47 || proto === 50) return null
|
||||
const app = applicationName(proto, dstPort, srcPort)
|
||||
if (app === "WireGuard" || app === "DNS" || app === "SSH" || app === "BGP") return null
|
||||
const brand = resolveFlowBrand(dst, ripe?.asn ?? 0, ripe?.holder ?? "", proto, dstPort, srcPort)
|
||||
if (!brand || !isNamedInternetService(brand.service, brand.category)) return null
|
||||
return brand
|
||||
}
|
||||
|
||||
async function resolveMinSharePct(q: FlowMapHopsQuery): Promise<number> {
|
||||
if (q.minSharePct != null) return clampMapServiceMinSharePct(q.minSharePct)
|
||||
try {
|
||||
@@ -202,6 +407,7 @@ async function resolveMinSharePct(q: FlowMapHopsQuery): Promise<number> {
|
||||
}
|
||||
|
||||
async function buildFlowMapHopsUncached(q: FlowMapHopsQuery, minSharePct: number): Promise<FlowMapHopsDto> {
|
||||
refreshFlowCatalogInBackground()
|
||||
const windowSec = Math.max(60, q.minutes * 60)
|
||||
const raw = await listFlowRowsForWindow(q.minutes)
|
||||
const allow = q.userId ? await userIfaceAllow(q.userId) : null
|
||||
@@ -238,6 +444,21 @@ async function buildFlowMapHopsUncached(q: FlowMapHopsQuery, minSharePct: number
|
||||
const enIds = new Set(topo.enNodes.map((n) => n.id))
|
||||
let totalBytes = 0
|
||||
|
||||
function rowClient(r: (typeof working)[number]) {
|
||||
const inName = resolveIfaceName(r.serverId, r.inIface).name
|
||||
const outName = resolveIfaceName(r.serverId, r.outIface).name
|
||||
return resolveMapClient(topo, r.serverId, inName, outName)
|
||||
}
|
||||
|
||||
const payloadRows = wantDedup
|
||||
? dedupFlowRowsAcrossExporters(working, (a, b) => {
|
||||
const aCli = Boolean(rowClient(a))
|
||||
const bCli = Boolean(rowClient(b))
|
||||
if (aCli !== bCli) return aCli ? a : b
|
||||
return a.bytes >= b.bytes ? a : b
|
||||
})
|
||||
: working
|
||||
|
||||
for (const r of working) {
|
||||
const inRes = resolveIfaceName(r.serverId, r.inIface)
|
||||
const outRes = resolveIfaceName(r.serverId, r.outIface)
|
||||
@@ -323,11 +544,28 @@ async function buildFlowMapHopsUncached(q: FlowMapHopsQuery, minSharePct: number
|
||||
}, r.bytes, "fwd")
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
for (const r of payloadRows) {
|
||||
const inName = resolveIfaceName(r.serverId, r.inIface).name
|
||||
const outName = resolveIfaceName(r.serverId, r.outIface).name
|
||||
totalBytes += r.bytes
|
||||
const peer = pickInternetPeer(r.src, r.dst, r.srcPort, r.dstPort)
|
||||
const client = resolveClient(topo, r.serverId, inName)
|
||||
const prevDst = dstAcc.get(peer)
|
||||
const ep = resolveFlowEndpoints({
|
||||
src: r.src,
|
||||
dst: r.dst,
|
||||
srcPort: r.srcPort,
|
||||
dstPort: r.dstPort,
|
||||
ctx: destCtxForIface(topo, r.serverId, inName, {
|
||||
natSrc: r.natSrc,
|
||||
natDst: r.natDst,
|
||||
natSrcPort: r.natSrcPort,
|
||||
natDstPort: r.natDstPort,
|
||||
}),
|
||||
})
|
||||
const dest = ep.internetPeer
|
||||
const destKey = dest || "__other__"
|
||||
const client = resolveMapClient(topo, r.serverId, inName, outName)
|
||||
const prevDst = dstAcc.get(destKey)
|
||||
if (prevDst) {
|
||||
prevDst.bytes += r.bytes
|
||||
bumpFrom(prevDst, String(r.serverId), r.bytes, client)
|
||||
@@ -335,34 +573,24 @@ async function buildFlowMapHopsUncached(q: FlowMapHopsQuery, minSharePct: number
|
||||
const acc: DstAcc = {
|
||||
bytes: r.bytes,
|
||||
proto: r.proto,
|
||||
dstPort: r.dstPort,
|
||||
srcPort: r.srcPort,
|
||||
dstPort: ep.peerPort || r.dstPort,
|
||||
srcPort: ep.otherPort || r.srcPort,
|
||||
fromBytes: new Map(),
|
||||
}
|
||||
bumpFrom(acc, String(r.serverId), r.bytes, client)
|
||||
dstAcc.set(peer, acc)
|
||||
dstAcc.set(destKey, acc)
|
||||
}
|
||||
}
|
||||
|
||||
const svcTotals = new Map<string, { label: string; category: string; bytes: number }>()
|
||||
const svcEdges = new Map<string, {
|
||||
fromId: string
|
||||
toId: string
|
||||
bytes: number
|
||||
bytesFwd: number
|
||||
bytesRev: number
|
||||
clients: Map<string, string>
|
||||
}>()
|
||||
const svcPaths = new Map<string, {
|
||||
clientId: string
|
||||
clientName: string
|
||||
viaId: string
|
||||
viaName: string
|
||||
enId: string
|
||||
enName: string
|
||||
serviceId: string
|
||||
bytes: number
|
||||
}>()
|
||||
const svcTotals = new Map<string, DestTotal>()
|
||||
const svcEdges = new Map<string, DestEdgeAcc>()
|
||||
const svcPaths = new Map<string, DestPathAcc>()
|
||||
const ccTotals = new Map<string, DestTotal>()
|
||||
const ccEdges = new Map<string, DestEdgeAcc>()
|
||||
const ccPaths = new Map<string, DestPathAcc>()
|
||||
const nestedTotals = new Map<string, DestTotal>()
|
||||
const nestedEdges = new Map<string, DestEdgeAcc>()
|
||||
const nestedPaths = new Map<string, DestPathAcc>()
|
||||
|
||||
for (const h of hops.values()) {
|
||||
if (h.kind !== "gre" || !h.toId) continue
|
||||
@@ -395,108 +623,45 @@ async function buildFlowMapHopsUncached(q: FlowMapHopsQuery, minSharePct: number
|
||||
}
|
||||
|
||||
for (const [dst, acc] of dstAcc) {
|
||||
const ripe = resolveFlowIp(dst)
|
||||
const classified = classifyMapDstLite(dst, acc.proto, acc.dstPort, acc.srcPort, ripe)
|
||||
if (!classified) continue
|
||||
const toId = mapServiceNodeId(classified.service)
|
||||
const prevSvc = svcTotals.get(toId)
|
||||
if (prevSvc) prevSvc.bytes += acc.bytes
|
||||
else svcTotals.set(toId, { label: classified.service, category: classified.category, bytes: acc.bytes })
|
||||
const ripe = dst && dst !== "__other__" ? resolveFlowIp(dst) : null
|
||||
const classified = dst && dst !== "__other__"
|
||||
? mapInternetBrand(dst, acc.proto, acc.dstPort, acc.srcPort, ripe)
|
||||
: { service: OTHER_SERVICE, category: OTHER_SERVICE }
|
||||
const svcId = mapServiceNodeId(classified.service)
|
||||
bumpDestTotal(svcTotals, svcId, classified.service, classified.category, acc.bytes)
|
||||
const country = countryDestFromRipe(ripe, dst)
|
||||
bumpDestTotal(ccTotals, country.id, country.label, country.category, acc.bytes)
|
||||
const nestedId = mapCountryServiceNodeId(country.id, svcId)
|
||||
bumpDestTotal(nestedTotals, nestedId, classified.service, classified.category, acc.bytes)
|
||||
for (const [exporterId, from] of acc.fromBytes) {
|
||||
const fromId = anchorEnId(exporterId)
|
||||
if (!fromId) continue
|
||||
const edgeKey = `${fromId}|${toId}`
|
||||
const prevEdge = svcEdges.get(edgeKey)
|
||||
const namedClients = new Map<string, string>()
|
||||
for (const [id, c] of from.clients) {
|
||||
if (id !== "—") namedClients.set(id, c.name)
|
||||
}
|
||||
if (prevEdge) {
|
||||
prevEdge.bytes += from.bytes
|
||||
prevEdge.bytesFwd += from.bytes
|
||||
for (const [id, name] of namedClients) prevEdge.clients.set(id, name)
|
||||
} else {
|
||||
svcEdges.set(edgeKey, {
|
||||
fromId,
|
||||
toId,
|
||||
bytes: from.bytes,
|
||||
bytesFwd: from.bytes,
|
||||
bytesRev: 0,
|
||||
clients: namedClients,
|
||||
})
|
||||
}
|
||||
const enName = nodeName(fromId)
|
||||
const viaName = nodeName(exporterId)
|
||||
for (const [clientId, c] of from.clients) {
|
||||
const pathKey = `${clientId}|${exporterId}|${fromId}|${toId}`
|
||||
const prevPath = svcPaths.get(pathKey)
|
||||
if (prevPath) {
|
||||
prevPath.bytes += c.bytes
|
||||
} else {
|
||||
svcPaths.set(pathKey, {
|
||||
clientId,
|
||||
clientName: c.name,
|
||||
viaId: exporterId,
|
||||
viaName,
|
||||
enId: fromId,
|
||||
enName,
|
||||
serviceId: toId,
|
||||
bytes: c.bytes,
|
||||
})
|
||||
}
|
||||
}
|
||||
bumpDestFrom(svcEdges, svcPaths, svcId, from, exporterId, fromId, enName, viaName)
|
||||
bumpDestFrom(ccEdges, ccPaths, country.id, from, exporterId, fromId, enName, viaName)
|
||||
bumpDestFrom(nestedEdges, nestedPaths, nestedId, from, exporterId, fromId, enName, viaName, country.id)
|
||||
}
|
||||
}
|
||||
|
||||
const namedBytes = [...svcTotals.values()].reduce((n, s) => n + s.bytes, 0)
|
||||
const services = pickMapServices(
|
||||
[...svcTotals.entries()]
|
||||
.map(([id, s]) => ({
|
||||
id,
|
||||
label: s.label,
|
||||
category: s.category,
|
||||
bytes: s.bytes,
|
||||
bps: (s.bytes * 8) / windowSec,
|
||||
share: namedBytes > 0 ? s.bytes / namedBytes : 0,
|
||||
}))
|
||||
.sort((a, b) => b.bytes - a.bytes),
|
||||
const namedBytes = [...svcTotals.values()]
|
||||
.filter((s) => isNamedInternetService(s.label, s.category))
|
||||
.reduce((n, s) => n + s.bytes, 0)
|
||||
const unclassifiedBytes = Math.max(0, totalBytes - namedBytes)
|
||||
const shareBase = totalBytes > 0 ? totalBytes : namedBytes
|
||||
const servicesOut = finalizeDestLayer(svcTotals, svcEdges, svcPaths, windowSec, minSharePct, shareBase)
|
||||
const countriesOut = finalizeDestLayer(ccTotals, ccEdges, ccPaths, windowSec, minSharePct, shareBase)
|
||||
const countryServiceGroups = buildCountryServiceGroups(
|
||||
nestedTotals,
|
||||
nestedEdges,
|
||||
nestedPaths,
|
||||
countriesOut.nodes,
|
||||
windowSec,
|
||||
minSharePct,
|
||||
)
|
||||
const keepSvc = new Set(services.map((s) => s.id))
|
||||
const serviceEdges: FlowMapServiceEdge[] = [...svcEdges.values()]
|
||||
.filter((e) => keepSvc.has(e.toId))
|
||||
.map((e) => {
|
||||
const clients = [...e.clients.entries()].map(([id, name]) => ({ id, name }))
|
||||
const first = clients[0]
|
||||
return {
|
||||
fromId: e.fromId,
|
||||
toId: e.toId,
|
||||
bytes: e.bytes,
|
||||
bps: (e.bytes * 8) / windowSec,
|
||||
bpsFwd: (e.bytesFwd * 8) / windowSec,
|
||||
bpsRev: (e.bytesRev * 8) / windowSec,
|
||||
...(first ? { clientId: first.id, clientName: first.name } : {}),
|
||||
...(clients.length ? { clients } : {}),
|
||||
}
|
||||
})
|
||||
.sort((a, b) => b.bytes - a.bytes)
|
||||
|
||||
const servicePaths: FlowMapServicePath[] = [...svcPaths.values()]
|
||||
.filter((p) => keepSvc.has(p.serviceId))
|
||||
.map((p) => ({
|
||||
clientId: p.clientId,
|
||||
clientName: p.clientName,
|
||||
viaId: p.viaId,
|
||||
viaName: p.viaName,
|
||||
enId: p.enId,
|
||||
enName: p.enName,
|
||||
serviceId: p.serviceId,
|
||||
bytes: p.bytes,
|
||||
bps: (p.bytes * 8) / windowSec,
|
||||
}))
|
||||
.sort((a, b) => b.bps - a.bps)
|
||||
|
||||
const listener = getFlowListenerState()
|
||||
const geo = geoipReadersStatus()
|
||||
return {
|
||||
hops: [...hops.values()]
|
||||
.map((a) => toHop(a, windowSec))
|
||||
@@ -505,9 +670,17 @@ async function buildFlowMapHopsUncached(q: FlowMapHopsQuery, minSharePct: number
|
||||
rangeMinutes: q.minutes,
|
||||
windowSec,
|
||||
totalBytes,
|
||||
services,
|
||||
serviceEdges,
|
||||
servicePaths,
|
||||
namedBytes,
|
||||
unclassifiedBytes,
|
||||
asnLoaded: geo.asnLoaded,
|
||||
countryLoaded: geo.countryLoaded,
|
||||
services: servicesOut.nodes,
|
||||
serviceEdges: servicesOut.edges,
|
||||
servicePaths: servicesOut.paths,
|
||||
countries: countriesOut.nodes,
|
||||
countryEdges: countriesOut.edges,
|
||||
countryPaths: countriesOut.paths,
|
||||
countryServiceGroups,
|
||||
mapServiceMinSharePct: minSharePct,
|
||||
dedupApplied: wantDedup,
|
||||
excludeMeshApplied: excludeMesh,
|
||||
|
||||
@@ -114,6 +114,8 @@ async function ensureIpfixFields(client: MikrotikClient): Promise<void> {
|
||||
"last-forwarded": "yes",
|
||||
"nat-src-address": "yes",
|
||||
"nat-dst-address": "yes",
|
||||
"nat-src-port": "yes",
|
||||
"nat-dst-port": "yes",
|
||||
})
|
||||
const rows = asRosArray<Record<string, unknown>>(await client.get("/ip/traffic-flow/ipfix"))
|
||||
const id = rows[0] ? rosRowId(rows[0]) : ""
|
||||
@@ -124,6 +126,9 @@ async function ensureIpfixFields(client: MikrotikClient): Promise<void> {
|
||||
await client.post("/ip/traffic-flow/ipfix/set", body)
|
||||
}
|
||||
|
||||
/** Максимум flows в RAM (docs: overflow обрезает новые 5-tuple). */
|
||||
export const FLOW_CACHE_ENTRIES = "256k"
|
||||
|
||||
async function ensureTrafficFlow(
|
||||
client: MikrotikClient,
|
||||
collectorIp: string,
|
||||
@@ -132,6 +137,7 @@ async function ensureTrafficFlow(
|
||||
const body = toRosBody({
|
||||
enabled: "yes",
|
||||
interfaces: "all",
|
||||
"cache-entries": FLOW_CACHE_ENTRIES,
|
||||
"active-flow-timeout": "1m",
|
||||
"inactive-flow-timeout": "15s",
|
||||
})
|
||||
@@ -165,6 +171,23 @@ async function ensureTrafficFlow(
|
||||
await client.put("/ip/traffic-flow/target", targetBody)
|
||||
}
|
||||
|
||||
/** GRE allow-fast-path=no: inner пакеты идут через CPU и попадают в Traffic Flow. Нагрузка на CPU. */
|
||||
export async function ensureGreSlowPath(client: MikrotikClient): Promise<number> {
|
||||
const rows = asRosArray<Record<string, unknown>>(await client.get("/interface/gre"))
|
||||
let patched = 0
|
||||
for (const row of rows) {
|
||||
const id = rosRowId(row)
|
||||
if (!id) continue
|
||||
const current = String(row["allow-fast-path"] ?? "true").toLowerCase()
|
||||
if (current === "false" || current === "no") continue
|
||||
await patchRosPath(client, `/interface/gre/${encodeRosId(id)}`, toRosBody({
|
||||
"allow-fast-path": "no",
|
||||
}))
|
||||
patched += 1
|
||||
}
|
||||
return patched
|
||||
}
|
||||
|
||||
export function usablePublicHost(raw: string | undefined): string {
|
||||
if (!raw) return ""
|
||||
const host = raw.split(",")[0]?.trim().replace(/^\[/, "").replace(/\]:\d+$/, "").split(":")[0]?.trim() ?? ""
|
||||
@@ -178,7 +201,7 @@ export function usablePublicHost(raw: string | undefined): string {
|
||||
|
||||
export async function applyFlowOverlay(
|
||||
serverIdRaw: string | number,
|
||||
opts?: { publicEndpoint?: string; requestHost?: string },
|
||||
opts?: { publicEndpoint?: string; requestHost?: string; disableGreFastPath?: boolean },
|
||||
): Promise<TrafficFlowOverlayResult> {
|
||||
const steps: string[] = []
|
||||
const keys = await ensureHostKeys()
|
||||
@@ -273,7 +296,20 @@ export async function applyFlowOverlay(
|
||||
}
|
||||
|
||||
await ensureTrafficFlow(client, settings.collectorIp, settings.flowListenPort)
|
||||
steps.push(`Traffic Flow → ${settings.collectorIp}:${settings.flowListenPort} ipfix (src auto)`)
|
||||
steps.push(`Traffic Flow → ${settings.collectorIp}:${settings.flowListenPort} ipfix (src auto, cache ${FLOW_CACHE_ENTRIES})`)
|
||||
|
||||
if (opts?.disableGreFastPath) {
|
||||
try {
|
||||
const n = await ensureGreSlowPath(client)
|
||||
steps.push(
|
||||
n > 0
|
||||
? `GRE allow-fast-path=no (${n}) — inner IPFIX через CPU`
|
||||
: "GRE already allow-fast-path=no",
|
||||
)
|
||||
} catch {
|
||||
steps.push("GRE allow-fast-path не изменён (нет /interface/gre)")
|
||||
}
|
||||
}
|
||||
|
||||
const listed = await listWireGuardInterfaces({ serverId: String(server.id), includePrivateKey: false })
|
||||
const created = listed.interfaces.find((i) => i.name === IFACE_NAME)
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
import assert from "node:assert/strict"
|
||||
import { parseFlowPacket, protoName, resetFlowTemplatesForTests, templateExporterCountForTests } from "./traffic-flow-parse.js"
|
||||
import { allocateOverlayAddress, FLOW_TARGET_SRC_AUTO, usablePublicHost } from "./traffic-flow-overlay.js"
|
||||
import { allocateOverlayAddress, FLOW_CACHE_ENTRIES, FLOW_TARGET_SRC_AUTO, usablePublicHost } from "./traffic-flow-overlay.js"
|
||||
|
||||
function netflowV5One(): Buffer {
|
||||
const buf = Buffer.alloc(24 + 48)
|
||||
@@ -38,6 +38,7 @@ assert.equal(usablePublicHost("192.168.1.10"), "")
|
||||
assert.equal(usablePublicHost("mm.example.com:443"), "mm.example.com")
|
||||
assert.equal(usablePublicHost("203.0.113.10"), "203.0.113.10")
|
||||
assert.equal(FLOW_TARGET_SRC_AUTO, "0.0.0.0")
|
||||
assert.equal(FLOW_CACHE_ENTRIES, "256k")
|
||||
|
||||
resetFlowTemplatesForTests()
|
||||
{
|
||||
@@ -182,6 +183,95 @@ resetFlowTemplatesForTests()
|
||||
assert.equal(extra[0]?.bytes, 1500)
|
||||
}
|
||||
|
||||
resetFlowTemplatesForTests()
|
||||
{
|
||||
const fieldSpecs: Array<[number, number]> = [
|
||||
[8, 4],
|
||||
[12, 4],
|
||||
[225, 4],
|
||||
[226, 4],
|
||||
[227, 2],
|
||||
[228, 2],
|
||||
[1, 4],
|
||||
]
|
||||
const tplSetLen = 4 + 4 + fieldSpecs.length * 4
|
||||
const tpl = Buffer.alloc(16 + tplSetLen)
|
||||
tpl.writeUInt16BE(10, 0)
|
||||
tpl.writeUInt16BE(tpl.length, 2)
|
||||
tpl.writeUInt16BE(2, 16)
|
||||
tpl.writeUInt16BE(tplSetLen, 18)
|
||||
tpl.writeUInt16BE(256, 20)
|
||||
tpl.writeUInt16BE(fieldSpecs.length, 22)
|
||||
let off = 24
|
||||
for (const [type, len] of fieldSpecs) {
|
||||
tpl.writeUInt16BE(type, off)
|
||||
tpl.writeUInt16BE(len, off + 2)
|
||||
off += 4
|
||||
}
|
||||
const recLen = fieldSpecs.reduce((n, [, len]) => n + len, 0)
|
||||
const data = Buffer.alloc(16 + 4 + recLen)
|
||||
data.writeUInt16BE(10, 0)
|
||||
data.writeUInt16BE(data.length, 2)
|
||||
data.writeUInt16BE(256, 16)
|
||||
data.writeUInt16BE(4 + recLen, 18)
|
||||
let d = 20
|
||||
data[d] = 10; data[d + 1] = 200; data[d + 2] = 100; data[d + 3] = 53; d += 4
|
||||
data[d] = 10; data[d + 1] = 200; data[d + 2] = 100; data[d + 3] = 1; d += 4
|
||||
data[d] = 0; data[d + 1] = 0; data[d + 2] = 0; data[d + 3] = 0; d += 4
|
||||
data[d] = 8; data[d + 1] = 8; data[d + 2] = 8; data[d + 3] = 8; d += 4
|
||||
data.writeUInt16BE(53880, d); d += 2
|
||||
data.writeUInt16BE(443, d); d += 2
|
||||
data.writeUInt32BE(900, d)
|
||||
parseFlowPacket(tpl, "10.255.254.9")
|
||||
const nat = parseFlowPacket(data, "10.255.254.9")
|
||||
assert.equal(nat.length, 1)
|
||||
assert.equal(nat[0]?.src, "10.200.100.53")
|
||||
assert.equal(nat[0]?.dst, "10.200.100.1")
|
||||
assert.equal(nat[0]?.natSrc, "0.0.0.0")
|
||||
assert.equal(nat[0]?.natDst, "8.8.8.8")
|
||||
assert.equal(nat[0]?.natSrcPort, 53880)
|
||||
assert.equal(nat[0]?.natDstPort, 443)
|
||||
assert.equal(nat[0]?.bytes, 900)
|
||||
}
|
||||
|
||||
resetFlowTemplatesForTests()
|
||||
{
|
||||
const fieldSpecs: Array<[number, number]> = [
|
||||
[225, 4],
|
||||
[12, 4],
|
||||
[1, 4],
|
||||
]
|
||||
const tplSetLen = 4 + 4 + fieldSpecs.length * 4
|
||||
const tpl = Buffer.alloc(16 + tplSetLen)
|
||||
tpl.writeUInt16BE(10, 0)
|
||||
tpl.writeUInt16BE(tpl.length, 2)
|
||||
tpl.writeUInt16BE(2, 16)
|
||||
tpl.writeUInt16BE(tplSetLen, 18)
|
||||
tpl.writeUInt16BE(256, 20)
|
||||
tpl.writeUInt16BE(fieldSpecs.length, 22)
|
||||
let off = 24
|
||||
for (const [type, len] of fieldSpecs) {
|
||||
tpl.writeUInt16BE(type, off)
|
||||
tpl.writeUInt16BE(len, off + 2)
|
||||
off += 4
|
||||
}
|
||||
const recLen = fieldSpecs.reduce((n, [, len]) => n + len, 0)
|
||||
const data = Buffer.alloc(16 + 4 + recLen)
|
||||
data.writeUInt16BE(10, 0)
|
||||
data.writeUInt16BE(data.length, 2)
|
||||
data.writeUInt16BE(256, 16)
|
||||
data.writeUInt16BE(4 + recLen, 18)
|
||||
let d = 20
|
||||
data[d] = 0; data[d + 1] = 0; data[d + 2] = 0; data[d + 3] = 0; d += 4
|
||||
data[d] = 8; data[d + 1] = 8; data[d + 2] = 8; data[d + 3] = 8; d += 4
|
||||
data.writeUInt32BE(10, d)
|
||||
parseFlowPacket(tpl, "10.255.254.10")
|
||||
const zeroNat = parseFlowPacket(data, "10.255.254.10")
|
||||
assert.equal(zeroNat[0]?.src, "")
|
||||
assert.equal(zeroNat[0]?.natSrc, "0.0.0.0")
|
||||
assert.equal(zeroNat[0]?.dst, "8.8.8.8")
|
||||
}
|
||||
|
||||
resetFlowTemplatesForTests()
|
||||
{
|
||||
const tpl = Buffer.alloc(16 + 16 + 20)
|
||||
|
||||
@@ -1,3 +1,5 @@
|
||||
import { canonicalIp } from "./traffic-flow-ip.js"
|
||||
|
||||
export interface ParsedFlow {
|
||||
src: string
|
||||
dst: string
|
||||
@@ -13,6 +15,8 @@ export interface ParsedFlow {
|
||||
flowEndMs: number
|
||||
natSrc: string
|
||||
natDst: string
|
||||
natSrcPort: number
|
||||
natDstPort: number
|
||||
}
|
||||
|
||||
export type ParsedFlowInput = Partial<ParsedFlow> & Pick<ParsedFlow, "src" | "dst" | "proto" | "bytes">
|
||||
@@ -33,6 +37,8 @@ export function emptyParsedFlow(): ParsedFlow {
|
||||
flowEndMs: 0,
|
||||
natSrc: "",
|
||||
natDst: "",
|
||||
natSrcPort: 0,
|
||||
natDstPort: 0,
|
||||
}
|
||||
}
|
||||
|
||||
@@ -40,11 +46,15 @@ export function normalizeParsedFlow(flow: ParsedFlowInput): ParsedFlow {
|
||||
return {
|
||||
...emptyParsedFlow(),
|
||||
...flow,
|
||||
nextHop: flow.nextHop ?? "",
|
||||
src: canonicalIp(flow.src) || flow.src || "",
|
||||
dst: canonicalIp(flow.dst) || flow.dst || "",
|
||||
nextHop: canonicalIp(flow.nextHop ?? ""),
|
||||
flowStartMs: flow.flowStartMs ?? 0,
|
||||
flowEndMs: flow.flowEndMs ?? 0,
|
||||
natSrc: flow.natSrc ?? "",
|
||||
natDst: flow.natDst ?? "",
|
||||
natSrc: canonicalIp(flow.natSrc ?? ""),
|
||||
natDst: canonicalIp(flow.natDst ?? ""),
|
||||
natSrcPort: flow.natSrcPort ?? 0,
|
||||
natDstPort: flow.natDstPort ?? 0,
|
||||
inIface: flow.inIface ?? "",
|
||||
outIface: flow.outIface ?? "",
|
||||
srcPort: flow.srcPort ?? 0,
|
||||
@@ -86,6 +96,12 @@ function ipv4(buf: Buffer, offset: number): string {
|
||||
return `${buf[offset]}.${buf[offset + 1]}.${buf[offset + 2]}.${buf[offset + 3]}`
|
||||
}
|
||||
|
||||
function usableIpfixIp(ip: string): boolean {
|
||||
const t = String(ip ?? "").trim()
|
||||
if (!t) return false
|
||||
return t !== "0.0.0.0" && t.toLowerCase() !== "::" && t.toLowerCase() !== "::0"
|
||||
}
|
||||
|
||||
function ipv6(buf: Buffer, offset: number): string {
|
||||
const parts: string[] = []
|
||||
for (let i = 0; i < 8; i++) parts.push(buf.readUInt16BE(offset + i * 2).toString(16))
|
||||
@@ -214,6 +230,8 @@ function recordFromFields(
|
||||
let flowEndMs = 0
|
||||
let natSrc = ""
|
||||
let natDst = ""
|
||||
let natSrcPort = 0
|
||||
let natDstPort = 0
|
||||
for (const f of fields) {
|
||||
const field = consumeField(buf, off, f.length, limit)
|
||||
if (!field) return null
|
||||
@@ -243,15 +261,21 @@ function recordFromFields(
|
||||
case 225:
|
||||
if (data.length === 4) {
|
||||
natSrc = ipv4(data, 0)
|
||||
if (!src) src = natSrc
|
||||
if (!usableIpfixIp(src) && usableIpfixIp(natSrc)) src = natSrc
|
||||
}
|
||||
break
|
||||
case 226:
|
||||
if (data.length === 4) {
|
||||
natDst = ipv4(data, 0)
|
||||
if (!dst) dst = natDst
|
||||
if (!usableIpfixIp(dst) && usableIpfixIp(natDst)) dst = natDst
|
||||
}
|
||||
break
|
||||
case 227:
|
||||
natSrcPort = readUint(data, 0, data.length)
|
||||
break
|
||||
case 228:
|
||||
natDstPort = readUint(data, 0, data.length)
|
||||
break
|
||||
case 4:
|
||||
proto = readUint(data, 0, data.length)
|
||||
break
|
||||
@@ -308,7 +332,7 @@ function recordFromFields(
|
||||
if (ifaceName && !inIface) inIface = ifaceName
|
||||
return {
|
||||
flow: normalizeParsedFlow({
|
||||
src, dst, proto, srcPort, dstPort, bytes, packets, inIface, outIface, nextHop, flowStartMs, flowEndMs, natSrc, natDst,
|
||||
src, dst, proto, srcPort, dstPort, bytes, packets, inIface, outIface, nextHop, flowStartMs, flowEndMs, natSrc, natDst, natSrcPort, natDstPort,
|
||||
}),
|
||||
next: off,
|
||||
}
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
import { dbAll, dbQuery } from "../db/index.js"
|
||||
import { ipv4ToInt, isNonPublicIp, parseCidrV4 } from "./traffic-flow-ip.js"
|
||||
import { canonicalIp, ipv4ToInt, isNonPublicIp, parseCidrV4 } from "./traffic-flow-ip.js"
|
||||
import { resolveRipeCountry } from "./traffic-flow-brands.js"
|
||||
|
||||
export interface FlowIpMeta {
|
||||
@@ -264,7 +264,7 @@ function negative(prefix: string): FlowIpMeta {
|
||||
|
||||
export function lookupRipeCached(ip: string): FlowIpMeta | null {
|
||||
loadSqlite()
|
||||
const trimmed = String(ip ?? "").trim()
|
||||
const trimmed = canonicalIp(ip)
|
||||
lastCandidateCount = 0
|
||||
if (!trimmed) return null
|
||||
if (isNonPublicIp(trimmed)) {
|
||||
|
||||
@@ -1,7 +1,8 @@
|
||||
import { db, dbAll } from "../db/index.js"
|
||||
import { parseJsonArray } from "../db/json.js"
|
||||
import { appUsers, servers, userInterfaceBindings } from "../db/schema.js"
|
||||
import { mapRosInterfaceType } from "../modules/users/iface-type.js"
|
||||
import { mapRosInterfaceType, parseRawInterfaces } from "../modules/users/iface-type.js"
|
||||
import { canonicalIp } from "./traffic-flow-ip.js"
|
||||
import type { PlaneTopology } from "./traffic-flow-planes.js"
|
||||
|
||||
export interface FlowClientBinding {
|
||||
@@ -25,6 +26,8 @@ export interface FlowTopology {
|
||||
enHosts: Set<string>
|
||||
jhHosts: Set<string>
|
||||
wanIfaces: Map<number, Set<string>>
|
||||
/** GRE/WG из последнего снимка RouterOS (`type`), без mgmt. */
|
||||
tunnelIfaces?: Map<number, Set<string>>
|
||||
plane: PlaneTopology
|
||||
}
|
||||
|
||||
@@ -48,6 +51,15 @@ export function invalidateFlowCatalogCache(): void {
|
||||
serverCatalogCache = null
|
||||
}
|
||||
|
||||
export function peekFlowTopology(): FlowTopology | null {
|
||||
if (seeded) return seeded
|
||||
return topologyCache?.topo ?? null
|
||||
}
|
||||
|
||||
export function peekServerCatalog(): { list: ServerCatalogEntry[]; byId: Map<number, ServerCatalogEntry> } | null {
|
||||
return serverCatalogCache
|
||||
}
|
||||
|
||||
export async function getServerCatalog(): Promise<{ list: ServerCatalogEntry[]; byId: Map<number, ServerCatalogEntry> }> {
|
||||
const now = Date.now()
|
||||
if (serverCatalogCache && now - serverCatalogCache.at < CATALOG_TTL_MS) {
|
||||
@@ -76,6 +88,25 @@ function ifaceKey(serverId: number, name: string): string {
|
||||
return `${serverId}|${name}`
|
||||
}
|
||||
|
||||
async function loadTunnelIfacesFromSnapshots(): Promise<Map<number, Set<string>>> {
|
||||
const rows = await dbAll<{ serverId: number; rawInterfaces: unknown }>(`
|
||||
SELECT DISTINCT ON (server_id) server_id AS "serverId", raw_interfaces AS "rawInterfaces"
|
||||
FROM server_snapshots
|
||||
ORDER BY server_id, polled_at DESC
|
||||
`)
|
||||
const map = new Map<number, Set<string>>()
|
||||
for (const r of rows) {
|
||||
const set = new Set<string>()
|
||||
for (const iface of parseRawInterfaces(r.rawInterfaces)) {
|
||||
if (iface.type !== "gre" && iface.type !== "wg") continue
|
||||
if (iface.name.toLowerCase() === "wg-flow") continue
|
||||
set.add(iface.name)
|
||||
}
|
||||
if (set.size) map.set(r.serverId, set)
|
||||
}
|
||||
return map
|
||||
}
|
||||
|
||||
export async function loadFlowTopology(): Promise<FlowTopology> {
|
||||
if (seeded) return seeded
|
||||
const now = Date.now()
|
||||
@@ -118,6 +149,7 @@ export async function loadFlowTopology(): Promise<FlowTopology> {
|
||||
for (const h of hosts) jhHosts.add(h)
|
||||
}
|
||||
}
|
||||
const tunnelIfaces = await loadTunnelIfacesFromSnapshots()
|
||||
const topo: FlowTopology = {
|
||||
clientIfaces,
|
||||
clientByIface,
|
||||
@@ -125,6 +157,7 @@ export async function loadFlowTopology(): Promise<FlowTopology> {
|
||||
enHosts,
|
||||
jhHosts,
|
||||
wanIfaces,
|
||||
tunnelIfaces,
|
||||
plane: {
|
||||
clientIfaceNames: allClientNames,
|
||||
enHosts,
|
||||
@@ -140,6 +173,20 @@ export function seedFlowTopologyForTests(topo: FlowTopology | null): void {
|
||||
invalidateFlowCatalogCache()
|
||||
}
|
||||
|
||||
export function flowOursHosts(topo: FlowTopology | null | undefined): Set<string> {
|
||||
const ours = new Set<string>()
|
||||
if (!topo) return ours
|
||||
for (const h of topo.enHosts) {
|
||||
const ip = canonicalIp(h)
|
||||
if (ip) ours.add(ip)
|
||||
}
|
||||
for (const h of topo.jhHosts) {
|
||||
const ip = canonicalIp(h)
|
||||
if (ip) ours.add(ip)
|
||||
}
|
||||
return ours
|
||||
}
|
||||
|
||||
export function resolveClient(
|
||||
topo: FlowTopology,
|
||||
serverId: number,
|
||||
@@ -168,10 +215,14 @@ export function resolveEn(
|
||||
|
||||
export function enGreIfaceNames(topo: FlowTopology, serverId: number, ifaceNames: string[]): string[] {
|
||||
const client = topo.clientIfaces.get(serverId) ?? new Set<string>()
|
||||
const wan = topo.wanIfaces.get(serverId) ?? new Set<string>()
|
||||
const typed = topo.tunnelIfaces?.get(serverId)
|
||||
return ifaceNames.filter((name) => {
|
||||
if (client.has(name)) return false
|
||||
if (client.has(name) || wan.has(name)) return false
|
||||
if (name === "wg-flow") return false
|
||||
return mapRosInterfaceType("", name) === "gre"
|
||||
if (typed && typed.size > 0) return typed.has(name)
|
||||
const t = mapRosInterfaceType("", name)
|
||||
return t === "gre" || t === "wg"
|
||||
})
|
||||
}
|
||||
|
||||
|
||||
@@ -0,0 +1,43 @@
|
||||
import assert from "node:assert/strict"
|
||||
import { mapVxlanRow } from "./vxlan-live.js"
|
||||
|
||||
const server = {
|
||||
id: 7,
|
||||
name: "mt-msk",
|
||||
host: "10.0.0.1",
|
||||
site: "MSK",
|
||||
country: "RU",
|
||||
} as Parameters<typeof mapVxlanRow>[0]
|
||||
|
||||
const row = mapVxlanRow(
|
||||
server,
|
||||
{
|
||||
".id": "*3",
|
||||
name: "vxlan-10",
|
||||
vni: "10010",
|
||||
port: "8472",
|
||||
"local-address": "10.0.0.1",
|
||||
running: "true",
|
||||
disabled: "false",
|
||||
l2mtu: "1500",
|
||||
"mac-learning": "true",
|
||||
"arp-proxy": "true",
|
||||
comment: "overlay",
|
||||
},
|
||||
[
|
||||
{ interface: "vxlan-10", "remote-ip": "10.0.1.1" },
|
||||
{ interface: "other", "remote-ip": "1.1.1.1" },
|
||||
{ interface: "vxlan-10", "remote-ip": "10.0.2.1" },
|
||||
],
|
||||
0,
|
||||
)
|
||||
|
||||
assert.equal(row.serverId, "7")
|
||||
assert.equal(row.vni, 10010)
|
||||
assert.equal(row.dstPort, 8472)
|
||||
assert.equal(row.status, "up")
|
||||
assert.equal(row.enabled, true)
|
||||
assert.deepEqual(row.remoteVteps, ["10.0.1.1", "10.0.2.1"])
|
||||
assert.equal(row.vtepIp, "10.0.0.1")
|
||||
|
||||
console.log("vxlan-live.test.ts: ok")
|
||||
@@ -0,0 +1,136 @@
|
||||
import { eq } from "drizzle-orm"
|
||||
import { db } from "../db/index.js"
|
||||
import { servers } from "../db/schema.js"
|
||||
import { MikrotikClient } from "./mikrotik.js"
|
||||
|
||||
type ServerRow = typeof servers.$inferSelect
|
||||
|
||||
interface RosVxlan {
|
||||
".id"?: string
|
||||
name?: string
|
||||
vni?: string
|
||||
port?: string
|
||||
"local-address"?: string
|
||||
"vtep-address"?: string
|
||||
running?: string
|
||||
disabled?: string
|
||||
comment?: string
|
||||
l2mtu?: string
|
||||
arp?: string
|
||||
"arp-proxy"?: string
|
||||
"mac-learning"?: string
|
||||
learning?: string
|
||||
}
|
||||
|
||||
interface RosVxlanVtep {
|
||||
".id"?: string
|
||||
interface?: string
|
||||
"remote-ip"?: string
|
||||
}
|
||||
|
||||
export type VxlanTunnelLive = {
|
||||
id: string
|
||||
rosId: string
|
||||
name: string
|
||||
vni: number
|
||||
port: number
|
||||
dstPort: number
|
||||
serverId: string
|
||||
vtepIp: string
|
||||
remoteVteps: string[]
|
||||
l2mtu: number
|
||||
arpProxy: boolean
|
||||
macLearning: boolean
|
||||
comment: string
|
||||
enabled: boolean
|
||||
status: "up" | "down"
|
||||
}
|
||||
|
||||
function rosYes(v: string | undefined): boolean {
|
||||
return v === "true" || v === "yes"
|
||||
}
|
||||
|
||||
function parseIntSafe(v: string | undefined, fallback: number): number {
|
||||
const n = Number.parseInt(v ?? "", 10)
|
||||
return Number.isFinite(n) ? n : fallback
|
||||
}
|
||||
|
||||
export function mapVxlanRow(
|
||||
server: ServerRow,
|
||||
vx: RosVxlan,
|
||||
vteps: RosVxlanVtep[],
|
||||
idx: number,
|
||||
): VxlanTunnelLive {
|
||||
const name = (vx.name ?? "").trim() || `vxlan-${idx + 1}`
|
||||
const rosId = String(vx[".id"] ?? name)
|
||||
const disabled = rosYes(vx.disabled)
|
||||
const running = rosYes(vx.running)
|
||||
const port = parseIntSafe(vx.port, 8472)
|
||||
const remoteVteps = vteps
|
||||
.filter((v) => (v.interface ?? "").trim() === name)
|
||||
.map((v) => (v["remote-ip"] ?? "").trim())
|
||||
.filter(Boolean)
|
||||
return {
|
||||
id: `${server.id}-${rosId}`,
|
||||
rosId,
|
||||
name,
|
||||
vni: parseIntSafe(vx.vni, 0),
|
||||
port: 0,
|
||||
dstPort: port,
|
||||
serverId: String(server.id),
|
||||
vtepIp: (vx["local-address"] ?? vx["vtep-address"] ?? "").trim(),
|
||||
remoteVteps,
|
||||
l2mtu: parseIntSafe(vx.l2mtu, 1500),
|
||||
arpProxy: rosYes(vx["arp-proxy"]) || vx.arp === "proxy-arp" || vx.arp === "enabled",
|
||||
macLearning: vx["mac-learning"] != null ? rosYes(vx["mac-learning"]) : vx.learning !== "false",
|
||||
comment: vx.comment ?? "",
|
||||
enabled: !disabled,
|
||||
status: !disabled && running ? "up" : "down",
|
||||
}
|
||||
}
|
||||
|
||||
async function fetchVxlanForServer(server: ServerRow): Promise<VxlanTunnelLive[]> {
|
||||
const client = MikrotikClient.fromServer(server)
|
||||
const [vxRaw, vtepRaw] = await Promise.all([
|
||||
client.get<RosVxlan[]>("/interface/vxlan"),
|
||||
client.get<RosVxlanVtep[]>("/interface/vxlan/vteps").catch(() => [] as RosVxlanVtep[]),
|
||||
])
|
||||
const list = Array.isArray(vxRaw) ? vxRaw : []
|
||||
const vteps = Array.isArray(vtepRaw) ? vtepRaw : []
|
||||
return list.map((vx, idx) => mapVxlanRow(server, vx, vteps, idx))
|
||||
}
|
||||
|
||||
export async function listVxlanTunnels(): Promise<VxlanTunnelLive[]> {
|
||||
const enabledServers = await db.select().from(servers).where(eq(servers.enabled, true))
|
||||
const results = await Promise.all(
|
||||
enabledServers.map(async (server) => {
|
||||
try {
|
||||
return await fetchVxlanForServer(server)
|
||||
} catch {
|
||||
return [] as VxlanTunnelLive[]
|
||||
}
|
||||
}),
|
||||
)
|
||||
return results.flat()
|
||||
}
|
||||
|
||||
export async function listVxlanTunnelsForServer(server: ServerRow): Promise<VxlanTunnelLive[]> {
|
||||
try {
|
||||
return await fetchVxlanForServer(server)
|
||||
} catch {
|
||||
return []
|
||||
}
|
||||
}
|
||||
|
||||
export async function countVxlanTunnels(): Promise<number> {
|
||||
try {
|
||||
const result = await Promise.race([
|
||||
listVxlanTunnels(),
|
||||
new Promise<null>((resolve) => setTimeout(() => resolve(null), 8_000)),
|
||||
])
|
||||
if (!result) return 0
|
||||
return result.length
|
||||
} catch {
|
||||
return 0
|
||||
}
|
||||
}
|
||||
@@ -3,6 +3,13 @@ import { db } from "../db/index.js"
|
||||
import { servers } from "../db/schema.js"
|
||||
import { MikrotikClient } from "./mikrotik.js"
|
||||
import type { WgIfaceDto, WgPeerDto } from "@mmapp/contracts/wireguard"
|
||||
import {
|
||||
canonicalWireguardSnapshot,
|
||||
type WgLiveAddr,
|
||||
type WgLiveIface,
|
||||
type WgLivePeer,
|
||||
type WgSnapshot,
|
||||
} from "./entity-snapshots.js"
|
||||
|
||||
type ServerRow = typeof servers.$inferSelect
|
||||
|
||||
@@ -35,6 +42,7 @@ interface RosWireGuardPeer {
|
||||
"client-address"?: string
|
||||
"client-dns"?: string
|
||||
"client-endpoint"?: string
|
||||
"private-key"?: string
|
||||
}
|
||||
|
||||
interface RosIpAddress {
|
||||
@@ -152,6 +160,85 @@ async function fetchForServer(
|
||||
})
|
||||
}
|
||||
|
||||
export async function fetchWireguardRestoreState(server: ServerRow): Promise<{
|
||||
client: MikrotikClient
|
||||
ifaces: WgLiveIface[]
|
||||
peers: WgLivePeer[]
|
||||
addrs: WgLiveAddr[]
|
||||
snapshot: WgSnapshot
|
||||
}> {
|
||||
const client = MikrotikClient.fromServer(server)
|
||||
const [ifacesRaw, peersRaw, addrsRaw] = await Promise.all([
|
||||
client.get<RosWireGuard[]>("/interface/wireguard"),
|
||||
client.get<RosWireGuardPeer[]>("/interface/wireguard/peers"),
|
||||
client.get<RosIpAddress[]>("/ip/address").catch(() => [] as RosIpAddress[]),
|
||||
])
|
||||
|
||||
const ifaces: WgLiveIface[] = (Array.isArray(ifacesRaw) ? ifacesRaw : []).map((w) => ({
|
||||
name: (w.name ?? "").trim(),
|
||||
rosId: String(w[".id"] ?? w.name ?? ""),
|
||||
listenPort: Number.parseInt(w["listen-port"] ?? "13231", 10) || 13231,
|
||||
mtu: Number.parseInt(w.mtu ?? "1420", 10) || 1420,
|
||||
privateKey: w["private-key"] ?? "",
|
||||
comment: w.comment ?? "",
|
||||
disabled: w.disabled === "true" || w.disabled === "yes",
|
||||
}))
|
||||
|
||||
const peers: WgLivePeer[] = (Array.isArray(peersRaw) ? peersRaw : []).map((p, idx) => {
|
||||
const mapped = mapPeer(p, idx)
|
||||
const ep = (p["endpoint-address"] ?? "").trim()
|
||||
const port = (p["endpoint-port"] ?? "").trim()
|
||||
const ka = p["persistent-keepalive"] ? Number.parseInt(p["persistent-keepalive"], 10) : NaN
|
||||
return {
|
||||
rosId: mapped.rosId,
|
||||
interfaceName: (p.interface ?? "").trim(),
|
||||
publicKey: mapped.publicKey,
|
||||
allowedAddresses: mapped.allowedIps,
|
||||
endpointAddress: ep,
|
||||
endpointPort: port,
|
||||
persistentKeepalive: Number.isFinite(ka) ? ka : null,
|
||||
comment: mapped.comment ?? "",
|
||||
name: mapped.name ?? "",
|
||||
disabled: mapped.disabled === true,
|
||||
privateKey: p["private-key"] ?? "",
|
||||
clientAddress: mapped.clientAddress ?? "",
|
||||
clientDns: mapped.clientDns ?? "",
|
||||
clientEndpoint: mapped.clientEndpoint ?? "",
|
||||
}
|
||||
})
|
||||
|
||||
const addrs: WgLiveAddr[] = []
|
||||
for (const a of Array.isArray(addrsRaw) ? addrsRaw : []) {
|
||||
if (a.disabled === "true" || a.disabled === "yes") continue
|
||||
const iface = (a.interface ?? "").trim()
|
||||
const address = (a.address ?? "").trim()
|
||||
const rosId = String(a[".id"] ?? "")
|
||||
if (!iface || !address || !rosId) continue
|
||||
if (!ifaces.some((i) => i.name === iface)) continue
|
||||
addrs.push({ rosId, interfaceName: iface, address })
|
||||
}
|
||||
|
||||
const snapshot = canonicalWireguardSnapshot({
|
||||
interfaces: ifaces.map((iface) => ({
|
||||
name: iface.name,
|
||||
listenPort: iface.listenPort,
|
||||
mtu: iface.mtu,
|
||||
privateKey: iface.privateKey,
|
||||
address: addrs.find((a) => a.interfaceName === iface.name)?.address ?? "",
|
||||
comment: iface.comment,
|
||||
disabled: iface.disabled,
|
||||
peers: peers.filter((p) => p.interfaceName === iface.name),
|
||||
})),
|
||||
})
|
||||
|
||||
return { client, ifaces, peers, addrs, snapshot }
|
||||
}
|
||||
|
||||
export async function captureWireguardSnapshot(server: ServerRow): Promise<WgSnapshot> {
|
||||
const state = await fetchWireguardRestoreState(server)
|
||||
return state.snapshot
|
||||
}
|
||||
|
||||
export type WgListResult = {
|
||||
interfaces: WgIfaceDto[]
|
||||
failures: Array<{ serverId: string; serverName?: string; error: string }>
|
||||
|
||||
@@ -285,6 +285,19 @@ export interface OspfInterfaceRead {
|
||||
useBfd: boolean
|
||||
}
|
||||
|
||||
export interface OspfRouteRead {
|
||||
id: string
|
||||
serverId: number
|
||||
serverName: string
|
||||
serverSite: string
|
||||
destination: string
|
||||
type: "O" | "O IA" | "O E1" | "O E2"
|
||||
cost: number
|
||||
nextHop: string
|
||||
via: string
|
||||
area: string
|
||||
}
|
||||
|
||||
export interface OspfInstanceRead {
|
||||
id: string
|
||||
serverId: number
|
||||
@@ -306,6 +319,7 @@ export interface RosIpRoute {
|
||||
"dst-address": string
|
||||
"pref-src"?: string
|
||||
"gateway"?: string
|
||||
"immediate-gw"?: string
|
||||
"distance"?: string
|
||||
"scope"?: string
|
||||
"active"?: string // "true"
|
||||
@@ -314,6 +328,9 @@ export interface RosIpRoute {
|
||||
"connect"?: string
|
||||
"bgp"?: string
|
||||
"ospf"?: string
|
||||
"ospf-type"?: string
|
||||
"ospf-metric"?: string
|
||||
"ospf-area"?: string
|
||||
"rip"?: string
|
||||
"blackhole"?: string
|
||||
"unreachable"?: string
|
||||
|
||||
@@ -13,6 +13,7 @@ import {
|
||||
import {
|
||||
LayoutDashboardIcon,
|
||||
ActivityIcon,
|
||||
ChartColumnIcon,
|
||||
MapIcon,
|
||||
HeartPulseIcon,
|
||||
GlobeIcon,
|
||||
@@ -54,6 +55,7 @@ const navStructure: { label: string; items: NavItemBase[] }[] = [
|
||||
items: [
|
||||
{ title: "Дашборд", url: "/dashboard", icon: <LayoutDashboardIcon /> },
|
||||
{ title: "Трафик", url: "/traffic", icon: <ActivityIcon /> },
|
||||
{ title: "Статистика", url: "/statistics", icon: <ChartColumnIcon /> },
|
||||
{ title: "Карта сети", url: "/network-map", icon: <MapIcon /> },
|
||||
{ title: "Мониторинг", url: "/uptime", icon: <HeartPulseIcon /> },
|
||||
],
|
||||
@@ -104,7 +106,15 @@ const navStructure: { label: string; items: NavItemBase[] }[] = [
|
||||
},
|
||||
]
|
||||
|
||||
type LiveSidebarCounts = SidebarCountsDto & { greTunnels?: number; certificates?: number; wireguard?: number; users?: number }
|
||||
type LiveSidebarCounts = SidebarCountsDto & {
|
||||
greTunnels?: number
|
||||
certificates?: number
|
||||
wireguard?: number
|
||||
users?: number
|
||||
bgpSessions?: number
|
||||
vxlan?: number
|
||||
containers?: number
|
||||
}
|
||||
|
||||
export function AppSidebar({ ...props }: React.ComponentProps<typeof Sidebar>) {
|
||||
const { mode, backendUrl, prefsHydrated } = useDataSource()
|
||||
@@ -169,10 +179,9 @@ export function AppSidebar({ ...props }: React.ComponentProps<typeof Sidebar>) {
|
||||
if (url === "/gre") return formatSidebarBadgeCount(liveCounts.greTunnels ?? 0)
|
||||
if (url === "/certificates") return formatSidebarBadgeCount(liveCounts.certificates ?? 0)
|
||||
if (url === "/wireguard") return formatSidebarBadgeCount(liveCounts.wireguard ?? 0)
|
||||
|
||||
if (url === "/containers" || url === "/bgp") {
|
||||
return undefined
|
||||
}
|
||||
if (url === "/bgp") return formatSidebarBadgeCount(liveCounts.bgpSessions ?? 0)
|
||||
if (url === "/vxlan") return formatSidebarBadgeCount(liveCounts.vxlan ?? 0)
|
||||
if (url === "/containers") return formatSidebarBadgeCount(liveCounts.containers ?? 0)
|
||||
|
||||
return undefined
|
||||
}
|
||||
|
||||
@@ -4,7 +4,7 @@ import { useEffect, useState, useRef, useMemo } from "react"
|
||||
import { useRouter, usePathname } from "next/navigation"
|
||||
import { cn } from "@/lib/utils"
|
||||
import {
|
||||
SearchIcon, LayoutDashboardIcon, ActivityIcon, MapIcon, HeartPulseIcon,
|
||||
SearchIcon, LayoutDashboardIcon, ActivityIcon, ChartColumnIcon, MapIcon, HeartPulseIcon,
|
||||
GlobeIcon, NetworkIcon, LayersIcon, TagIcon, ServerIcon, FilterIcon,
|
||||
ShieldIcon, ShieldCheckIcon, CableIcon, BoxIcon, BadgeCheckIcon,
|
||||
HardDriveIcon, RouteIcon, GitForkIcon, GitMergeIcon, ScanLineIcon,
|
||||
@@ -27,6 +27,7 @@ const ALL_ITEMS: CommandItem[] = [
|
||||
// Обзор
|
||||
{ id: "dashboard", title: "Дашборд", group: "Обзор", url: "/dashboard", icon: <LayoutDashboardIcon />, keywords: ["главная","home","overview"] },
|
||||
{ id: "traffic", title: "Трафик", group: "Обзор", url: "/traffic", icon: <ActivityIcon />, keywords: ["bandwidth","traffic","клиенты","интерфейсы"] },
|
||||
{ id: "statistics", title: "Статистика", group: "Обзор", url: "/statistics", icon: <ChartColumnIcon />, keywords: ["stats","отчёт","куб","страны","asn","ipfix"] },
|
||||
{ id: "network-map", title: "Карта сети", group: "Обзор", url: "/network-map", icon: <MapIcon />, keywords: ["topology","топология","map"] },
|
||||
{ id: "uptime", title: "Мониторинг / Uptime", group: "Обзор", url: "/uptime", icon: <HeartPulseIcon />, keywords: ["ping","uptime","мониторинг","проверка"] },
|
||||
// Данные
|
||||
|
||||
@@ -0,0 +1,185 @@
|
||||
"use client"
|
||||
|
||||
import { useMemo, useState, type ComponentProps } from "react"
|
||||
import { HistoryIcon, LoaderCircleIcon, RotateCcwIcon } from "lucide-react"
|
||||
import {
|
||||
Sheet,
|
||||
SheetContent,
|
||||
SheetDescription,
|
||||
SheetHeader,
|
||||
SheetTitle,
|
||||
} from "@/components/ui/sheet"
|
||||
import {
|
||||
AlertDialog,
|
||||
AlertDialogAction,
|
||||
AlertDialogCancel,
|
||||
AlertDialogContent,
|
||||
AlertDialogDescription,
|
||||
AlertDialogFooter,
|
||||
AlertDialogHeader,
|
||||
AlertDialogMedia,
|
||||
AlertDialogTitle,
|
||||
} from "@/components/ui/alert-dialog"
|
||||
import { Button } from "@/components/ui/button"
|
||||
import { Badge } from "@/components/reui/badge"
|
||||
import { Frame, FramePanel } from "@/components/reui/frame"
|
||||
import {
|
||||
Timeline,
|
||||
TimelineContent,
|
||||
TimelineDate,
|
||||
TimelineHeader,
|
||||
TimelineIndicator,
|
||||
TimelineItem,
|
||||
TimelineSeparator,
|
||||
TimelineTitle,
|
||||
} from "@/components/reui/timeline"
|
||||
import { EmptyState } from "@/components/empty-state"
|
||||
import { Alert, AlertDescription, AlertTitle } from "@/components/reui/alert"
|
||||
import { TriangleAlertIcon } from "lucide-react"
|
||||
import type { ConfigRevisionDto } from "@/lib/config-revisions"
|
||||
|
||||
const SOURCE_LABEL: Record<ConfigRevisionDto["source"], string> = {
|
||||
apply: "Изменение",
|
||||
rollback: "Откат",
|
||||
observed: "С роутера",
|
||||
copy: "Копирование",
|
||||
}
|
||||
|
||||
const SOURCE_BADGE: Record<ConfigRevisionDto["source"], ComponentProps<typeof Badge>["variant"]> = {
|
||||
apply: "primary-light",
|
||||
rollback: "warning-light",
|
||||
observed: "secondary",
|
||||
copy: "success-light",
|
||||
}
|
||||
|
||||
function formatWhen(iso: string): string {
|
||||
const d = new Date(iso)
|
||||
if (Number.isNaN(d.getTime())) return iso
|
||||
return d.toLocaleString("ru-RU", { dateStyle: "short", timeStyle: "short" })
|
||||
}
|
||||
|
||||
export function ConfigHistorySheet({
|
||||
open,
|
||||
onOpenChange,
|
||||
title,
|
||||
itemLabel,
|
||||
revisions,
|
||||
loading,
|
||||
restoring,
|
||||
onRestore,
|
||||
}: {
|
||||
open: boolean
|
||||
onOpenChange: (open: boolean) => void
|
||||
title: string
|
||||
itemLabel: string
|
||||
revisions: ConfigRevisionDto[]
|
||||
loading: boolean
|
||||
restoring: boolean
|
||||
onRestore: (id: string) => Promise<void> | void
|
||||
}) {
|
||||
const [pending, setPending] = useState<ConfigRevisionDto | null>(null)
|
||||
const newestFirst = useMemo(() => revisions, [revisions])
|
||||
|
||||
return (
|
||||
<>
|
||||
<Sheet open={open} onOpenChange={onOpenChange}>
|
||||
<SheetContent side="right" className="w-full sm:max-w-md flex flex-col gap-0 p-0">
|
||||
<SheetHeader className="px-6 pt-6 pb-4 border-b shrink-0">
|
||||
<SheetTitle>{title}</SheetTitle>
|
||||
<SheetDescription>
|
||||
Снапшоты managed-объектов. Откат применяет выбранную версию на CHR.
|
||||
</SheetDescription>
|
||||
</SheetHeader>
|
||||
<div className="flex-1 min-h-0 overflow-y-auto px-6 py-5">
|
||||
{loading ? (
|
||||
<div className="flex items-center justify-center gap-2 py-16 text-sm text-muted-foreground">
|
||||
<LoaderCircleIcon className="size-4 animate-spin" />
|
||||
Загрузка истории…
|
||||
</div>
|
||||
) : newestFirst.length === 0 ? (
|
||||
<EmptyState
|
||||
icon={<HistoryIcon className="size-4" />}
|
||||
title="Истории пока нет"
|
||||
description="Снапшот появится после первого чтения или изменения на роутере"
|
||||
className="py-12"
|
||||
/>
|
||||
) : (
|
||||
<Frame>
|
||||
<FramePanel>
|
||||
<Timeline value={newestFirst.length} className="px-1">
|
||||
{newestFirst.map((rev, idx) => (
|
||||
<TimelineItem key={rev.id} step={newestFirst.length - idx}>
|
||||
<TimelineHeader>
|
||||
<TimelineDate dateTime={rev.createdAt}>{formatWhen(rev.createdAt)}</TimelineDate>
|
||||
<TimelineTitle className="flex items-center gap-2 flex-wrap">
|
||||
<Badge variant={SOURCE_BADGE[rev.source]} size="sm">
|
||||
{SOURCE_LABEL[rev.source]}
|
||||
</Badge>
|
||||
<span className="text-muted-foreground font-normal tabular-nums">
|
||||
{rev.itemCount} {itemLabel}
|
||||
</span>
|
||||
</TimelineTitle>
|
||||
</TimelineHeader>
|
||||
<TimelineIndicator />
|
||||
<TimelineSeparator />
|
||||
<TimelineContent>
|
||||
<Button
|
||||
type="button"
|
||||
variant="outline"
|
||||
size="sm"
|
||||
disabled={restoring}
|
||||
onClick={() => setPending(rev)}
|
||||
>
|
||||
<RotateCcwIcon className="size-3.5" />
|
||||
Откатить
|
||||
</Button>
|
||||
</TimelineContent>
|
||||
</TimelineItem>
|
||||
))}
|
||||
</Timeline>
|
||||
</FramePanel>
|
||||
</Frame>
|
||||
)}
|
||||
</div>
|
||||
</SheetContent>
|
||||
</Sheet>
|
||||
|
||||
<AlertDialog open={Boolean(pending)} onOpenChange={(v) => { if (!v && !restoring) setPending(null) }}>
|
||||
<AlertDialogContent>
|
||||
<AlertDialogHeader>
|
||||
<AlertDialogMedia className="bg-warning/10 text-warning">
|
||||
<TriangleAlertIcon />
|
||||
</AlertDialogMedia>
|
||||
<AlertDialogTitle>Откатить на эту версию?</AlertDialogTitle>
|
||||
<AlertDialogDescription className="flex flex-col gap-3">
|
||||
<span>
|
||||
На CHR будут применены {pending?.itemCount ?? 0} {itemLabel} от{" "}
|
||||
{pending ? formatWhen(pending.createdAt) : ""}.
|
||||
</span>
|
||||
<Alert variant="warning">
|
||||
<TriangleAlertIcon />
|
||||
<AlertTitle>Изменятся только объекты MikrotikManager</AlertTitle>
|
||||
<AlertDescription>
|
||||
Чужие правила и маршруты на роутере не удаляются.
|
||||
</AlertDescription>
|
||||
</Alert>
|
||||
</AlertDialogDescription>
|
||||
</AlertDialogHeader>
|
||||
<AlertDialogFooter>
|
||||
<AlertDialogCancel disabled={restoring} onClick={() => setPending(null)}>Отмена</AlertDialogCancel>
|
||||
<AlertDialogAction
|
||||
disabled={restoring || !pending}
|
||||
onClick={() => {
|
||||
if (!pending) return
|
||||
void Promise.resolve(onRestore(pending.id)).finally(() => setPending(null))
|
||||
}}
|
||||
>
|
||||
{restoring ? <LoaderCircleIcon className="size-4 animate-spin" /> : null}
|
||||
Откатить
|
||||
</AlertDialogAction>
|
||||
</AlertDialogFooter>
|
||||
</AlertDialogContent>
|
||||
</AlertDialog>
|
||||
</>
|
||||
)
|
||||
}
|
||||
@@ -11,7 +11,6 @@ import type { FilterRule, GreTunnel, Server } from "@/lib/data"
|
||||
import { Flag } from "@/components/flag"
|
||||
import { cn } from "@/lib/utils"
|
||||
import { Button } from "@/components/ui/button"
|
||||
import { Tooltip, TooltipContent, TooltipTrigger } from "@/components/ui/tooltip"
|
||||
import { DataGridShell } from "@/components/data-grids/shared/data-grid-shell"
|
||||
import {
|
||||
DATA_GRID_CELL_PAD,
|
||||
@@ -22,21 +21,15 @@ import { DataGridSortHeader } from "@/components/data-grids/shared/data-grid-sor
|
||||
import { EmptyState } from "@/components/empty-state"
|
||||
import {
|
||||
AlertCircleIcon,
|
||||
AlertTriangleIcon,
|
||||
CheckCircle2Icon,
|
||||
ChevronDownIcon,
|
||||
ChevronUpIcon,
|
||||
CircleDashedIcon,
|
||||
PencilIcon,
|
||||
RouteIcon,
|
||||
StarIcon,
|
||||
TrashIcon,
|
||||
XCircleIcon,
|
||||
FilterIcon,
|
||||
} from "lucide-react"
|
||||
|
||||
export type FilterRouterSyncStatus = "synced" | "drift" | "missing"
|
||||
|
||||
export interface RecursiveRouteLite {
|
||||
id: string
|
||||
dstAddress: string
|
||||
@@ -47,42 +40,6 @@ export interface RecursiveRouteLite {
|
||||
disabled: boolean
|
||||
}
|
||||
|
||||
function RouterSyncMarker({
|
||||
status,
|
||||
}: {
|
||||
status: FilterRouterSyncStatus | null | "skip"
|
||||
}) {
|
||||
if (status === "skip") {
|
||||
return <span className="size-3.5 shrink-0 block" aria-hidden />
|
||||
}
|
||||
const icon =
|
||||
status === "synced"
|
||||
? <CheckCircle2Icon className="size-3.5 text-emerald-600 dark:text-emerald-500 shrink-0" />
|
||||
: status === "drift"
|
||||
? <AlertTriangleIcon className="size-3.5 text-amber-500 shrink-0" />
|
||||
: status === "missing"
|
||||
? <XCircleIcon className="size-3.5 text-destructive shrink-0" />
|
||||
: <CircleDashedIcon className="size-3.5 text-muted-foreground/35 shrink-0" />
|
||||
const title =
|
||||
status === "synced"
|
||||
? "Совпадает с цепочкой bgp-in на MikroTik"
|
||||
: status === "drift"
|
||||
? "В БД и на роутере разное действие (gateway, blackhole или out-interface)"
|
||||
: status === "missing"
|
||||
? "Эта community не найдена в правиле bgp-in на роутере"
|
||||
: "Не проверено — нажмите «Сверить с роутером»"
|
||||
return (
|
||||
<Tooltip>
|
||||
<TooltipTrigger className="inline-flex cursor-default border-0 bg-transparent p-0">
|
||||
{icon}
|
||||
</TooltipTrigger>
|
||||
<TooltipContent side="top" className="max-w-xs">
|
||||
{title}
|
||||
</TooltipContent>
|
||||
</Tooltip>
|
||||
)
|
||||
}
|
||||
|
||||
function innerIpToGateway(ip: string) {
|
||||
return ip.split("/")[0]
|
||||
}
|
||||
@@ -234,8 +191,6 @@ interface FiltersDataGridProps {
|
||||
serversList: Server[]
|
||||
communityNameMap: Record<string, string>
|
||||
recursiveRoutes: RecursiveRouteLite[]
|
||||
routerSyncByCommunity?: Record<string, FilterRouterSyncStatus> | null
|
||||
isLive?: boolean
|
||||
enableSorting?: boolean
|
||||
onEdit: (rule: FilterRule) => void
|
||||
onDelete: (id: string) => void
|
||||
@@ -249,8 +204,6 @@ function FiltersDataGrid({
|
||||
serversList,
|
||||
communityNameMap,
|
||||
recursiveRoutes,
|
||||
routerSyncByCommunity,
|
||||
isLive,
|
||||
enableSorting = false,
|
||||
onEdit,
|
||||
onDelete,
|
||||
@@ -308,33 +261,6 @@ function FiltersDataGrid({
|
||||
size: 28,
|
||||
meta: { headerClassName: DATA_GRID_CELL_PAD, cellClassName: DATA_GRID_CELL_PAD },
|
||||
},
|
||||
{
|
||||
id: "routerSync",
|
||||
header: () => (
|
||||
<Tooltip>
|
||||
<TooltipTrigger className="cursor-help font-mono text-xs text-muted-foreground border-0 bg-transparent p-0">
|
||||
MT
|
||||
</TooltipTrigger>
|
||||
<TooltipContent side="top" className="max-w-xs">
|
||||
Совпадение с MikroTik (bgp-in)
|
||||
</TooltipContent>
|
||||
</Tooltip>
|
||||
),
|
||||
enableSorting: false,
|
||||
cell: ({ row }) => (
|
||||
<RouterSyncMarker
|
||||
status={
|
||||
!isLive
|
||||
? "skip"
|
||||
: !routerSyncByCommunity
|
||||
? null
|
||||
: routerSyncByCommunity[row.original.community.trim()] ?? null
|
||||
}
|
||||
/>
|
||||
),
|
||||
size: 32,
|
||||
meta: { headerClassName: DATA_GRID_CELL_PAD, cellClassName: DATA_GRID_CELL_PAD },
|
||||
},
|
||||
{
|
||||
id: "community",
|
||||
accessorKey: "community",
|
||||
@@ -413,13 +339,11 @@ function FiltersDataGrid({
|
||||
[
|
||||
communityNameMap,
|
||||
enableSorting,
|
||||
isLive,
|
||||
onDelete,
|
||||
onEdit,
|
||||
onMoveDown,
|
||||
onMoveUp,
|
||||
recursiveRoutes,
|
||||
routerSyncByCommunity,
|
||||
serversList,
|
||||
tunnelsList,
|
||||
],
|
||||
@@ -454,4 +378,4 @@ function FiltersDataGrid({
|
||||
)
|
||||
}
|
||||
|
||||
export { FiltersDataGrid, RouterSyncMarker, type FiltersDataGridProps }
|
||||
export { FiltersDataGrid, type FiltersDataGridProps }
|
||||
|
||||
@@ -75,7 +75,7 @@ const STATUS_MAP: Record<GreStatus, { label: string; dot: string }> = {
|
||||
}
|
||||
|
||||
function TunnelStatus({ status }: { status: GreStatus }) {
|
||||
const s = STATUS_MAP[status]
|
||||
const s = STATUS_MAP[status] ?? STATUS_MAP.degraded
|
||||
return (
|
||||
<span className="inline-flex items-center gap-1.5 text-sm">
|
||||
<span className={cn("size-1.5 rounded-full", s.dot)} />
|
||||
@@ -101,6 +101,10 @@ interface GreTunnelsDataGridProps {
|
||||
servers: Server[]
|
||||
pools: GrePool[]
|
||||
onCodePreview: (tunnel: GreTunnel) => void
|
||||
onEdit?: (tunnel: GreTunnel) => void
|
||||
onToggle?: (tunnel: GreTunnel) => void
|
||||
onDelete?: (tunnel: GreTunnel) => void
|
||||
mutationsLocked?: boolean
|
||||
}
|
||||
|
||||
function GreTunnelsDataGrid({
|
||||
@@ -108,6 +112,10 @@ function GreTunnelsDataGrid({
|
||||
servers,
|
||||
pools,
|
||||
onCodePreview,
|
||||
onEdit,
|
||||
onToggle,
|
||||
onDelete,
|
||||
mutationsLocked = false,
|
||||
}: GreTunnelsDataGridProps) {
|
||||
const serverMap = useMemo(() => new Map(servers.map((s) => [s.id, s])), [servers])
|
||||
const poolMap = useMemo(() => new Map(pools.map((p) => [p.id, p])), [pools])
|
||||
@@ -200,14 +208,21 @@ function GreTunnelsDataGrid({
|
||||
cell: ({ row }) => {
|
||||
const t = row.original
|
||||
if (!t.ipsec) return <span className="text-xs text-muted-foreground">—</span>
|
||||
const enc = t.ipsec.encAlg ? ENC_LABELS[t.ipsec.encAlg] : undefined
|
||||
const auth = t.ipsec.authAlg ? AUTH_LABELS[t.ipsec.authAlg] : undefined
|
||||
const dh = t.ipsec.dhGroup ? DH_LABELS[t.ipsec.dhGroup] : undefined
|
||||
const ike = t.ipsec.ikeVersion ? IKE_LABELS[t.ipsec.ikeVersion] : undefined
|
||||
if (!enc && !auth && !dh && !ike) {
|
||||
return <span className="text-xs text-muted-foreground">PSK · auto</span>
|
||||
}
|
||||
return (
|
||||
<div className="flex flex-col gap-0.5">
|
||||
<span className="text-xs font-mono">
|
||||
{ENC_LABELS[t.ipsec.encAlg]} / {AUTH_LABELS[t.ipsec.authAlg]}
|
||||
{[enc, auth].filter(Boolean).join(" / ") || "PSK"}
|
||||
</span>
|
||||
<span className="text-xs text-muted-foreground font-mono">
|
||||
{DH_LABELS[t.ipsec.dhGroup].split(" ")[0]} · {IKE_LABELS[t.ipsec.ikeVersion]}
|
||||
{t.ipsec.pfs && " · PFS"}
|
||||
{[dh?.split(" ")[0], ike].filter(Boolean).join(" · ")}
|
||||
{t.ipsec.pfs ? " · PFS" : ""}
|
||||
</span>
|
||||
</div>
|
||||
)
|
||||
@@ -297,16 +312,20 @@ function GreTunnelsDataGrid({
|
||||
<DropdownMenuItem onClick={() => onCodePreview(t)}>
|
||||
<CodeXmlIcon className="size-4" /> Просмотр кода
|
||||
</DropdownMenuItem>
|
||||
<DropdownMenuItem>
|
||||
<DropdownMenuItem disabled={mutationsLocked || !onEdit} onClick={() => onEdit?.(t)}>
|
||||
<PencilIcon className="size-4" /> Редактировать
|
||||
</DropdownMenuItem>
|
||||
<DropdownMenuSeparator />
|
||||
<DropdownMenuItem>
|
||||
<DropdownMenuItem disabled={mutationsLocked || !onToggle} onClick={() => onToggle?.(t)}>
|
||||
<PowerIcon className="size-4" />
|
||||
{t.enabled ? "Выключить" : "Включить"}
|
||||
</DropdownMenuItem>
|
||||
<DropdownMenuSeparator />
|
||||
<DropdownMenuItem variant="destructive">
|
||||
<DropdownMenuItem
|
||||
variant="destructive"
|
||||
disabled={mutationsLocked || !onDelete}
|
||||
onClick={() => onDelete?.(t)}
|
||||
>
|
||||
<Trash2Icon className="size-4" /> Удалить туннель
|
||||
</DropdownMenuItem>
|
||||
</DropdownMenuContent>
|
||||
@@ -318,7 +337,7 @@ function GreTunnelsDataGrid({
|
||||
meta: { headerClassName: DATA_GRID_CELL_PAD_LAST, cellClassName: DATA_GRID_CELL_PAD_LAST },
|
||||
},
|
||||
],
|
||||
[onCodePreview, poolMap, serverMap],
|
||||
[onCodePreview, onEdit, onToggle, onDelete, mutationsLocked, poolMap, serverMap],
|
||||
)
|
||||
|
||||
const table = useReactTable({
|
||||
|
||||
@@ -0,0 +1,101 @@
|
||||
"use client"
|
||||
|
||||
import { CompactDataGrid, type CompactDataGridColumn } from "@/components/data-grids/compact-data-grid"
|
||||
import { Flag } from "@/components/flag"
|
||||
import { Badge } from "@/components/reui/badge"
|
||||
import { fmtBps, formatBytes } from "@/lib/fmt-rate"
|
||||
import { cn } from "@/lib/utils"
|
||||
import {
|
||||
STATISTICS_DUP_MARK,
|
||||
STATISTICS_UNBOUND_USER_ID,
|
||||
STATISTICS_WAN_MARK,
|
||||
type StatisticsBreakdownRow,
|
||||
} from "@mmapp/contracts/statistics"
|
||||
|
||||
export type StatisticsSliceKind = "users" | "servers" | "interfaces" | "countries" | "services" | "asns"
|
||||
|
||||
export function StatisticsBreakdownDataGrid({
|
||||
rows,
|
||||
kind,
|
||||
selectedId,
|
||||
onRowClick,
|
||||
isLoading,
|
||||
density = "full",
|
||||
}: {
|
||||
rows: StatisticsBreakdownRow[]
|
||||
kind: StatisticsSliceKind
|
||||
selectedId?: string
|
||||
onRowClick?: (row: StatisticsBreakdownRow) => void
|
||||
isLoading?: boolean
|
||||
density?: "full" | "mini"
|
||||
}) {
|
||||
const mini = density === "mini"
|
||||
const columns: CompactDataGridColumn<StatisticsBreakdownRow>[] = [
|
||||
{
|
||||
id: "label",
|
||||
header: "Имя",
|
||||
accessorKey: "label",
|
||||
cell: (row) => (
|
||||
<span className={cn("flex items-center gap-2", selectedId === row.id && "font-medium")}>
|
||||
{kind === "countries" && row.id !== "XX" && row.id !== STATISTICS_UNBOUND_USER_ID ? (
|
||||
<Flag code={row.id} size={16} />
|
||||
) : null}
|
||||
<span className="truncate">{row.label || row.id}</span>
|
||||
{selectedId === row.id ? (
|
||||
<Badge variant="outline" size="sm">
|
||||
слайс
|
||||
</Badge>
|
||||
) : null}
|
||||
</span>
|
||||
),
|
||||
},
|
||||
{
|
||||
id: "bytes",
|
||||
header: "Байты",
|
||||
accessorKey: "bytes",
|
||||
cell: (row) => <span className="tabular-nums">{formatBytes(row.bytes)}</span>,
|
||||
},
|
||||
...(!mini
|
||||
? [
|
||||
{
|
||||
id: "packets",
|
||||
header: "Пакеты",
|
||||
accessorKey: "packets" as const,
|
||||
cell: (row: StatisticsBreakdownRow) => (
|
||||
<span className="tabular-nums">{row.packets.toLocaleString("ru-RU")}</span>
|
||||
),
|
||||
},
|
||||
{
|
||||
id: "bps",
|
||||
header: "Средний bitrate",
|
||||
accessorKey: "bps" as const,
|
||||
cell: (row: StatisticsBreakdownRow) => <span className="tabular-nums">{fmtBps(row.bps)}</span>,
|
||||
},
|
||||
]
|
||||
: []),
|
||||
{
|
||||
id: "percent",
|
||||
header: "Доля",
|
||||
accessorKey: "percent",
|
||||
cell: (row) => (
|
||||
<span className="tabular-nums">
|
||||
{row.percent === 0
|
||||
&& (row.label.includes(STATISTICS_WAN_MARK) || row.label.includes(`· ${STATISTICS_DUP_MARK}`))
|
||||
? "—"
|
||||
: `${row.percent.toFixed(1)}%`}
|
||||
</span>
|
||||
),
|
||||
},
|
||||
]
|
||||
|
||||
return (
|
||||
<CompactDataGrid
|
||||
data={rows}
|
||||
columns={columns}
|
||||
isLoading={isLoading}
|
||||
emptyTitle="Нет трафика"
|
||||
emptyDescription="За выбранный период и слайсы нет данных куба."
|
||||
onRowClick={onRowClick}
|
||||
/>
|
||||
)
|
||||
}
|
||||
@@ -3,6 +3,7 @@
|
||||
import { useMemo } from "react"
|
||||
import { type ColumnDef, getCoreRowModel, useReactTable } from "@tanstack/react-table"
|
||||
import type { FlowTalkerDto } from "@mmapp/contracts/traffic-flow"
|
||||
import { ArrowDownIcon, ArrowUpIcon, MinusIcon } from "lucide-react"
|
||||
import { DataGridShell } from "@/components/data-grids/shared/data-grid-shell"
|
||||
import {
|
||||
DATA_GRID_CELL_PAD,
|
||||
@@ -19,6 +20,40 @@ function formatBytes(n: number): string {
|
||||
return `${n} Б`
|
||||
}
|
||||
|
||||
function formatEndpoint(ip: string | undefined, port: number | undefined): string {
|
||||
const host = String(ip ?? "").trim()
|
||||
if (!host) return "—"
|
||||
return port ? `${host}:${port}` : host
|
||||
}
|
||||
|
||||
function packetTuple(row: FlowTalkerDto): string {
|
||||
const src = formatEndpoint(row.src, row.srcPort)
|
||||
const dst = formatEndpoint(row.dst, row.dstPort)
|
||||
return `${src} → ${dst}`
|
||||
}
|
||||
|
||||
function FlowDirectionMark({ direction }: { direction: FlowTalkerDto["direction"] }) {
|
||||
if (direction === "to_client") {
|
||||
return (
|
||||
<span className="inline-flex text-muted-foreground" title="Download: интернет → клиент">
|
||||
<ArrowDownIcon className="size-3" />
|
||||
</span>
|
||||
)
|
||||
}
|
||||
if (direction === "from_client") {
|
||||
return (
|
||||
<span className="inline-flex text-muted-foreground" title="Upload: клиент → интернет">
|
||||
<ArrowUpIcon className="size-3" />
|
||||
</span>
|
||||
)
|
||||
}
|
||||
return (
|
||||
<span className="inline-flex text-muted-foreground" title="Транзит">
|
||||
<MinusIcon className="size-3" />
|
||||
</span>
|
||||
)
|
||||
}
|
||||
|
||||
function TrafficFlowsDataGrid({
|
||||
rows,
|
||||
emptyHint,
|
||||
@@ -30,9 +65,16 @@ function TrafficFlowsDataGrid({
|
||||
() => [
|
||||
{
|
||||
id: "client",
|
||||
accessorFn: (r) => r.clientName ?? "",
|
||||
accessorFn: (r) => r.clientName || r.clientIp || "",
|
||||
header: () => <span className="text-xs font-medium text-muted-foreground">Клиент</span>,
|
||||
cell: ({ row }) => <span className="text-xs">{row.original.clientName || "—"}</span>,
|
||||
cell: ({ row }) => (
|
||||
<span className="flex min-w-0 flex-col gap-0.5">
|
||||
<span className="text-xs truncate">{row.original.clientName || "—"}</span>
|
||||
{row.original.clientIp ? (
|
||||
<span className="font-mono text-[10px] text-muted-foreground truncate">{row.original.clientIp}</span>
|
||||
) : null}
|
||||
</span>
|
||||
),
|
||||
meta: { headerClassName: DATA_GRID_CELL_PAD_FIRST, cellClassName: DATA_GRID_CELL_PAD_FIRST },
|
||||
},
|
||||
{
|
||||
@@ -43,27 +85,26 @@ function TrafficFlowsDataGrid({
|
||||
meta: { headerClassName: DATA_GRID_CELL_PAD, cellClassName: DATA_GRID_CELL_PAD },
|
||||
},
|
||||
{
|
||||
id: "src",
|
||||
accessorKey: "src",
|
||||
header: () => <span className="text-xs font-medium text-muted-foreground">Src</span>,
|
||||
cell: ({ row }) => (
|
||||
<span className="font-mono text-xs">
|
||||
{row.original.src}
|
||||
{row.original.srcPort ? `:${row.original.srcPort}` : ""}
|
||||
</span>
|
||||
),
|
||||
id: "direction",
|
||||
accessorFn: (r) => r.direction ?? "",
|
||||
header: () => <span className="text-xs font-medium text-muted-foreground">Напр.</span>,
|
||||
cell: ({ row }) => <FlowDirectionMark direction={row.original.direction} />,
|
||||
meta: { headerClassName: DATA_GRID_CELL_PAD, cellClassName: DATA_GRID_CELL_PAD },
|
||||
},
|
||||
{
|
||||
id: "dst",
|
||||
accessorKey: "dst",
|
||||
header: () => <span className="text-xs font-medium text-muted-foreground">Dst</span>,
|
||||
cell: ({ row }) => (
|
||||
<span className="font-mono text-xs">
|
||||
{row.original.dst}
|
||||
{row.original.dstPort ? `:${row.original.dstPort}` : ""}
|
||||
</span>
|
||||
),
|
||||
id: "internet",
|
||||
accessorFn: (r) => r.internetPeer ?? r.dst,
|
||||
header: () => <span className="text-xs font-medium text-muted-foreground">Интернет</span>,
|
||||
cell: ({ row }) => {
|
||||
const r = row.original
|
||||
const label = formatEndpoint(r.internetPeer, r.internetPeerPort)
|
||||
const tuple = packetTuple(r)
|
||||
return (
|
||||
<span className="font-mono text-xs truncate max-w-[220px]" title={tuple}>
|
||||
{label}
|
||||
</span>
|
||||
)
|
||||
},
|
||||
meta: { headerClassName: DATA_GRID_CELL_PAD, cellClassName: DATA_GRID_CELL_PAD },
|
||||
},
|
||||
{
|
||||
|
||||
+29
-2
@@ -20,9 +20,29 @@ const COUNTRY_NAMES: Record<string, string> = {
|
||||
HK: "Гонконг",
|
||||
}
|
||||
|
||||
/** Country name in Russian (fallback to code) */
|
||||
let regionNames: Intl.DisplayNames | null | undefined
|
||||
|
||||
function regionDisplayName(iso: string): string | undefined {
|
||||
try {
|
||||
if (regionNames === undefined) {
|
||||
regionNames = typeof Intl !== "undefined" && "DisplayNames" in Intl
|
||||
? new Intl.DisplayNames(["ru"], { type: "region" })
|
||||
: null
|
||||
}
|
||||
return regionNames?.of(iso) ?? undefined
|
||||
} catch {
|
||||
return undefined
|
||||
}
|
||||
}
|
||||
|
||||
/** Country name in Russian (fallback to ISO code) */
|
||||
export function countryName(code: string): string {
|
||||
return COUNTRY_NAMES[code.toUpperCase()] ?? code
|
||||
const iso = code.toUpperCase()
|
||||
if (!iso) return code
|
||||
if (COUNTRY_NAMES[iso]) return COUNTRY_NAMES[iso]
|
||||
const intl = regionDisplayName(iso)
|
||||
if (intl && intl !== iso) return intl
|
||||
return iso
|
||||
}
|
||||
|
||||
interface FlagProps {
|
||||
@@ -39,6 +59,13 @@ function nearestCdnSize(px: number): number {
|
||||
return CDN_SIZES.find(s => s >= px) ?? CDN_SIZES[CDN_SIZES.length - 1]
|
||||
}
|
||||
|
||||
/** CDN URL for SVG `<image href>` (flagcdn widths only). */
|
||||
export function flagCdnUrl(code: string, size = 40): string | null {
|
||||
const lower = code.toLowerCase()
|
||||
if (!/^[a-z]{2}$/.test(lower)) return null
|
||||
return `https://flagcdn.com/w${nearestCdnSize(size)}/${lower}.png`
|
||||
}
|
||||
|
||||
/**
|
||||
* Renders a flag <img> for a given ISO 3166-1 alpha-2 country code.
|
||||
* Source: https://flagcdn.com — free CDN, no API key needed.
|
||||
|
||||
@@ -0,0 +1,17 @@
|
||||
"use client"
|
||||
|
||||
import { flagCdnUrl } from "@/components/flag"
|
||||
|
||||
export function CountryFlagSvg({ iso, size = 22 }: { iso: string; size?: number }) {
|
||||
const url = flagCdnUrl(iso, size)
|
||||
if (!url) return null
|
||||
const h = Math.round(size * 0.75)
|
||||
return (
|
||||
<image
|
||||
href={url}
|
||||
width={size}
|
||||
height={h}
|
||||
preserveAspectRatio="xMidYMid meet"
|
||||
/>
|
||||
)
|
||||
}
|
||||
@@ -74,6 +74,14 @@ export function ServiceBrandIcon({ label, size = 22 }: { label: string; size?: n
|
||||
<path d="M10.2 9.2v5.6L15.6 12Z" fill="#fff" />
|
||||
</BrandSvg>
|
||||
)
|
||||
case "instagram":
|
||||
return (
|
||||
<BrandSvg size={size}>
|
||||
<rect x="3" y="3" width="18" height="18" rx="5" fill="#E4405F" />
|
||||
<circle cx="12" cy="12.2" r="4.1" fill="none" stroke="#fff" strokeWidth="1.8" />
|
||||
<circle cx="16.3" cy="7.7" r="1.15" fill="#fff" />
|
||||
</BrandSvg>
|
||||
)
|
||||
case "netflix":
|
||||
return (
|
||||
<BrandSvg size={size}>
|
||||
|
||||
File diff suppressed because it is too large
Load Diff
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user