Compare commits

...
9 Commits
Author SHA1 Message Date
DenozordecandCursor 0c0dfa1df7 fix(statistics): не двоить overlay и транзит в кубе
Docker images / prepare-release (push) Successful in 8s
Docker images / backend-test (push) Successful in 2m18s
Docker images / frontend-image (push) Successful in 3m25s
Docker images / updater-image (push) Successful in 48s
Docker images / backend-image (push) Successful in 3m9s
Docker images / notify-webhook (push) Skipped
Docker images / publish-release (push) Successful in 14s
Co-authored-by: Cursor <[email protected]>
2026-09-11 00:46:25 +07:00
DenozordecandCursor c162a41bc0 fix(network-map): подписать unbound-пути парой серверов
Docker images / prepare-release (push) Successful in 7s
Docker images / backend-test (push) Successful in 2m22s
Docker images / frontend-image (push) Successful in 3m29s
Docker images / updater-image (push) Successful in 52s
Docker images / backend-image (push) Successful in 2m41s
Docker images / notify-webhook (push) Skipped
Docker images / publish-release (push) Successful in 12s
Co-authored-by: Cursor <[email protected]>
2026-09-10 22:19:05 +07:00
DenozordecandCursor 97e43b2335 feat(statistics): enhance interface handling and data aggregation
Docker images / prepare-release (push) Successful in 9s
Docker images / backend-test (push) Successful in 2m29s
Docker images / frontend-image (push) Successful in 3m24s
Docker images / updater-image (push) Successful in 44s
Docker images / backend-image (push) Successful in 2m39s
Docker images / notify-webhook (push) Skipped
Docker images / publish-release (push) Successful in 15s
Updated the statistics aggregation service to improve interface resolution and data handling. Introduced new functions for managing interface aliases and collapsing server interface rows, ensuring accurate data representation. Enhanced test coverage for interface resolution and added checks for new functionality.

- Implemented `factIfaceAliases` and `collapseServerIfaceRows` for better interface data management.
- Updated `resolveIfaceName` to handle additional cases for interface indexing.
- Enhanced tests for interface resolution and aggregation logic.

Co-authored-by: Cursor <[email protected]>
2026-09-10 21:49:50 +07:00
DenozordecandCursor db21e1217c feat(network-map): enhance service selection logic and UI updates
Docker images / prepare-release (push) Successful in 9s
Docker images / backend-test (push) Successful in 2m19s
Docker images / frontend-image (push) Successful in 2m52s
Docker images / updater-image (push) Successful in 46s
Docker images / backend-image (push) Successful in 2m20s
Docker images / notify-webhook (push) Skipped
Docker images / publish-release (push) Successful in 11s
Refactor the service selection mechanism to utilize a live reference for selected services, ensuring accurate display of service details. Update UI components to reflect changes in service selection, including dynamic updates for service metrics and paths. Additionally, introduce new utility functions for deduplicating flow rows across exporters in the backend, improving data handling in traffic flow analysis.

- Updated service selection logic in NetworkMapPage component.
- Enhanced UI to display live service data.
- Added deduplication functions in traffic-flow-dedup module for improved data integrity.

Co-authored-by: Cursor <[email protected]>
2026-09-10 19:54:16 +07:00
DenozordecandCursor 5bb9066be8 feat(statistics): добавить BI-разрез и сводную матрицу
Docker images / prepare-release (push) Successful in 10s
Docker images / backend-test (push) Successful in 2m23s
Docker images / frontend-image (push) Successful in 3m30s
Docker images / updater-image (push) Successful in 52s
Docker images / backend-image (push) Successful in 3m8s
Docker images / notify-webhook (push) Skipped
Docker images / publish-release (push) Successful in 14s
Сопоставить клиентов с ifIndex как на карте трафика, чтобы KPI пользователей не обнулялся. На экране — разрез остальных измерений и сводная матрица.

Co-authored-by: Cursor <[email protected]>
2026-09-10 16:07:25 +07:00
DenozordecandCursor b1fd259f10 fix(statistics): убрать параметры хранения с родительских партиций
Docker images / prepare-release (push) Successful in 9s
Docker images / backend-test (push) Successful in 2m25s
Docker images / frontend-image (push) Successful in 3m26s
Docker images / updater-image (push) Successful in 45s
Docker images / backend-image (push) Successful in 2m32s
Docker images / notify-webhook (push) Skipped
Docker images / publish-release (push) Successful in 13s
PostgreSQL 42809: FILLFACTOR и autovacuum нельзя задавать на partitioned parent — только на листьях.

Co-authored-by: Cursor <[email protected]>
2026-09-10 13:36:35 +07:00
DenozordecandCursor 3687bb8fa2 feat(statistics): добавить раздел статистики трафика
Docker images / prepare-release (push) Successful in 10s
Docker images / backend-test (push) Successful in 2m19s
Docker images / frontend-image (push) Successful in 4m47s
Docker images / updater-image (push) Successful in 48s
Docker images / backend-image (push) Successful in 2m53s
Docker images / notify-webhook (push) Skipped
Docker images / publish-release (push) Successful in 13s
Куб IPFIX час+день с AND-слайсами и экраном отчётности /statistics, живой /traffic не меняем.

Co-authored-by: Cursor <[email protected]>
2026-09-10 11:57:47 +07:00
Denozordec a80caf5676 feat(netflow): локальные GeoLite2-базы для стран и ASN потоков
Docker images / prepare-release (push) Successful in 13s
Docker images / backend-test (push) Successful in 2m13s
Docker images / frontend-image (push) Successful in 4m31s
Docker images / updater-image (push) Successful in 48s
Docker images / backend-image (push) Successful in 3m1s
Docker images / notify-webhook (push) Skipped
Docker images / publish-release (push) Successful in 15s
Lookup страны/ASN при ingest теперь идёт сначала по локальным mmdb
MaxMind GeoLite2 (Country + ASN, зеркало P3TERX, без ключей), с
мгновенным синхронным ответом для IPv4/IPv6; RIPEstat остаётся
fallback до первой загрузки баз и при промахе.

- backend/src/services/traffic-flow-geoip.ts: ридеры maxmind, фасад
  resolveFlowIp (geoip-first → RIPE-кэш), статус ридеров
- geoip-update-collector.ts: conditional GET по ETag, валидация пробоем
  8.8.8.8 (US / AS15169), атомарная подмена с .prev-откатом
- джоба планировщика geoip_update (по умолчанию раз в 7 дней),
  настройки geoip_settings + миграция 0004, API /api/geoip
  (GET/PUT/update), контракты @mmapp/contracts/geoip
- engine/analytics/map-hops переведены на resolveFlowIp
- секция «GeoIP-базы» в настройках NetFlow, метки джобы на странице
  сбора данных, тесты test:geoip
2026-09-09 23:33:31 +07:00
DenozordecandCursor d39e3454aa feat(network-map): показывать ASN-бренды по доле среди сервисов
Docker images / prepare-release (push) Successful in 11s
Docker images / backend-test (push) Successful in 2m5s
Docker images / frontend-image (push) Successful in 3m15s
Docker images / updater-image (push) Successful in 48s
Docker images / backend-image (push) Successful in 2m44s
Docker images / notify-webhook (push) Skipped
Docker images / publish-release (push) Successful in 15s
Долю считать среди именованных брендов, а не от всего окна. На карте всегда оставлять топ-8 и добавить иконки новых брендов.

Co-authored-by: Cursor <[email protected]>
2026-09-09 16:53:06 +07:00
71 changed files with 6849 additions and 93 deletions
@@ -0,0 +1,63 @@
# Локальные GeoLite2-базы (Country + ASN) для потоков по странам и ASN
## Контекст
Сейчас страна и ASN для netflow-потоков резолвятся через внешний RIPEstat API (`backend/src/services/traffic-flow-ripe.ts`): лимит 30 новых префиксов/мин, очередь на 90, кэш в PG `flow_ip_meta`. Новые IP «дозревают» с задержкой, IPv6 не покрывается (кэш индексируется только по IPv4). Локальные mmdb-базы дают мгновенный синхронный lookup всех IP без внешних вызовов.
Решения (подтверждены):
- Источник — **MaxMind GeoLite2 через P3TERX-зеркало**: `https://github.com/P3TERX/GeoLite.mmdb/raw/download/GeoLite2-Country.mmdb` и `.../GeoLite2-ASN.mmdb`. Без регистрации, ключей и tar-распаковки. Точность по стране у GeoLite2 и IPinfo паритетная (<1% ошибок у обоих, arXiv 2026); выбран P3TERX за надёжность зеркала (5.2k звёзд) и преемственность: текущий RIPE-путь и так читает GeoLite (`maxmind-geo-lite`), история в кэше остаётся консистентной.
- **RIPEstat остаётся fallback** (до первой загрузки баз / если lookup не дал результата).
- City-базу не качаем (lat/lng фронтенд не использует).
## Изменения
### 1. Зависимость
- `npm install -w mikrotik-manager-backend maxmind` — sync-чтение mmdb, встроенные TS-типы, без транзитивных зависимостей, Node 22 ок.
### 2. Новый сервис `backend/src/services/traffic-flow-geoip.ts`
(по конвенциям окружения traffic-flow-*: контракт → маршрут → сервис, без БД-логики в маршрутах)
- Каталог: `backend/storage/geoip/` (конвенция `storage/backups`), файлы `GeoLite2-Country.mmdb`, `GeoLite2-ASN.mmdb`.
- `initGeoip()` — открыть ридеры best-effort при старте (из `index.ts` рядом с `startTrafficFlowListener`), независимо от настроек автообновления: файлы есть — работают.
- `lookupGeoip(ip): FlowIpMeta | null` — синхронно: `country.iso_code` (fallback `registered_country.iso_code`) с валидацией `isIsoCountry`, ASN = `autonomous_system_number`, holder = `autonomous_system_organization`; приватные IP → negative-запись как в RIPE (`isNonPublicIp`); в PG не пишем (lookup и так быстрый). IPv6 поддержан ридером.
- `resolveFlowIp(ip)` — фасад: `lookupGeoip(ip) ?? lookupRipeCached(ip)`; главный экспорт для потребителей.
- `geoipStatus()` — loaded, даты сборки баз (метаданные mmdb). Тест-хук `setGeoipReadersForTests`. Смена ридеров после обновления — атомарная замена ссылок.
### 3. Коллектор `backend/src/services/geoip-update-collector.ts`
`collectGeoipUpdateOnce()` по образцу `certificate-renew-collector.ts`:
1. Conditional GET с ETag/If-None-Match из настроек → 304 = skip (фолбэк-сравнение: размер/содержимое).
2. Скачивание в `*.tmp` через глобальный `fetch` + AbortController с таймаутом (внешний HTTP из service-слоя — по правилу fastify-backend-drizzle).
3. Валидация: открыть ридер из tmp-файла, пробой 8.8.8.8 (страна US, ASN 15169).
4. `fs.rename` атомарная подмена, старый файл → `*.prev` (откат, если новый ридер не открылся).
5. Перезагрузка ридеров, статус в настройках; snapshot для `scheduler_runs` (checked/downloaded/skipped/bytes/error).
### 4. Планировщик (`backend/src/services/scheduler.ts`)
- `JOB_KEYS` += `geoip_update`; case в `runSchedulerJobBody`; блок в `refreshScheduler()` по образцу `certificates_renew`: интервал `Math.max(6ч, updateIntervalSec*1000)`, по умолчанию 7 дней (upstream обновляется еженедельно) + немедленный первый запуск при включённой настройке.
### 5. Схема и миграция
- `backend/src/db/schema.ts`: singleton `geoip_settings``enabled` (default true), `updateIntervalSec` (default 604800), `lastCheckAt`, `lastSuccessAt`, `lastError`, `countryBuildAt`, `asnBuildAt`, `etagsJson` (jsonb), `createdAt/updatedAt`.
- Миграция: `npm run db:generate` → файл в `backend/drizzle/`.
### 6. API + контракты
- `packages/contracts/src/geoip.ts`: zod-схемы настроек/статуса (все входы — Zod, по правилам проекта).
- Новый `backend/src/routes/geoip.ts`, регистрация в `index.ts` с prefix `/api`:
- `GET /api/geoip` — настройки + статус (ready, даты сборки, последняя проверка/ошибка);
- `PUT /api/geoip` — сохранить настройки, затем `refreshScheduler()`;
- `POST /api/geoip/update` — запустить загрузку сейчас (409, если уже идёт; флаг-гард как в коллекторах).
### 7. Интеграция в пайплайн (geoip-first, RIPE-fallback)
- `traffic-flow-engine.ts` (`queueParsedFlows`, ~строка 336): `lookupRipeCached``resolveFlowIp`. Логика misses не меняется: при готовом mmdb публичные IP (v4+v6) резолвятся сразу, очередь RIPE пустеет; до скачивания баз — прежнее поведение.
- Остальные вызовы `lookupRipeCached``resolveFlowIp` (grep: как минимум `traffic-flow-analytics.ts` ~258271).
- `classifyFlowDst`/бренды не трогаем: holder из mmdb (org name) встаёт в существующие `HOLDER_BRANDS`-регулярки как есть.
### 8. Frontend (по next-shadcn-production / ui-guardian: только переиспользование)
- Секция «GeoIP-базы (GeoLite2)» внутри существующей `components/traffic/netflow-settings-panel.tsx`: статус (готово/не скачано, даты сборки Country/ASN, последняя проверка, ошибка), тумблер автообновления, интервал, кнопка «Обновить сейчас» с индикатором. Только уже используемые в панели примитивы (Switch/Button/поля) — никаких новых визуальных паттернов и Card-shell. API-клиент через существующие http-хелперы.
### 9. Хаускипинг, тесты, проверки
- `backend/.gitignore`: `storage/geoip/`.
- Тесты `backend/src/services/traffic-flow-geoip.test.ts` + скрипт `test:geoip` (по образцу `test:traffic-flow`): приоритет фасада (geoip hit → RIPE не зовётся; miss → fallback), negative на приватных IP, фильтрация EU/ZZ через `isIsoCountry`, коллектор с мокнутым fetch (304-skip, битый файл → подмены нет, `.prev` сохранён), dims по стране/ASN с засеянным ридером.
- Проверки после реализации (обязательно по правилам): типы/сборка бэка (`npm run build -w mikrotik-manager-backend`), типы фронта при правке UI (`npx tsc --noEmit`), `npm run test:geoip` и `test:traffic-flow`; предупреждения не игнорировать.
- Коммит: `feat(netflow): <subject по-русски>` — новая пользовательская фича (мгновенные страна/ASN в потоках), по commit-messages-ru.
- README: короткий раздел о GeoIP; примечание, что в Docker `storage/geoip` ephemeral без тома — базы перекачаются после пересоздания контейнера (~17 МБ); при желании смонтировать volume.
## Что это даёт
Страна и ASN появляются у потока мгновенно при ingest (включая IPv6), без ограничения скорости RIPE; dims `country`/`asn` в `flow_daily_dims`, аналитика (карта, топы, monthly) становятся полными сразу. Внешняя зависимость от stat.ripe.net остаётся только как fallback до первой загрузки баз.
+8
View File
@@ -190,6 +190,14 @@ npm run build -w @mmapp/contracts
npm --prefix backend run db:migrate-from-sqlite
```
### GeoIP-базы GeoLite2 (страны и ASN для NetFlow)
Backend держит локальные mmdb-базы MaxMind GeoLite2 (Country + ASN) в `backend/storage/geoip/` и скачивает их с зеркала [P3TERX/GeoLite.mmdb](https://github.com/P3TERX/GeoLite.mmdb) — без регистрации и ключей. Lookup страны/ASN потока при ingest становится мгновенным (включая IPv6) и не упирается в лимиты RIPEstat; пока базы не скачаны или lookup промахнулся, работает прежний RIPE-fallback.
Управление — секция «GeoIP-базы (GeoLite2)» в настройках NetFlow (страница «Сбор данных»): автообновление (по умолчанию проверка раз в 7 дней, upstream обновляется еженедельно), статус сборки баз и кнопка «Обновить сейчас». Джоба планировщика — `geoip_update`. Атрибуция: данные MaxMind GeoLite2, CC BY-SA 4.0.
Примечание для Docker: каталог `storage/geoip` внутри контейнера ephemeral — без смонтированного volume базы (~17 МБ) перекачаются после пересоздания контейнера. Каталог переопределяется переменной `GEOIP_DIR`.
## CI/CD (Gitea Actions)
Файл: `.gitea/workflows/docker.yml` (имя workflow: **Docker images**).
+36
View File
@@ -34,6 +34,7 @@ import {
type InternetPathRunSnapshot,
type CertificatesRenewRunSnapshot,
type BackupsRunSnapshot,
type GeoipUpdateRunSnapshot,
type PingRunSnapshot,
type ResourcesRunSnapshot,
type SchedulerRunSnapshot,
@@ -340,6 +341,41 @@ function SnapshotTables({ snap }: { snap: SchedulerRunSnapshot }) {
</div>
)
}
if (snap.job === "geoip_update") {
const g = snap as GeoipUpdateRunSnapshot
return (
<div className="flex flex-col gap-3">
{g.skipped ? (
<p className="text-xs text-amber-600 dark:text-amber-400">Прогон пропущен: обновление уже выполнялось или задача отключена.</p>
) : null}
<dl className="grid grid-cols-2 gap-3 text-xs sm:grid-cols-4">
<div>
<dt className="text-muted-foreground">Баз проверено</dt>
<dd className="font-mono font-medium">{g.checked}</dd>
</div>
<div>
<dt className="text-muted-foreground">Скачано</dt>
<dd className="font-mono font-medium">{g.downloaded}</dd>
</div>
<div>
<dt className="text-muted-foreground">Без изменений</dt>
<dd className="font-mono font-medium">{g.skippedUnchanged}</dd>
</div>
<div>
<dt className="text-muted-foreground">Объём, МБ</dt>
<dd className="font-mono font-medium">{(g.bytes / 1024 / 1024).toFixed(1)}</dd>
</div>
</dl>
{g.errors.length ? (
<div className="rounded-md border border-amber-500/40 bg-amber-500/10 px-3 py-2 text-xs text-amber-800 dark:text-amber-200 flex flex-col gap-1">
{g.errors.map((e, i) => (
<p key={i} className="break-words">{e}</p>
))}
</div>
) : null}
</div>
)
}
if (snap.job === "alert_engine") {
const a = snap as AlertEngineRunSnapshot
return (
+48 -33
View File
@@ -67,6 +67,7 @@ import {
CableIcon, CopyIcon, ActivityIcon, ExternalLinkIcon,
} from "lucide-react"
import { cn } from "@/lib/utils"
import { formatServicePathLabel, formatServicePathTitle } from "@/lib/format-service-path-label"
import Link from "next/link"
import { Flag } from "@/components/flag"
@@ -820,9 +821,15 @@ function ServicePathList({
{paths.map((p) => {
const rowKey = servicePathKey(p)
const via = servers.find((s) => s.id === p.viaId)
const viaLabel = via?.site || p.viaName
const en = servers.find((s) => s.id === p.enId)
const svc = services.find((s) => s.id === p.serviceId)
const mid = viaMode === "via" ? viaLabel : (svc?.label ?? p.serviceId)
const label = formatServicePathLabel(p, viaMode, {
viaName: via?.name,
viaSite: via?.site,
enName: en?.name,
serviceLabel: svc?.label,
})
const title = formatServicePathTitle(label, svc?.label ?? p.serviceId)
const active = Boolean(
highlight
&& highlight.viaId === p.viaId
@@ -833,13 +840,14 @@ function ServicePathList({
<button
key={rowKey}
type="button"
title={title}
onClick={() => onToggle(p)}
className={cn(
"flex items-center justify-between gap-2 rounded-md px-2 py-1.5 text-left text-xs transition-colors",
active ? "bg-cyan-500/15 ring-1 ring-cyan-500/40" : "hover:bg-muted/50",
)}
>
<span className="font-mono truncate min-w-0">{p.clientName} · {mid}</span>
<span className="font-mono truncate min-w-0">{label}</span>
<span className="font-mono text-emerald-400 tabular-nums shrink-0">
{formatNetflowRate({ bytes: p.bytes, bps: p.bps, bpsFwd: p.bps, bpsRev: 0 })}
</span>
@@ -1216,6 +1224,9 @@ export default function NetworkMapPage() {
// ── Interaction ─────────────────────────────────────────────────────────────
const [selected, setSelected] = useState<Server | null>(null)
const [selectedService, setSelectedService] = useState<FlowMapService | null>(null)
const liveSelectedService = selectedService
? (mapServices.find((s) => s.id === selectedService.id) ?? selectedService)
: null
const [highlightedPath, setHighlightedPath] = useState<{ viaId: string; enId: string; serviceId: string } | null>(null)
const [selWanIdx, setSelWanIdx] = useState<number | null>(null)
const [hoveredId, setHoveredId] = useState<string | null>(null)
@@ -2702,16 +2713,16 @@ export default function NetworkMapPage() {
})()}
</div>
</>
) : selectedService ? (
) : liveSelectedService ? (
<>
<div className="flex items-start gap-2 px-4 py-3 border-b">
<div className="mt-0.5">
<ServiceBrandIcon label={selectedService.label} size={22} />
<ServiceBrandIcon label={liveSelectedService.label} size={22} />
</div>
<div className="flex-1 min-w-0">
<p className="font-mono font-semibold text-sm truncate">{selectedService.label}</p>
<p className="font-mono font-semibold text-sm truncate">{liveSelectedService.label}</p>
<p className="text-xs text-muted-foreground mt-0.5">
Конечный сервис · {selectedService.category}
Конечный сервис · {liveSelectedService.category}
</p>
</div>
<button
@@ -2726,15 +2737,15 @@ export default function NetworkMapPage() {
<div className="flex flex-col gap-0">
<div className="flex items-center justify-between py-2 border-b border-border/50">
<span className="text-xs text-muted-foreground">Доля окна</span>
<span className="text-xs font-mono font-medium text-cyan-400">{serviceSharePct(selectedService.share)}</span>
<span className="text-xs font-mono font-medium text-cyan-400">{serviceSharePct(liveSelectedService.share)}</span>
</div>
<div className="flex items-center justify-between py-2 border-b border-border/50">
<span className="text-xs text-muted-foreground">Скорость</span>
<span className="text-xs font-mono font-medium">
{formatNetflowRate({
bytes: selectedService.bytes,
bps: selectedService.bps,
bpsFwd: selectedService.bps,
bytes: liveSelectedService.bytes,
bps: liveSelectedService.bps,
bpsFwd: liveSelectedService.bps,
bpsRev: 0,
})}
</span>
@@ -2742,34 +2753,34 @@ export default function NetworkMapPage() {
</div>
<div>
<p className="text-xs font-semibold text-muted-foreground uppercase tracking-wider mb-2">Выход</p>
<div className="flex flex-col gap-1.5">
{visibleServiceEdges.filter((e) => e.toId === selectedService.id).map((e) => {
<div className="flex flex-col gap-3">
{visibleServiceEdges.filter((e) => e.toId === liveSelectedService.id).map((e) => {
const src = mapServers.find((s) => s.id === e.fromId)
const enPaths = mapServicePaths
.filter((p) => p.serviceId === liveSelectedService.id && p.enId === e.fromId)
.slice()
.sort((a, b) => b.bps - a.bps)
return (
<div key={`${e.fromId}|${e.toId}`} className="flex items-center justify-between text-xs">
<span className="font-mono truncate">{src?.name ?? e.fromId}</span>
<span className="font-mono text-emerald-400 tabular-nums">
{formatNetflowRate({ bytes: e.bytes, bps: e.bps, bpsFwd: e.bpsFwd, bpsRev: e.bpsRev })}
</span>
<div key={`${e.fromId}|${e.toId}`} className="flex flex-col gap-1.5">
<div className="flex items-center justify-between text-xs">
<span className="font-mono truncate">{src?.name ?? e.fromId}</span>
<span className="font-mono text-emerald-400 tabular-nums">
{formatNetflowRate({ bytes: e.bytes, bps: e.bps, bpsFwd: e.bpsFwd, bpsRev: e.bpsRev })}
</span>
</div>
<ServicePathList
paths={enPaths}
servers={mapServers}
services={mapServices}
highlight={highlightedPath}
viaMode="via"
onToggle={togglePathHighlight}
/>
</div>
)
})}
</div>
</div>
<div>
<p className="text-xs font-semibold text-muted-foreground uppercase tracking-wider mb-2">Пути</p>
<ServicePathList
paths={mapServicePaths
.filter((p) => p.serviceId === selectedService.id)
.slice()
.sort((a, b) => b.bps - a.bps)}
servers={mapServers}
services={mapServices}
highlight={highlightedPath}
viaMode="via"
onToggle={togglePathHighlight}
/>
</div>
</div>
</>
) : selected ? (
@@ -3002,7 +3013,11 @@ export default function NetworkMapPage() {
<p className="text-xs font-semibold text-muted-foreground uppercase tracking-wider mb-2">Пути</p>
<ServicePathList
paths={mapServicePaths
.filter((p) => p.viaId === selected.id || p.enId === selected.id)
.filter((p) => (
selected.type === "exit-node"
? p.enId === selected.id
: p.viaId === selected.id
))
.slice()
.sort((a, b) => b.bps - a.bps)}
servers={mapServers}
+533
View File
@@ -0,0 +1,533 @@
"use client"
import { useCallback, useEffect, useMemo, useState } from "react"
import { useRouter, useSearchParams } from "next/navigation"
import {
ActivityIcon,
DatabaseIcon,
GaugeIcon,
ServerIcon,
UsersIcon,
} from "lucide-react"
import { PageHeader } from "@/components/page-header"
import { KpiStatGrid } from "@/components/reui-kit/kpi-stat-grid"
import { DataPageCard } from "@/components/data-page-card"
import { DataPageToolbar } from "@/components/data-page-toolbar"
import { SegmentedControl } from "@/components/form-kit"
import { EmptyState } from "@/components/empty-state"
import { PeriodSelector, rangeForPreset, type DateRangeYmd } from "@/components/statistics/period-selector"
import { StatisticsVolumeChart } from "@/components/statistics/statistics-volume-chart"
import { DimensionSelect, PivotDimSelect } from "@/components/statistics/dimension-select"
import { SliceChips } from "@/components/statistics/slice-chips"
import { BreakdownDashboard } from "@/components/statistics/breakdown-dashboard"
import { StatisticsPivotGrid } from "@/components/statistics/statistics-pivot-grid"
import {
StatisticsBreakdownDataGrid,
type StatisticsSliceKind,
} from "@/components/data-grids/statistics-breakdown-data-grid"
import { Alert, AlertDescription, AlertTitle } from "@/components/reui/alert"
import type { Filter } from "@/components/reui/filters"
import { STATISTICS_FILTER_FIELDS } from "@/lib/data-filters/statistics-filter-fields"
import {
isStatisticsPivotDim,
isStatisticsSliceKind,
STATISTICS_DIMS,
} from "@/lib/statistics-dims"
import { useDataSource } from "@/lib/data-source"
import { fmtBps, formatBytes } from "@/lib/fmt-rate"
import {
getStatistics,
getStatisticsPivot,
STATISTICS_UNBOUND_USER_ID,
type StatisticsDto,
type StatisticsPivotDto,
type StatisticsQuery,
} from "@/shared/api/statistics"
import type { StatisticsBreakdownRow, StatisticsPivotDim } from "@mmapp/contracts/statistics"
/**
* BI-куб трафика: критерий → остальные разрезы + pivot.
* Preview: https://reui.io/preview/base/dashboard-1 · https://reui.io/preview/base/stats-12
* · https://reui.io/preview/base/data-grid-filtering-2 · https://reui.io/preview/base/solution-analytics-8
* · https://reui.io/docs/components/base/frame · https://reui.io/docs/components/base/data-grid
*/
const EMPTY: StatisticsDto = {
from: "",
to: "",
grain: "day",
kpis: {
bytes: 0,
packets: 0,
avgBps: 0,
users: 0,
servers: 0,
ifaces: 0,
topCountry: "",
topService: "",
},
series: [],
users: [],
servers: [],
interfaces: [],
countries: [],
services: [],
asns: [],
}
const EMPTY_PIVOT: StatisticsPivotDto = {
rowDim: "country",
colDim: "service",
metric: "bytes",
columns: [],
rows: [],
otherBytes: 0,
}
interface CubeSlices {
country?: string
service?: string
asn?: string
serverId?: string
userId?: string
iface?: string
}
const SLICE_KEYS = ["country", "service", "asn", "serverId", "userId", "iface"] as const
function readRange(sp: URLSearchParams): DateRangeYmd {
const from = sp.get("from")
const to = sp.get("to")
if (from && to && from <= to) return { from, to }
return rangeForPreset("7d")
}
function readDim(sp: URLSearchParams): StatisticsSliceKind {
const t = sp.get("dim") ?? sp.get("tab")
return t && isStatisticsSliceKind(t) ? t : "users"
}
function readView(sp: URLSearchParams): "explore" | "pivot" {
return sp.get("view") === "pivot" ? "pivot" : "explore"
}
function readPivotDim(sp: URLSearchParams, key: string, fallback: StatisticsPivotDim): StatisticsPivotDim {
const v = sp.get(key)
return v && isStatisticsPivotDim(v) ? v : fallback
}
function readSlices(sp: URLSearchParams): CubeSlices {
const next: CubeSlices = {}
for (const key of SLICE_KEYS) {
const v = sp.get(key)?.trim()
if (v) next[key] = v
}
return next
}
function slicesToFilters(slices: CubeSlices): Filter[] {
return SLICE_KEYS.flatMap((key) => {
const val = slices[key]
if (!val) return []
return [{ id: key, field: key, operator: "is", values: [val] }]
})
}
function filtersToSlices(filters: Filter[]): CubeSlices {
const next: CubeSlices = {}
for (const f of filters) {
const raw = String(f.values[0] ?? "").trim()
if (!raw) continue
if (f.field === "country") next.country = raw.toUpperCase().slice(0, 2)
else if (f.field === "service") next.service = raw
else if (f.field === "asn") next.asn = raw.replace(/[^\d]/g, "")
else if (f.field === "serverId") next.serverId = raw
else if (f.field === "userId") next.userId = raw
else if (f.field === "iface") next.iface = raw
}
return next
}
function toQuery(range: DateRangeYmd, slices: CubeSlices): StatisticsQuery {
const serverId = slices.serverId ? Number(slices.serverId) : undefined
const asn = slices.asn != null && slices.asn !== "" ? Number(slices.asn) : undefined
return {
from: range.from,
to: range.to,
serverId: Number.isFinite(serverId) && (serverId ?? 0) > 0 ? serverId : undefined,
userId: slices.userId,
iface: slices.iface,
country: slices.country && slices.country.length === 2 ? slices.country : undefined,
service: slices.service,
asn: Number.isFinite(asn) ? asn : undefined,
}
}
function selectedIdForKind(kind: StatisticsSliceKind, slices: CubeSlices): string | undefined {
if (kind === "users") return slices.userId
if (kind === "servers") return slices.serverId
if (kind === "countries") return slices.country
if (kind === "services") return slices.service
if (kind === "asns") return slices.asn
if (kind === "interfaces" && slices.serverId && slices.iface) {
return `${slices.serverId}:${slices.iface}`
}
if (kind === "interfaces") return slices.iface
return undefined
}
function rowsForKind(data: StatisticsDto, kind: StatisticsSliceKind) {
if (kind === "users") return data.users
if (kind === "servers") return data.servers
if (kind === "interfaces") return data.interfaces
if (kind === "countries") return data.countries
if (kind === "services") return data.services
return data.asns
}
function hasAnySlice(slices: CubeSlices): boolean {
return SLICE_KEYS.some((k) => Boolean(slices[k]))
}
function hiddenKinds(slices: CubeSlices): Set<StatisticsSliceKind> {
const hidden = new Set<StatisticsSliceKind>()
if (slices.userId) hidden.add("users")
if (slices.serverId) hidden.add("servers")
if (slices.iface) hidden.add("interfaces")
if (slices.country) hidden.add("countries")
if (slices.service) hidden.add("services")
if (slices.asn) hidden.add("asns")
return hidden
}
function applyDimValue(slices: CubeSlices, kind: StatisticsSliceKind, rowId: string): CubeSlices {
const next: CubeSlices = { ...slices }
if (kind === "users") {
if (rowId === STATISTICS_UNBOUND_USER_ID) return next
if (next.userId === rowId) delete next.userId
else next.userId = rowId
} else if (kind === "servers") {
if (next.serverId === rowId) delete next.serverId
else next.serverId = rowId
} else if (kind === "countries") {
if (next.country === rowId) delete next.country
else next.country = rowId
} else if (kind === "services") {
if (next.service === rowId) delete next.service
else next.service = rowId
} else if (kind === "asns") {
if (next.asn === rowId) delete next.asn
else next.asn = rowId
} else {
const colon = rowId.indexOf(":")
const sid = colon >= 0 ? rowId.slice(0, colon) : undefined
const iface = colon >= 0 ? rowId.slice(colon + 1) : rowId
if (next.iface === iface && next.serverId === sid) {
delete next.iface
delete next.serverId
} else {
next.iface = iface
if (sid) next.serverId = sid
}
}
return next
}
function applyPivotDim(slices: CubeSlices, dim: StatisticsPivotDim, id: string): CubeSlices {
const kind = STATISTICS_DIMS.find((d) => d.pivot === dim)?.id ?? "users"
return applyDimValue(slices, kind, id)
}
function chipList(slices: CubeSlices): { key: string; label: string }[] {
const chips: { key: string; label: string }[] = []
if (slices.country) chips.push({ key: "country", label: `страна ${slices.country}` })
if (slices.service) chips.push({ key: "service", label: `сервис ${slices.service}` })
if (slices.asn) chips.push({ key: "asn", label: `ASN ${slices.asn}` })
if (slices.serverId) chips.push({ key: "serverId", label: `сервер ${slices.serverId}` })
if (slices.userId) chips.push({ key: "userId", label: `пользователь ${slices.userId}` })
if (slices.iface) chips.push({ key: "iface", label: `iface ${slices.iface}` })
return chips
}
export default function StatisticsPage() {
const router = useRouter()
const searchParams = useSearchParams()
const { mode, backendUrl, prefsHydrated } = useDataSource()
const isLive = mode === "live"
const range = useMemo(() => readRange(searchParams), [searchParams])
const slices = useMemo(() => readSlices(searchParams), [searchParams])
const filters = useMemo(() => slicesToFilters(slices), [slices])
const dim = useMemo(() => readDim(searchParams), [searchParams])
const view = useMemo(() => readView(searchParams), [searchParams])
const pivotRow = useMemo(() => readPivotDim(searchParams, "pivotRow", "country"), [searchParams])
const pivotCol = useMemo(() => readPivotDim(searchParams, "pivotCol", "service"), [searchParams])
const [data, setData] = useState<StatisticsDto>(EMPTY)
const [pivot, setPivot] = useState<StatisticsPivotDto>(EMPTY_PIVOT)
const [loading, setLoading] = useState(false)
const [error, setError] = useState<string | null>(null)
const replaceParams = useCallback(
(patch: Record<string, string | undefined>) => {
const sp = new URLSearchParams(searchParams.toString())
for (const [k, v] of Object.entries(patch)) {
if (v) sp.set(k, v)
else sp.delete(k)
}
const qs = sp.toString()
router.replace(qs ? `/statistics?${qs}` : "/statistics")
},
[router, searchParams],
)
const setRange = useCallback(
(next: DateRangeYmd) => {
replaceParams({ from: next.from, to: next.to })
},
[replaceParams],
)
const setSlices = useCallback(
(next: CubeSlices) => {
replaceParams({
country: next.country,
service: next.service,
asn: next.asn,
serverId: next.serverId,
userId: next.userId,
iface: next.iface,
})
},
[replaceParams],
)
useEffect(() => {
if (!prefsHydrated || !isLive) return
let cancelled = false
void (async () => {
setLoading(true)
setError(null)
try {
const query = toQuery(range, slices)
const dto = await getStatistics(backendUrl, query)
if (!cancelled) setData(dto)
if (view === "pivot" && pivotRow !== pivotCol) {
const matrix = await getStatisticsPivot(backendUrl, {
...query,
row: pivotRow,
col: pivotCol,
metric: "bytes",
})
if (!cancelled) setPivot(matrix)
}
} catch (e: unknown) {
if (!cancelled) {
setData(EMPTY)
setPivot(EMPTY_PIVOT)
setError(e instanceof Error ? e.message : "Не удалось загрузить статистику")
}
} finally {
if (!cancelled) setLoading(false)
}
})()
return () => {
cancelled = true
}
}, [backendUrl, isLive, prefsHydrated, range, slices, view, pivotRow, pivotCol])
const viewData = isLive ? data : EMPTY
const sliced = hasAnySlice(slices)
const emptyCube = !isLive || (!loading && viewData.kpis.bytes === 0)
function handleRowClick(kind: StatisticsSliceKind, row: StatisticsBreakdownRow) {
if (kind === "users" && row.id === STATISTICS_UNBOUND_USER_ID) return
setSlices(applyDimValue(slices, kind, row.id))
}
function handlePivotCell(rowId: string, colId: string) {
if (rowId === "__other__" || colId === "__other__") return
let next = applyPivotDim(slices, pivotRow, rowId)
next = applyPivotDim(next, pivotCol, colId)
replaceParams({
country: next.country,
service: next.service,
asn: next.asn,
serverId: next.serverId,
userId: next.userId,
iface: next.iface,
view: "explore",
})
}
const kpis = viewData.kpis
const chips = chipList(slices)
const countLabel =
view === "pivot"
? `${pivot.rows.length} × ${pivot.columns.length}`
: sliced
? `${STATISTICS_DIMS.filter((d) => !hiddenKinds(slices).has(d.id)).length} разрезов`
: `${rowsForKind(viewData, dim).length} строк`
return (
<div className="flex h-full flex-col">
<PageHeader
crumbs={[{ label: "Обзор", href: "/dashboard" }, { label: "Статистика" }]}
actions={<PeriodSelector range={range} onChange={setRange} />}
/>
<div className="flex flex-1 flex-col gap-4 overflow-y-auto px-4 py-4 md:gap-6 md:px-6 md:py-5">
{!isLive ? (
<Alert>
<AlertTitle>Живые данные выключены</AlertTitle>
<AlertDescription>
Куб статистики строится из IPFIX. Переключитесь на живой источник, чтобы увидеть отчёт.
</AlertDescription>
</Alert>
) : null}
{error ? (
<Alert variant="destructive">
<AlertTitle>Ошибка загрузки</AlertTitle>
<AlertDescription>{error}</AlertDescription>
</Alert>
) : null}
{!slices.serverId && isLive && !emptyCube ? (
<Alert>
<AlertTitle>Интернет сети</AlertTitle>
<AlertDescription>
KPI уникальный payload без overlay и транзита EN. Сумма WAN по серверам не равна интернету сети; для uplink откройте сервер.
</AlertDescription>
</Alert>
) : null}
<KpiStatGrid
aria-label="Сводка трафика"
isLoading={loading}
skeletonCount={5}
items={[
{
id: "bytes",
label: "Объём",
value: formatBytes(kpis.bytes),
hint: kpis.topCountry ? `топ: ${kpis.topCountry}` : undefined,
icon: <DatabaseIcon />,
iconClassName: "text-muted-foreground",
},
{
id: "packets",
label: "Пакеты",
value: kpis.packets.toLocaleString("ru-RU"),
hint: kpis.topService ? `топ: ${kpis.topService}` : undefined,
icon: <ActivityIcon />,
iconClassName: "text-muted-foreground",
},
{
id: "bps",
label: "Средний bitrate",
value: fmtBps(kpis.avgBps),
icon: <GaugeIcon />,
iconClassName: "text-muted-foreground",
},
{
id: "users",
label: "Пользователи",
value: String(kpis.users),
icon: <UsersIcon />,
iconClassName: "text-muted-foreground",
},
{
id: "servers",
label: "Серверы",
value: String(kpis.servers),
hint: slices.serverId
? (kpis.ifaces ? `${kpis.ifaces} iface` : undefined)
: "WAN — в слайсе сервера",
icon: <ServerIcon />,
iconClassName: "text-muted-foreground",
},
]}
/>
<StatisticsVolumeChart series={viewData.series} grain={viewData.grain} />
<DataPageCard>
<DataPageToolbar
leading={
<div className="flex flex-wrap items-center gap-3">
<SegmentedControl
value={view}
onChange={(next) => replaceParams({ view: next === "pivot" ? "pivot" : "explore" })}
options={[
{ value: "explore", label: "Разрез" },
{ value: "pivot", label: "Сводка" },
]}
/>
{view === "explore" && !sliced ? (
<DimensionSelect
label="Критерий"
value={dim}
onChange={(next) => replaceParams({ dim: next })}
/>
) : null}
{view === "pivot" ? (
<>
<PivotDimSelect
label="Строки"
value={pivotRow}
exclude={pivotCol}
onChange={(next) => replaceParams({ pivotRow: next })}
/>
<PivotDimSelect
label="Колонки"
value={pivotCol}
exclude={pivotRow}
onChange={(next) => replaceParams({ pivotCol: next })}
/>
</>
) : null}
</div>
}
filters={filters}
onFiltersChange={(next) => setSlices(filtersToSlices(next))}
filterFields={STATISTICS_FILTER_FIELDS}
countLabel={countLabel}
/>
<SliceChips
chips={chips}
onRemove={(key) => {
const next = { ...slices }
delete next[key as keyof CubeSlices]
setSlices(next)
}}
/>
{emptyCube ? (
<EmptyState
title="Нет данных куба"
description="За выбранный период нет IPFIX-фактов. Куб заполняется с момента деплоя, без бэкфилла за год."
/>
) : view === "pivot" ? (
<StatisticsPivotGrid data={isLive ? pivot : EMPTY_PIVOT} onCellClick={handlePivotCell} isLoading={loading} />
) : sliced ? (
<BreakdownDashboard
data={viewData}
hidden={hiddenKinds(slices)}
selectedIdFor={(kind) => selectedIdForKind(kind, slices)}
onRowClick={handleRowClick}
isLoading={loading}
/>
) : (
<StatisticsBreakdownDataGrid
rows={rowsForKind(viewData, dim)}
kind={dim}
selectedId={selectedIdForKind(dim, slices)}
onRowClick={(row) => handleRowClick(dim, row)}
isLoading={loading}
/>
)}
</DataPageCard>
</div>
</div>
)
}
+1
View File
@@ -5,3 +5,4 @@ dist/
*.db-wal
.env
storage/backups/
storage/geoip/
+19
View File
@@ -0,0 +1,19 @@
-- GeoLite2 mmdb (страна/ASN для netflow): настройки автообновления зеркала P3TERX
CREATE TABLE IF NOT EXISTS geoip_settings (
id BIGINT PRIMARY KEY CHECK (id = 1),
enabled BOOLEAN NOT NULL DEFAULT TRUE,
update_interval_sec INTEGER NOT NULL DEFAULT 604800,
last_check_at TIMESTAMPTZ,
last_success_at TIMESTAMPTZ,
last_error TEXT,
country_build_at TIMESTAMPTZ,
asn_build_at TIMESTAMPTZ,
etags_json JSONB NOT NULL DEFAULT '{}'::jsonb,
created_at TIMESTAMPTZ NOT NULL DEFAULT now(),
updated_at TIMESTAMPTZ NOT NULL DEFAULT now()
);
INSERT INTO geoip_settings (id)
VALUES (1)
ON CONFLICT (id) DO NOTHING;
+28
View File
@@ -0,0 +1,28 @@
-- Statistics cube: hour + daily facts (server × iface × country × service × ASN).
-- Compact types. FILLFACTOR/autovacuum нельзя на partitioned parent (PG 42809) —
-- задаются на листовых партициях в ensurePartitionFor.
-- Retention: DROP partitions only (see PARTITION_SPECS).
CREATE TABLE IF NOT EXISTS flow_hour_facts (
server_id BIGINT NOT NULL REFERENCES servers(id) ON DELETE CASCADE,
bucket_at TIMESTAMPTZ NOT NULL,
iface TEXT NOT NULL,
country CHAR(2) NOT NULL,
service TEXT NOT NULL,
asn INTEGER NOT NULL,
bytes BIGINT NOT NULL DEFAULT 0,
packets BIGINT NOT NULL DEFAULT 0,
PRIMARY KEY (server_id, bucket_at, iface, country, service, asn)
) PARTITION BY RANGE (bucket_at);
CREATE TABLE IF NOT EXISTS flow_daily_facts (
server_id BIGINT NOT NULL REFERENCES servers(id) ON DELETE CASCADE,
day DATE NOT NULL,
iface TEXT NOT NULL,
country CHAR(2) NOT NULL,
service TEXT NOT NULL,
asn INTEGER NOT NULL,
bytes BIGINT NOT NULL DEFAULT 0,
packets BIGINT NOT NULL DEFAULT 0,
PRIMARY KEY (server_id, day, iface, country, service, asn)
) PARTITION BY RANGE (day);
+4 -2
View File
@@ -15,11 +15,12 @@
"test:auth": "tsx src/lib/permissions.test.ts && tsx src/plugins/auth.smoke.test.ts",
"test:wireguard": "npx tsx src/services/wireguard-config.test.ts",
"test:traffic-rate": "tsx src/services/traffic-rate.test.ts",
"test:traffic-flow": "tsx src/services/traffic-flow-parse.test.ts && tsx src/services/traffic-flow-map-exporter.test.ts && tsx src/services/traffic-flow-ifaces.test.ts && tsx src/services/traffic-flow-dedup.test.ts && tsx src/services/traffic-flow-planes.test.ts && tsx src/services/traffic-flow-ip.test.ts && tsx src/services/traffic-flow-classify.test.ts && tsx src/services/traffic-flow-ripe.test.ts && tsx src/services/traffic-flow-brands.test.ts && tsx src/services/traffic-flow-ingest.test.ts && tsx src/services/traffic-flow-analytics.test.ts && tsx src/services/traffic-flow-map-hops.test.ts && tsx src/services/traffic-flow-purge.test.ts",
"test:traffic-flow": "tsx src/services/traffic-flow-parse.test.ts && tsx src/services/traffic-flow-map-exporter.test.ts && tsx src/services/traffic-flow-ifaces.test.ts && tsx src/services/traffic-flow-ifindex.test.ts && tsx src/services/traffic-flow-dedup.test.ts && tsx src/services/traffic-flow-planes.test.ts && tsx src/services/traffic-flow-ip.test.ts && tsx src/services/traffic-flow-classify.test.ts && tsx src/services/traffic-flow-ripe.test.ts && tsx src/services/traffic-flow-brands.test.ts && tsx src/services/traffic-flow-ingest.test.ts && tsx src/services/traffic-flow-analytics.test.ts && tsx src/services/traffic-flow-map-hops.test.ts && tsx src/services/traffic-flow-purge.test.ts && tsx src/services/traffic-flow-geoip.test.ts && tsx src/services/traffic-flow-facts.test.ts && tsx src/services/traffic-flow-facts-filter.test.ts && tsx src/services/statistics-aggregate.test.ts",
"test:users": "tsx src/modules/users/iface-type.test.ts && tsx src/modules/users/bindings.test.ts",
"test:pg": "tsx src/db/sql-bind.test.ts && tsx src/db/sqlite-json.test.ts && tsx src/db/traffic-flags.test.ts && tsx src/db/pg-schema.test.ts",
"test:backups": "tsx src/services/s3-backup-client.test.ts",
"test": "npm run test:alert-engine && npm run test:auth && npm run test:wireguard && npm run test:traffic-rate && npm run test:traffic-flow && npm run test:users && npm run test:pg && npm run test:backups"
"test": "npm run test:alert-engine && npm run test:auth && npm run test:wireguard && npm run test:traffic-rate && npm run test:traffic-flow && npm run test:users && npm run test:pg && npm run test:backups",
"test:geoip": "tsx src/services/traffic-flow-geoip.test.ts"
},
"dependencies": {
"@aws-sdk/client-s3": "^3.888.0",
@@ -33,6 +34,7 @@
"drizzle-orm": "^0.45.2",
"fastify": "^5.8.5",
"fastify-plugin": "^5.1.0",
"maxmind": "^5.0.7",
"pg": "^8.23.0",
"undici": "^8.1.0",
"zod": "^4.4.1"
+11
View File
@@ -8,11 +8,19 @@ export interface PartitionSpec {
keepDays: number
}
/** Leaf-only: PG forbids storage params on partitioned parents (SQLSTATE 42809). */
const FACT_LEAF_STORAGE =
"fillfactor = 70, autovacuum_vacuum_scale_factor = 0.05, autovacuum_vacuum_cost_limit = 2000"
const FACT_PARENTS = new Set(["flow_hour_facts", "flow_daily_facts"])
export const PARTITION_SPECS: PartitionSpec[] = [
{ parent: "flow_buckets", kind: "day", keepDays: 4 },
{ parent: "flow_minute_stats", kind: "day", keepDays: 4 },
{ parent: "flow_minute_dims", kind: "day", keepDays: 4 },
{ parent: "flow_daily_dims", kind: "month", keepDays: 420 },
{ parent: "flow_hour_facts", kind: "day", keepDays: 3 },
{ parent: "flow_daily_facts", kind: "month", keepDays: 420 },
{ parent: "traffic_samples", kind: "week", keepDays: 21 },
{ parent: "servers_rest_ping_samples", kind: "week", keepDays: 35 },
{ parent: "uptime_probe_samples", kind: "week", keepDays: 21 },
@@ -110,6 +118,9 @@ export async function ensurePartitionFor(
await pool.query(
`CREATE TABLE IF NOT EXISTS ${name} PARTITION OF ${parent} FOR VALUES FROM ('${from}') TO ('${to}')`,
)
if (FACT_PARENTS.has(parent)) {
await pool.query(`ALTER TABLE ${name} SET (${FACT_LEAF_STORAGE})`)
}
return name
}
+44
View File
@@ -208,6 +208,36 @@ export const flowDailyDims = pgTable("flow_daily_dims", {
index("idx_flow_daily_dims_day").on(t.day, t.dim),
])
/** Hour-grain traffic cube for statistics (≤48h). No secondary indexes. */
export const flowHourFacts = pgTable("flow_hour_facts", {
serverId: bigint("server_id", { mode: "number" }).notNull()
.references(() => servers.id, { onDelete: "cascade" }),
bucketAt: ts("bucket_at").notNull(),
iface: text("iface").notNull(),
country: text("country").notNull(),
service: text("service").notNull(),
asn: integer("asn").notNull(),
bytes: bigint("bytes", { mode: "number" }).notNull().default(0),
packets: bigint("packets", { mode: "number" }).notNull().default(0),
}, (t) => [
primaryKey({ columns: [t.serverId, t.bucketAt, t.iface, t.country, t.service, t.asn] }),
])
/** Daily-grain traffic cube for statistics (long window). No secondary indexes. */
export const flowDailyFacts = pgTable("flow_daily_facts", {
serverId: bigint("server_id", { mode: "number" }).notNull()
.references(() => servers.id, { onDelete: "cascade" }),
day: date("day", { mode: "string" }).notNull(),
iface: text("iface").notNull(),
country: text("country").notNull(),
service: text("service").notNull(),
asn: integer("asn").notNull(),
bytes: bigint("bytes", { mode: "number" }).notNull().default(0),
packets: bigint("packets", { mode: "number" }).notNull().default(0),
}, (t) => [
primaryKey({ columns: [t.serverId, t.day, t.iface, t.country, t.service, t.asn] }),
])
export const flowBuckets = pgTable("flow_buckets", {
serverId: intPkRef().references(() => servers.id, { onDelete: "cascade" }),
bucketAt: ts("bucket_at").notNull(),
@@ -231,6 +261,20 @@ export const flowBuckets = pgTable("flow_buckets", {
index("idx_flow_buckets_server_time").on(t.serverId, t.bucketAt),
])
export const geoipSettings = pgTable("geoip_settings", {
id: idSingleton(),
enabled: boolean("enabled").notNull().default(true),
updateIntervalSec: integer("update_interval_sec").notNull().default(604800),
lastCheckAt: ts("last_check_at"),
lastSuccessAt: ts("last_success_at"),
lastError: text("last_error"),
countryBuildAt: ts("country_build_at"),
asnBuildAt: ts("asn_build_at"),
etagsJson: jsonb("etags_json").notNull().default(sql`'{}'::jsonb`),
createdAt: ts("created_at").notNull().defaultNow(),
updatedAt: ts("updated_at").notNull().defaultNow(),
})
export const flowIpMeta = pgTable("flow_ip_meta", {
prefix: text("prefix").primaryKey(),
asn: integer("asn").notNull().default(0),
+6
View File
@@ -14,6 +14,7 @@ import filtersRoutes from "./routes/filters.js"
import recursiveRoutes from "./routes/recursive-routes.js"
import trafficRoutes from "./routes/traffic.js"
import trafficFlowRoutes from "./routes/traffic-flow.js"
import geoipRoutes from "./routes/geoip.js"
import serversApiPingRoutes from "./routes/servers-api-ping.js"
import uptimeRoutes from "./routes/uptime.js"
import networkRoutes from "./routes/network.js"
@@ -30,8 +31,10 @@ import eventsRoutes from "./routes/events.js"
import wireguardRoutes from "./routes/wireguard.js"
import firewallRoutes from "./routes/firewall.js"
import usersRoutes from "./routes/users.js"
import statisticsRoutes from "./routes/statistics.js"
import { refreshScheduler, stopScheduler } from "./services/scheduler.js"
import { getFlowWorkerHealth, startTrafficFlowListener, stopTrafficFlowListener } from "./services/traffic-flow-ingest.js"
import { initGeoip } from "./services/traffic-flow-geoip.js"
const eventLoopDelay = monitorEventLoopDelay({ resolution: 20 })
eventLoopDelay.enable()
@@ -116,6 +119,7 @@ export async function buildApp(opts?: {
await app.register(recursiveRoutes, { prefix: "/api" })
await app.register(trafficRoutes, { prefix: "/api" })
await app.register(trafficFlowRoutes, { prefix: "/api" })
await app.register(geoipRoutes, { prefix: "/api" })
await app.register(serversApiPingRoutes, { prefix: "/api" })
await app.register(uptimeRoutes, { prefix: "/api" })
await app.register(networkRoutes, { prefix: "/api" })
@@ -132,9 +136,11 @@ export async function buildApp(opts?: {
await app.register(wireguardRoutes, { prefix: "/api" })
await app.register(firewallRoutes, { prefix: "/api" })
await app.register(usersRoutes, { prefix: "/api" })
await app.register(statisticsRoutes, { prefix: "/api" })
if (opts?.startScheduler !== false) {
await refreshScheduler()
await initGeoip()
await startTrafficFlowListener()
app.addHook("onClose", async () => {
stopScheduler()
+1 -1
View File
@@ -18,7 +18,7 @@ assert.equal(
"mm:settings:admin",
)
assert.equal(
permissionForRequest("GET", "/api/traffic/servers/1/live"),
permissionForRequest("GET", "/api/statistics"),
"mm:traffic:read",
)
assert.equal(
+1 -1
View File
@@ -107,7 +107,7 @@ const RULES: Rule[] = [
},
{
methods: ["GET"],
match: (p) => p.startsWith("/api/traffic"),
match: (p) => p.startsWith("/api/traffic") || p.startsWith("/api/statistics"),
permission: "mm:traffic:read",
},
{
+57
View File
@@ -0,0 +1,57 @@
import type { FastifyPluginAsyncZod } from "@fastify/type-provider-zod"
import { geoipSettingsPatchSchema } from "@mmapp/contracts/geoip"
import { refreshScheduler } from "../services/scheduler.js"
import { getGeoipSettings, updateGeoipSettings } from "../services/geoip-settings.js"
import {
GEOIP_ASN_FILE,
GEOIP_COUNTRY_FILE,
geoipReadersStatus,
initGeoip,
} from "../services/traffic-flow-geoip.js"
import { collectGeoipUpdateOnce, getGeoipUpdateState } from "../services/geoip-update-collector.js"
async function buildGeoipStatus() {
await initGeoip()
const readers = geoipReadersStatus()
return {
ready: readers.countryLoaded && readers.asnLoaded,
countryLoaded: readers.countryLoaded,
asnLoaded: readers.asnLoaded,
countryFile: GEOIP_COUNTRY_FILE,
asnFile: GEOIP_ASN_FILE,
dir: readers.dir,
running: getGeoipUpdateState().running,
settings: await getGeoipSettings(),
}
}
const geoipRoutes: FastifyPluginAsyncZod = async (app) => {
app.get("/geoip", async (_req, reply) => {
return reply.send(await buildGeoipStatus())
})
app.put("/geoip", async (req, reply) => {
const parsed = geoipSettingsPatchSchema.safeParse(req.body ?? {})
if (!parsed.success) {
return reply
.status(400)
.send({ error: "Некорректное тело запроса", details: parsed.error.flatten() })
}
await updateGeoipSettings(parsed.data)
await refreshScheduler()
return reply.send({ ok: true, status: await buildGeoipStatus() })
})
app.post("/geoip/update", async (_req, reply) => {
try {
const snapshot = await collectGeoipUpdateOnce({ force: true })
return reply.send({ ok: !snapshot.fatalError && snapshot.errors.length === 0, snapshot })
} catch (e) {
const status = (e as { statusCode?: number }).statusCode ?? 502
const msg = e instanceof Error ? e.message : String(e)
return reply.status(status).send({ error: msg })
}
})
}
export default geoipRoutes
+26
View File
@@ -0,0 +1,26 @@
import type { FastifyPluginAsyncZod } from "@fastify/type-provider-zod"
import { statisticsPivotQuerySchema, statisticsQuerySchema } from "@mmapp/contracts/statistics"
import { getStatistics, getStatisticsPivot, pivotDimsConflict } from "../services/statistics-aggregate.js"
const statisticsRoutes: FastifyPluginAsyncZod = async (app) => {
app.get("/statistics", async (req, reply) => {
const parsed = statisticsQuerySchema.safeParse(req.query ?? {})
if (!parsed.success) {
return reply.status(400).send({ error: "Некорректный период или фильтры", details: parsed.error.flatten() })
}
return reply.send(await getStatistics(parsed.data))
})
app.get("/statistics/pivot", async (req, reply) => {
const parsed = statisticsPivotQuerySchema.safeParse(req.query ?? {})
if (!parsed.success) {
return reply.status(400).send({ error: "Некорректный период или измерения", details: parsed.error.flatten() })
}
if (pivotDimsConflict(parsed.data.row, parsed.data.col)) {
return reply.status(400).send({ error: "Строки и колонки должны отличаться" })
}
return reply.send(await getStatisticsPivot(parsed.data))
})
}
export default statisticsRoutes
+100
View File
@@ -0,0 +1,100 @@
import { eq } from "drizzle-orm"
import { db } from "../db/index.js"
import { geoipSettings } from "../db/schema.js"
import type { GeoipSettingsDto, GeoipSettingsPatch } from "@mmapp/contracts/geoip"
const SETTINGS_ID = 1
const DEFAULT_INTERVAL_SEC = 604800
let dbEnabled = true
/** Тесты без PostgreSQL: геттеры отдают дефолты, touch/update — no-op. */
export function disableGeoipDbForTests(): void {
dbEnabled = false
}
export function resetGeoipSettingsForTests(): void {
dbEnabled = true
}
type GeoipSettingsRow = typeof geoipSettings.$inferSelect
async function getGeoipSettingsRow(): Promise<GeoipSettingsRow | undefined> {
if (!dbEnabled) return undefined
return (
(await db.select().from(geoipSettings).where(eq(geoipSettings.id, SETTINGS_ID)).limit(1))[0]
)
}
function toDto(row: GeoipSettingsRow | undefined): GeoipSettingsDto {
return {
enabled: row?.enabled ?? true,
updateIntervalSec: row?.updateIntervalSec ?? DEFAULT_INTERVAL_SEC,
lastCheckAt: row?.lastCheckAt ?? null,
lastSuccessAt: row?.lastSuccessAt ?? null,
lastError: row?.lastError ?? null,
countryBuildAt: row?.countryBuildAt ?? null,
asnBuildAt: row?.asnBuildAt ?? null,
updatedAt: row?.updatedAt ?? new Date().toISOString(),
}
}
export async function getGeoipSettings(): Promise<GeoipSettingsDto> {
return toDto(await getGeoipSettingsRow())
}
/** ETag'и зеркала для conditional GET (ключ — имя файла базы). */
export async function getGeoipEtags(): Promise<Record<string, string>> {
const row = await getGeoipSettingsRow()
if (!row) return {}
const raw = row?.etagsJson
if (!raw || typeof raw !== "object") return {}
return Object.fromEntries(
Object.entries(raw as Record<string, unknown>).filter(
(entry): entry is [string, string] => typeof entry[1] === "string",
),
)
}
export async function updateGeoipSettings(patch: GeoipSettingsPatch): Promise<GeoipSettingsDto> {
const prev = await getGeoipSettingsRow()
const next = {
enabled: patch.enabled ?? prev?.enabled ?? true,
updateIntervalSec: patch.updateIntervalSec ?? prev?.updateIntervalSec ?? DEFAULT_INTERVAL_SEC,
updatedAt: new Date().toISOString(),
}
if (prev) {
await db.update(geoipSettings).set(next).where(eq(geoipSettings.id, SETTINGS_ID))
} else {
await db.insert(geoipSettings).values({ id: SETTINGS_ID, ...next })
}
return getGeoipSettings()
}
export async function touchGeoipRunMeta(patch: {
lastCheckAt?: string
lastSuccessAt?: string | null
lastError?: string | null
countryBuildAt?: string | null
asnBuildAt?: string | null
etags?: Record<string, string>
}): Promise<void> {
const prev = await getGeoipSettingsRow()
const set: Partial<typeof geoipSettings.$inferInsert> = {
updatedAt: new Date().toISOString(),
}
if (patch.lastCheckAt !== undefined) set.lastCheckAt = patch.lastCheckAt
if (patch.lastSuccessAt !== undefined) set.lastSuccessAt = patch.lastSuccessAt
if (patch.lastError !== undefined) set.lastError = patch.lastError
if (patch.countryBuildAt !== undefined) set.countryBuildAt = patch.countryBuildAt
if (patch.asnBuildAt !== undefined) set.asnBuildAt = patch.asnBuildAt
if (patch.etags !== undefined) {
const prevEtags = (prev?.etagsJson as Record<string, string> | null) ?? {}
set.etagsJson = { ...prevEtags, ...patch.etags }
}
if (prev) {
await db.update(geoipSettings).set(set).where(eq(geoipSettings.id, SETTINGS_ID))
} else {
await db.insert(geoipSettings).values({ id: SETTINGS_ID, ...set })
}
}
@@ -0,0 +1,198 @@
import { rename, rm, mkdir, writeFile } from "node:fs/promises"
import path from "node:path"
import { open, type AsnResponse, type CountryResponse } from "maxmind"
import type { GeoipUpdateRunSnapshot } from "../types/scheduler-run-snapshot.js"
import { SCHEDULER_RUN_SNAPSHOT_VERSION } from "../types/scheduler-run-snapshot.js"
import { getGeoipEtags, getGeoipSettings, touchGeoipRunMeta } from "./geoip-settings.js"
import {
GEOIP_ASN_FILE,
GEOIP_COUNTRY_FILE,
geoipDir,
reloadGeoipReaders,
} from "./traffic-flow-geoip.js"
/** Зеркало GeoLite2 без регистрации и ключей (см. README: GeoIP). */
const MIRROR_BASE = "https://github.com/P3TERX/GeoLite.mmdb/raw/download"
const DOWNLOAD_TIMEOUT_MS = 120_000
/** Пробный IP для валидации скачанной базы: Google DNS. */
const PROBE_IP = "8.8.8.8"
type GeoipDbKind = "country" | "asn"
let updating = false
let fetchImpl: typeof fetch = globalThis.fetch.bind(globalThis)
export function getGeoipUpdateState(): { running: boolean } {
return { running: updating }
}
async function validateCountryFile(filePath: string): Promise<string> {
const reader = await open<CountryResponse>(filePath)
const rec = reader.get(PROBE_IP)
const iso = rec?.country?.iso_code ?? rec?.registered_country?.iso_code ?? ""
if (iso !== "US") {
throw new Error(`база Country не распознала ${PROBE_IP} как US (${iso || "нет записи"})`)
}
return reader.metadata.buildEpoch.toISOString()
}
async function validateAsnFile(filePath: string): Promise<string> {
const reader = await open<AsnResponse>(filePath)
const rec = reader.get(PROBE_IP)
const asn = rec?.autonomous_system_number ?? 0
if (asn !== 15169) {
throw new Error(`база ASN не распознала ${PROBE_IP} как AS15169 (${asn ? `AS${asn}` : "нет записи"})`)
}
return reader.metadata.buildEpoch.toISOString()
}
let validateCountry = validateCountryFile
let validateAsn = validateAsnFile
export function setGeoipFetchForTests(fn: typeof fetch): void {
fetchImpl = fn
}
export function setGeoipValidateForTests(opts: {
country?: (filePath: string) => Promise<string>
asn?: (filePath: string) => Promise<string>
}): void {
validateCountry = opts.country ?? validateCountryFile
validateAsn = opts.asn ?? validateAsnFile
}
export function resetGeoipUpdateForTests(): void {
updating = false
fetchImpl = globalThis.fetch.bind(globalThis)
validateCountry = validateCountryFile
validateAsn = validateAsnFile
}
/**
* Разовая проверка/доставка баз с зеркала P3TERX. Conditional GET по ETag
* (304 = не меняем файл), валидация пробоем 8.8.8.8, атомарная подмена через rename.
*/
export async function collectGeoipUpdateOnce(
opts: { force?: boolean } = {},
): Promise<GeoipUpdateRunSnapshot> {
const sampledAt = new Date().toISOString()
if (updating) {
if (opts.force) {
throw Object.assign(new Error("Обновление GeoIP уже выполняется"), { statusCode: 409 })
}
return emptySnapshot(sampledAt, true)
}
const settings = await getGeoipSettings()
if (!settings.enabled && !opts.force) {
return emptySnapshot(sampledAt, true)
}
updating = true
const snapshot: GeoipUpdateRunSnapshot = {
v: SCHEDULER_RUN_SNAPSHOT_VERSION,
job: "geoip_update",
sampledAt,
checked: 0,
downloaded: 0,
skippedUnchanged: 0,
bytes: 0,
errors: [],
}
try {
const dir = geoipDir()
await mkdir(dir, { recursive: true })
const storedEtags = await getGeoipEtags()
const etags: Record<string, string> = {}
const buildAt: Partial<Record<GeoipDbKind, string>> = {}
for (const kind of ["country", "asn"] as const) {
snapshot.checked += 1
const file = kind === "country" ? GEOIP_COUNTRY_FILE : GEOIP_ASN_FILE
const target = path.join(dir, file)
const tmp = `${target}.tmp`
const prevEtag = storedEtags[file]
try {
const ac = new AbortController()
const timer = setTimeout(() => ac.abort(), DOWNLOAD_TIMEOUT_MS)
let res: Response
try {
res = await fetchImpl(`${MIRROR_BASE}/${file}`, {
headers: prevEtag ? { "If-None-Match": prevEtag } : {},
signal: ac.signal,
})
} finally {
clearTimeout(timer)
}
if (res.status === 304) {
snapshot.skippedUnchanged += 1
if (prevEtag) etags[file] = prevEtag
continue
}
if (!res.ok) throw new Error(`HTTP ${res.status}`)
const etag = res.headers.get("etag") ?? ""
const body = Buffer.from(await res.arrayBuffer())
snapshot.bytes += body.byteLength
await writeFile(tmp, body)
buildAt[kind] =
kind === "country" ? await validateCountry(tmp) : await validateAsn(tmp)
const prevFile = `${target}.prev`
await rm(prevFile, { force: true })
await rename(target, prevFile).catch(() => {
/* текущего файла могло ещё не быть */
})
await rename(tmp, target)
snapshot.downloaded += 1
if (etag) etags[file] = etag
} catch (e) {
await rm(tmp, { force: true }).catch(() => {
/* best-effort */
})
const message = e instanceof Error ? e.message : String(e)
snapshot.errors.push(`${file}: ${message}`)
}
}
if (snapshot.downloaded > 0) {
await reloadGeoipReaders()
}
await touchGeoipRunMeta({
lastCheckAt: sampledAt,
lastSuccessAt: snapshot.errors.length ? null : sampledAt,
lastError: snapshot.errors.length ? snapshot.errors.join("; ") : null,
countryBuildAt: buildAt.country,
asnBuildAt: buildAt.asn,
etags,
})
return snapshot
} catch (e) {
const message = e instanceof Error ? e.message : String(e)
snapshot.fatalError = message
await touchGeoipRunMeta({
lastCheckAt: sampledAt,
lastError: message,
}).catch(() => {
/* best-effort */
})
return snapshot
} finally {
updating = false
}
}
function emptySnapshot(sampledAt: string, skipped: boolean): GeoipUpdateRunSnapshot {
return {
v: SCHEDULER_RUN_SNAPSHOT_VERSION,
job: "geoip_update",
sampledAt,
skipped,
checked: 0,
downloaded: 0,
skippedUnchanged: 0,
bytes: 0,
errors: [],
}
}
+20
View File
@@ -46,6 +46,8 @@ import { collectCertificatesRenewOnce } from "./certificate-renew-collector.js"
import { getCertificateRenewSettings } from "./certificates-service.js"
import { collectScheduledBackupsOnce } from "./backup-scheduler-collector.js"
import { getBackupScheduleSettings } from "./backup-service.js"
import { collectGeoipUpdateOnce } from "./geoip-update-collector.js"
import { getGeoipSettings } from "./geoip-settings.js"
import {
endSchedulerJob,
isSchedulerJobRunning,
@@ -63,6 +65,7 @@ export const JOB_KEYS = [
"gre_bgp",
"certificates_renew",
"backups",
"geoip_update",
"alert_engine",
] as const
export type SchedulerJobKey = (typeof JOB_KEYS)[number]
@@ -148,6 +151,9 @@ async function runSchedulerJobBody(jobKey: SchedulerJobKey): Promise<void> {
case "backups":
snapshot = await collectScheduledBackupsOnce()
break
case "geoip_update":
snapshot = await collectGeoipUpdateOnce()
break
case "alert_engine": {
const r = await runAlertEngineOnce()
snapshot = {
@@ -377,6 +383,18 @@ export async function refreshScheduler(): Promise<void> {
)
}
const geoip = await getGeoipSettings()
if (geoip.enabled) {
const geoipMs = Math.max(6 * 3600_000, geoip.updateIntervalSec * 1000)
void executeSchedulerJob("geoip_update").catch(() => {})
timers.set(
"geoip_update",
setInterval(() => {
void executeSchedulerJob("geoip_update").catch(() => {})
}, geoipMs),
)
}
const alertMs = 20_000
void executeSchedulerJob("alert_engine").catch(() => {})
timers.set(
@@ -409,6 +427,7 @@ export async function getSchedulerStatus() {
const internetPath = await getInternetPathSettings()
const certRenew = await getCertificateRenewSettings()
const backupSchedule = await getBackupScheduleSettings()
const geoip = await getGeoipSettings()
const resOn = uptime.resourcesEnabled ?? uptime.enabled
const pingOn = uptime.pingEnabled ?? uptime.enabled
@@ -424,6 +443,7 @@ export async function getSchedulerStatus() {
gre_bgp: { enabled: true, intervalSec: 30 },
certificates_renew: { enabled: certRenew.enabled, intervalSec: certRenew.intervalSec },
backups: { enabled: backupSchedule.enabled, intervalSec: 60 },
geoip_update: { enabled: geoip.enabled, intervalSec: geoip.updateIntervalSec },
alert_engine: { enabled: true, intervalSec: 20 },
}
@@ -0,0 +1,187 @@
import assert from "node:assert/strict"
import { getStatistics, getStatisticsPivot, parseStatisticsPeriod, pivotDimsConflict } from "./statistics-aggregate.js"
import { rememberServerIfaces, resetIfaceCacheForTests } from "./traffic-flow-ifindex.js"
import { setRefreshIfacesForTests } from "./traffic-flow-ifaces.js"
import { invalidateFlowCatalogCache } from "./traffic-flow-topology.js"
import { withPgOrSkip } from "../test/pg.js"
import { dbQuery } from "../db/index.js"
import { ensurePartitionFor } from "../db/partitions.js"
import { pool } from "../db/index.js"
import { STATISTICS_UNBOUND_USER_ID } from "@mmapp/contracts/statistics"
{
const sameDay = parseStatisticsPeriod("2026-09-10", "2026-09-10")
assert.ok(sameDay)
assert.equal(sameDay.fromDay, "2026-09-10")
assert.equal(sameDay.toDayExclusive, "2026-09-11")
assert.equal(sameDay.grain, "hour")
const month = parseStatisticsPeriod("2026-08-01", "2026-08-31")
assert.ok(month)
assert.equal(month.grain, "day")
assert.equal(month.toDayExclusive, "2026-09-01")
assert.equal(parseStatisticsPeriod("2026-09-10", "2026-09-09"), null)
assert.equal(pivotDimsConflict("country", "country"), true)
assert.equal(pivotDimsConflict("country", "service"), false)
}
if (!(await withPgOrSkip())) {
console.log("statistics-aggregate.test.ts: skip")
process.exit(0)
}
const inserted = await dbQuery<{ id: number }>(`
INSERT INTO servers (name, host, type, wan_uplinks)
VALUES ('stats-cube', '127.0.0.1', 'jump-host', '[{"iface":"wan1"}]'::jsonb)
RETURNING id
`)
const serverId = inserted.rows[0]?.id
if (serverId == null) throw new Error("no server")
const enInserted = await dbQuery<{ id: number }>(`
INSERT INTO servers (name, host, type)
VALUES ('stats-en', '198.51.100.1', 'exit-node')
RETURNING id
`)
const enId = enInserted.rows[0]?.id
if (enId == null) throw new Error("no en server")
await ensurePartitionFor(pool, "flow_daily_facts", "month", new Date("2026-09-01T00:00:00Z"))
await ensurePartitionFor(pool, "flow_hour_facts", "day", new Date("2026-09-10T00:00:00Z"))
await dbQuery(`DELETE FROM flow_daily_facts WHERE server_id IN ($1, $2)`, [serverId, enId])
await dbQuery(`DELETE FROM flow_hour_facts WHERE server_id IN ($1, $2)`, [serverId, enId])
await dbQuery(`DELETE FROM user_interface_bindings WHERE server_id = $1`, [serverId])
await dbQuery(`DELETE FROM app_users WHERE id = 'u-stats-1'`)
await dbQuery(`
INSERT INTO app_users (id, name, login, role, active)
VALUES ('u-stats-1', 'Клиент', 'stats-user', 'viewer', TRUE)
ON CONFLICT (id) DO NOTHING
`)
await dbQuery(`
INSERT INTO user_interface_bindings (id, user_id, server_id, interface_name, interface_type)
VALUES ('bind-stats-1', 'u-stats-1', $1, 'gre-client', 'gre')
`, [serverId])
resetIfaceCacheForTests()
rememberServerIfaces(serverId, [
{ name: "gre-client", ifindex: "2" },
{ name: "wan1", ifindex: "8" },
{ name: "gre-en", ifindex: "9" },
])
rememberServerIfaces(enId, [
{ name: "ether1", ifindex: "2" },
{ name: "gre-jh", ifindex: "5" },
])
setRefreshIfacesForTests(async () => {})
invalidateFlowCatalogCache()
await dbQuery(`
INSERT INTO flow_daily_facts (server_id, day, iface, country, service, asn, bytes, packets)
VALUES
($1, '2026-09-10', '2', 'US', 'https', 15169, 800, 10),
($1, '2026-09-10', '2', 'DE', 'dns', 15133, 200, 4),
($1, '2026-09-10', 'wan1', 'NL', 'other', 0, 70, 1),
($1, '2026-09-10', '0', 'US', 'https', 0, 999, 3),
($1, '2026-09-10', 'gre-en', 'US', 'https', 15169, 400, 2),
($2, '2026-09-10', 'gre-jh', 'US', 'https', 15169, 500, 5),
($2, '2026-09-10', 'ether1', 'US', 'https', 15169, 200, 2)
`, [serverId, enId])
try {
const all = await getStatistics({ from: "2026-09-01", to: "2026-09-30" })
assert.equal(all.grain, "day")
assert.equal(all.kpis.bytes, 1070)
assert.equal(all.kpis.users, 1)
assert.ok(all.countries.some((r) => r.id === "US"))
assert.ok(all.users.some((r) => r.id === "u-stats-1"))
const unbound = all.users.find((r) => r.id === STATISTICS_UNBOUND_USER_ID)
assert.equal(unbound, undefined, "WAN не в Прочие / Без привязки")
assert.ok(all.servers.some((r) => r.id === String(serverId)))
assert.ok(!all.servers.some((r) => r.id === String(enId)), "EN-транзит не в сетевом KPI")
const greIface = all.interfaces.find((r) => r.label.includes("gre-client"))
assert.ok(greIface)
assert.equal(greIface.bytes, 1000)
assert.equal(greIface.id, `${serverId}:gre-client`)
assert.ok(!all.interfaces.some((r) => /· (?:#)?\d+$/.test(r.label)))
assert.ok(!all.interfaces.some((r) => r.label.includes(" · —") || r.label.endsWith("· —")))
assert.ok(!all.interfaces.some((r) => r.label.includes("gre-en")))
const wanRow = all.interfaces.find((r) => r.id === `${serverId}:wan1`)
assert.ok(wanRow)
assert.ok(wanRow.label.includes("WAN · интернет"))
assert.equal(wanRow.bytes, 70)
const nodeSlice = await getStatistics({
from: "2026-09-01",
to: "2026-09-30",
serverId,
})
assert.equal(nodeSlice.kpis.bytes, 1070)
const nodeWan = nodeSlice.interfaces.find((r) => r.id === `${serverId}:wan1`)
assert.ok(nodeWan)
assert.ok(nodeWan.label.includes("WAN · интернет"))
assert.ok(!nodeSlice.interfaces.some((r) => r.label.includes(" · —")))
assert.ok(!nodeSlice.users.some((r) => r.id === STATISTICS_UNBOUND_USER_ID))
const enSlice = await getStatistics({
from: "2026-09-01",
to: "2026-09-30",
serverId: enId,
})
assert.equal(enSlice.kpis.bytes, 200)
assert.ok(enSlice.interfaces.some((r) => r.label.includes("WAN · интернет") && r.label.includes("ether1")))
assert.ok(!enSlice.interfaces.some((r) => r.label.includes("gre-jh")))
const sliced = await getStatistics({
from: "2026-09-01",
to: "2026-09-30",
country: "US",
service: "https",
asn: 15169,
})
assert.equal(sliced.kpis.bytes, 800)
assert.equal(sliced.countries.length, 1)
assert.equal(sliced.countries[0]?.id, "US")
assert.ok(sliced.users.some((r) => r.id === "u-stats-1"))
const byUser = await getStatistics({
from: "2026-09-01",
to: "2026-09-30",
userId: "u-stats-1",
})
assert.equal(byUser.kpis.bytes, 1000)
const pivot = await getStatisticsPivot({
from: "2026-09-01",
to: "2026-09-30",
row: "country",
col: "service",
metric: "bytes",
})
const us = pivot.rows.find((r) => r.id === "US")
const de = pivot.rows.find((r) => r.id === "DE")
assert.ok(us)
assert.ok(de)
assert.equal(us.cells.https, 800)
assert.equal(de.cells.dns, 200)
await dbQuery(`
INSERT INTO flow_hour_facts (server_id, bucket_at, iface, country, service, asn, bytes, packets)
VALUES ($1, '2026-09-10T10:00:00Z', '2', 'US', 'https', 15169, 40, 2)
`, [serverId])
const hourly = await getStatistics({
from: "2026-09-10T00:00:00.000Z",
to: "2026-09-10T23:00:00.000Z",
})
assert.equal(hourly.grain, "hour")
assert.equal(hourly.kpis.bytes, 40)
assert.ok(hourly.users.some((r) => r.id === "u-stats-1"))
} finally {
setRefreshIfacesForTests(null)
resetIfaceCacheForTests()
invalidateFlowCatalogCache()
await dbQuery(`DELETE FROM flow_daily_facts WHERE server_id IN ($1, $2)`, [serverId, enId])
await dbQuery(`DELETE FROM flow_hour_facts WHERE server_id IN ($1, $2)`, [serverId, enId])
await dbQuery(`DELETE FROM servers WHERE id IN ($1, $2)`, [serverId, enId])
}
console.log("statistics-aggregate.test.ts: ok")
@@ -0,0 +1,841 @@
import { eq } from "drizzle-orm"
import { db, dbAll } from "../db/index.js"
import { appUsers, flowAsnMeta, servers, userInterfaceBindings } from "../db/schema.js"
import {
STATISTICS_UNBOUND_USER_ID,
type StatisticsBreakdownRow,
type StatisticsDto,
type StatisticsPivotDim,
type StatisticsPivotDto,
type StatisticsPivotQuery,
type StatisticsQuery,
} from "@mmapp/contracts/statistics"
import {
collapseServerIfaceRows,
displayFactIface,
expandBindingIfaces,
factIfaceAliases,
listCachedIfaceNames,
} from "./traffic-flow-ifindex.js"
import { refreshServerIfaces } from "./traffic-flow-ifaces.js"
import {
isDashDisplayIface,
isJunkFactIface,
isOverlayGreIface,
isWanFactIface,
wanIfaceLabel,
} from "./traffic-flow-facts-filter.js"
import { getServerCatalog, loadFlowTopology, type FlowTopology } from "./traffic-flow-topology.js"
const TOP_N = 200
const HOUR_WINDOW_MS = 48 * 3600_000
const PIVOT_ROW_CAP = 50
const PIVOT_COL_CAP = 15
const PIVOT_OTHER_ID = "__other__"
export interface ParsedPeriod {
fromIso: string
toIso: string
fromDay: string
toDayExclusive: string
grain: "hour" | "day"
windowSec: number
}
function pad2(n: number): string {
return String(n).padStart(2, "0")
}
function toUtcDay(d: Date): string {
return `${d.getUTCFullYear()}-${pad2(d.getUTCMonth() + 1)}-${pad2(d.getUTCDate())}`
}
function addUtcDays(day: string, n: number): string {
const d = new Date(`${day}T00:00:00Z`)
d.setUTCDate(d.getUTCDate() + n)
return toUtcDay(d)
}
/** Parse from/to. Date-only `to` is inclusive (end of that UTC day). */
export function parseStatisticsPeriod(fromRaw: string, toRaw: string): ParsedPeriod | null {
const from = Date.parse(fromRaw.includes("T") ? fromRaw : `${fromRaw}T00:00:00Z`)
const toHasTime = toRaw.includes("T")
const to = Date.parse(toHasTime ? toRaw : `${toRaw}T00:00:00Z`)
if (!Number.isFinite(from) || !Number.isFinite(to)) return null
const fromDate = new Date(from)
let toDate = new Date(to)
let toDayExclusive: string
if (toHasTime) {
toDayExclusive = toUtcDay(toDate)
if (toDate.getUTCHours() !== 0 || toDate.getUTCMinutes() !== 0 || toDate.getUTCSeconds() !== 0) {
toDayExclusive = addUtcDays(toDayExclusive, 1)
}
} else {
toDayExclusive = addUtcDays(toUtcDay(toDate), 1)
toDate = new Date(`${toDayExclusive}T00:00:00Z`)
}
if (toDate.getTime() <= from) return null
const windowSec = Math.max(1, Math.round((toDate.getTime() - from) / 1000))
const grain: "hour" | "day" = toDate.getTime() - from <= HOUR_WINDOW_MS ? "hour" : "day"
return {
fromIso: fromDate.toISOString(),
toIso: toDate.toISOString(),
fromDay: toUtcDay(fromDate),
toDayExclusive,
grain,
windowSec,
}
}
interface FilterCtx {
fromIso: string
toIso: string
fromDay: string
toDayExclusive: string
serverId?: number
iface?: string
country?: string
service?: string
asn?: number
userIfaces: Array<{ serverId: number; iface: string }> | null
unboundOnly: boolean
boundIfaces: Array<{ serverId: number; iface: string }>
overlayIfaces: Array<{ serverId: number; iface: string }>
wanIfaces: Array<{ serverId: number; iface: string }>
excludeServerIds: number[]
topo: FlowTopology | null
}
function ifaceFilterAliases(iface: string, serverId?: number): string[] {
return factIfaceAliases(iface.trim(), serverId)
}
function looksLikeIfIndex(iface: string): boolean {
const raw = iface.trim()
return /^\d+$/.test(raw) || /^#\d+$/.test(raw)
}
async function warmIfaceCache(ids: Iterable<number>): Promise<void> {
const uniq = [...new Set(ids)].filter((id) => Number.isFinite(id) && id > 0)
if (!uniq.length) return
await Promise.all(uniq.map((id) => refreshServerIfaces(id)))
}
async function warmBindingIfaceCache(): Promise<void> {
const rows = await db.select({ serverId: userInterfaceBindings.serverId }).from(userInterfaceBindings)
await warmIfaceCache(rows.map((r) => r.serverId))
}
function canonicalIfaceDimId(id: string): string {
const colon = id.indexOf(":")
if (colon < 0) return id
const sid = Number(id.slice(0, colon))
if (!Number.isFinite(sid)) return id
return `${sid}:${displayFactIface(sid, id.slice(colon + 1))}`
}
function factWhere(alias: string, grain: "hour" | "day", ctx: FilterCtx): { sql: string; params: unknown[] } {
const params: unknown[] = []
const parts: string[] = []
if (grain === "hour") {
params.push(ctx.fromIso, ctx.toIso)
parts.push(`${alias}.bucket_at >= ? AND ${alias}.bucket_at < ?`)
} else {
params.push(ctx.fromDay, ctx.toDayExclusive)
parts.push(`${alias}.day >= ? AND ${alias}.day < ?`)
}
if (ctx.serverId != null) {
parts.push(`${alias}.server_id = ?`)
params.push(ctx.serverId)
}
if (ctx.iface) {
const aliases = ifaceFilterAliases(ctx.iface, ctx.serverId)
if (aliases.length <= 1) {
parts.push(`${alias}.iface = ?`)
params.push(aliases[0] ?? ctx.iface)
} else {
parts.push(`${alias}.iface IN (${aliases.map(() => "?").join(", ")})`)
params.push(...aliases)
}
}
if (ctx.country) {
parts.push(`${alias}.country = ?`)
params.push(ctx.country.toUpperCase())
}
if (ctx.service) {
parts.push(`${alias}.service = ?`)
params.push(ctx.service)
}
if (ctx.asn != null) {
parts.push(`${alias}.asn = ?`)
params.push(ctx.asn)
}
if (ctx.userIfaces) {
if (ctx.userIfaces.length === 0) {
parts.push("FALSE")
} else {
const tuples = ctx.userIfaces.map(() => "(?, ?)").join(", ")
parts.push(`(${alias}.server_id, ${alias}.iface) IN (${tuples})`)
for (const u of ctx.userIfaces) {
params.push(u.serverId, u.iface)
}
}
}
if (ctx.unboundOnly) {
const skip = [...ctx.boundIfaces, ...ctx.wanIfaces]
if (skip.length) {
const tuples = skip.map(() => "(?, ?)").join(", ")
parts.push(`(${alias}.server_id, ${alias}.iface) NOT IN (${tuples})`)
for (const u of skip) params.push(u.serverId, u.iface)
}
}
parts.push(`${alias}.iface NOT IN ('0', '—', '__unknown__', 'wg-flow', '')`)
const greLike = `(LOWER(${alias}.iface) LIKE 'gre%' OR LOWER(${alias}.iface) LIKE '%gre-tunnel%')`
if (ctx.boundIfaces.length) {
const tuples = ctx.boundIfaces.map(() => "(?, ?)").join(", ")
parts.push(`(NOT ${greLike} OR (${alias}.server_id, ${alias}.iface) IN (${tuples}))`)
for (const u of ctx.boundIfaces) params.push(u.serverId, u.iface)
} else {
parts.push(`NOT ${greLike}`)
}
if (ctx.overlayIfaces.length) {
const tuples = ctx.overlayIfaces.map(() => "(?, ?)").join(", ")
parts.push(`(${alias}.server_id, ${alias}.iface) NOT IN (${tuples})`)
for (const u of ctx.overlayIfaces) params.push(u.serverId, u.iface)
}
if (ctx.excludeServerIds.length) {
parts.push(`${alias}.server_id NOT IN (${ctx.excludeServerIds.map(() => "?").join(", ")})`)
params.push(...ctx.excludeServerIds)
}
return { sql: parts.join(" AND "), params }
}
function emptyDto(period: ParsedPeriod): StatisticsDto {
return {
from: period.fromIso,
to: period.toIso,
grain: period.grain,
kpis: {
bytes: 0,
packets: 0,
avgBps: 0,
users: 0,
servers: 0,
ifaces: 0,
topCountry: "—",
topService: "—",
},
series: [],
users: [],
servers: [],
interfaces: [],
countries: [],
services: [],
asns: [],
}
}
function toBreakdown(
rows: Array<{ id: string; label: string; bytes: number; packets: number }>,
totalBytes: number,
windowSec: number,
): StatisticsBreakdownRow[] {
const denom = totalBytes || 1
return rows
.sort((a, b) => b.bytes - a.bytes)
.slice(0, TOP_N)
.map((r) => ({
id: r.id,
label: r.label,
bytes: r.bytes,
packets: r.packets,
bps: (r.bytes * 8) / windowSec,
percent: (r.bytes / denom) * 100,
}))
}
interface UserBindTuple {
userId: string
serverId: number
iface: string
}
async function loadBindUserTuples(): Promise<UserBindTuple[]> {
const binds = await db.select().from(userInterfaceBindings)
const seen = new Set<string>()
const out: UserBindTuple[] = []
for (const b of binds) {
for (const iface of factIfaceAliases(b.interfaceName, b.serverId)) {
const k = `${b.userId}\0${b.serverId}\0${iface}`
if (seen.has(k)) continue
seen.add(k)
out.push({ userId: b.userId, serverId: b.serverId, iface })
}
}
return out
}
function uniqueBoundIfaces(tuples: UserBindTuple[]): Array<{ serverId: number; iface: string }> {
const seen = new Set<string>()
const out: Array<{ serverId: number; iface: string }> = []
for (const t of tuples) {
const k = `${t.serverId}\0${t.iface}`
if (seen.has(k)) continue
seen.add(k)
out.push({ serverId: t.serverId, iface: t.iface })
}
return out
}
async function resolveUserIfaces(userId?: string): Promise<Array<{ serverId: number; iface: string }> | null> {
if (!userId || userId === STATISTICS_UNBOUND_USER_ID) return null
const binds = await db.select().from(userInterfaceBindings).where(eq(userInterfaceBindings.userId, userId))
return expandBindingIfaces(binds.map((b) => ({ serverId: b.serverId, iface: b.interfaceName })))
}
function userBindJoinSql(tuples: UserBindTuple[]): { sql: string; params: unknown[] } {
const values = tuples.map(() => "(?::text, ?::int, ?::text)").join(", ")
const params = tuples.flatMap((t) => [t.userId, t.serverId, t.iface])
return {
sql: `JOIN (VALUES ${values}) AS b(user_id, server_id, iface) ON b.server_id = f.server_id AND b.iface = f.iface`,
params,
}
}
function expandIfaceTuples(
items: Array<{ serverId: number; iface: string }>,
): Array<{ serverId: number; iface: string }> {
const seen = new Set<string>()
const out: Array<{ serverId: number; iface: string }> = []
for (const t of items) {
for (const iface of factIfaceAliases(t.iface, t.serverId)) {
const k = `${t.serverId}\0${iface}`
if (seen.has(k)) continue
seen.add(k)
out.push({ serverId: t.serverId, iface })
}
}
return out
}
async function loadPayloadScope(serverId?: number): Promise<{
overlayIfaces: Array<{ serverId: number; iface: string }>
wanIfaces: Array<{ serverId: number; iface: string }>
excludeServerIds: number[]
topo: FlowTopology
}> {
const topo = await loadFlowTopology()
const catalog = await getServerCatalog()
await warmIfaceCache(catalog.list.map((s) => s.id))
const overlayRaw: Array<{ serverId: number; iface: string }> = []
const wanRaw: Array<{ serverId: number; iface: string }> = []
for (const s of catalog.list) {
if (serverId != null && s.id !== serverId) continue
const wanSet = topo.wanIfaces.get(s.id)
const wanNames = wanSet && wanSet.size > 0
? [...wanSet]
: s.type === "home-router" ? [] : ["ether1"]
for (const name of wanNames) wanRaw.push({ serverId: s.id, iface: name })
for (const name of listCachedIfaceNames(s.id)) {
if (isOverlayGreIface(topo, s.id, name)) overlayRaw.push({ serverId: s.id, iface: name })
}
}
return {
overlayIfaces: expandIfaceTuples(overlayRaw),
wanIfaces: expandIfaceTuples(wanRaw),
excludeServerIds: serverId != null
? []
: catalog.list.filter((s) => s.type === "exit-node").map((s) => s.id),
topo,
}
}
async function buildFilterCtx(query: StatisticsQuery, period: ParsedPeriod): Promise<FilterCtx | null> {
const bindTuples = await loadBindUserTuples()
const boundIfaces = uniqueBoundIfaces(bindTuples)
const unboundOnly = query.userId === STATISTICS_UNBOUND_USER_ID
const userIfaces = unboundOnly ? null : await resolveUserIfaces(query.userId)
if (userIfaces && userIfaces.length === 0) return null
const scope = await loadPayloadScope(query.serverId)
return {
...period,
serverId: query.serverId,
iface: query.iface,
country: query.country,
service: query.service,
asn: query.asn,
userIfaces,
unboundOnly,
boundIfaces,
overlayIfaces: scope.overlayIfaces,
wanIfaces: scope.wanIfaces,
excludeServerIds: scope.excludeServerIds,
topo: scope.topo,
}
}
export async function getStatistics(query: StatisticsQuery): Promise<StatisticsDto> {
const period = parseStatisticsPeriod(query.from, query.to)
if (!period) return emptyDto({
fromIso: query.from,
toIso: query.to,
fromDay: query.from.slice(0, 10),
toDayExclusive: query.to.slice(0, 10),
grain: "day",
windowSec: 1,
})
await warmBindingIfaceCache()
if (query.serverId) await warmIfaceCache([query.serverId])
const bindTuples = await loadBindUserTuples()
const ctx = await buildFilterCtx(query, period)
if (!ctx) return emptyDto(period)
const table = period.grain === "hour" ? "flow_hour_facts" : "flow_daily_facts"
const timeCol = period.grain === "hour" ? "bucket_at" : "day"
const where = factWhere("f", period.grain, ctx)
const totals = await dbAll<{ bytes: number; packets: number; servers: number }>(`
SELECT
COALESCE(SUM(f.bytes), 0) AS bytes,
COALESCE(SUM(f.packets), 0) AS packets,
COUNT(DISTINCT f.server_id)::int AS servers
FROM ${table} f
WHERE ${where.sql}
`, where.params)
const bytes = Number(totals[0]?.bytes) || 0
const packets = Number(totals[0]?.packets) || 0
const serverCount = Number(totals[0]?.servers) || 0
const seriesRows = await dbAll<{ t: string; bytes: number }>(`
SELECT ${timeCol}::text AS t, SUM(f.bytes) AS bytes
FROM ${table} f
WHERE ${where.sql}
GROUP BY ${timeCol}
ORDER BY ${timeCol}
`, where.params)
const countryRows = await dbAll<{ id: string; bytes: number; packets: number }>(`
SELECT f.country AS id, SUM(f.bytes) AS bytes, SUM(f.packets) AS packets
FROM ${table} f
WHERE ${where.sql}
GROUP BY f.country
`, where.params)
const serviceRows = await dbAll<{ id: string; bytes: number; packets: number }>(`
SELECT f.service AS id, SUM(f.bytes) AS bytes, SUM(f.packets) AS packets
FROM ${table} f
WHERE ${where.sql}
GROUP BY f.service
`, where.params)
const asnRows = await dbAll<{ id: number; bytes: number; packets: number }>(`
SELECT f.asn AS id, SUM(f.bytes) AS bytes, SUM(f.packets) AS packets
FROM ${table} f
WHERE ${where.sql}
GROUP BY f.asn
`, where.params)
const serverRows = await dbAll<{ id: number; bytes: number; packets: number }>(`
SELECT f.server_id AS id, SUM(f.bytes) AS bytes, SUM(f.packets) AS packets
FROM ${table} f
WHERE ${where.sql}
GROUP BY f.server_id
`, where.params)
const ifaceRowsRaw = await dbAll<{ serverId: number; iface: string; bytes: number; packets: number }>(`
SELECT f.server_id AS "serverId", f.iface AS iface, SUM(f.bytes) AS bytes, SUM(f.packets) AS packets
FROM ${table} f
WHERE ${where.sql}
GROUP BY f.server_id, f.iface
`, where.params)
await warmIfaceCache(ifaceRowsRaw.filter((r) => looksLikeIfIndex(r.iface)).map((r) => r.serverId))
const ifaceRows = collapseServerIfaceRows(ifaceRowsRaw).filter((r) => {
if (isJunkFactIface(r.iface) || isDashDisplayIface(r.iface)) return false
if (ctx.topo && isOverlayGreIface(ctx.topo, r.serverId, r.iface)) return false
return true
})
const ifaceCount = ifaceRows.length
let userRows: Array<{ id: string; bytes: number; packets: number }> = []
if (bindTuples.length && !ctx.unboundOnly) {
const join = userBindJoinSql(bindTuples)
userRows = await dbAll<{ id: string; bytes: number; packets: number }>(`
SELECT b.user_id AS id, SUM(f.bytes) AS bytes, SUM(f.packets) AS packets
FROM ${table} f
${join.sql}
WHERE ${where.sql}
GROUP BY b.user_id
`, [...join.params, ...where.params])
}
const serverNames = new Map<number, string>()
const allServers = await db.select({ id: servers.id, name: servers.name, host: servers.host }).from(servers)
for (const s of allServers) serverNames.set(s.id, s.name || s.host)
const userNames = new Map<string, string>()
const allUsers = await db.select({ id: appUsers.id, name: appUsers.name, login: appUsers.login }).from(appUsers)
for (const u of allUsers) userNames.set(u.id, u.name || u.login)
const asnHolders = new Map<number, string>()
const asnMeta = await db.select({ asn: flowAsnMeta.asn, holder: flowAsnMeta.holder }).from(flowAsnMeta)
for (const a of asnMeta) asnHolders.set(a.asn, a.holder)
const countries = toBreakdown(
countryRows.map((r) => ({
id: r.id,
label: r.id === "XX" ? "Неизвестно" : r.id,
bytes: Number(r.bytes) || 0,
packets: Number(r.packets) || 0,
})),
bytes,
period.windowSec,
)
const services = toBreakdown(
serviceRows.map((r) => ({
id: r.id,
label: r.id,
bytes: Number(r.bytes) || 0,
packets: Number(r.packets) || 0,
})),
bytes,
period.windowSec,
)
const asns = toBreakdown(
asnRows.map((r) => {
const id = Number(r.id) || 0
const holder = asnHolders.get(id)
return {
id: String(id),
label: id === 0 ? "other" : holder ? `AS${id} · ${holder}` : `AS${id}`,
bytes: Number(r.bytes) || 0,
packets: Number(r.packets) || 0,
}
}),
bytes,
period.windowSec,
)
const serverBreakdown = toBreakdown(
serverRows.map((r) => ({
id: String(r.id),
label: serverNames.get(r.id) || String(r.id),
bytes: Number(r.bytes) || 0,
packets: Number(r.packets) || 0,
})),
bytes,
period.windowSec,
)
const interfaces = toBreakdown(
ifaceRows.map((r) => {
const serverName = serverNames.get(r.serverId) || String(r.serverId)
const wan = ctx.topo ? isWanFactIface(ctx.topo, r.serverId, r.iface) : false
return {
id: `${r.serverId}:${r.iface}`,
label: wan ? wanIfaceLabel(serverName, r.iface) : `${serverName} · ${r.iface}`,
bytes: Number(r.bytes) || 0,
packets: Number(r.packets) || 0,
}
}),
bytes,
period.windowSec,
)
const matchedUsers = toBreakdown(
userRows.map((r) => ({
id: r.id,
label: userNames.get(r.id) || r.id,
bytes: Number(r.bytes) || 0,
packets: Number(r.packets) || 0,
})),
bytes,
period.windowSec,
)
const users = [...matchedUsers]
if (!ctx.unboundOnly && !ctx.userIfaces) {
let unboundBytes = 0
let unboundPackets = 0
const skipUnbound = [...ctx.boundIfaces, ...ctx.wanIfaces]
if (skipUnbound.length === 0) {
unboundBytes = bytes
unboundPackets = packets
} else {
const tuples = skipUnbound.map(() => "(?, ?)").join(", ")
const unboundParams = [...where.params]
for (const u of skipUnbound) unboundParams.push(u.serverId, u.iface)
const unboundRows = await dbAll<{ bytes: number; packets: number }>(`
SELECT COALESCE(SUM(f.bytes), 0) AS bytes, COALESCE(SUM(f.packets), 0) AS packets
FROM ${table} f
WHERE ${where.sql}
AND (f.server_id, f.iface) NOT IN (${tuples})
`, unboundParams)
unboundBytes = Number(unboundRows[0]?.bytes) || 0
unboundPackets = Number(unboundRows[0]?.packets) || 0
}
if (unboundBytes > 0) {
const denom = bytes || 1
users.push({
id: STATISTICS_UNBOUND_USER_ID,
label: "Прочие",
bytes: unboundBytes,
packets: unboundPackets,
bps: (unboundBytes * 8) / period.windowSec,
percent: (unboundBytes / denom) * 100,
})
users.sort((a, b) => b.bytes - a.bytes)
}
}
return {
from: period.fromIso,
to: period.toIso,
grain: period.grain,
kpis: {
bytes,
packets,
avgBps: (bytes * 8) / period.windowSec,
users: matchedUsers.length,
servers: serverCount,
ifaces: ifaceCount,
topCountry: countries[0]?.label || "—",
topService: services[0]?.label || "—",
},
series: seriesRows.map((r) => ({ t: r.t, bytes: Number(r.bytes) || 0 })),
users,
servers: serverBreakdown,
interfaces,
countries,
services,
asns,
}
}
function dimSql(dim: StatisticsPivotDim, factAlias: string, bindAlias: string): string {
if (dim === "country") return `${factAlias}.country`
if (dim === "service") return `${factAlias}.service`
if (dim === "asn") return `${factAlias}.asn::text`
if (dim === "server") return `${factAlias}.server_id::text`
if (dim === "iface") return `(${factAlias}.server_id::text || ':' || ${factAlias}.iface)`
return `${bindAlias}.user_id`
}
function emptyPivot(query: StatisticsPivotQuery): StatisticsPivotDto {
return {
rowDim: query.row,
colDim: query.col,
metric: query.metric,
columns: [],
rows: [],
otherBytes: 0,
}
}
export function pivotDimsConflict(row: StatisticsPivotDim, col: StatisticsPivotDim): boolean {
return row === col
}
export async function getStatisticsPivot(query: StatisticsPivotQuery): Promise<StatisticsPivotDto> {
if (pivotDimsConflict(query.row, query.col)) return emptyPivot(query)
const period = parseStatisticsPeriod(query.from, query.to)
if (!period) return emptyPivot(query)
await warmBindingIfaceCache()
if (query.serverId) await warmIfaceCache([query.serverId])
const bindTuples = await loadBindUserTuples()
const ctx = await buildFilterCtx(query, period)
if (!ctx) return emptyPivot(query)
const needsUser = query.row === "user" || query.col === "user"
if (needsUser && bindTuples.length === 0) return emptyPivot(query)
const table = period.grain === "hour" ? "flow_hour_facts" : "flow_daily_facts"
const where = factWhere("f", period.grain, ctx)
const rowExpr = dimSql(query.row, "f", "b")
const colExpr = dimSql(query.col, "f", "b")
const join = needsUser ? userBindJoinSql(bindTuples) : { sql: "", params: [] as unknown[] }
const raw = await dbAll<{ row_id: string; col_id: string; bytes: number; packets: number }>(`
SELECT ${rowExpr} AS row_id, ${colExpr} AS col_id,
SUM(f.bytes) AS bytes, SUM(f.packets) AS packets
FROM ${table} f
${join.sql}
WHERE ${where.sql}
GROUP BY 1, 2
`, [...join.params, ...where.params])
if (query.row === "iface" || query.col === "iface") {
const ifaceServerIds: number[] = []
for (const r of raw) {
for (const dim of [query.row, query.col] as const) {
if (dim !== "iface") continue
const id = dim === query.row ? String(r.row_id ?? "") : String(r.col_id ?? "")
const colon = id.indexOf(":")
if (colon < 0) continue
const sid = Number(id.slice(0, colon))
if (looksLikeIfIndex(id.slice(colon + 1)) && Number.isFinite(sid)) ifaceServerIds.push(sid)
}
}
await warmIfaceCache(ifaceServerIds)
for (const r of raw) {
if (query.row === "iface") r.row_id = canonicalIfaceDimId(String(r.row_id ?? ""))
if (query.col === "iface") r.col_id = canonicalIfaceDimId(String(r.col_id ?? ""))
}
}
const metric = query.metric
type Acc = { bytes: number; packets: number }
const cell = new Map<string, Map<string, Acc>>()
const colTotals = new Map<string, number>()
for (const r of raw) {
const rid = String(r.row_id ?? "")
const cid = String(r.col_id ?? "")
const acc: Acc = { bytes: Number(r.bytes) || 0, packets: Number(r.packets) || 0 }
const val = metric === "packets" ? acc.packets : acc.bytes
let rowMap = cell.get(rid)
if (!rowMap) {
rowMap = new Map()
cell.set(rid, rowMap)
}
const prev = rowMap.get(cid)
if (prev) {
prev.bytes += acc.bytes
prev.packets += acc.packets
} else {
rowMap.set(cid, acc)
}
colTotals.set(cid, (colTotals.get(cid) ?? 0) + val)
}
const topCols = [...colTotals.entries()]
.sort((a, b) => b[1] - a[1])
.slice(0, PIVOT_COL_CAP)
.map(([id]) => id)
const topColSet = new Set(topCols)
const folded = new Map<string, Map<string, number>>()
const foldedColTotals = new Map<string, number>()
let otherBytes = 0
for (const [rid, cols] of cell) {
const rowMap = new Map<string, number>()
for (const [cid, acc] of cols) {
const val = metric === "packets" ? acc.packets : acc.bytes
const dest = topColSet.has(cid) ? cid : PIVOT_OTHER_ID
if (dest === PIVOT_OTHER_ID) otherBytes += val
rowMap.set(dest, (rowMap.get(dest) ?? 0) + val)
foldedColTotals.set(dest, (foldedColTotals.get(dest) ?? 0) + val)
}
folded.set(rid, rowMap)
}
const rowTotals = new Map<string, number>()
for (const [rid, cols] of folded) {
let t = 0
for (const v of cols.values()) t += v
rowTotals.set(rid, t)
}
const topRows = [...rowTotals.entries()]
.sort((a, b) => b[1] - a[1])
.slice(0, PIVOT_ROW_CAP)
.map(([id]) => id)
const topRowSet = new Set(topRows)
const finalRows = new Map<string, Map<string, number>>()
const finalRowTotals = new Map<string, number>()
for (const [rid, cols] of folded) {
const dest = topRowSet.has(rid) ? rid : PIVOT_OTHER_ID
if (dest === PIVOT_OTHER_ID) {
for (const [cid, v] of cols) {
if (cid !== PIVOT_OTHER_ID) otherBytes += v
}
}
let rowMap = finalRows.get(dest)
if (!rowMap) {
rowMap = new Map()
finalRows.set(dest, rowMap)
}
for (const [cid, v] of cols) {
rowMap.set(cid, (rowMap.get(cid) ?? 0) + v)
}
}
for (const [rid, cols] of finalRows) {
let t = 0
for (const v of cols.values()) t += v
finalRowTotals.set(rid, t)
}
const colIds = [...topCols]
if (foldedColTotals.has(PIVOT_OTHER_ID)) colIds.push(PIVOT_OTHER_ID)
const rowIds = [...topRows]
if (finalRows.has(PIVOT_OTHER_ID) && !topRowSet.has(PIVOT_OTHER_ID)) rowIds.push(PIVOT_OTHER_ID)
const labels = await loadPivotLabels(query.row, query.col, rowIds, colIds)
return {
rowDim: query.row,
colDim: query.col,
metric,
columns: colIds.map((id) => ({
id,
label: labels.col.get(id) ?? (id === PIVOT_OTHER_ID ? "Прочие" : id),
total: foldedColTotals.get(id) ?? 0,
})),
rows: rowIds.map((id) => {
const cols = finalRows.get(id) ?? new Map()
const cells: Record<string, number> = {}
for (const cid of colIds) cells[cid] = cols.get(cid) ?? 0
return {
id,
label: labels.row.get(id) ?? (id === PIVOT_OTHER_ID ? "Прочие" : id),
total: finalRowTotals.get(id) ?? 0,
cells,
}
}),
otherBytes,
}
}
async function loadPivotLabels(
rowDim: StatisticsPivotDim,
colDim: StatisticsPivotDim,
rowIds: string[],
colIds: string[],
): Promise<{ row: Map<string, string>; col: Map<string, string> }> {
const topo = await loadFlowTopology()
const serverNames = new Map<string, string>()
const allServers = await db.select({ id: servers.id, name: servers.name, host: servers.host }).from(servers)
for (const s of allServers) serverNames.set(String(s.id), s.name || s.host)
const userNames = new Map<string, string>()
const allUsers = await db.select({ id: appUsers.id, name: appUsers.name, login: appUsers.login }).from(appUsers)
for (const u of allUsers) userNames.set(u.id, u.name || u.login)
const asnHolders = new Map<string, string>()
const asnMeta = await db.select({ asn: flowAsnMeta.asn, holder: flowAsnMeta.holder }).from(flowAsnMeta)
for (const a of asnMeta) asnHolders.set(String(a.asn), a.holder)
function label(dim: StatisticsPivotDim, id: string): string {
if (id === PIVOT_OTHER_ID) return "Прочие"
if (dim === "country") return id === "XX" ? "Неизвестно" : id
if (dim === "server") return serverNames.get(id) || id
if (dim === "user") return userNames.get(id) || id
if (dim === "asn") {
if (id === "0") return "other"
const holder = asnHolders.get(id)
return holder ? `AS${id} · ${holder}` : `AS${id}`
}
if (dim === "iface") {
const colon = id.indexOf(":")
if (colon < 0) return id
const sid = id.slice(0, colon)
const iface = id.slice(colon + 1)
const sidNum = Number(sid)
const name = Number.isFinite(sidNum) ? displayFactIface(sidNum, iface) : iface
const serverName = serverNames.get(sid) || sid
if (Number.isFinite(sidNum) && isWanFactIface(topo, sidNum, name)) {
return wanIfaceLabel(serverName, name)
}
return `${serverName} · ${name}`
}
return id
}
const row = new Map<string, string>()
const col = new Map<string, string>()
for (const id of rowIds) row.set(id, label(rowDim, id))
for (const id of colIds) col.set(id, label(colDim, id))
return { row, col }
}
@@ -26,7 +26,8 @@ import { resolveIfaceName } from "./traffic-flow-ifaces.js"
import { getTrafficFlowSettingsRow, listHostPeers } from "./traffic-flow-settings.js"
import { applicationName, flowRowMatchesFilter } from "./traffic-flow-apps.js"
import { dedupFlowRowsMaxBytes, flowTupleKey } from "./traffic-flow-dedup.js"
import { enqueueRipeMisses, lookupRipeCached } from "./traffic-flow-ripe.js"
import { enqueueRipeMisses } from "./traffic-flow-ripe.js"
import { resolveFlowIp } from "./traffic-flow-geoip.js"
import { classifyFlowDst, refreshFlowCatalogInBackground } from "./traffic-flow-classify.js"
import { isIsoCountry } from "./traffic-flow-brands.js"
import { classifyFlowPlane, flowBps, shouldKeepPlane } from "./traffic-flow-planes.js"
@@ -255,7 +256,7 @@ async function buildFlowAnalyticsUncached(q: FlowAnalyticsQuery): Promise<FlowAn
const peer = pickInternetPeer(r.src, r.dst, r.srcPort, r.dstPort)
peers.add(peer)
const app = applicationName(r.proto, r.dstPort, r.srcPort)
const ripe = lookupRipeCached(peer)
const ripe = resolveFlowIp(peer)
const classified = classifyFlowDst(peer, r.proto, r.dstPort, r.srcPort, ripe)
bump(applications, app, r.bytes, r.packets)
bump(protocols, protoName(r.proto), r.bytes, r.packets)
@@ -1,5 +1,10 @@
import assert from "node:assert/strict"
import { dedupFlowRowsMaxBytes, flowTupleKey } from "./traffic-flow-dedup.js"
import {
dedupFlowRowsAcrossExporters,
dedupFlowRowsMaxBytes,
flowConversationKey,
flowTupleKey,
} from "./traffic-flow-dedup.js"
const a = {
serverId: 7,
@@ -22,4 +27,13 @@ assert.equal(flowTupleKey(a), flowTupleKey(b))
const sameIface = dedupFlowRowsMaxBytes([a, { ...a, bytes: 3_000, packets: 2 }])
assert.equal(sameIface[0]?.bytes, 15_000)
const jh = { ...a, serverId: 7, bytes: 9_000 }
const en = { ...a, serverId: 9, bytes: 11_000, inIface: "1" }
assert.equal(flowConversationKey(jh), flowConversationKey(en))
assert.notEqual(flowTupleKey(jh), flowTupleKey(en))
const across = dedupFlowRowsAcrossExporters([en, jh], (x, y) => (x.serverId === 7 ? x : y))
assert.equal(across.length, 1)
assert.equal(across[0]?.serverId, 7)
assert.equal(across[0]?.bytes, 9_000)
console.log("traffic-flow-dedup.test.ts: ok")
@@ -14,6 +14,32 @@ export function flowTupleKey(r: Pick<FlowTupleRow, "serverId" | "src" | "dst" |
return `${r.serverId}|${r.src}|${r.dst}|${r.proto}|${r.srcPort}|${r.dstPort}`
}
/** Один разговор на всех экспортёрах (JH+EN), без serverId. */
export function flowConversationKey(r: Pick<FlowTupleRow, "src" | "dst" | "proto" | "srcPort" | "dstPort">): string {
return `${r.src}|${r.dst}|${r.proto}|${r.srcPort}|${r.dstPort}`
}
/**
* Схлопнуть копии одного 5-tuple с разных серверов.
* `prefer` выбирает ряд (клиент на JH важнее голого EN).
*/
export function dedupFlowRowsAcrossExporters<T extends FlowTupleRow>(
rows: T[],
prefer: (a: T, b: T) => T,
): T[] {
const byConv = new Map<string, T>()
for (const row of rows) {
const key = flowConversationKey(row)
const prev = byConv.get(key)
if (!prev) {
byConv.set(key, row)
continue
}
byConv.set(key, prefer(prev, row))
}
return [...byConv.values()]
}
function ifaceKey(r: FlowTupleRow): string {
return `${flowTupleKey(r)}|${r.inIface}`
}
+76 -3
View File
@@ -5,11 +5,27 @@ import { classifyFlowPlaneLite } from "./traffic-flow-planes.js"
import { pickServerIdForExporter, type OverlayPeerRef } from "./traffic-flow-map-exporter.js"
import { applicationName } from "./traffic-flow-apps.js"
import { classifyFlowDst } from "./traffic-flow-classify.js"
import { enqueueRipeMisses, lookupRipeCached, pruneRipeSqlite } from "./traffic-flow-ripe.js"
import { enqueueRipeMisses, pruneRipeSqlite } from "./traffic-flow-ripe.js"
import { resolveFlowIp } from "./traffic-flow-geoip.js"
import { invalidateTrafficFlowSettingsCache } from "./traffic-flow-settings.js"
import { isIsoCountry } from "./traffic-flow-brands.js"
import { maybeRefreshIfaces } from "./traffic-flow-ifaces.js"
import { canonicalFactIface } from "./traffic-flow-ifindex.js"
import { shouldWriteFlowFact } from "./traffic-flow-facts-filter.js"
import { pickInternetPeer } from "./traffic-flow-ip.js"
import {
getServerCatalog,
loadFlowTopology,
peekFlowTopology,
peekServerCatalog,
} from "./traffic-flow-topology.js"
import {
bumpFlowFact,
factsPendingSize,
flushFlowFacts,
hourBucketIso,
resetFactsForTests,
} from "./traffic-flow-facts.js"
export const TICK_MS = 2_000
export const PERSIST_MS = 10_000
@@ -327,13 +343,19 @@ export function getEngineStats(): EngineStats {
export function queueParsedFlows(serverId: number, flows: ParsedFlowInput[]): void {
if (flows.length) bumpDataEpoch()
const bucketAt = minuteBucketIso()
const hourAt = hourBucketIso()
const ripeMisses: string[] = []
const topo = peekFlowTopology()
const catalog = peekServerCatalog()
if (!topo) void loadFlowTopology().catch(() => {})
if (!catalog) void getServerCatalog().catch(() => {})
const serverType = catalog?.byId.get(serverId)?.type
for (const raw of flows) {
const flow = normalizeParsedFlow(raw)
addToTick(serverId, flow, flow.bytes)
bumpRollup(serverId, bucketAt, flow, flow.bytes, flow.packets)
const peer = pickInternetPeer(flow.src, flow.dst, flow.srcPort, flow.dstPort)
const ripe = lookupRipeCached(peer)
const ripe = resolveFlowIp(peer)
if (peer && !ripe) ripeMisses.push(peer)
const classified = classifyFlowDst(peer, flow.proto, flow.dstPort, flow.srcPort, ripe)
const app = applicationName(flow.proto, flow.dstPort, flow.srcPort)
@@ -348,6 +370,29 @@ export function queueParsedFlows(serverId: number, flows: ParsedFlowInput[]): vo
bumpDim(serverId, bucketAt, "service", classified.service, flow.bytes, flow.packets)
if (country) bumpDim(serverId, bucketAt, "country", country, flow.bytes, flow.packets)
bumpDim(serverId, bucketAt, "asn", asnKey, flow.bytes, flow.packets)
if (shouldWriteFlowFact({
serverId,
serverType,
inIface: flow.inIface,
outIface: flow.outIface,
proto: flow.proto,
srcPort: flow.srcPort,
dstPort: flow.dstPort,
src: flow.src,
dst: flow.dst,
topo,
})) {
bumpFlowFact({
serverId,
bucketAt: hourAt,
iface: canonicalFactIface(serverId, flow.inIface),
country: country || "XX",
service: classified.service,
asn: ripe?.ok && ripe.asn ? ripe.asn : 0,
bytes: flow.bytes,
packets: flow.packets,
})
}
const key = pendingKey(serverId, bucketAt, flow)
const prev = pending.get(key)
@@ -826,7 +871,7 @@ export async function flushPending(opts?: { force?: boolean }): Promise<void> {
pruneRecent()
rollFlowRings()
const force = Boolean(opts?.force)
const hasWork = pending.size > 0 || minuteRollup.size > 0 || minuteDims.size > 0
const hasWork = pending.size > 0 || minuteRollup.size > 0 || minuteDims.size > 0 || factsPendingSize() > 0
const due = persistDue(force, hasWork)
try {
await persistListenerStats(force)
@@ -884,6 +929,11 @@ export async function flushPending(opts?: { force?: boolean }): Promise<void> {
} catch {
/* rollup best-effort */
}
try {
await flushFlowFacts()
} catch {
/* statistics cube best-effort */
}
try {
await pruneStored()
} catch {
@@ -915,6 +965,7 @@ export function resetEngineForTests(): void {
rings.clear()
minuteRollup.clear()
minuteDims.clear()
resetFactsForTests()
packetsReceived = 0
lastExporterIp = null
lastError = ""
@@ -939,3 +990,25 @@ export function pendingSizeForTests(): number {
export function droppedForTests(): number {
return dropped
}
/** Снимок минутных dims (dim → key → bytes) для тестов обогащения потоков. */
export function minuteDimsSnapshotForTests(): Map<string, Map<string, { bytes: number; packets: number }>> {
const out = new Map<string, Map<string, { bytes: number; packets: number }>>()
for (const [k, acc] of minuteDims) {
// dimKey: serverId\0bucketAt\0dim\0key
const parts = k.split("\0")
const dim = parts[2] ?? ""
const key = parts.slice(3).join("\0")
let byKey = out.get(dim)
if (!byKey) {
byKey = new Map()
out.set(dim, byKey)
}
const prev = byKey.get(key)
byKey.set(key, {
bytes: (prev?.bytes ?? 0) + acc.bytes,
packets: (prev?.packets ?? 0) + acc.packets,
})
}
return out
}
@@ -0,0 +1,132 @@
import assert from "node:assert/strict"
import {
isJunkFactIface,
isOverlayGreIface,
isWanFactIface,
shouldWriteFlowFact,
} from "./traffic-flow-facts-filter.js"
import { seedFlowTopologyForTests, type FlowTopology } from "./traffic-flow-topology.js"
import { rememberServerIfaces, resetIfaceCacheForTests } from "./traffic-flow-ifindex.js"
function topo(partial: Partial<FlowTopology> = {}): FlowTopology {
const wanIfaces = partial.wanIfaces ?? new Map([[1, new Set(["ether1"])]])
const clientIfaces = partial.clientIfaces ?? new Map([[1, new Set(["gre-client"])]])
const clientByIface = partial.clientByIface ?? new Map()
const enHosts = partial.enHosts ?? new Set(["198.51.100.1"])
const jhHosts = partial.jhHosts ?? new Set(["203.0.113.10"])
return {
clientIfaces,
clientByIface,
enNodes: partial.enNodes ?? [{ id: 2, name: "en", hosts: ["198.51.100.1"] }],
enHosts,
jhHosts,
wanIfaces,
plane: partial.plane ?? {
clientIfaceNames: new Set(["gre-client"]),
enHosts,
jhHosts,
},
}
}
resetIfaceCacheForTests()
rememberServerIfaces(1, [{ name: "ether1", ifindex: "2" }])
seedFlowTopologyForTests(topo())
assert.equal(isJunkFactIface("0"), true)
assert.equal(isJunkFactIface(""), true)
assert.equal(isJunkFactIface("ether1"), false)
assert.equal(isWanFactIface(topo(), 1, "ether1"), true)
assert.equal(isOverlayGreIface(topo(), 1, "gre-en"), true)
assert.equal(isOverlayGreIface(topo(), 1, "gre-client"), false)
assert.equal(isOverlayGreIface(topo(), 1, "ether1"), false)
const overlayOuter = shouldWriteFlowFact({
serverId: 1,
serverType: "jump-host",
inIface: "ether1",
outIface: "gre-en",
proto: 47,
srcPort: 0,
dstPort: 0,
src: "203.0.113.10",
dst: "198.51.100.1",
topo: topo(),
})
assert.equal(overlayOuter, false, "overlay proto 47 на ether1 не в facts")
const payloadGre = shouldWriteFlowFact({
serverId: 1,
serverType: "jump-host",
inIface: "gre-client",
outIface: "gre-en",
proto: 6,
srcPort: 51234,
dstPort: 443,
src: "10.100.1.17",
dst: "8.8.8.8",
topo: topo(),
})
assert.equal(payloadGre, true, "payload на GRE — да")
const payloadWan = shouldWriteFlowFact({
serverId: 1,
serverType: "jump-host",
inIface: "ether1",
outIface: "gre-client",
proto: 6,
srcPort: 443,
dstPort: 51234,
src: "8.8.8.8",
dst: "10.100.1.17",
topo: topo(),
})
assert.equal(payloadWan, true, "payload на ether1 WAN — да")
const junkZero = shouldWriteFlowFact({
serverId: 1,
serverType: "jump-host",
inIface: "0",
proto: 6,
srcPort: 443,
dstPort: 80,
src: "1.1.1.1",
dst: "8.8.8.8",
topo: topo(),
})
assert.equal(junkZero, false)
const enTopo = topo({
clientIfaces: new Map([[2, new Set()]]),
wanIfaces: new Map([[2, new Set(["ether1"])]]),
})
const enTransit = shouldWriteFlowFact({
serverId: 2,
serverType: "exit-node",
inIface: "gre-jh",
outIface: "ether1",
proto: 6,
srcPort: 51234,
dstPort: 443,
src: "10.100.1.17",
dst: "8.8.8.8",
topo: enTopo,
})
assert.equal(enTransit, false, "EN-транзит без клиента — нет")
const enWan = shouldWriteFlowFact({
serverId: 2,
serverType: "exit-node",
inIface: "ether1",
proto: 6,
srcPort: 443,
dstPort: 80,
src: "8.8.8.8",
dst: "198.51.100.1",
topo: enTopo,
})
assert.equal(enWan, true, "WAN payload на EN — да")
seedFlowTopologyForTests(null)
resetIfaceCacheForTests()
console.log("traffic-flow-facts-filter.test.ts: ok")
@@ -0,0 +1,85 @@
import { mapRosInterfaceType } from "../modules/users/iface-type.js"
import { canonicalFactIface } from "./traffic-flow-ifindex.js"
import { classifyFlowPlane } from "./traffic-flow-planes.js"
import { resolveClient, type FlowTopology } from "./traffic-flow-topology.js"
const JUNK_IFACE = new Set(["", "0", "—", "__unknown__", "wg-flow"])
export function isJunkFactIface(iface: string | null | undefined): boolean {
const n = String(iface ?? "").trim()
if (JUNK_IFACE.has(n)) return true
return /^#?0$/.test(n)
}
export function isDashDisplayIface(iface: string): boolean {
return String(iface ?? "").trim() === "—"
}
export function isGreIfaceName(name: string): boolean {
return mapRosInterfaceType("", name) === "gre"
}
/** WAN uplink: `wanIfaces` топологии, иначе ether1 у JH/EN без wan_uplinks. */
export function isWanFactIface(
topo: FlowTopology | null | undefined,
serverId: number,
iface: string,
): boolean {
const name = String(iface ?? "").trim()
if (!name || isJunkFactIface(name) || isDashDisplayIface(name)) return false
const wan = topo?.wanIfaces.get(serverId)
if (wan && wan.size > 0) return wan.has(name)
return /^ether1$/i.test(name)
}
/** GRE между своими серверами / транзит EN: gre-имя, не клиент, не WAN. */
export function isOverlayGreIface(
topo: FlowTopology | null | undefined,
serverId: number,
iface: string,
): boolean {
const name = String(iface ?? "").trim()
if (!name || isWanFactIface(topo, serverId, name)) return false
if (topo?.clientIfaces.get(serverId)?.has(name)) return false
if (name === "wg-flow") return false
return isGreIfaceName(name)
}
export function shouldWriteFlowFact(opts: {
serverId: number
serverType?: string
inIface: string
outIface?: string
proto: number
srcPort: number
dstPort: number
src: string
dst: string
topo?: FlowTopology | null
}): boolean {
const inName = canonicalFactIface(opts.serverId, opts.inIface) || String(opts.inIface ?? "").trim()
if (isJunkFactIface(inName) || isJunkFactIface(opts.inIface)) return false
const outRaw = String(opts.outIface ?? "").trim()
const outName = outRaw ? (canonicalFactIface(opts.serverId, outRaw) || outRaw) : ""
const plane = classifyFlowPlane({
src: opts.src,
dst: opts.dst,
proto: opts.proto,
srcPort: opts.srcPort,
dstPort: opts.dstPort,
inIface: inName,
outIface: outName || undefined,
}, opts.topo?.plane)
if (plane !== "payload") return false
if (opts.serverType === "exit-node" && opts.topo) {
const client =
resolveClient(opts.topo, opts.serverId, inName)
?? (outName ? resolveClient(opts.topo, opts.serverId, outName) : null)
if (!client && isOverlayGreIface(opts.topo, opts.serverId, inName)) return false
}
return true
}
export function wanIfaceLabel(serverName: string, iface: string): string {
return `${serverName} · ${iface} · WAN · интернет`
}
@@ -0,0 +1,71 @@
import assert from "node:assert/strict"
import {
bumpFlowFact,
collectCappedFacts,
FACT_ASN_TOP,
FACT_TUPLE_CAP,
resetFactsForTests,
} from "./traffic-flow-facts.js"
resetFactsForTests()
bumpFlowFact({
serverId: 1,
bucketAt: "2026-09-10T10:00:00.000Z",
iface: "ether1",
country: "US",
service: "https",
asn: 15169,
bytes: 100,
packets: 2,
})
bumpFlowFact({
serverId: 1,
bucketAt: "2026-09-10T10:00:00.000Z",
iface: "ether1",
country: "us",
service: "https",
asn: 15169,
bytes: 50,
packets: 1,
})
const merged = collectCappedFacts()
assert.equal(merged.length, 1)
assert.equal(merged[0]?.bytes, 150)
assert.equal(merged[0]?.country, "US")
assert.equal(merged[0]?.asn, 15169)
resetFactsForTests()
for (let i = 1; i <= FACT_ASN_TOP + 20; i++) {
bumpFlowFact({
serverId: 2,
bucketAt: "2026-09-10T11:00:00.000Z",
iface: "ether1",
country: "DE",
service: "https",
asn: i,
bytes: FACT_ASN_TOP + 21 - i,
packets: 1,
})
}
const cappedAsn = collectCappedFacts()
const asns = new Set(cappedAsn.map((r) => r.asn))
assert.ok(asns.has(0))
assert.ok(asns.size <= FACT_ASN_TOP + 1)
resetFactsForTests()
for (let i = 0; i < FACT_TUPLE_CAP + 30; i++) {
bumpFlowFact({
serverId: 3,
bucketAt: "2026-09-10T12:00:00.000Z",
iface: `ether${i % 3}`,
country: "NL",
service: `svc-${i}`,
asn: 1,
bytes: 10,
packets: 1,
})
}
const cappedTuples = collectCappedFacts()
assert.ok(cappedTuples.length <= FACT_TUPLE_CAP + 3)
console.log("traffic-flow-facts.test.ts: ok")
+285
View File
@@ -0,0 +1,285 @@
import { pool } from "../db/index.js"
import { ensurePartitionFor, specForParent } from "../db/partitions.js"
export const FACT_ASN_TOP = 200
export const FACT_TUPLE_CAP = 8000
export const FACT_SERVICE_MAX_LEN = 64
export const UNKNOWN_COUNTRY = "XX"
export const OTHER_SERVICE = "other"
export const UNKNOWN_IFACE = "__unknown__"
export interface FactAcc {
bytes: number
packets: number
}
export interface FactRow {
serverId: number
bucketAt: string
iface: string
country: string
service: string
asn: number
bytes: number
packets: number
}
const hourFacts = new Map<string, FactAcc>()
const ensuredParts = new Set<string>()
export function hourBucketIso(at = Date.now()): string {
const d = new Date(at)
d.setMinutes(0, 0, 0)
return d.toISOString()
}
export function normalizeFactCountry(raw: string): string {
const iso = raw.trim().toUpperCase()
if (/^[A-Z]{2}$/.test(iso)) return iso
return UNKNOWN_COUNTRY
}
export function normalizeFactService(raw: string): string {
const s = raw.trim().slice(0, FACT_SERVICE_MAX_LEN)
return s || OTHER_SERVICE
}
export function normalizeFactIface(raw: string): string {
return raw.trim() || UNKNOWN_IFACE
}
export function normalizeFactAsn(raw: number): number {
if (!Number.isFinite(raw) || raw <= 0) return 0
return Math.trunc(raw)
}
function factKey(
serverId: number,
bucketAt: string,
iface: string,
country: string,
service: string,
asn: number,
): string {
return `${serverId}\0${bucketAt}\0${iface}\0${country}\0${service}\0${asn}`
}
function parseFactKey(k: string, acc: FactAcc): FactRow | null {
const parts = k.split("\0")
if (parts.length !== 6) return null
const serverId = Number(parts[0])
const asn = Number(parts[5])
if (!Number.isFinite(serverId) || !Number.isFinite(asn)) return null
return {
serverId,
bucketAt: parts[1] ?? "",
iface: parts[2] ?? UNKNOWN_IFACE,
country: parts[3] ?? UNKNOWN_COUNTRY,
service: parts[4] ?? OTHER_SERVICE,
asn,
bytes: acc.bytes,
packets: acc.packets,
}
}
export function bumpFlowFact(row: {
serverId: number
bucketAt: string
iface: string
country: string
service: string
asn: number
bytes: number
packets: number
}): void {
const iface = normalizeFactIface(row.iface)
const country = normalizeFactCountry(row.country)
const service = normalizeFactService(row.service)
const asn = normalizeFactAsn(row.asn)
const k = factKey(row.serverId, row.bucketAt, iface, country, service, asn)
const prev = hourFacts.get(k)
if (prev) {
prev.bytes += row.bytes
prev.packets += row.packets
return
}
hourFacts.set(k, { bytes: row.bytes, packets: row.packets })
}
function groupKey(row: FactRow): string {
return `${row.serverId}\0${row.bucketAt}`
}
function mergeRow(map: Map<string, FactRow>, row: FactRow): void {
const k = factKey(row.serverId, row.bucketAt, row.iface, row.country, row.service, row.asn)
const prev = map.get(k)
if (prev) {
prev.bytes += row.bytes
prev.packets += row.packets
return
}
map.set(k, { ...row })
}
/** Cap ASN tail and tuple count per server×hour before persist. */
export function collectCappedFacts(): FactRow[] {
const parsed: FactRow[] = []
for (const [k, acc] of hourFacts) {
const row = parseFactKey(k, acc)
if (row) parsed.push(row)
}
hourFacts.clear()
const groups = new Map<string, FactRow[]>()
for (const row of parsed) {
const g = groupKey(row)
const list = groups.get(g) ?? []
list.push(row)
groups.set(g, list)
}
const out = new Map<string, FactRow>()
for (const list of groups.values()) {
const byAsn = new Map<number, number>()
for (const row of list) {
byAsn.set(row.asn, (byAsn.get(row.asn) ?? 0) + row.bytes)
}
const asnKeep = new Set(
[...byAsn.entries()]
.sort((a, b) => b[1] - a[1])
.slice(0, FACT_ASN_TOP)
.map(([asn]) => asn),
)
const afterAsn: FactRow[] = []
for (const row of list) {
if (asnKeep.has(row.asn) || row.asn === 0) {
afterAsn.push(row)
continue
}
afterAsn.push({ ...row, asn: 0 })
}
const collapsed = new Map<string, FactRow>()
for (const row of afterAsn) mergeRow(collapsed, row)
const tuples = [...collapsed.values()].sort((a, b) => b.bytes - a.bytes)
const keep = tuples.slice(0, FACT_TUPLE_CAP)
const tail = tuples.slice(FACT_TUPLE_CAP)
for (const row of keep) mergeRow(out, row)
for (const row of tail) {
mergeRow(out, {
...row,
country: UNKNOWN_COUNTRY,
service: OTHER_SERVICE,
asn: 0,
})
}
}
return [...out.values()]
}
async function ensureParentPartition(parent: string, ts: string): Promise<void> {
const spec = specForParent(parent)
if (!spec) return
const iso = ts.length === 10 ? `${ts}T00:00:00Z` : ts
const key = `${parent}:${iso.slice(0, 10)}`
if (ensuredParts.has(key)) return
await ensurePartitionFor(pool, parent, spec.kind, new Date(iso))
ensuredParts.add(key)
}
function dayKey(bucketAt: string): string {
return bucketAt.slice(0, 10)
}
export async function flushFlowFacts(): Promise<number> {
const rows = collectCappedFacts()
if (rows.length === 0) return 0
const hours = new Set(rows.map((r) => r.bucketAt))
const days = new Set(rows.map((r) => dayKey(r.bucketAt)))
for (const h of hours) await ensureParentPartition("flow_hour_facts", h)
for (const d of days) await ensureParentPartition("flow_daily_facts", d)
await pool.query({
text: `
INSERT INTO flow_hour_facts (
server_id, bucket_at, iface, country, service, asn, bytes, packets
)
SELECT *
FROM UNNEST(
$1::bigint[],
$2::timestamptz[],
$3::text[],
$4::char(2)[],
$5::text[],
$6::int[],
$7::bigint[],
$8::bigint[]
) AS t(server_id, bucket_at, iface, country, service, asn, bytes, packets)
ON CONFLICT (server_id, bucket_at, iface, country, service, asn)
DO UPDATE SET
bytes = flow_hour_facts.bytes + excluded.bytes,
packets = flow_hour_facts.packets + excluded.packets
`,
values: [
rows.map((r) => r.serverId),
rows.map((r) => r.bucketAt),
rows.map((r) => r.iface),
rows.map((r) => r.country),
rows.map((r) => r.service),
rows.map((r) => r.asn),
rows.map((r) => r.bytes),
rows.map((r) => r.packets),
],
})
await pool.query({
text: `
INSERT INTO flow_daily_facts (
server_id, day, iface, country, service, asn, bytes, packets
)
SELECT *
FROM UNNEST(
$1::bigint[],
$2::date[],
$3::text[],
$4::char(2)[],
$5::text[],
$6::int[],
$7::bigint[],
$8::bigint[]
) AS t(server_id, day, iface, country, service, asn, bytes, packets)
ON CONFLICT (server_id, day, iface, country, service, asn)
DO UPDATE SET
bytes = flow_daily_facts.bytes + excluded.bytes,
packets = flow_daily_facts.packets + excluded.packets
`,
values: [
rows.map((r) => r.serverId),
rows.map((r) => dayKey(r.bucketAt)),
rows.map((r) => r.iface),
rows.map((r) => r.country),
rows.map((r) => r.service),
rows.map((r) => r.asn),
rows.map((r) => r.bytes),
rows.map((r) => r.packets),
],
})
return rows.length
}
export function factsPendingSize(): number {
return hourFacts.size
}
export function resetFactsForTests(): void {
hourFacts.clear()
ensuredParts.clear()
}
export function factsSnapshotForTests(): FactRow[] {
const parsed: FactRow[] = []
for (const [k, acc] of hourFacts) {
const row = parseFactKey(k, acc)
if (row) parsed.push(row)
}
return parsed
}
@@ -0,0 +1,186 @@
import assert from "node:assert/strict"
import { existsSync, mkdtempSync, readFileSync, rmSync, writeFileSync } from "node:fs"
import { tmpdir } from "node:os"
import path from "node:path"
import type { AsnResponse, CountryResponse, Reader } from "maxmind"
import {
disableRipeEnqueueForTests,
disableRipePersistForTests,
resetRipeCacheForTests,
seedRipeCacheForTests,
} from "./traffic-flow-ripe.js"
import {
lookupGeoip,
resetGeoipForTests,
resolveFlowIp,
setGeoipReadersForTests,
} from "./traffic-flow-geoip.js"
import {
resetEngineForTests,
ingestParsedFlowsForServerForTests,
minuteDimsSnapshotForTests,
} from "./traffic-flow-engine.js"
import { classifyFlowDst } from "./traffic-flow-classify.js"
import { disableGeoipDbForTests } from "./geoip-settings.js"
import {
collectGeoipUpdateOnce,
resetGeoipUpdateForTests,
setGeoipFetchForTests,
setGeoipValidateForTests,
} from "./geoip-update-collector.js"
disableRipePersistForTests()
disableRipeEnqueueForTests()
resetRipeCacheForTests()
resetGeoipForTests()
// ── lookupGeoip: приватные IP → negative без ридеров ──────────────────────────
assert.equal(lookupGeoip("10.1.1.8")?.ok, false)
assert.equal(lookupGeoip("192.168.0.1")?.prefix, "192.168.0.1/32")
assert.equal(lookupGeoip("100.64.1.2")?.ok, false)
assert.equal(lookupGeoip("fe80::1")?.prefix, "fe80::1/128")
// ── без ридеров публичный IP → null, resolveFlowIp уходит в RIPE-кэш ─────────
assert.equal(lookupGeoip("1.2.3.10"), null)
seedRipeCacheForTests({
prefix: "1.2.3.0/24",
asn: 64500,
country: "NL",
lat: null,
lng: null,
holder: "TEST",
ok: true,
fetchedAt: Date.now(),
})
assert.equal(resolveFlowIp("1.2.3.10")?.country, "NL")
assert.equal(resolveFlowIp("1.2.3.10")?.asn, 64500)
// ── fake-ридеры: geoip приоритетнее RIPE ──────────────────────────────────────
function fakeCountryReader(byIp: Record<string, string>): Reader<CountryResponse> {
return {
get(ip: string) {
const iso = byIp[ip]
return iso ? ({ country: { iso_code: iso } } as CountryResponse) : null
},
metadata: { buildEpoch: new Date("2026-09-02T00:00:00Z") },
} as unknown as Reader<CountryResponse>
}
function fakeAsnReader(byIp: Record<string, { asn: number; org: string }>): Reader<AsnResponse> {
return {
get(ip: string) {
const hit = byIp[ip]
return hit
? ({ autonomous_system_number: hit.asn, autonomous_system_organization: hit.org } as AsnResponse)
: null
},
metadata: { buildEpoch: new Date("2026-09-02T00:00:00Z") },
} as unknown as Reader<AsnResponse>
}
setGeoipReadersForTests({
country: fakeCountryReader({ "8.8.8.8": "US", "6.6.6.6": "EU" }),
asn: fakeAsnReader({
"8.8.8.8": { asn: 15169, org: "GOOGLE" },
"6.6.6.6": { asn: 15169, org: "GOOGLE" },
}),
})
const hit = resolveFlowIp("8.8.8.8")
assert.equal(hit?.country, "US")
assert.equal(hit?.asn, 15169)
assert.equal(hit?.holder, "GOOGLE")
assert.equal(hit?.ok, true)
// 1.2.3.10 в fake-ридерах нет — по-прежнему из RIPE-кэша
assert.equal(resolveFlowIp("1.2.3.10")?.asn, 64500)
// EU не ISO-страна: отфильтрована, страна выведена из ASN (HQ Google → US)
assert.equal(lookupGeoip("6.6.6.6")?.country, "US")
// geoip-мета совместима с classifyFlowDst (бренд по ASN 15169)
const classified = classifyFlowDst("8.8.8.8", 6, 443, 51504, hit)
assert.equal(classified.service, "Google")
// ── движок: dims country/asn наполняются из geoip-ридеров ────────────────────
resetEngineForTests()
ingestParsedFlowsForServerForTests(1, [{
src: "192.168.88.10",
dst: "8.8.8.8",
proto: 6,
srcPort: 51504,
dstPort: 443,
bytes: 1000,
packets: 10,
inIface: "wg-flow",
outIface: "",
nextHop: "",
flowStartMs: 0,
flowEndMs: 0,
natSrc: "",
natDst: "",
}])
const dims = minuteDimsSnapshotForTests()
assert.equal(dims.get("country")?.get("US")?.bytes, 1000)
assert.equal(dims.get("asn")?.get("15169")?.bytes, 1000)
// ── коллектор: 304 → обе базы без изменений ───────────────────────────────────
disableGeoipDbForTests()
resetGeoipUpdateForTests()
const geoipDir = mkdtempSync(path.join(tmpdir(), "mm-geoip-test-"))
process.env.GEOIP_DIR = geoipDir
setGeoipFetchForTests(async () => new Response(null, { status: 304 }))
let snap = await collectGeoipUpdateOnce({ force: true })
assert.equal(snap.skippedUnchanged, 2)
assert.equal(snap.downloaded, 0)
assert.equal(existsSync(path.join(geoipDir, "GeoLite2-Country.mmdb")), false)
// ── коллектор: 200 + валидация ok → подмена, старый файл в .prev ─────────────
const countryPath = path.join(geoipDir, "GeoLite2-Country.mmdb")
const asnPath = path.join(geoipDir, "GeoLite2-ASN.mmdb")
writeFileSync(countryPath, "old-country")
setGeoipFetchForTests(async () =>
new Response(new Uint8Array([1, 2, 3]), { status: 200, headers: { etag: '"v1"' } }))
setGeoipValidateForTests({
country: async (p) => {
assert.ok(p.endsWith(".tmp"), "валидация должна идти по tmp-файлу")
return "2026-09-08T00:00:00.000Z"
},
asn: async () => "2026-09-08T00:00:00.000Z",
})
snap = await collectGeoipUpdateOnce({ force: true })
assert.equal(snap.downloaded, 2)
assert.equal(snap.errors.length, 0)
assert.deepEqual(readFileSync(countryPath), Buffer.from([1, 2, 3]))
assert.equal(readFileSync(`${countryPath}.prev`, "utf8"), "old-country")
assert.equal(existsSync(`${asnPath}.prev`), false, "prev у asn не бывает при первой загрузке")
assert.equal(existsSync(`${countryPath}.tmp`), false)
// ── коллектор: битая база → подмены нет, старый файл цел, tmp удалён ─────────
writeFileSync(asnPath, "good-asn")
setGeoipFetchForTests(async () =>
new Response(new Uint8Array([9, 9]), { status: 200 }))
setGeoipValidateForTests({
country: async () => {
throw new Error("битая база")
},
asn: async () => {
throw new Error("битая база")
},
})
snap = await collectGeoipUpdateOnce({ force: true })
assert.equal(snap.downloaded, 0)
assert.equal(snap.errors.length, 2)
assert.deepEqual(readFileSync(countryPath), Buffer.from([1, 2, 3]), "country не тронута")
assert.equal(readFileSync(asnPath, "utf8"), "good-asn", "asn не тронут")
assert.equal(existsSync(`${countryPath}.tmp`), false)
assert.equal(existsSync(`${asnPath}.tmp`), false)
rmSync(geoipDir, { recursive: true, force: true })
delete process.env.GEOIP_DIR
resetGeoipUpdateForTests()
resetGeoipForTests()
console.log("traffic-flow-geoip.test.ts: ok")
+162
View File
@@ -0,0 +1,162 @@
import { existsSync } from "node:fs"
import path from "node:path"
import { open, type AsnResponse, type CountryResponse, type Reader } from "maxmind"
import { isNonPublicIp } from "./traffic-flow-ip.js"
import { isIsoCountry, resolveRipeCountry } from "./traffic-flow-brands.js"
import { lookupRipeCached, type FlowIpMeta } from "./traffic-flow-ripe.js"
export const GEOIP_COUNTRY_FILE = "GeoLite2-Country.mmdb"
export const GEOIP_ASN_FILE = "GeoLite2-ASN.mmdb"
/** Каталог баз: `storage/geoip` рядом со storage/backups; переопределяется GEOIP_DIR. */
export function geoipDir(): string {
return path.resolve(process.env.GEOIP_DIR ?? path.join(process.cwd(), "storage", "geoip"))
}
export function geoipCountryPath(): string {
return path.join(geoipDir(), GEOIP_COUNTRY_FILE)
}
export function geoipAsnPath(): string {
return path.join(geoipDir(), GEOIP_ASN_FILE)
}
export interface GeoipReaders {
country: Reader<CountryResponse> | null
asn: Reader<AsnResponse> | null
}
let readers: GeoipReaders = { country: null, asn: null }
let initPromise: Promise<GeoipReaders> | null = null
/** Открывает оба файла best-effort: отсутствующий/битый файл не мешает второму. */
export async function openGeoipReaders(dir = geoipDir()): Promise<GeoipReaders> {
const next: GeoipReaders = { country: null, asn: null }
if (existsSync(path.join(dir, GEOIP_COUNTRY_FILE))) {
try {
next.country = await open<CountryResponse>(path.join(dir, GEOIP_COUNTRY_FILE))
} catch {
/* битый файл — работаем без country */
}
}
if (existsSync(path.join(dir, GEOIP_ASN_FILE))) {
try {
next.asn = await open<AsnResponse>(path.join(dir, GEOIP_ASN_FILE))
} catch {
/* битый файл — работаем без ASN */
}
}
return next
}
/** Открывает ридеры при старте; файлы есть — работают, нет — lookup уходит в RIPE-fallback. */
export async function initGeoip(): Promise<GeoipReaders> {
if (!initPromise) {
initPromise = openGeoipReaders().then((next) => {
readers = next
return next
})
}
return initPromise
}
/** Переоткрывает ридеры после обновления файлов (атомарная замена ссылок). */
export async function reloadGeoipReaders(): Promise<GeoipReaders> {
const next = await openGeoipReaders()
readers = next
initPromise = Promise.resolve(next)
return next
}
export function setGeoipReadersForTests(next: Partial<GeoipReaders>): void {
readers = { country: next.country ?? null, asn: next.asn ?? null }
}
export function resetGeoipForTests(): void {
readers = { country: null, asn: null }
initPromise = null
}
function negativeMeta(ip: string): FlowIpMeta {
const v6 = ip.includes(":")
return {
prefix: `${ip}/${v6 ? 128 : 32}`,
asn: 0,
country: "—",
lat: null,
lng: null,
holder: "",
ok: false,
fetchedAt: Date.now(),
}
}
function safeCountryIso(reader: Reader<CountryResponse>, ip: string): string {
try {
const rec = reader.get(ip)
return rec?.country?.iso_code ?? rec?.registered_country?.iso_code ?? ""
} catch {
return ""
}
}
function safeAsn(reader: Reader<AsnResponse>, ip: string): { asn: number; holder: string } {
try {
const rec = reader.get(ip)
return {
asn: rec?.autonomous_system_number ?? 0,
holder: rec?.autonomous_system_organization ?? "",
}
} catch {
return { asn: 0, holder: "" }
}
}
/**
* Синхронный lookup по локальным GeoLite2. Возвращает FlowIpMeta в семантике RIPE-кэша
* (ok=true когда есть страна или ASN; null данных нет, пусть пробует RIPE).
*/
export function lookupGeoip(ip: string): FlowIpMeta | null {
const trimmed = String(ip ?? "").trim()
if (!trimmed) return null
if (isNonPublicIp(trimmed)) return negativeMeta(trimmed)
const { country: countryReader, asn: asnReader } = readers
if (!countryReader && !asnReader) return null
const iso = countryReader ? safeCountryIso(countryReader, trimmed) : ""
const country = iso && isIsoCountry(iso) ? iso : ""
const { asn, holder } = asnReader ? safeAsn(asnReader, trimmed) : { asn: 0, holder: "" }
if (!asn && !country) return null
return {
prefix: `${trimmed}/${trimmed.includes(":") ? 128 : 32}`,
asn,
country: resolveRipeCountry(country, asn, holder) || "—",
lat: null,
lng: null,
holder,
ok: true,
fetchedAt: Date.now(),
}
}
/** Главный вход для потребителей пайплайна: локальные базы первыми, RIPE-кэш fallback. */
export function resolveFlowIp(ip: string): FlowIpMeta | null {
return lookupGeoip(ip) ?? lookupRipeCached(ip)
}
export interface GeoipReadersStatus {
countryLoaded: boolean
asnLoaded: boolean
countryBuildAt: string | null
asnBuildAt: string | null
dir: string
}
export function geoipReadersStatus(): GeoipReadersStatus {
return {
countryLoaded: Boolean(readers.country),
asnLoaded: Boolean(readers.asn),
countryBuildAt: readers.country?.metadata.buildEpoch.toISOString() ?? null,
asnBuildAt: readers.asn?.metadata.buildEpoch.toISOString() ?? null,
dir: geoipDir(),
}
}
@@ -22,8 +22,9 @@ rememberServerIfaces(7, [
{ ".id": "*A", name: "wg-flow" },
{ ".id": "*D", name: "bridge" },
])
assert.equal(resolveIfaceName(7, "2").name, "ether1")
assert.equal(resolveIfaceName(7, "10").name, "wg-flow")
assert.equal(resolveIfaceName(7, "2").name, "ether1")
assert.equal(resolveIfaceName(7, "#2").name, "ether1")
assert.equal(resolveIfaceName(7, "10").name, "wg-flow")
assert.equal(resolveIfaceName(7, "13").name, "bridge")
assert.equal(resolveIfaceName(7, "0").name, "—")
assert.equal(resolveIfaceName(7, "ether1").name, "ether1")
@@ -0,0 +1,58 @@
import assert from "node:assert/strict"
import {
bindingIfaceAliases,
bindingIfaceAliasesAllServers,
canonicalFactIface,
collapseServerIfaceRows,
displayFactIface,
expandBindingIfaces,
factIfaceAliases,
rememberServerIfaces,
resetIfaceCacheForTests,
resolveIfaceName,
} from "./traffic-flow-ifindex.js"
resetIfaceCacheForTests()
assert.equal(canonicalFactIface(1, "2"), "2")
assert.deepEqual(bindingIfaceAliases(1, "gre-client"), ["gre-client"])
rememberServerIfaces(1, [{ name: "gre-client", ifindex: "2" }])
assert.equal(canonicalFactIface(1, "2"), "gre-client")
assert.equal(canonicalFactIface(1, "gre-client"), "gre-client")
assert.equal(canonicalFactIface(1, "9"), "9")
assert.equal(resolveIfaceName(1, "9").name, "#9")
assert.equal(resolveIfaceName(1, "2").name, "gre-client")
assert.equal(resolveIfaceName(1, "#2").name, "gre-client")
assert.equal(displayFactIface(1, "2"), "gre-client")
const aliases = bindingIfaceAliases(1, "gre-client")
assert.ok(aliases.includes("gre-client"))
assert.ok(aliases.includes("2"))
assert.ok(aliases.includes("#2"))
const fromIndex = factIfaceAliases("2", 1)
assert.ok(fromIndex.includes("gre-client"))
assert.ok(fromIndex.includes("2"))
assert.ok(fromIndex.includes("#2"))
const all = bindingIfaceAliasesAllServers("gre-client")
assert.ok(all.includes("2"))
const expanded = expandBindingIfaces([{ serverId: 1, iface: "gre-client" }])
assert.ok(expanded.some((x) => x.iface === "2"))
assert.ok(expanded.some((x) => x.iface === "gre-client"))
const collapsed = collapseServerIfaceRows([
{ serverId: 1, iface: "2", bytes: 10, packets: 1 },
{ serverId: 1, iface: "gre-client", bytes: 5, packets: 2 },
{ serverId: 1, iface: "wan1", bytes: 3, packets: 1 },
])
assert.equal(collapsed.length, 2)
const gre = collapsed.find((r) => r.iface === "gre-client")
assert.ok(gre)
assert.equal(gre.bytes, 15)
assert.equal(gre.packets, 3)
assert.ok(collapsed.some((r) => r.iface === "wan1"))
resetIfaceCacheForTests()
console.log("traffic-flow-ifindex.test.ts: ok")
+129 -6
View File
@@ -36,12 +36,135 @@ export function rememberServerIfaces(serverId: number, rows: RosIfaceIndexRow[])
export function resolveIfaceName(serverId: number, indexOrName: string): { name: string; index: string } {
const trimmed = String(indexOrName ?? "").trim()
if (!trimmed || trimmed === "0") return { name: "—", index: trimmed }
if (!/^\d+$/.test(trimmed)) return { name: trimmed, index: "" }
const idx = Number(trimmed)
const name = cache.get(serverId)?.get(idx)
if (name) return { name, index: trimmed }
return { name: `#${trimmed}`, index: trimmed }
const asIndex = trimmed.startsWith("#") && /^\d+$/.test(trimmed.slice(1)) ? trimmed.slice(1) : trimmed
if (!asIndex || asIndex === "0") return { name: "—", index: asIndex }
if (!/^\d+$/.test(asIndex)) return { name: trimmed, index: "" }
const name = cache.get(serverId)?.get(Number(asIndex))
if (name) return { name, index: asIndex }
return { name: `#${asIndex}`, index: asIndex }
}
/** Имя iface для факта куба: ifIndex→имя, без `#13` при пустом кэше. */
export function canonicalFactIface(serverId: number, inIface: string): string {
const trimmed = String(inIface ?? "").trim()
if (!trimmed) return trimmed
if (!/^\d+$/.test(trimmed)) return trimmed
const name = cache.get(serverId)?.get(Number(trimmed))
return name || trimmed
}
function numericIfaceIndex(iface: string): string | null {
const raw = String(iface ?? "").trim()
if (/^\d+$/.test(raw)) return raw
if (raw.startsWith("#") && /^\d+$/.test(raw.slice(1))) return raw.slice(1)
return null
}
/** Имя для UI: ifIndex → RouterOS name; `0` → «—»; miss → `#n`. */
export function displayFactIface(serverId: number, iface: string): string {
return resolveIfaceName(serverId, iface).name
}
/** Склеить факты `2` + `ether1` в одну строку после резолва ifIndex. */
export function collapseServerIfaceRows(
rows: Array<{ serverId: number; iface: string; bytes: number; packets: number }>,
): Array<{ serverId: number; iface: string; bytes: number; packets: number }> {
const acc = new Map<string, { serverId: number; iface: string; bytes: number; packets: number }>()
for (const r of rows) {
const name = displayFactIface(r.serverId, r.iface)
const k = `${r.serverId}\0${name}`
const prev = acc.get(k)
const bytes = Number(r.bytes) || 0
const packets = Number(r.packets) || 0
if (prev) {
prev.bytes += bytes
prev.packets += packets
} else {
acc.set(k, { serverId: r.serverId, iface: name, bytes, packets })
}
}
return [...acc.values()]
}
/** Ключи факта для фильтра: имя, ifIndex и `#n`. */
export function factIfaceAliases(iface: string, serverId?: number): string[] {
const raw = String(iface ?? "").trim()
if (!raw) return []
const out = new Set<string>([raw])
const idx = numericIfaceIndex(raw)
if (idx) {
out.add(idx)
out.add(`#${idx}`)
const n = Number(idx)
if (serverId != null) {
const name = cache.get(serverId)?.get(n)
if (name) out.add(name)
} else {
for (const map of cache.values()) {
const name = map.get(n)
if (name) out.add(name)
}
}
}
if (serverId != null) {
for (const a of bindingIfaceAliases(serverId, raw)) out.add(a)
} else {
for (const a of bindingIfaceAliasesAllServers(raw)) out.add(a)
}
return [...out]
}
/** Имя + ifIndex + `#n` — тот же матч, что карта `/traffic`. */
export function bindingIfaceAliases(serverId: number, interfaceName: string): string[] {
const name = String(interfaceName ?? "").trim()
if (!name) return []
const out = new Set<string>([name])
const map = cache.get(serverId)
const idx = numericIfaceIndex(name)
const canonical = (idx && map?.get(Number(idx))) || name
out.add(canonical)
if (idx) {
out.add(idx)
out.add(`#${idx}`)
}
if (!map) return [...out]
for (const [i, n] of map) {
if (n !== canonical && n !== name) continue
out.add(String(i))
out.add(`#${i}`)
}
return [...out]
}
export function bindingIfaceAliasesAllServers(interfaceName: string): string[] {
const name = String(interfaceName ?? "").trim()
const out = new Set<string>(name ? [name] : [])
for (const serverId of cache.keys()) {
for (const alias of bindingIfaceAliases(serverId, name)) out.add(alias)
}
return [...out]
}
export function expandBindingIfaces(
binds: Array<{ serverId: number; iface: string }>,
): Array<{ serverId: number; iface: string }> {
const seen = new Set<string>()
const out: Array<{ serverId: number; iface: string }> = []
for (const b of binds) {
for (const iface of bindingIfaceAliases(b.serverId, b.iface)) {
const k = `${b.serverId}\0${iface}`
if (seen.has(k)) continue
seen.add(k)
out.push({ serverId: b.serverId, iface })
}
}
return out
}
export function listCachedIfaceNames(serverId: number): string[] {
const map = cache.get(serverId)
if (!map) return []
return [...new Set(map.values())]
}
export function ifaceCacheHas(serverId: number): boolean {
@@ -467,11 +467,15 @@ export async function purgeTrafficFlowStore(): Promise<FlowPurgeDto> {
minuteStats: await tableCount("flow_minute_stats"),
minuteDims: await tableCount("flow_minute_dims"),
dailyDims: await tableCount("flow_daily_dims"),
hourFacts: await tableCount("flow_hour_facts"),
dailyFacts: await tableCount("flow_daily_facts"),
}
await dbQuery(`DELETE FROM flow_buckets`)
await dbQuery(`DELETE FROM flow_minute_stats`)
await dbQuery(`DELETE FROM flow_minute_dims`)
await dbQuery(`DELETE FROM flow_daily_dims`)
await dbQuery(`DELETE FROM flow_hour_facts`)
await dbQuery(`DELETE FROM flow_daily_facts`)
await resetFlowIngestCounters()
await dropExpiredPartitions(pool)
try {
@@ -4,7 +4,13 @@ import {
ingestParsedFlowsForServerForTests,
resetFlowRingsForTests,
} from "./traffic-flow-ingest.js"
import { buildFlowMapHops, resetFlowMapHopsCacheForTests } from "./traffic-flow-map-hops.js"
import {
buildFlowMapHops,
MAP_SERVICE_MIN_NODES,
MAP_SERVICE_NODE_CAP,
pickMapServices,
resetFlowMapHopsCacheForTests,
} from "./traffic-flow-map-hops.js"
import { withPgOrSkip } from "../test/pg.js"
import { seedFlowTopologyForTests, type FlowTopology } from "./traffic-flow-topology.js"
import { disableCatalogFetchForTests, resetFlowCatalogForTests } from "./traffic-flow-classify.js"
@@ -15,6 +21,55 @@ import {
seedRipeCacheForTests,
} from "./traffic-flow-ripe.js"
{
const googleOnly = pickMapServices(
[{ id: "svc:google", label: "Google", category: "Веб", bytes: 400, bps: 0, share: 1 }],
5,
)
assert.equal(googleOnly.length, 1)
assert.equal(googleOnly[0]?.share, 1)
const twoNamed = pickMapServices(
[
{ id: "svc:google", label: "Google", category: "Веб", bytes: 400, bps: 0, share: 0.5 },
{ id: "svc:cloudflare", label: "Cloudflare", category: "CDN", bytes: 400, bps: 0, share: 0.5 },
],
5,
)
assert.equal(twoNamed.length, 2)
const tinyTail = pickMapServices(
[
{ id: "svc:google", label: "Google", category: "Веб", bytes: 9000, bps: 0, share: 0.9 },
...Array.from({ length: 9 }, (_, i) => ({
id: `svc:t${i}`,
label: `T${i}`,
category: "Веб",
bytes: 100,
bps: 0,
share: 0.01,
})),
],
5,
)
assert.equal(tinyTail.length, MAP_SERVICE_MIN_NODES)
assert.equal(tinyTail.at(-1)?.id, "svc:t6")
const allOff = pickMapServices(
Array.from({ length: 25 }, (_, i) => ({
id: `svc:n${i}`,
label: `N${i}`,
category: "Веб",
bytes: 25 - i,
bps: 0,
share: 0.04,
})),
0,
)
assert.equal(allOff.length, MAP_SERVICE_NODE_CAP)
console.log("traffic-flow-map-hops.test.ts: pickMapServices ok")
}
if (!(await withPgOrSkip())) {
console.log("traffic-flow-map-hops.test.ts: skip")
process.exit(0)
@@ -178,6 +233,19 @@ function googleRipe() {
})
}
function seedRipeAsn(ip: string, asn: number, holder: string) {
seedRipeCacheForTests({
prefix: `${ip}/32`,
asn,
country: "US",
lat: 37.4,
lng: -122.1,
holder,
ok: true,
fetchedAt: Date.now(),
})
}
function payloadFlow(dst: string, bytes: number) {
return {
src: "10.100.1.17",
@@ -241,10 +309,106 @@ try {
resetFlowMapHopsCacheForTests()
const four = await buildFlowMapHops({ minutes: 5, minSharePct: 5 })
assert.equal(four.totalBytes, 10_000)
assert.ok(!(four.services ?? []).some((s) => s.id === "svc:google"), "Google < 5% hidden")
const googleFour = four.services?.find((s) => s.id === "svc:google")
assert.ok(googleFour, "единственный бренд виден при 4% от окна")
assert.ok(googleFour.share >= 0.99, "доля среди брендов ≈ 1")
resetFlowMapHopsCacheForTests()
const off = await buildFlowMapHops({ minutes: 5, minSharePct: 0 })
assert.ok(off.services?.some((s) => s.id === "svc:google"), "порог 0 показывает Google 4%")
assert.ok(off.services?.some((s) => s.id === "svc:google"), "порог 0 показывает Google")
} finally {
resetFlowRingsForTests()
resetIfaceCacheForTests()
resetRipeCacheForTests()
}
resetFlowRingsForTests()
resetIfaceCacheForTests()
resetRipeCacheForTests()
disableRipeEnqueueForTests()
seedFlowTopologyForTests(topo)
rememberServerIfaces(7, [
{ ".id": "*2", name: "gre-client" },
{ ".id": "*3", name: "gre-jh-en" },
])
ingestParsedFlowsForServerForTests(7, [
payloadFlow("8.8.8.8", 400),
payloadFlow("104.18.35.51", 400),
payloadFlow("203.0.113.50", 9200),
])
try {
resetFlowMapHopsCacheForTests()
const two = await buildFlowMapHops({ minutes: 5, minSharePct: 5 })
const googleTwo = two.services?.find((s) => s.id === "svc:google")
const cfTwo = two.services?.find((s) => s.id === "svc:cloudflare")
assert.ok(googleTwo, "Google среди брендов")
assert.ok(cfTwo, "Cloudflare среди брендов")
assert.ok(googleTwo.share >= 0.05)
assert.ok(cfTwo.share >= 0.05)
} finally {
resetFlowRingsForTests()
resetIfaceCacheForTests()
resetRipeCacheForTests()
}
resetFlowRingsForTests()
resetIfaceCacheForTests()
resetRipeCacheForTests()
disableRipeEnqueueForTests()
seedFlowTopologyForTests(topo)
rememberServerIfaces(7, [
{ ".id": "*2", name: "gre-client" },
{ ".id": "*3", name: "gre-jh-en" },
])
seedRipeAsn("162.254.192.71", 32590, "VALVE-CORP")
ingestParsedFlowsForServerForTests(7, [
payloadFlow("162.254.192.71", 2000),
])
try {
resetFlowMapHopsCacheForTests()
const steam = await buildFlowMapHops({ minutes: 5, minSharePct: 5 })
assert.ok(steam.services?.some((s) => s.id === "svc:steam"), "Steam AS32590 на карте")
} finally {
resetFlowRingsForTests()
resetIfaceCacheForTests()
resetRipeCacheForTests()
}
const smallBrands: Array<{ ip: string; asn: number; holder: string; bytes: number; id: string }> = [
{ ip: "203.0.113.1", asn: 714, holder: "APPLE-ENGINEERING", bytes: 400, id: "svc:apple" },
{ ip: "203.0.113.2", asn: 36459, holder: "GITHUB", bytes: 390, id: "svc:github" },
{ ip: "203.0.113.3", asn: 54876, holder: "GITLAB", bytes: 380, id: "svc:gitlab" },
{ ip: "203.0.113.4", asn: 8403, holder: "SPOTIFY", bytes: 370, id: "svc:spotify" },
{ ip: "203.0.113.5", asn: 13414, holder: "TWITTER", bytes: 360, id: "svc:x" },
{ ip: "203.0.113.6", asn: 47541, holder: "VKONTAKTE", bytes: 350, id: "svc:vk" },
{ ip: "203.0.113.7", asn: 30103, holder: "ZOOM", bytes: 340, id: "svc:zoom" },
{ ip: "203.0.113.8", asn: 395701, holder: "EPIC-GAMES", bytes: 330, id: "svc:epic" },
{ ip: "203.0.113.9", asn: 6507, holder: "RIOT-GAMES", bytes: 320, id: "svc:riot" },
]
resetFlowRingsForTests()
resetIfaceCacheForTests()
resetRipeCacheForTests()
disableRipeEnqueueForTests()
seedFlowTopologyForTests(topo)
rememberServerIfaces(7, [
{ ".id": "*2", name: "gre-client" },
{ ".id": "*3", name: "gre-jh-en" },
])
googleRipe()
for (const b of smallBrands) seedRipeAsn(b.ip, b.asn, b.holder)
ingestParsedFlowsForServerForTests(7, [
payloadFlow("8.8.8.8", 5000),
...smallBrands.map((b) => payloadFlow(b.ip, b.bytes)),
])
try {
resetFlowMapHopsCacheForTests()
const top = await buildFlowMapHops({ minutes: 5, minSharePct: 5 })
const ids = new Set((top.services ?? []).map((s) => s.id))
assert.equal(top.services?.length, MAP_SERVICE_MIN_NODES, "топ-8 брендов на карте")
assert.ok(ids.has("svc:google"))
for (const b of smallBrands.slice(0, 7)) assert.ok(ids.has(b.id), b.id)
assert.ok(!ids.has("svc:epic"), "хвост ниже ранга 8 скрыт")
assert.ok(!ids.has("svc:riot"))
} finally {
resetFlowRingsForTests()
resetIfaceCacheForTests()
@@ -420,4 +584,98 @@ try {
resetFlowCatalogForTests()
}
resetFlowRingsForTests()
resetIfaceCacheForTests()
resetRipeCacheForTests()
disableRipeEnqueueForTests()
seedFlowTopologyForTests(topo)
rememberServerIfaces(7, [
{ ".id": "*2", name: "gre-client" },
{ ".id": "*3", name: "gre-jh-en" },
])
rememberServerIfaces(9, [
{ ".id": "*1", name: "ether1" },
])
googleRipe()
ingestParsedFlowsForServerForTests(7, [payloadFlow("8.8.8.8", 12_000)])
ingestParsedFlowsForServerForTests(9, [{
src: "10.100.1.17",
dst: "8.8.8.8",
proto: 6,
srcPort: 51234,
dstPort: 443,
bytes: 12_000,
packets: 10,
inIface: "1",
outIface: "1",
nextHop: "",
}])
try {
resetFlowMapHopsCacheForTests()
const dual = await buildFlowMapHops({ minutes: 5, minSharePct: 0 })
const gre = dual.hops.find((h) => h.kind === "gre" && h.fromId === "7" && h.toId === "9")
assert.ok(gre, "GRE JH→EN сохранён")
assert.equal(gre.bytes, 12_000)
const googlePaths = (dual.servicePaths ?? []).filter((p) => p.serviceId === "svc:google")
assert.equal(googlePaths.length, 1, "один путь без копии EN")
assert.equal(googlePaths[0]?.clientId, "u1")
assert.equal(googlePaths[0]?.viaId, "7")
const googleEdge = dual.serviceEdges?.find((e) => e.toId === "svc:google" && e.fromId === "9")
assert.ok(googleEdge)
assert.equal(googleEdge.bytes, 12_000)
assert.equal(googleEdge.bps, googlePaths[0]?.bps)
} finally {
seedFlowTopologyForTests(null)
resetFlowRingsForTests()
resetIfaceCacheForTests()
resetRipeCacheForTests()
resetFlowCatalogForTests()
}
resetFlowRingsForTests()
resetIfaceCacheForTests()
resetRipeCacheForTests()
disableRipeEnqueueForTests()
seedFlowTopologyForTests(topo)
rememberServerIfaces(7, [
{ ".id": "*2", name: "gre-client" },
{ ".id": "*3", name: "gre-jh-en" },
])
googleRipe()
ingestParsedFlowsForServerForTests(7, [
payloadFlow("8.8.8.8", 8_000),
{
src: "8.8.8.8",
dst: "10.100.1.17",
proto: 6,
srcPort: 443,
dstPort: 51234,
bytes: 4_000,
packets: 8,
inIface: "3",
outIface: "2",
nextHop: "",
},
])
try {
resetFlowMapHopsCacheForTests()
const bothDir = await buildFlowMapHops({ minutes: 5, minSharePct: 0 })
const gre = bothDir.hops.find((h) => h.kind === "gre" && h.fromId === "7" && h.toId === "9")
assert.ok(gre, "GRE-hop при fwd/rev")
const googlePaths = (bothDir.servicePaths ?? []).filter((p) => p.serviceId === "svc:google")
assert.equal(googlePaths.length, 1, "fwd+rev — один клиент")
assert.equal(googlePaths[0]?.clientId, "u1")
assert.ok(!(bothDir.servicePaths ?? []).some((p) => p.serviceId === "svc:google" && p.clientId === "—"))
const googleEdge = bothDir.serviceEdges?.find((e) => e.toId === "svc:google" && e.fromId === "9")
assert.ok(googleEdge)
assert.equal(googleEdge.bytes, 12_000)
assert.equal(googleEdge.bps, googlePaths[0]?.bps)
} finally {
seedFlowTopologyForTests(null)
resetFlowRingsForTests()
resetIfaceCacheForTests()
resetRipeCacheForTests()
resetFlowCatalogForTests()
}
console.log("traffic-flow-map-hops.test.ts: ok")
+66 -21
View File
@@ -8,18 +8,21 @@ import {
mapServiceNodeId,
resolveFlowBrand,
} from "./traffic-flow-brands.js"
import { dedupFlowRowsMaxBytes } from "./traffic-flow-dedup.js"
import { dedupFlowRowsAcrossExporters, dedupFlowRowsMaxBytes } from "./traffic-flow-dedup.js"
import { getFlowListenerState, listFlowRowsForWindow } from "./traffic-flow-ingest.js"
import { resolveIfaceName } from "./traffic-flow-ifaces.js"
import { classifyFlowPlane, shouldKeepPlane } from "./traffic-flow-planes.js"
import { pickInternetPeer } from "./traffic-flow-ip.js"
import { lookupRipeCached, type FlowIpMeta } from "./traffic-flow-ripe.js"
import { type FlowIpMeta } from "./traffic-flow-ripe.js"
import { resolveFlowIp } from "./traffic-flow-geoip.js"
import { getTrafficFlowSettingsRow } from "./traffic-flow-settings.js"
import { loadFlowTopology, resolveClient, resolveEn, getServerCatalog } from "./traffic-flow-topology.js"
import { loadFlowTopology, resolveClient, resolveEn, getServerCatalog, type FlowTopology } from "./traffic-flow-topology.js"
import { flowDataEpoch } from "./traffic-flow-engine.js"
export const DEFAULT_MAP_SERVICE_MIN_SHARE_PCT = 5
export const MAP_SERVICE_NODE_CAP = 20
/** Минимум узлов-брендов на карте, даже если доля ниже порога. */
export const MAP_SERVICE_MIN_NODES = 8
const HOPS_CACHE_TTL_MS = 2000
export interface FlowMapHopsQuery {
@@ -99,6 +102,15 @@ export function clampMapServiceMinSharePct(n: unknown): number {
return Math.min(100, Math.max(0, v))
}
/** Доля среди именованных брендов; порог ИЛИ топ-N, затем cap. */
export function pickMapServices(ranked: FlowMapService[], minSharePct: number): FlowMapService[] {
if (minSharePct <= 0) return ranked.slice(0, MAP_SERVICE_NODE_CAP)
const minShare = minSharePct / 100
return ranked
.filter((s, i) => s.share >= minShare || i < MAP_SERVICE_MIN_NODES)
.slice(0, MAP_SERVICE_NODE_CAP)
}
function hopsQueryKey(q: FlowMapHopsQuery, minSharePct: number): string {
return JSON.stringify({
epoch: flowDataEpoch(),
@@ -129,6 +141,16 @@ function ifaceUsable(name: string): boolean {
return Boolean(name) && name !== "—"
}
function resolveMapClient(
topo: FlowTopology,
serverId: number,
inName: string,
outName: string,
) {
return resolveClient(topo, serverId, inName)
?? (ifaceUsable(outName) ? resolveClient(topo, serverId, outName) : null)
}
function bump(acc: Map<string, HopAcc>, key: string, seed: Omit<HopAcc, "bytes" | "bytesFwd" | "bytesRev">, bytes: number, dir: "fwd" | "rev" | "both"): void {
const prev = acc.get(key)
const addFwd = dir === "fwd" || dir === "both" ? bytes : 0
@@ -226,6 +248,21 @@ async function buildFlowMapHopsUncached(q: FlowMapHopsQuery, minSharePct: number
const enIds = new Set(topo.enNodes.map((n) => n.id))
let totalBytes = 0
function rowClient(r: (typeof working)[number]) {
const inName = resolveIfaceName(r.serverId, r.inIface).name
const outName = resolveIfaceName(r.serverId, r.outIface).name
return resolveMapClient(topo, r.serverId, inName, outName)
}
const payloadRows = wantDedup
? dedupFlowRowsAcrossExporters(working, (a, b) => {
const aCli = Boolean(rowClient(a))
const bCli = Boolean(rowClient(b))
if (aCli !== bCli) return aCli ? a : b
return a.bytes >= b.bytes ? a : b
})
: working
for (const r of working) {
const inRes = resolveIfaceName(r.serverId, r.inIface)
const outRes = resolveIfaceName(r.serverId, r.outIface)
@@ -311,10 +348,14 @@ async function buildFlowMapHopsUncached(q: FlowMapHopsQuery, minSharePct: number
}, r.bytes, "fwd")
}
}
}
for (const r of payloadRows) {
const inName = resolveIfaceName(r.serverId, r.inIface).name
const outName = resolveIfaceName(r.serverId, r.outIface).name
totalBytes += r.bytes
const peer = pickInternetPeer(r.src, r.dst, r.srcPort, r.dstPort)
const client = resolveClient(topo, r.serverId, inName)
const client = resolveMapClient(topo, r.serverId, inName, outName)
const prevDst = dstAcc.get(peer)
if (prevDst) {
prevDst.bytes += r.bytes
@@ -383,7 +424,7 @@ async function buildFlowMapHopsUncached(q: FlowMapHopsQuery, minSharePct: number
}
for (const [dst, acc] of dstAcc) {
const ripe = lookupRipeCached(dst)
const ripe = resolveFlowIp(dst)
const classified = classifyMapDstLite(dst, acc.proto, acc.dstPort, acc.srcPort, ripe)
if (!classified) continue
const toId = mapServiceNodeId(classified.service)
@@ -416,10 +457,15 @@ async function buildFlowMapHopsUncached(q: FlowMapHopsQuery, minSharePct: number
const enName = nodeName(fromId)
const viaName = nodeName(exporterId)
for (const [clientId, c] of from.clients) {
const pathKey = `${clientId}|${exporterId}|${fromId}|${toId}`
const pathKey = `${clientId}|${fromId}|${toId}`
const prevPath = svcPaths.get(pathKey)
if (prevPath) {
prevPath.bytes += c.bytes
if (exporterId !== fromId && prevPath.viaId === fromId) {
prevPath.viaId = exporterId
prevPath.viaName = viaName
}
if (prevPath.clientName === "—" && c.name !== "—") prevPath.clientName = c.name
} else {
svcPaths.set(pathKey, {
clientId,
@@ -436,21 +482,20 @@ async function buildFlowMapHopsUncached(q: FlowMapHopsQuery, minSharePct: number
}
}
const minShare = minSharePct / 100
let services: FlowMapService[] = [...svcTotals.entries()]
.map(([id, s]) => ({
id,
label: s.label,
category: s.category,
bytes: s.bytes,
bps: (s.bytes * 8) / windowSec,
share: totalBytes > 0 ? s.bytes / totalBytes : 0,
}))
.sort((a, b) => b.bytes - a.bytes)
if (minSharePct > 0) {
services = services.filter((s) => s.share >= minShare)
}
services = services.slice(0, MAP_SERVICE_NODE_CAP)
const namedBytes = [...svcTotals.values()].reduce((n, s) => n + s.bytes, 0)
const services = pickMapServices(
[...svcTotals.entries()]
.map(([id, s]) => ({
id,
label: s.label,
category: s.category,
bytes: s.bytes,
bps: (s.bytes * 8) / windowSec,
share: namedBytes > 0 ? s.bytes / namedBytes : 0,
}))
.sort((a, b) => b.bytes - a.bytes),
minSharePct,
)
const keepSvc = new Set(services.map((s) => s.id))
const serviceEdges: FlowMapServiceEdge[] = [...svcEdges.values()]
.filter((e) => keepSvc.has(e.toId))
@@ -48,6 +48,15 @@ export function invalidateFlowCatalogCache(): void {
serverCatalogCache = null
}
export function peekFlowTopology(): FlowTopology | null {
if (seeded) return seeded
return topologyCache?.topo ?? null
}
export function peekServerCatalog(): { list: ServerCatalogEntry[]; byId: Map<number, ServerCatalogEntry> } | null {
return serverCatalogCache
}
export async function getServerCatalog(): Promise<{ list: ServerCatalogEntry[]; byId: Map<number, ServerCatalogEntry> }> {
const now = Date.now()
if (serverCatalogCache && now - serverCatalogCache.at < CATALOG_TTL_MS) {
@@ -226,6 +226,19 @@ export interface BackupsRunSnapshot {
fatalError?: string
}
export interface GeoipUpdateRunSnapshot {
v: typeof SCHEDULER_RUN_SNAPSHOT_VERSION
job: "geoip_update"
sampledAt: string
skipped?: boolean
fatalError?: string
checked: number
downloaded: number
skippedUnchanged: number
bytes: number
errors: string[]
}
export type SchedulerRunSnapshot =
| TrafficRunSnapshot
| ResourcesRunSnapshot
@@ -236,4 +249,5 @@ export type SchedulerRunSnapshot =
| InternetPathRunSnapshot
| CertificatesRenewRunSnapshot
| BackupsRunSnapshot
| GeoipUpdateRunSnapshot
| AlertEngineRunSnapshot
+2
View File
@@ -13,6 +13,7 @@ import {
import {
LayoutDashboardIcon,
ActivityIcon,
ChartColumnIcon,
MapIcon,
HeartPulseIcon,
GlobeIcon,
@@ -54,6 +55,7 @@ const navStructure: { label: string; items: NavItemBase[] }[] = [
items: [
{ title: "Дашборд", url: "/dashboard", icon: <LayoutDashboardIcon /> },
{ title: "Трафик", url: "/traffic", icon: <ActivityIcon /> },
{ title: "Статистика", url: "/statistics", icon: <ChartColumnIcon /> },
{ title: "Карта сети", url: "/network-map", icon: <MapIcon /> },
{ title: "Мониторинг", url: "/uptime", icon: <HeartPulseIcon /> },
],
+2 -1
View File
@@ -4,7 +4,7 @@ import { useEffect, useState, useRef, useMemo } from "react"
import { useRouter, usePathname } from "next/navigation"
import { cn } from "@/lib/utils"
import {
SearchIcon, LayoutDashboardIcon, ActivityIcon, MapIcon, HeartPulseIcon,
SearchIcon, LayoutDashboardIcon, ActivityIcon, ChartColumnIcon, MapIcon, HeartPulseIcon,
GlobeIcon, NetworkIcon, LayersIcon, TagIcon, ServerIcon, FilterIcon,
ShieldIcon, ShieldCheckIcon, CableIcon, BoxIcon, BadgeCheckIcon,
HardDriveIcon, RouteIcon, GitForkIcon, GitMergeIcon, ScanLineIcon,
@@ -27,6 +27,7 @@ const ALL_ITEMS: CommandItem[] = [
// Обзор
{ id: "dashboard", title: "Дашборд", group: "Обзор", url: "/dashboard", icon: <LayoutDashboardIcon />, keywords: ["главная","home","overview"] },
{ id: "traffic", title: "Трафик", group: "Обзор", url: "/traffic", icon: <ActivityIcon />, keywords: ["bandwidth","traffic","клиенты","интерфейсы"] },
{ id: "statistics", title: "Статистика", group: "Обзор", url: "/statistics", icon: <ChartColumnIcon />, keywords: ["stats","отчёт","куб","страны","asn","ipfix"] },
{ id: "network-map", title: "Карта сети", group: "Обзор", url: "/network-map", icon: <MapIcon />, keywords: ["topology","топология","map"] },
{ id: "uptime", title: "Мониторинг / Uptime", group: "Обзор", url: "/uptime", icon: <HeartPulseIcon />, keywords: ["ping","uptime","мониторинг","проверка"] },
// Данные
@@ -0,0 +1,89 @@
"use client"
import { CompactDataGrid, type CompactDataGridColumn } from "@/components/data-grids/compact-data-grid"
import { Flag } from "@/components/flag"
import { Badge } from "@/components/reui/badge"
import { fmtBps, formatBytes } from "@/lib/fmt-rate"
import { cn } from "@/lib/utils"
import { STATISTICS_UNBOUND_USER_ID, type StatisticsBreakdownRow } from "@mmapp/contracts/statistics"
export type StatisticsSliceKind = "users" | "servers" | "interfaces" | "countries" | "services" | "asns"
export function StatisticsBreakdownDataGrid({
rows,
kind,
selectedId,
onRowClick,
isLoading,
density = "full",
}: {
rows: StatisticsBreakdownRow[]
kind: StatisticsSliceKind
selectedId?: string
onRowClick?: (row: StatisticsBreakdownRow) => void
isLoading?: boolean
density?: "full" | "mini"
}) {
const mini = density === "mini"
const columns: CompactDataGridColumn<StatisticsBreakdownRow>[] = [
{
id: "label",
header: "Имя",
accessorKey: "label",
cell: (row) => (
<span className={cn("flex items-center gap-2", selectedId === row.id && "font-medium")}>
{kind === "countries" && row.id !== "XX" && row.id !== STATISTICS_UNBOUND_USER_ID ? (
<Flag code={row.id} size={16} />
) : null}
<span className="truncate">{row.label || row.id}</span>
{selectedId === row.id ? (
<Badge variant="outline" size="sm">
слайс
</Badge>
) : null}
</span>
),
},
{
id: "bytes",
header: "Байты",
accessorKey: "bytes",
cell: (row) => <span className="tabular-nums">{formatBytes(row.bytes)}</span>,
},
...(!mini
? [
{
id: "packets",
header: "Пакеты",
accessorKey: "packets" as const,
cell: (row: StatisticsBreakdownRow) => (
<span className="tabular-nums">{row.packets.toLocaleString("ru-RU")}</span>
),
},
{
id: "bps",
header: "Средний bitrate",
accessorKey: "bps" as const,
cell: (row: StatisticsBreakdownRow) => <span className="tabular-nums">{fmtBps(row.bps)}</span>,
},
]
: []),
{
id: "percent",
header: "Доля",
accessorKey: "percent",
cell: (row) => <span className="tabular-nums">{row.percent.toFixed(1)}%</span>,
},
]
return (
<CompactDataGrid
data={rows}
columns={columns}
isLoading={isLoading}
emptyTitle="Нет трафика"
emptyDescription="За выбранный период и слайсы нет данных куба."
onRowClick={onRowClick}
/>
)
}
@@ -125,6 +125,117 @@ export function ServiceBrandIcon({ label, size = 22 }: { label: string; size?: n
<path d="M13.4 4v9.1a3.3 3.3 0 1 1-2.8-3.3V7.2c1.6.9 3.2 1.4 5 1.5V5.4c-1.4-.1-2.7-.6-3.8-1.4H13.4Z" fill="#FE2C55" transform="translate(1.2 1)" />
</BrandSvg>
)
case "apple":
return (
<BrandSvg size={size}>
<path d="M14.7 6.2c.8-.9 1.3-2.2 1.2-3.5-1.2.1-2.6.8-3.4 1.8-.8.9-1.5 2.2-1.3 3.5 1.3 0 2.6-.8 3.5-1.8Z" fill="#111" />
<path d="M16.8 12.2c0-2.2 1.8-3.3 1.9-3.4-1.1-1.6-2.7-1.8-3.3-1.8-1.4-.1-2.7.8-3.4.8s-1.8-.8-3-.8c-1.5 0-3 .9-3.8 2.3-1.6 2.8-.4 7 1.2 9.3.8 1.1 1.7 2.3 2.9 2.3 1.2 0 1.6-.7 3-.7s1.8.7 3 .7 2-.1 2.9-2.2c1.1-1.5 1.5-3 1.5-3.1-.1 0-2.9-1.1-2.9-4.4Z" fill="#111" />
</BrandSvg>
)
case "github":
return (
<BrandSvg size={size}>
<circle cx="12" cy="12" r="10" fill="#181717" />
<path d="M12 6.4c-3.1 0-5.6 2.5-5.6 5.6 0 2.5 1.6 4.6 3.8 5.3.3.1.4-.1.4-.3v-1.1c-1.6.3-1.9-.7-1.9-.7-.3-.6-.6-.8-.6-.8-.5-.4 0-.4 0-.4.6 0 .9.6.9.6.5.9 1.4.6 1.7.5.1-.4.2-.6.4-.8-1.2-.1-2.5-.6-2.5-2.8 0-.6.2-1.1.6-1.5-.1-.1-.3-.7 0-1.4 0 0 .5-.2 1.6.6.5-.1 1-.2 1.5-.2s1 .1 1.5.2c1.1-.8 1.6-.6 1.6-.6.3.7.1 1.3 0 1.4.4.4.6.9.6 1.5 0 2.2-1.3 2.6-2.5 2.8.2.2.4.5.4 1.1v1.6c0 .2.1.4.4.3 2.2-.7 3.8-2.8 3.8-5.3 0-3.1-2.5-5.6-5.6-5.6Z" fill="#fff" />
</BrandSvg>
)
case "gitlab":
return (
<BrandSvg size={size}>
<path d="M12 19.2 8.4 8.4h7.2L12 19.2Z" fill="#E24329" />
<path d="M12 19.2 8.4 8.4 5.2 16.2 12 19.2Z" fill="#FC6D26" />
<path d="M12 19.2 15.6 8.4 18.8 16.2 12 19.2Z" fill="#FC6D26" />
<path d="M5.2 16.2 3 8.4h5.4L5.2 16.2Z" fill="#FCA326" />
<path d="M18.8 16.2 21 8.4h-5.4l3.2 7.8Z" fill="#FCA326" />
</BrandSvg>
)
case "spotify":
return (
<BrandSvg size={size}>
<circle cx="12" cy="12" r="10" fill="#1DB954" />
<path d="M7.2 10.4c3.2-1 6.8-.8 9.6.8M7.6 13c2.6-.8 5.6-.6 8 .6M8 15.4c2-.6 4.4-.4 6.2.4" fill="none" stroke="#fff" strokeWidth="1.5" strokeLinecap="round" />
</BrandSvg>
)
case "x":
return (
<BrandSvg size={size}>
<rect width="24" height="24" rx="5" fill="#111" />
<path d="M6.2 5.6h3.2l3 4.2 3.6-4.2H18l-5.2 6.1 5.4 6.7h-3.2l-3.4-4.4-4 4.4H6.4l5.6-6.4Z" fill="#fff" />
</BrandSvg>
)
case "vk":
return (
<BrandSvg size={size}>
<rect width="24" height="24" rx="5" fill="#0077FF" />
<path d="M4.8 7.8h2.6c.1 4.2 1.9 6.7 5.4 6.7V7.8h2.4v3.9c1.5-.2 2.9-1.7 3.4-3.9h2.4c-.6 3.3-2.6 5.4-4.4 6.2 1.8.6 4.1 2.4 5 5.2h-2.8c-.7-1.9-2.2-3.4-4-3.6v3.6h-2.4v-3.6c-3.5.1-6.1-2.4-6.6-6.8Z" fill="#fff" />
</BrandSvg>
)
case "zoom":
return (
<BrandSvg size={size}>
<rect width="24" height="24" rx="5" fill="#2D8CFF" />
<path d="M5.2 9.2h7.2a2.2 2.2 0 0 1 2.2 2.2v5.2H7.4A2.2 2.2 0 0 1 5.2 14.4Z" fill="#fff" />
<path d="M16.2 11.2 20 9.4v7.4l-3.8-1.8Z" fill="#fff" />
</BrandSvg>
)
case "epic":
return (
<BrandSvg size={size}>
<circle cx="12" cy="12" r="10" fill="#111" />
<path d="M8.2 7.4h7.6v2H10.6v2h4.6v2h-4.6v3.2H8.2Z" fill="#fff" />
</BrandSvg>
)
case "riot":
return (
<BrandSvg size={size}>
<path d="M5 19.2 12 4.2 19 19.2h-3.2L12 10.6 8.2 19.2Z" fill="#D32936" />
<path d="M9.4 19.2h5.2l-2.6-5.2Z" fill="#EB0029" />
</BrandSvg>
)
case "playstation":
return (
<BrandSvg size={size}>
<circle cx="12" cy="12" r="10" fill="#003087" />
<path d="M8.2 14.6c-1 .4-1.8.2-2-.4s.4-1.2 1.6-1.6l2-.7v1.6l-1.2.4c-.6.2-.8.4-.7.6.1.2.4.2.9 0l1-.4v1.5Zm3-6.4v8.2c-1.1.4-2.1.5-2.8.2-.9-.4-.9-1.3 0-1.7.5-.2 1.2-.3 2-.2V9.4c0-1.2.5-1.8 1.4-1.5.4.2.7.6.8 1.2Zm5.2 7.6c-1.1.4-2.2.4-3 0-.8-.4-.8-1.2 0-1.6.5-.2 1.2-.3 2-.2v-2.2l-2.4.8V11l4.2-1.5v6.3Z" fill="#fff" />
</BrandSvg>
)
case "roblox":
return (
<BrandSvg size={size}>
<rect width="24" height="24" rx="5" fill="#111" />
<path d="M8.4 6.2 17.6 8.8 15.6 17.8 6.4 15.2Z" fill="#fff" />
</BrandSvg>
)
case "digitalocean":
return (
<BrandSvg size={size}>
<circle cx="12" cy="12" r="10" fill="#0080FF" />
<path d="M12.4 6.2A5.8 5.8 0 0 0 7.8 16l1.6-1.5A3.6 3.6 0 1 1 16 12h-3.6Z" fill="#fff" />
<path d="M12.4 16.2h-1.6v1.6h1.6zm-1.6-2h-1.4v1.4h1.4z" fill="#fff" />
</BrandSvg>
)
case "hetzner":
return (
<BrandSvg size={size}>
<rect width="24" height="24" rx="4" fill="#D50C2D" />
<path d="M7.2 6.4h2.6v4.4h4.4V6.4h2.6v11.2h-2.6v-4.4H9.8v4.4H7.2Z" fill="#fff" />
</BrandSvg>
)
case "ovh":
return (
<BrandSvg size={size}>
<rect width="24" height="24" rx="4" fill="#123F6D" />
<path d="M4.6 15.6 8.4 8.4h3.2L7.8 15.6Zm6.4 0 3.8-7.2h3.2l-3.8 7.2Zm2.2 0h3.4l1.8-3.4h-3.4Z" fill="#00A2E2" />
</BrandSvg>
)
case "chatgpt":
return (
<BrandSvg size={size}>
<circle cx="12" cy="12" r="10" fill="#10A37F" />
<path d="M12.2 6.2c.9-.5 2-.5 2.9 0l2.2 1.3c.9.5 1.4 1.4 1.4 2.4v2.6c0 1-.5 1.9-1.4 2.4l-2.2 1.3c-.9.5-2 .5-2.9 0l-.4-.2c.6-.4 1-1 1.1-1.7l.5.3c.4.2.9.2 1.3 0l2.2-1.3c.4-.2.6-.6.6-1.1V10c0-.4-.2-.8-.6-1.1l-2.2-1.3c-.4-.2-.9-.2-1.3 0L10.2 9c-.4.2-.6.6-.6 1.1v.4h-2V10c0-1 .5-1.9 1.4-2.4Z" fill="#fff" />
<path d="M8.8 9.6c.6-.4 1.3-.5 2-.3v2.1c0 .4.2.8.6 1.1l2.2 1.3c.4.2.9.2 1.3 0l.5-.3c.2.7.6 1.3 1.1 1.7l-.4.2c-.9.5-2 .5-2.9 0l-2.2-1.3c-.9-.5-1.4-1.4-1.4-2.4Z" fill="#fff" opacity="0.85" />
</BrandSvg>
)
default:
return <GenericCloud size={size} />
}
File diff suppressed because it is too large Load Diff
@@ -0,0 +1,57 @@
"use client"
import { Frame, FrameHeader, FramePanel, FrameTitle } from "@/components/reui/frame"
import {
StatisticsBreakdownDataGrid,
type StatisticsSliceKind,
} from "@/components/data-grids/statistics-breakdown-data-grid"
import { STATISTICS_DIMS } from "@/lib/statistics-dims"
import type { StatisticsBreakdownRow, StatisticsDto } from "@mmapp/contracts/statistics"
const MINI_ROWS = 12
function rowsForKind(data: StatisticsDto, kind: StatisticsSliceKind): StatisticsBreakdownRow[] {
if (kind === "users") return data.users
if (kind === "servers") return data.servers
if (kind === "interfaces") return data.interfaces
if (kind === "countries") return data.countries
if (kind === "services") return data.services
return data.asns
}
export function BreakdownDashboard({
data,
hidden,
selectedIdFor,
onRowClick,
isLoading,
}: {
data: StatisticsDto
hidden: Set<StatisticsSliceKind>
selectedIdFor: (kind: StatisticsSliceKind) => string | undefined
onRowClick: (kind: StatisticsSliceKind, row: StatisticsBreakdownRow) => void
isLoading?: boolean
}) {
const dims = STATISTICS_DIMS.filter((d) => !hidden.has(d.id))
return (
<div className="grid grid-cols-1 gap-4 p-4 md:grid-cols-2">
{dims.map((d) => (
<Frame key={d.id} dense>
<FrameHeader className="border-b">
<FrameTitle>{d.label}</FrameTitle>
</FrameHeader>
<FramePanel className="max-h-80 overflow-auto p-0">
<StatisticsBreakdownDataGrid
rows={rowsForKind(data, d.id).slice(0, MINI_ROWS)}
kind={d.id}
selectedId={selectedIdFor(d.id)}
onRowClick={(row) => onRowClick(d.id, row)}
isLoading={isLoading}
density="mini"
/>
</FramePanel>
</Frame>
))}
</div>
)
}
@@ -0,0 +1,65 @@
import type { DateSelectorI18nConfig } from "@/components/reui/date-selector"
/** Русские подписи ReUI DateSelector (шапка /statistics). */
export const DATE_SELECTOR_RU: DateSelectorI18nConfig = {
selectDate: "Выбрать дату",
apply: "Применить",
cancel: "Отмена",
clear: "Сбросить",
today: "Сегодня",
filterTypes: {
is: "равно",
before: "до",
after: "после",
between: "между",
},
periodTypes: {
day: "День",
month: "Месяц",
quarter: "Квартал",
halfYear: "Полугодие",
year: "Год",
},
months: [
"Январь",
"Февраль",
"Март",
"Апрель",
"Май",
"Июнь",
"Июль",
"Август",
"Сентябрь",
"Октябрь",
"Ноябрь",
"Декабрь",
],
monthsShort: [
"янв",
"фев",
"мар",
"апр",
"май",
"июн",
"июл",
"авг",
"сен",
"окт",
"ноя",
"дек",
],
quarters: ["I кв.", "II кв.", "III кв.", "IV кв."],
halfYears: ["1-е полугодие", "2-е полугодие"],
weekdays: [
"Воскресенье",
"Понедельник",
"Вторник",
"Среда",
"Четверг",
"Пятница",
"Суббота",
],
weekdaysShort: ["вс", "пн", "вт", "ср", "чт", "пт", "сб"],
placeholder: "Выберите дату…",
rangePlaceholder: "Выберите период…",
}
@@ -0,0 +1,73 @@
"use client"
import {
Select,
SelectContent,
SelectItem,
SelectTrigger,
SelectValue,
} from "@/components/ui/select"
import { STATISTICS_DIMS } from "@/lib/statistics-dims"
import type { StatisticsPivotDim } from "@mmapp/contracts/statistics"
import type { StatisticsSliceKind } from "@/components/data-grids/statistics-breakdown-data-grid"
export function DimensionSelect({
value,
onChange,
label,
}: {
value: StatisticsSliceKind
onChange: (value: StatisticsSliceKind) => void
label?: string
}) {
const current = STATISTICS_DIMS.find((d) => d.id === value)
return (
<label className="flex items-center gap-2">
{label ? <span className="text-muted-foreground text-xs whitespace-nowrap">{label}</span> : null}
<Select value={value} onValueChange={(v) => onChange(String(v ?? value) as StatisticsSliceKind)}>
<SelectTrigger size="sm" className="min-w-40">
<SelectValue>{current?.label ?? "Измерение"}</SelectValue>
</SelectTrigger>
<SelectContent align="start">
{STATISTICS_DIMS.map((d) => (
<SelectItem key={d.id} value={d.id}>
{d.label}
</SelectItem>
))}
</SelectContent>
</Select>
</label>
)
}
export function PivotDimSelect({
value,
onChange,
exclude,
label,
}: {
value: StatisticsPivotDim
onChange: (value: StatisticsPivotDim) => void
exclude?: StatisticsPivotDim
label: string
}) {
const options = STATISTICS_DIMS.filter((d) => d.pivot !== exclude)
const current = STATISTICS_DIMS.find((d) => d.pivot === value)
return (
<label className="flex items-center gap-2">
<span className="text-muted-foreground text-xs whitespace-nowrap">{label}</span>
<Select value={value} onValueChange={(v) => onChange(String(v ?? value) as StatisticsPivotDim)}>
<SelectTrigger size="sm" className="min-w-40">
<SelectValue>{current?.label ?? label}</SelectValue>
</SelectTrigger>
<SelectContent align="start">
{options.map((d) => (
<SelectItem key={d.pivot} value={d.pivot}>
{d.label}
</SelectItem>
))}
</SelectContent>
</Select>
</label>
)
}
+188
View File
@@ -0,0 +1,188 @@
"use client"
import { useMemo } from "react"
import { format } from "date-fns"
import { ru } from "date-fns/locale"
import { CalendarIcon } from "lucide-react"
import {
DateSelector,
type DateSelectorValue,
} from "@/components/reui/date-selector"
import { DATE_SELECTOR_RU } from "@/components/statistics/date-selector-i18n"
import { Button } from "@/components/ui/button"
import { Popover, PopoverContent, PopoverTrigger } from "@/components/ui/popover"
/**
* Период отчёта DateSelector в Popover (c-date-selector-2).
* @see https://reui.io/preview/base/components/c-date-selector-2
* @see https://reui.io/docs/components/base/date-selector
*/
export type PeriodPreset = "today" | "24h" | "7d" | "30d" | "month"
export interface DateRangeYmd {
from: string
to: string
}
const PRESETS: { id: PeriodPreset; label: string }[] = [
{ id: "today", label: "Сегодня" },
{ id: "24h", label: "24 ч" },
{ id: "7d", label: "7 д" },
{ id: "30d", label: "30 д" },
{ id: "month", label: "Месяц" },
]
function pad2(n: number): string {
return String(n).padStart(2, "0")
}
export function formatYmd(d: Date): string {
return `${d.getFullYear()}-${pad2(d.getMonth() + 1)}-${pad2(d.getDate())}`
}
export function parseYmd(s: string): Date | null {
const m = /^(\d{4})-(\d{2})-(\d{2})$/.exec(s)
if (!m) return null
const d = new Date(Number(m[1]), Number(m[2]) - 1, Number(m[3]))
return Number.isNaN(d.getTime()) ? null : d
}
export function rangeForPreset(preset: PeriodPreset, now = new Date()): DateRangeYmd {
const to = formatYmd(now)
if (preset === "today") return { from: to, to }
if (preset === "24h") {
const from = new Date(now)
from.setDate(from.getDate() - 1)
return { from: formatYmd(from), to }
}
if (preset === "7d") {
const from = new Date(now)
from.setDate(from.getDate() - 6)
return { from: formatYmd(from), to }
}
if (preset === "30d") {
const from = new Date(now)
from.setDate(from.getDate() - 29)
return { from: formatYmd(from), to }
}
const start = new Date(now.getFullYear(), now.getMonth(), 1)
return { from: formatYmd(start), to }
}
export function dateSelectorToRange(value: DateSelectorValue): DateRangeYmd | null {
if (value.period === "day") {
if (value.operator === "between") {
if (!value.startDate || !value.endDate) return null
const a = formatYmd(value.startDate)
const b = formatYmd(value.endDate)
return a <= b ? { from: a, to: b } : { from: b, to: a }
}
if (value.startDate) {
const d = formatYmd(value.startDate)
return { from: d, to: d }
}
}
if (value.period === "month" && value.year != null && value.month != null) {
const start = new Date(value.year, value.month, 1)
const end = new Date(value.year, value.month + 1, 0)
return { from: formatYmd(start), to: formatYmd(end) }
}
if (value.period === "year" && value.year != null) {
return { from: `${value.year}-01-01`, to: `${value.year}-12-31` }
}
if (value.period === "quarter" && value.year != null && value.quarter != null) {
const startMonth = value.quarter * 3
const start = new Date(value.year, startMonth, 1)
const end = new Date(value.year, startMonth + 3, 0)
return { from: formatYmd(start), to: formatYmd(end) }
}
if (value.period === "half-year" && value.year != null && value.halfYear != null) {
const startMonth = value.halfYear * 6
const start = new Date(value.year, startMonth, 1)
const end = new Date(value.year, startMonth + 6, 0)
return { from: formatYmd(start), to: formatYmd(end) }
}
return null
}
export function rangeToSelector(range: DateRangeYmd): DateSelectorValue {
return {
period: "day",
operator: "between",
startDate: parseYmd(range.from) ?? undefined,
endDate: parseYmd(range.to) ?? undefined,
}
}
function formatRangeLabel(range: DateRangeYmd): string {
const from = parseYmd(range.from)
const to = parseYmd(range.to)
if (!from || !to) return "Период"
if (range.from === range.to) return format(from, "d MMM yyyy", { locale: ru })
return `${format(from, "d MMM", { locale: ru })} ${format(to, "d MMM yyyy", { locale: ru })}`
}
export function PeriodSelector({
range,
onChange,
}: {
range: DateRangeYmd
onChange: (next: DateRangeYmd) => void
}) {
const selectorValue = useMemo(() => rangeToSelector(range), [range])
function handleSelectorChange(value: DateSelectorValue) {
const next = dateSelectorToRange(value)
if (!next) return
if (next.from === range.from && next.to === range.to) return
onChange(next)
}
const activePreset = PRESETS.find((p) => {
const r = rangeForPreset(p.id)
return r.from === range.from && r.to === range.to
})?.id
return (
<div className="flex flex-wrap items-center gap-1.5">
{PRESETS.map((p) => (
<Button
key={p.id}
type="button"
variant={activePreset === p.id ? "secondary" : "ghost"}
size="sm"
onClick={() => onChange(rangeForPreset(p.id))}
>
{p.label}
</Button>
))}
<Popover>
<PopoverTrigger
render={
<Button type="button" variant="outline" size="sm" className="min-w-40 justify-between" />
}
>
<CalendarIcon className="size-3.5" />
<span className="tabular-nums">{formatRangeLabel(range)}</span>
</PopoverTrigger>
<PopoverContent align="end" className="w-auto min-w-[32rem] max-w-[min(100vw-2rem,42rem)] p-3">
<DateSelector
value={selectorValue}
onChange={handleSelectorChange}
allowRange
defaultPeriodType="day"
defaultFilterType="between"
periodTypes={["day", "month", "year"]}
showTwoMonths
weekStartsOn={1}
maxYear={2035}
dayDateFormat="dd.MM.yyyy"
i18n={DATE_SELECTOR_RU}
className="sm:w-[470px]"
/>
</PopoverContent>
</Popover>
</div>
)
}
+38
View File
@@ -0,0 +1,38 @@
"use client"
import { XIcon } from "lucide-react"
import { Badge } from "@/components/reui/badge"
import { Button } from "@/components/ui/button"
export interface SliceChip {
key: string
label: string
}
export function SliceChips({
chips,
onRemove,
}: {
chips: SliceChip[]
onRemove: (key: string) => void
}) {
if (!chips.length) return null
return (
<div className="flex flex-wrap items-center gap-1.5 border-b px-5 py-2">
{chips.map((chip) => (
<Badge key={chip.key} variant="outline" size="sm" className="gap-1 pr-0.5">
{chip.label}
<Button
type="button"
variant="ghost"
size="icon-xs"
aria-label={`Снять ${chip.label}`}
onClick={() => onRemove(chip.key)}
>
<XIcon />
</Button>
</Badge>
))}
</div>
)
}
@@ -0,0 +1,89 @@
"use client"
import { useMemo } from "react"
import { CompactDataGrid, type CompactDataGridColumn } from "@/components/data-grids/compact-data-grid"
import { formatBytes } from "@/lib/fmt-rate"
import { cn } from "@/lib/utils"
import type { StatisticsPivotDto } from "@mmapp/contracts/statistics"
interface PivotGridRow {
id: string
label: string
total: number
[key: string]: string | number
}
export function StatisticsPivotGrid({
data,
onCellClick,
isLoading,
}: {
data: StatisticsPivotDto
onCellClick?: (rowId: string, colId: string) => void
isLoading?: boolean
}) {
const rows: PivotGridRow[] = useMemo(
() =>
data.rows.map((r) => {
const next: PivotGridRow = { id: r.id, label: r.label, total: r.total }
for (const col of data.columns) {
next[`c:${col.id}`] = r.cells[col.id] ?? 0
}
return next
}),
[data],
)
const columns: CompactDataGridColumn<PivotGridRow>[] = [
{
id: "label",
header: "Измерение",
accessorKey: "label",
cell: (row) => <span className="truncate font-medium">{row.label}</span>,
},
...data.columns.map((col) => ({
id: `c:${col.id}`,
header: col.label,
accessorKey: `c:${col.id}` as const,
cell: (row: PivotGridRow) => {
const value = Number(row[`c:${col.id}`] ?? 0)
return (
<button
type="button"
className={cn(
"tabular-nums text-left hover:underline",
col.id === "__other__" || row.id === "__other__" ? "text-muted-foreground" : "",
)}
onClick={(e) => {
e.stopPropagation()
if (row.id === "__other__" || col.id === "__other__") return
onCellClick?.(row.id, col.id)
}}
>
{data.metric === "packets" ? value.toLocaleString("ru-RU") : formatBytes(value)}
</button>
)
},
})),
{
id: "total",
header: "Итого",
accessorKey: "total",
cell: (row) => (
<span className="tabular-nums font-medium">
{data.metric === "packets" ? row.total.toLocaleString("ru-RU") : formatBytes(row.total)}
</span>
),
},
]
return (
<CompactDataGrid
data={rows}
columns={columns}
isLoading={isLoading}
emptyTitle="Нет данных сводной"
emptyDescription="Выберите разные измерения строк и колонок."
/>
)
}
@@ -0,0 +1,105 @@
"use client"
import { Area, AreaChart, CartesianGrid, XAxis, YAxis } from "recharts"
import { Frame, FramePanel } from "@/components/reui/frame"
import { ChartContainer, ChartTooltip, type ChartConfig } from "@/components/ui/chart"
import { formatBytes } from "@/lib/fmt-rate"
import type { StatisticsSeriesPoint } from "@mmapp/contracts/statistics"
/**
* Объём трафика за период adapt ReUI chart-23 (байты, не live bps).
* @see https://reui.io/preview/base/chart-23
*/
const chartConfig = {
bytes: { label: "Объём", color: "var(--chart-1)" },
} satisfies ChartConfig
function formatTick(iso: string, grain: "hour" | "day"): string {
const d = new Date(iso)
if (Number.isNaN(d.getTime())) return iso.slice(0, 10)
if (grain === "hour") {
return `${String(d.getHours()).padStart(2, "0")}:00`
}
return d.toLocaleDateString("ru-RU", { day: "2-digit", month: "short" })
}
function CustomTooltip({
active,
payload,
}: {
active?: boolean
payload?: { payload: { label: string; bytes: number } }[]
}) {
if (!active || !payload?.length) return null
const row = payload[0]?.payload
if (!row) return null
return (
<div className="flex min-w-[120px] flex-col gap-1.5 rounded-lg bg-popover p-3 text-popover-foreground shadow-lg ring-1 ring-foreground/10">
<div className="text-[10px] font-medium tracking-wider uppercase opacity-70">{row.label}</div>
<div className="text-sm font-semibold tabular-nums">{formatBytes(row.bytes)}</div>
</div>
)
}
export function StatisticsVolumeChart({
series,
grain,
}: {
series: StatisticsSeriesPoint[]
grain: "hour" | "day"
}) {
const data = series.map((p) => ({
t: p.t,
bytes: p.bytes,
label: formatTick(p.t, grain),
}))
const tickEvery = Math.max(1, Math.ceil(data.length / 8))
return (
<Frame className="w-full">
<FramePanel className="flex flex-col gap-4">
<div className="flex items-baseline justify-between gap-2">
<h2 className="text-sm font-medium">Объём по времени</h2>
<span className="text-muted-foreground text-xs">
{grain === "hour" ? "по часам" : "по суткам"}
</span>
</div>
{data.length === 0 ? (
<p className="text-muted-foreground py-10 text-center text-sm">Нет данных за выбранный период</p>
) : (
<ChartContainer config={chartConfig} className="-ms-4 aspect-auto h-[220px] w-full">
<AreaChart data={data} margin={{ top: 5, right: 5, left: 5, bottom: 5 }}>
<CartesianGrid strokeDasharray="4 8" vertical={false} stroke="var(--border)" />
<XAxis
dataKey="label"
axisLine={false}
tickLine={false}
tick={{ fontSize: 11 }}
tickMargin={10}
interval={tickEvery - 1}
/>
<YAxis
axisLine={false}
tickLine={false}
tick={{ fontSize: 11 }}
tickFormatter={(v: number) => formatBytes(Number(v))}
tickMargin={8}
width={72}
/>
<ChartTooltip content={<CustomTooltip />} />
<Area
dataKey="bytes"
type="monotone"
stroke="var(--chart-1)"
fill="var(--chart-1)"
fillOpacity={0.15}
strokeWidth={2}
/>
</AreaChart>
</ChartContainer>
)}
</FramePanel>
</Frame>
)
}
@@ -10,6 +10,7 @@ import { Button } from "@/components/ui/button"
import { Input } from "@/components/ui/input"
import { CodeExportSheet, type CodeExportFormat } from "@/components/reui-kit/code-export-sheet"
import type { TrafficFlowSettingsDto } from "@mmapp/contracts/traffic-flow"
import type { GeoipStatusDto } from "@mmapp/contracts/geoip"
import {
generateTrafficFlowKeys,
getTrafficFlowHostFiles,
@@ -17,6 +18,7 @@ import {
purgeTrafficFlowData,
putTrafficFlowSettings,
} from "@/shared/api/traffic-flow"
import { getGeoipStatus, putGeoipSettings, runGeoipUpdateNow } from "@/shared/api/geoip"
import { formatFlowPurgeResult, NetflowPurgeConfirm } from "@/components/traffic/netflow-purge-dialog"
import { KeyRoundIcon, DownloadIcon, InfoIcon } from "lucide-react"
@@ -29,6 +31,127 @@ const HOST_STEPS = [
"Проверка: wg show · ss -ulnp | grep 4739 · в этой панели — last datagram.",
]
function fmtDate(iso: string | null | undefined): string {
return iso ? new Date(iso).toLocaleString("ru-RU") : "—"
}
function GeoipSettingsSection({ backendUrl }: { backendUrl: string }) {
const [status, setStatus] = useState<GeoipStatusDto | null>(null)
const [busy, setBusy] = useState(false)
const [updating, setUpdating] = useState(false)
const [autoOn, setAutoOn] = useState(true)
const [intervalHours, setIntervalHours] = useState("168")
const load = useCallback(async () => {
const s = await getGeoipStatus(backendUrl)
setStatus(s)
setAutoOn(s.settings.enabled)
setIntervalHours(String(Math.round(s.settings.updateIntervalSec / 3600)))
}, [backendUrl])
useEffect(() => {
void load().catch((e: unknown) => {
toast.error(e instanceof Error ? e.message : "Не удалось загрузить GeoIP")
})
}, [load])
async function handleSave() {
setBusy(true)
try {
const hours = Math.min(720, Math.max(6, Number.parseInt(intervalHours, 10) || 168))
const res = await putGeoipSettings(backendUrl, {
enabled: autoOn,
updateIntervalSec: hours * 3600,
})
setStatus(res.status)
toast.success("Настройки GeoIP сохранены")
} catch (e) {
toast.error(e instanceof Error ? e.message : "Не удалось сохранить")
} finally {
setBusy(false)
}
}
async function handleUpdateNow() {
setUpdating(true)
try {
const res = await runGeoipUpdateNow(backendUrl)
if (res.ok || res.snapshot.downloaded > 0 || res.snapshot.skippedUnchanged > 0) {
toast.success(
res.snapshot.downloaded > 0
? `Скачано баз: ${res.snapshot.downloaded} (${(res.snapshot.bytes / 1024 / 1024).toFixed(1)} МБ)`
: "Базы актуальны, скачивание не требуется",
)
} else {
toast.error(res.snapshot.errors.join("; ") || "Обновление не выполнено")
}
await load()
} catch (e) {
toast.error(e instanceof Error ? e.message : "Не удалось обновить базы")
} finally {
setUpdating(false)
}
}
return (
<OpsPanel
title="GeoIP-базы (GeoLite2)"
description="Локальные mmdb MaxMind GeoLite2 с зеркала P3TERX: страна и ASN каждого потока при ingest — мгновенно, включая IPv6, без лимитов RIPEstat."
headerRight={
<div className="flex items-center gap-2">
<Badge variant={status?.countryLoaded ? "success" : "secondary"}>
country {status?.countryLoaded ? "ok" : "нет"}
</Badge>
<Badge variant={status?.asnLoaded ? "success" : "secondary"}>
asn {status?.asnLoaded ? "ok" : "нет"}
</Badge>
</div>
}
contentClassName="px-5 py-4 flex flex-col gap-4"
>
<div className="grid grid-cols-1 sm:grid-cols-2 gap-3">
<div className="flex items-center gap-3">
<FormToggle checked={autoOn} onChange={setAutoOn} />
<span className="text-sm">Автообновление</span>
</div>
<FormField label="Интервал проверки (часов)" hint="Upstream обновляется еженедельно; минимум 6 ч">
<Input
className="font-mono"
value={intervalHours}
onChange={(e) => setIntervalHours(e.target.value)}
inputMode="numeric"
disabled={!autoOn}
/>
</FormField>
</div>
<p className="text-xs text-muted-foreground">
Сборка Country: {fmtDate(status?.settings.countryBuildAt)} · ASN: {fmtDate(status?.settings.asnBuildAt)}
{" · "}последняя проверка: {fmtDate(status?.settings.lastCheckAt)}
{status?.settings.lastError ? ` · ошибка: ${status.settings.lastError}` : ""}
</p>
<p className="text-xs text-muted-foreground">
Каталог: <span className="font-mono">{status?.dir || "storage/geoip"}</span>. До загрузки баз
и при промахе lookup страна/ASN берутся из RIPEstat, как раньше.
</p>
<div className="flex flex-wrap gap-2">
<Button size="sm" disabled={busy || updating} onClick={() => { void handleSave() }}>
Сохранить GeoIP
</Button>
<Button size="sm" variant="outline" disabled={busy || updating} onClick={() => { void handleUpdateNow() }}>
<DownloadIcon className={updating ? "size-4 animate-spin" : "size-4"} />
{updating ? "Обновление…" : "Обновить сейчас"}
</Button>
</div>
<p className="text-xs text-muted-foreground">
Данные: MaxMind GeoLite2 (CC BY-SA 4.0), зеркало P3TERX/GeoLite.mmdb.
</p>
</OpsPanel>
)
}
function NetflowSettingsPanel({
backendUrl,
enabled,
@@ -276,6 +399,8 @@ function NetflowSettingsPanel({
</div>
</OpsPanel>
<GeoipSettingsSection backendUrl={backendUrl} />
<CodeExportSheet
open={exportOpen}
onClose={() => setExportOpen(false)}
+221
View File
@@ -0,0 +1,221 @@
"use client"
import * as React from "react"
import {
DayPicker,
getDefaultClassNames,
type DayButton,
type Locale,
} from "react-day-picker"
import { cn } from "@/lib/utils"
import { Button, buttonVariants } from "@/components/ui/button"
import { ChevronLeftIcon, ChevronRightIcon, ChevronDownIcon } from "lucide-react"
function Calendar({
className,
classNames,
showOutsideDays = true,
captionLayout = "label",
buttonVariant = "ghost",
locale,
formatters,
components,
...props
}: React.ComponentProps<typeof DayPicker> & {
buttonVariant?: React.ComponentProps<typeof Button>["variant"]
}) {
const defaultClassNames = getDefaultClassNames()
return (
<DayPicker
showOutsideDays={showOutsideDays}
className={cn(
"group/calendar bg-background p-2 [--cell-radius:var(--radius-md)] [--cell-size:--spacing(7)] in-data-[slot=card-content]:bg-transparent in-data-[slot=popover-content]:bg-transparent",
String.raw`rtl:**:[.rdp-button\_next>svg]:rotate-180`,
String.raw`rtl:**:[.rdp-button\_previous>svg]:rotate-180`,
className
)}
captionLayout={captionLayout}
locale={locale}
formatters={{
formatMonthDropdown: (date) =>
date.toLocaleString(locale?.code, { month: "short" }),
...formatters,
}}
classNames={{
root: cn("w-fit", defaultClassNames.root),
months: cn(
"relative flex flex-col gap-4 md:flex-row",
defaultClassNames.months
),
month: cn("flex w-full flex-col gap-4", defaultClassNames.month),
nav: cn(
"absolute inset-x-0 top-0 flex w-full items-center justify-between gap-1",
defaultClassNames.nav
),
button_previous: cn(
buttonVariants({ variant: buttonVariant }),
"size-(--cell-size) p-0 select-none aria-disabled:opacity-50",
defaultClassNames.button_previous
),
button_next: cn(
buttonVariants({ variant: buttonVariant }),
"size-(--cell-size) p-0 select-none aria-disabled:opacity-50",
defaultClassNames.button_next
),
month_caption: cn(
"flex h-(--cell-size) w-full items-center justify-center px-(--cell-size)",
defaultClassNames.month_caption
),
dropdowns: cn(
"flex h-(--cell-size) w-full items-center justify-center gap-1.5 text-sm font-medium",
defaultClassNames.dropdowns
),
dropdown_root: cn(
"relative rounded-(--cell-radius)",
defaultClassNames.dropdown_root
),
dropdown: cn(
"absolute inset-0 bg-popover opacity-0",
defaultClassNames.dropdown
),
caption_label: cn(
"font-medium select-none",
captionLayout === "label"
? "text-sm"
: "flex items-center gap-1 rounded-(--cell-radius) text-sm [&>svg]:size-3.5 [&>svg]:text-muted-foreground",
defaultClassNames.caption_label
),
month_grid: cn("w-full border-collapse", defaultClassNames.month_grid),
weekdays: cn("flex", defaultClassNames.weekdays),
weekday: cn(
"flex-1 rounded-(--cell-radius) text-[0.8rem] font-normal text-muted-foreground select-none",
defaultClassNames.weekday
),
week: cn("mt-2 flex w-full", defaultClassNames.week),
week_number_header: cn(
"w-(--cell-size) select-none",
defaultClassNames.week_number_header
),
week_number: cn(
"text-[0.8rem] text-muted-foreground select-none",
defaultClassNames.week_number
),
day: cn(
"group/day relative aspect-square h-full w-full rounded-(--cell-radius) p-0 text-center select-none [&:last-child[data-selected=true]_button]:rounded-r-(--cell-radius)",
props.showWeekNumber
? "[&:nth-child(2)[data-selected=true]_button]:rounded-l-(--cell-radius)"
: "[&:first-child[data-selected=true]_button]:rounded-l-(--cell-radius)",
defaultClassNames.day
),
range_start: cn(
"relative isolate z-0 rounded-l-(--cell-radius) bg-muted after:absolute after:inset-y-0 after:right-0 after:w-4 after:bg-muted",
defaultClassNames.range_start
),
range_middle: cn("rounded-none", defaultClassNames.range_middle),
range_end: cn(
"relative isolate z-0 rounded-r-(--cell-radius) bg-muted after:absolute after:inset-y-0 after:left-0 after:w-4 after:bg-muted",
defaultClassNames.range_end
),
today: cn(
"rounded-(--cell-radius) bg-muted text-foreground data-[selected=true]:rounded-none",
defaultClassNames.today
),
outside: cn(
"text-muted-foreground aria-selected:text-muted-foreground",
defaultClassNames.outside
),
disabled: cn(
"text-muted-foreground opacity-50",
defaultClassNames.disabled
),
hidden: cn("invisible", defaultClassNames.hidden),
...classNames,
}}
components={{
Root: ({ className, rootRef, ...props }) => {
return (
<div
data-slot="calendar"
ref={rootRef}
className={cn(className)}
{...props}
/>
)
},
Chevron: ({ className, orientation, ...props }) => {
if (orientation === "left") {
return (
<ChevronLeftIcon className={cn("size-4", className)} {...props} />
)
}
if (orientation === "right") {
return (
<ChevronRightIcon className={cn("size-4", className)} {...props} />
)
}
return (
<ChevronDownIcon className={cn("size-4", className)} {...props} />
)
},
DayButton: ({ ...props }) => (
<CalendarDayButton locale={locale} {...props} />
),
WeekNumber: ({ children, ...props }) => {
return (
<td {...props}>
<div className="flex size-(--cell-size) items-center justify-center text-center">
{children}
</div>
</td>
)
},
...components,
}}
{...props}
/>
)
}
function CalendarDayButton({
className,
day,
modifiers,
locale,
...props
}: React.ComponentProps<typeof DayButton> & { locale?: Partial<Locale> }) {
const defaultClassNames = getDefaultClassNames()
const ref = React.useRef<HTMLButtonElement>(null)
React.useEffect(() => {
if (modifiers.focused) ref.current?.focus()
}, [modifiers.focused])
return (
<Button
variant="ghost"
size="icon"
data-day={day.date.toLocaleDateString(locale?.code)}
data-selected-single={
modifiers.selected &&
!modifiers.range_start &&
!modifiers.range_end &&
!modifiers.range_middle
}
data-range-start={modifiers.range_start}
data-range-end={modifiers.range_end}
data-range-middle={modifiers.range_middle}
className={cn(
"relative isolate z-10 flex aspect-square size-auto w-full min-w-(--cell-size) flex-col gap-1 border-0 leading-none font-normal group-data-[focused=true]/day:relative group-data-[focused=true]/day:z-10 group-data-[focused=true]/day:border-ring group-data-[focused=true]/day:ring-[3px] group-data-[focused=true]/day:ring-ring/50 data-[range-end=true]:rounded-(--cell-radius) data-[range-end=true]:rounded-r-(--cell-radius) data-[range-end=true]:bg-primary data-[range-end=true]:text-primary-foreground data-[range-middle=true]:rounded-none data-[range-middle=true]:bg-muted data-[range-middle=true]:text-foreground data-[range-start=true]:rounded-(--cell-radius) data-[range-start=true]:rounded-l-(--cell-radius) data-[range-start=true]:bg-primary data-[range-start=true]:text-primary-foreground data-[selected-single=true]:bg-primary data-[selected-single=true]:text-primary-foreground dark:hover:text-foreground [&>span]:text-xs [&>span]:opacity-70",
defaultClassNames.day,
className
)}
{...props}
/>
)
}
export { Calendar, CalendarDayButton }
+4 -1
View File
@@ -233,7 +233,9 @@ export function permissionForPath(pathname: string): string | null {
}
if (pathname.startsWith("/bgp")) return "mm:bgp:read"
if (pathname.startsWith("/uptime")) return "mm:uptime:read"
if (pathname.startsWith("/traffic")) return "mm:traffic:read"
if (pathname.startsWith("/traffic") || pathname.startsWith("/statistics")) {
return "mm:traffic:read"
}
if (pathname.startsWith("/alerts")) return "mm:alerts:read"
if (pathname.startsWith("/backups")) return "mm:backups:read"
if (pathname.startsWith("/certificates")) return "mm:certificates:read"
@@ -257,6 +259,7 @@ export function firstAllowedPath(): string {
"/filters",
"/uptime",
"/traffic",
"/statistics",
"/alerts",
"/backups",
"/settings",
@@ -0,0 +1,46 @@
import type { FilterFieldConfig } from "@/components/reui/filters"
export const STATISTICS_FILTER_FIELDS: FilterFieldConfig[] = [
{
key: "country",
label: "Страна",
type: "text",
placeholder: "ISO, напр. US",
defaultOperator: "is",
},
{
key: "service",
label: "Сервис",
type: "text",
placeholder: "https, dns…",
defaultOperator: "is",
},
{
key: "asn",
label: "ASN",
type: "text",
placeholder: "номер ASN",
defaultOperator: "is",
},
{
key: "serverId",
label: "Сервер",
type: "text",
placeholder: "id сервера",
defaultOperator: "is",
},
{
key: "userId",
label: "Пользователь",
type: "text",
placeholder: "id пользователя",
defaultOperator: "is",
},
{
key: "iface",
label: "Интерфейс",
type: "text",
placeholder: "ether1",
defaultOperator: "is",
},
]
+14
View File
@@ -13,6 +13,20 @@ export function fmtGB(v: number): string {
return `${v.toFixed(1)} ГБ`
}
export function formatBytes(n: number): string {
if (!Number.isFinite(n) || n <= 0) return "0 Б"
if (n >= 1_000_000_000_000) return `${(n / 1_000_000_000_000).toFixed(2)} ТБ`
if (n >= 1_000_000_000) return `${(n / 1_000_000_000).toFixed(2)} ГБ`
if (n >= 1_000_000) return `${(n / 1_000_000).toFixed(1)} МБ`
if (n >= 1000) return `${(n / 1000).toFixed(1)} КБ`
return `${Math.round(n)} Б`
}
/** API bitrate is bits/s; fmtRate expects Мбит/с. */
export function fmtBps(bps: number): string {
return fmtRate(bps / 1_000_000)
}
export const TRAFFIC_RANGE_MINUTES: Record<string, number> = {
"5m": 5,
"15m": 15,
+62
View File
@@ -0,0 +1,62 @@
import assert from "node:assert/strict"
import {
formatServicePathLabel,
formatServicePathTitle,
isUnboundServicePath,
} from "./format-service-path-label.ts"
const bound = {
clientId: "u1",
clientName: "D",
viaId: "7",
viaName: "nsk-gw01",
enId: "9",
enName: "arn-gw01",
serviceId: "svc:cdn",
}
const jhEn = {
clientId: "—",
clientName: "—",
viaId: "7",
viaName: "msk-gw01",
enId: "9",
enName: "arn-gw01",
serviceId: "svc:cdn",
}
const enOnly = {
clientId: "—",
clientName: "—",
viaId: "9",
viaName: "arn-gw01",
enId: "9",
enName: "arn-gw01",
serviceId: "svc:cdn",
}
assert.equal(isUnboundServicePath(jhEn), true)
assert.equal(isUnboundServicePath(bound), false)
assert.equal(
formatServicePathLabel(jhEn, "via", { viaName: "msk-gw01.rtnt.top", enName: "arn-gw01.rtnt.top" }),
"msk-gw01.rtnt.top → arn-gw01.rtnt.top",
)
assert.equal(formatServicePathLabel(enOnly, "via"), "arn-gw01 · без привязки")
assert.equal(formatServicePathLabel(enOnly, "service"), "arn-gw01 · без привязки")
assert.equal(
formatServicePathLabel(bound, "via", { viaSite: "NSK" }),
"D · NSK",
)
assert.equal(
formatServicePathLabel(bound, "service", { serviceLabel: "CDN" }),
"D · CDN",
)
assert.equal(
formatServicePathTitle("msk-gw01 → arn-gw01", "CDN"),
"msk-gw01 → arn-gw01 · CDN",
)
console.log("format-service-path-label.test.ts: ok")
+45
View File
@@ -0,0 +1,45 @@
export const UNBOUND_PATH_CLIENT = "—"
export interface ServicePathLabelInput {
clientId: string
clientName: string
viaId: string
viaName: string
enId: string
enName: string
serviceId: string
}
export interface ServicePathLabelNames {
viaName?: string
viaSite?: string
enName?: string
serviceLabel?: string
}
export function isUnboundServicePath(p: Pick<ServicePathLabelInput, "clientId" | "clientName">): boolean {
return p.clientId === UNBOUND_PATH_CLIENT || p.clientName === UNBOUND_PATH_CLIENT
}
export function formatServicePathLabel(
p: ServicePathLabelInput,
viaMode: "via" | "service",
names: ServicePathLabelNames = {},
): string {
const viaName = names.viaName || p.viaName
const enName = names.enName || p.enName
if (isUnboundServicePath(p)) {
if (p.viaId !== p.enId) return `${viaName}${enName}`
return `${enName} · без привязки`
}
const viaLabel = names.viaSite || p.viaName
const mid = viaMode === "via" ? viaLabel : (names.serviceLabel || p.serviceId)
return `${p.clientName} · ${mid}`
}
export function formatServicePathTitle(label: string, serviceLabel: string): string {
const svc = serviceLabel.trim()
if (!svc) return label
if (label.includes(svc)) return label
return `${label} · ${svc}`
}
+14
View File
@@ -97,6 +97,19 @@ export interface BackupsRunSnapshot {
fatalError?: string
}
export interface GeoipUpdateRunSnapshot {
v: number
job: "geoip_update"
sampledAt: string
skipped?: boolean
fatalError?: string
checked: number
downloaded: number
skippedUnchanged: number
bytes: number
errors: string[]
}
export type SchedulerRunSnapshot =
| TrafficRunSnapshot
| ResourcesRunSnapshot
@@ -107,6 +120,7 @@ export type SchedulerRunSnapshot =
| InternetPathRunSnapshot
| CertificatesRenewRunSnapshot
| BackupsRunSnapshot
| GeoipUpdateRunSnapshot
| AlertEngineRunSnapshot
export interface TrafficServerSnapshot {
+4
View File
@@ -11,6 +11,7 @@ export const SCHEDULER_JOB_KEYS = [
"gre_bgp",
"certificates_renew",
"backups",
"geoip_update",
"alert_engine",
] as const
export type SchedulerJobKey = (typeof SCHEDULER_JOB_KEYS)[number]
@@ -25,6 +26,7 @@ export const SCHEDULER_JOB_LABELS: Record<string, string> = {
gre_bgp: "GRE + BGP",
certificates_renew: "Сертификаты: автообновление",
backups: "Бэкапы",
geoip_update: "GeoIP: базы GeoLite2",
alert_engine: "Оповещения",
}
@@ -43,6 +45,8 @@ export const SCHEDULER_JOB_DESCRIPTIONS: Record<string, string> = {
"Автообновление сертификатов, выпущенных через UI (ACME DNS-01 / Cloudflare). Отключается на странице «Сертификаты», если ACME ведёт RouterOS.",
backups:
"Плановые бэкапы RouterOS по расписанию со страницы «Бэкапы»; тик планировщика раз в минуту.",
geoip_update:
"Проверка и доставка GeoLite2 Country/ASN с зеркала P3TERX в backend/storage/geoip (ETag, атомарная подмена). Управление — в настройках NetFlow.",
alert_engine:
"Оценка правил по данным из PostgreSQL (сэмплы пишут джобы сбора, в т.ч. «GRE + BGP» и «Серверы: REST API»).",
}
+31
View File
@@ -0,0 +1,31 @@
import type { StatisticsPivotDim } from "@mmapp/contracts/statistics"
import type { StatisticsSliceKind } from "@/components/data-grids/statistics-breakdown-data-grid"
export const STATISTICS_DIMS: {
id: StatisticsSliceKind
pivot: StatisticsPivotDim
label: string
}[] = [
{ id: "users", pivot: "user", label: "Пользователи" },
{ id: "servers", pivot: "server", label: "Серверы" },
{ id: "interfaces", pivot: "iface", label: "Интерфейсы" },
{ id: "countries", pivot: "country", label: "Страны" },
{ id: "services", pivot: "service", label: "Сервисы" },
{ id: "asns", pivot: "asn", label: "ASN" },
]
export function isStatisticsSliceKind(v: string): v is StatisticsSliceKind {
return STATISTICS_DIMS.some((d) => d.id === v)
}
export function isStatisticsPivotDim(v: string): v is StatisticsPivotDim {
return STATISTICS_DIMS.some((d) => d.pivot === v)
}
export function sliceKindToPivot(kind: StatisticsSliceKind): StatisticsPivotDim {
return STATISTICS_DIMS.find((d) => d.id === kind)?.pivot ?? "user"
}
export function pivotToSliceKind(dim: StatisticsPivotDim): StatisticsSliceKind {
return STATISTICS_DIMS.find((d) => d.pivot === dim)?.id ?? "users"
}
+78 -15
View File
@@ -25,9 +25,11 @@
"clsx": "^2.1.1",
"cmdk": "^1.1.1",
"cn": "^0.2.5",
"date-fns": "^4.4.0",
"lucide-react": "^1.11.0",
"next": "16.2.4",
"react": "19.2.4",
"react-day-picker": "^10.0.1",
"react-dom": "19.2.4",
"recharts": "^3.8.0",
"shadcn": "^4.5.0",
@@ -61,6 +63,7 @@
"drizzle-orm": "^0.45.2",
"fastify": "^5.8.5",
"fastify-plugin": "^5.1.0",
"maxmind": "^5.0.7",
"pg": "^8.23.0",
"undici": "^8.1.0",
"zod": "^4.4.1"
@@ -893,6 +896,12 @@
}
}
},
"node_modules/@date-fns/tz": {
"version": "1.5.0",
"resolved": "https://registry.npmjs.org/@date-fns/tz/-/tz-1.5.0.tgz",
"integrity": "sha512-lwYN/vDPeNRULcepoE/LO2Pgx+7/RV+S9ARfbc9lr2DtGkOD7pAiruHvbR1RX3Qyf6ja47EWJDMsNK5vK08DJg==",
"license": "MIT"
},
"node_modules/@dnd-kit/accessibility": {
"version": "3.1.1",
"resolved": "https://registry.npmjs.org/@dnd-kit/accessibility/-/accessibility-3.1.1.tgz",
@@ -6690,6 +6699,16 @@
"url": "https://github.com/sponsors/ljharb"
}
},
"node_modules/date-fns": {
"version": "4.4.0",
"resolved": "https://registry.npmjs.org/date-fns/-/date-fns-4.4.0.tgz",
"integrity": "sha512-+1UMbeh68lH1SegH83CGWwpb6OHHbpSgr3+s5Eww5M4CAgswBpoWS0AjTOfEJ33HiYKz1hdj/KTFprzXHmq/6w==",
"license": "MIT",
"funding": {
"type": "github",
"url": "https://github.com/sponsors/kossnocorp"
}
},
"node_modules/dateformat": {
"version": "4.6.3",
"resolved": "https://registry.npmjs.org/dateformat/-/dateformat-4.6.3.tgz",
@@ -10386,6 +10405,20 @@
"node": ">= 0.4"
}
},
"node_modules/maxmind": {
"version": "5.0.7",
"resolved": "https://registry.npmjs.org/maxmind/-/maxmind-5.0.7.tgz",
"integrity": "sha512-+w637dwfv01MKjkrp4sKDBTEKHLPvWLYb647QTjiz3wG/teSemqudIKNShaS6eqZ7ffxC9oZlQQgIqY0rGojog==",
"license": "MIT",
"dependencies": {
"mmdb-lib": "3.0.3",
"tiny-lru": "13.0.0"
},
"engines": {
"node": ">=12",
"npm": ">=6"
}
},
"node_modules/media-typer": {
"version": "1.1.0",
"resolved": "https://registry.npmjs.org/media-typer/-/media-typer-1.1.0.tgz",
@@ -10531,6 +10564,16 @@
"integrity": "sha512-gKLcREMhtuZRwRAfqP3RFW+TK4JqApVBtOIftVgjuABpAtpxhPGaDcfvbhNvD0B8iD1oUr/txX35NjcaY6Ns/A==",
"license": "MIT"
},
"node_modules/mmdb-lib": {
"version": "3.0.3",
"resolved": "https://registry.npmjs.org/mmdb-lib/-/mmdb-lib-3.0.3.tgz",
"integrity": "sha512-xQPoBXcNjjHiOvOraFBKtA++uNWF6aCVHL9dRKFXEov8eI3QJwtgiw3qApsonFT5SpoqsEVISUTg3HIDs2DiXw==",
"license": "MIT",
"engines": {
"node": ">=10",
"npm": ">=6"
}
},
"node_modules/mnemonist": {
"version": "0.40.4",
"resolved": "https://registry.npmjs.org/mnemonist/-/mnemonist-0.40.4.tgz",
@@ -11848,6 +11891,32 @@
"node": ">=0.10.0"
}
},
"node_modules/react-day-picker": {
"version": "10.0.1",
"resolved": "https://registry.npmjs.org/react-day-picker/-/react-day-picker-10.0.1.tgz",
"integrity": "sha512-eNh6BlwcYInWaJtRv18mXQ06Ys/H6rdTZAnTaSdOYJuTpwP1JMCHNd1FDRadA+gbeinq+psdULN5Xnowy9mV8w==",
"license": "MIT",
"dependencies": {
"@date-fns/tz": "^1.4.1",
"date-fns": "^4.1.0"
},
"engines": {
"node": ">=18"
},
"funding": {
"type": "individual",
"url": "https://github.com/sponsors/gpbl"
},
"peerDependencies": {
"@types/react": ">=16.8.0",
"react": ">=16.8.0"
},
"peerDependenciesMeta": {
"@types/react": {
"optional": true
}
}
},
"node_modules/react-dom": {
"version": "19.2.4",
"resolved": "https://registry.npmjs.org/react-dom/-/react-dom-19.2.4.tgz",
@@ -13299,6 +13368,15 @@
"integrity": "sha512-+FbBPE1o9QAYvviau/qC5SE3caw21q3xkvWKBtja5vgqOWIHHJ3ioaq1VPfn/Szqctz2bU/oYeKd9/z5BL+PVg==",
"license": "MIT"
},
"node_modules/tiny-lru": {
"version": "13.0.0",
"resolved": "https://registry.npmjs.org/tiny-lru/-/tiny-lru-13.0.0.tgz",
"integrity": "sha512-xDHxKKS1FdF0Tv2P+QT7IeSEg74K/8cEDzbv3Tv6UyHHUgBOjOiQiBp818MGj66dhurQus/IBcoAbwIKtSGc6Q==",
"license": "BSD-3-Clause",
"engines": {
"node": ">=14"
}
},
"node_modules/tinyglobby": {
"version": "0.2.16",
"resolved": "https://registry.npmjs.org/tinyglobby/-/tinyglobby-0.2.16.tgz",
@@ -14797,21 +14875,6 @@
"dependencies": {
"zod": "^4.4.1"
}
},
"node_modules/@next/swc-win32-x64-msvc": {
"version": "16.2.4",
"resolved": "https://registry.npmjs.org/@next/swc-win32-x64-msvc/-/swc-win32-x64-msvc-16.2.4.tgz",
"integrity": "sha512-kMVGgsqhO5YTYODD9IPGGhA6iprWidQckK3LmPeW08PIFENRmgfb4MjXHO+p//d+ts2rpjvK5gXWzXSMrPl9cw==",
"cpu": [
"x64"
],
"optional": true,
"os": [
"win32"
],
"engines": {
"node": ">= 10"
}
}
}
}
+2
View File
@@ -27,9 +27,11 @@
"clsx": "^2.1.1",
"cmdk": "^1.1.1",
"cn": "^0.2.5",
"date-fns": "^4.4.0",
"lucide-react": "^1.11.0",
"next": "16.2.4",
"react": "19.2.4",
"react-day-picker": "^10.0.1",
"react-dom": "19.2.4",
"recharts": "^3.8.0",
"shadcn": "^4.5.0",
+8
View File
@@ -45,6 +45,14 @@
"./traffic-flow": {
"types": "./dist/traffic-flow.d.ts",
"default": "./dist/traffic-flow.js"
},
"./geoip": {
"types": "./dist/geoip.d.ts",
"default": "./dist/geoip.js"
},
"./statistics": {
"types": "./dist/statistics.d.ts",
"default": "./dist/statistics.js"
}
},
"dependencies": {
+51
View File
@@ -0,0 +1,51 @@
import { z } from "zod"
export const geoipSettingsDtoSchema = z.object({
enabled: z.boolean(),
updateIntervalSec: z.number().int().positive(),
lastCheckAt: z.string().nullable(),
lastSuccessAt: z.string().nullable(),
lastError: z.string().nullable(),
countryBuildAt: z.string().nullable(),
asnBuildAt: z.string().nullable(),
updatedAt: z.string(),
})
export const geoipSettingsPatchSchema = z.object({
enabled: z.boolean().optional(),
updateIntervalSec: z
.number()
.int()
.min(6 * 3600)
.max(30 * 86400)
.optional(),
})
export const geoipStatusDtoSchema = z.object({
ready: z.boolean(),
countryLoaded: z.boolean(),
asnLoaded: z.boolean(),
countryFile: z.string(),
asnFile: z.string(),
dir: z.string(),
running: z.boolean(),
settings: geoipSettingsDtoSchema,
})
export const geoipUpdateSnapshotDtoSchema = z.object({
v: z.number(),
job: z.literal("geoip_update"),
sampledAt: z.string(),
skipped: z.boolean().optional(),
fatalError: z.string().optional(),
checked: z.number().int(),
downloaded: z.number().int(),
skippedUnchanged: z.number().int(),
bytes: z.number().int().nonnegative(),
errors: z.array(z.string()),
})
export type GeoipSettingsDto = z.infer<typeof geoipSettingsDtoSchema>
export type GeoipSettingsPatch = z.infer<typeof geoipSettingsPatchSchema>
export type GeoipStatusDto = z.infer<typeof geoipStatusDtoSchema>
export type GeoipUpdateSnapshotDto = z.infer<typeof geoipUpdateSnapshotDtoSchema>
+2
View File
@@ -6,3 +6,5 @@ export * from "./backups.js"
export * from "./wireguard.js"
export * from "./users.js"
export * from "./traffic-flow.js"
export * from "./geoip.js"
export * from "./statistics.js"
+99
View File
@@ -0,0 +1,99 @@
import { z } from "zod"
export const STATISTICS_UNBOUND_USER_ID = "__unbound__"
export const statisticsPivotDimSchema = z.enum([
"country",
"service",
"asn",
"server",
"user",
"iface",
])
export const statisticsBreakdownRowSchema = z.object({
id: z.string(),
label: z.string(),
bytes: z.number().nonnegative(),
packets: z.number().nonnegative(),
bps: z.number().nonnegative(),
percent: z.number().nonnegative(),
})
export const statisticsSeriesPointSchema = z.object({
t: z.string(),
bytes: z.number().nonnegative(),
})
export const statisticsKpisSchema = z.object({
bytes: z.number().nonnegative(),
packets: z.number().nonnegative(),
avgBps: z.number().nonnegative(),
users: z.number().int().nonnegative(),
servers: z.number().int().nonnegative(),
ifaces: z.number().int().nonnegative(),
topCountry: z.string(),
topService: z.string(),
})
export const statisticsQuerySchema = z.object({
from: z.string().min(1),
to: z.string().min(1),
serverId: z.coerce.number().int().positive().optional(),
userId: z.string().min(1).optional(),
iface: z.string().min(1).optional(),
country: z.string().min(2).max(2).optional(),
service: z.string().min(1).optional(),
asn: z.coerce.number().int().optional(),
})
export const statisticsDtoSchema = z.object({
from: z.string(),
to: z.string(),
grain: z.enum(["hour", "day"]),
kpis: statisticsKpisSchema,
series: z.array(statisticsSeriesPointSchema),
users: z.array(statisticsBreakdownRowSchema),
servers: z.array(statisticsBreakdownRowSchema),
interfaces: z.array(statisticsBreakdownRowSchema),
countries: z.array(statisticsBreakdownRowSchema),
services: z.array(statisticsBreakdownRowSchema),
asns: z.array(statisticsBreakdownRowSchema),
})
export const statisticsPivotQuerySchema = statisticsQuerySchema.extend({
row: statisticsPivotDimSchema,
col: statisticsPivotDimSchema,
metric: z.enum(["bytes", "packets"]).default("bytes"),
})
export const statisticsPivotColumnSchema = z.object({
id: z.string(),
label: z.string(),
total: z.number().nonnegative(),
})
export const statisticsPivotRowSchema = z.object({
id: z.string(),
label: z.string(),
total: z.number().nonnegative(),
cells: z.record(z.string(), z.number().nonnegative()),
})
export const statisticsPivotDtoSchema = z.object({
rowDim: statisticsPivotDimSchema,
colDim: statisticsPivotDimSchema,
metric: z.enum(["bytes", "packets"]),
columns: z.array(statisticsPivotColumnSchema),
rows: z.array(statisticsPivotRowSchema),
otherBytes: z.number().nonnegative(),
})
export type StatisticsBreakdownRow = z.infer<typeof statisticsBreakdownRowSchema>
export type StatisticsSeriesPoint = z.infer<typeof statisticsSeriesPointSchema>
export type StatisticsKpis = z.infer<typeof statisticsKpisSchema>
export type StatisticsQuery = z.infer<typeof statisticsQuerySchema>
export type StatisticsDto = z.infer<typeof statisticsDtoSchema>
export type StatisticsPivotDim = z.infer<typeof statisticsPivotDimSchema>
export type StatisticsPivotQuery = z.infer<typeof statisticsPivotQuerySchema>
export type StatisticsPivotDto = z.infer<typeof statisticsPivotDtoSchema>
+2
View File
@@ -244,6 +244,8 @@ export const flowPurgeDtoSchema = z.object({
minuteStats: z.number().int().nonnegative(),
minuteDims: z.number().int().nonnegative(),
dailyDims: z.number().int().nonnegative(),
hourFacts: z.number().int().nonnegative().optional(),
dailyFacts: z.number().int().nonnegative().optional(),
}),
fileBytesBefore: z.number().int().nonnegative(),
fileBytesAfter: z.number().int().nonnegative(),
+27
View File
@@ -0,0 +1,27 @@
import type {
GeoipSettingsPatch,
GeoipStatusDto,
GeoipUpdateSnapshotDto,
} from "@mmapp/contracts/geoip"
import { requestJson } from "@/shared/api/http-client"
export async function getGeoipStatus(baseUrl: string): Promise<GeoipStatusDto> {
return requestJson<GeoipStatusDto>(baseUrl, "/api/geoip")
}
export async function putGeoipSettings(
baseUrl: string,
patch: GeoipSettingsPatch,
): Promise<{ ok: boolean; status: GeoipStatusDto }> {
return requestJson(baseUrl, "/api/geoip", {
method: "PUT",
body: JSON.stringify(patch),
})
}
export async function runGeoipUpdateNow(baseUrl: string): Promise<{
ok: boolean
snapshot: GeoipUpdateSnapshotDto
}> {
return requestJson(baseUrl, "/api/geoip/update", { method: "POST" })
}
+45
View File
@@ -0,0 +1,45 @@
import type {
StatisticsDto,
StatisticsPivotDto,
StatisticsPivotQuery,
StatisticsQuery,
} from "@mmapp/contracts/statistics"
import { requestJson } from "@/shared/api/http-client"
export type { StatisticsDto, StatisticsQuery, StatisticsPivotDto, StatisticsPivotQuery }
export { STATISTICS_UNBOUND_USER_ID } from "@mmapp/contracts/statistics"
export async function getStatistics(
baseUrl: string,
query: StatisticsQuery,
): Promise<StatisticsDto> {
const params = new URLSearchParams()
params.set("from", query.from)
params.set("to", query.to)
if (query.serverId != null) params.set("serverId", String(query.serverId))
if (query.userId) params.set("userId", query.userId)
if (query.iface) params.set("iface", query.iface)
if (query.country) params.set("country", query.country)
if (query.service) params.set("service", query.service)
if (query.asn != null) params.set("asn", String(query.asn))
return requestJson<StatisticsDto>(baseUrl, `/api/statistics?${params.toString()}`)
}
export async function getStatisticsPivot(
baseUrl: string,
query: StatisticsPivotQuery,
): Promise<StatisticsPivotDto> {
const params = new URLSearchParams()
params.set("from", query.from)
params.set("to", query.to)
params.set("row", query.row)
params.set("col", query.col)
params.set("metric", query.metric)
if (query.serverId != null) params.set("serverId", String(query.serverId))
if (query.userId) params.set("userId", query.userId)
if (query.iface) params.set("iface", query.iface)
if (query.country) params.set("country", query.country)
if (query.service) params.set("service", query.service)
if (query.asn != null) params.set("asn", String(query.asn))
return requestJson<StatisticsPivotDto>(baseUrl, `/api/statistics/pivot?${params.toString()}`)
}
+1 -1
View File
File diff suppressed because one or more lines are too long