Compare commits

..
6 Commits
Author SHA1 Message Date
Denozordec 564aae21f0 fix(statistics): выровнять трафик-данные с картой сети
Docker images / prepare-release (push) Successful in 10s
Docker images / backend-test (push) Successful in 2m40s
Docker images / frontend-image (push) Successful in 3m28s
Docker images / updater-image (push) Successful in 50s
Docker images / backend-image (push) Successful in 3m5s
Docker images / notify-webhook (push) Skipped
Docker images / publish-release (push) Successful in 10s
- Окно периода клампится к текущему моменту: «Сегодня»/«7 д» больше не делят байты на ещё не наступившие часы суток (avgBps/bps были занижены).
- Пресет «24 ч» шлёт точные ISO-границы «сейчас−24ч … сейчас» вместо дат «вчера…сегодня» (~48 ч); подпись диапазона и подсветка пресета сверяются по длительности окна.
- Таксономия сервисов как на карте: DNS/SSH/BGP/WireGuard/GRE и пустые метки сворачиваются в «Прочее» (normalizeFactService) в разбивке сервисов и pivot country×service.
- Тесты: кламп окна, ISO-24ч, будущее «to», невалидный диапазон, нормализация меток.
2026-09-12 13:22:06 +07:00
Denozordec 0fdd2fc1e1 feat(traffic-flow): раскрытие сервисов страны на карте сети
Docker images / prepare-release (push) Successful in 8s
Docker images / backend-test (push) Successful in 2m11s
Docker images / frontend-image (push) Successful in 3m14s
Docker images / updater-image (push) Successful in 43s
Docker images / backend-image (push) Successful in 2m44s
Docker images / notify-webhook (push) Skipped
Docker images / publish-release (push) Successful in 10s
- Контракт: countryServiceGroups (группа сервисов страны) в flowMapHopsDto, тип FlowMapCountryServiceGroup.
- Бэкенд: вложенная агрегация brand×ISO в том же проходе hops; рёбра страна→сервис, пути с реальным EN; cap 8 / min 4, доля от байтов страны.
- Хелпер mapCountryServiceNodeId: id вида cc:us|svc:google, сервисы разных стран не смешиваются.
- Карта сети: клик по стране сдвигает колонку стран и раскрывает столбец её сервисов; остальные страны приглушены; панели страны и вложенного сервиса.
- Тесты: группы US/US+NL, рёбра от страны, cap/min вложенного слоя; mock countryServiceGroups для демо-режима.
2026-09-12 12:47:30 +07:00
DenozordecandCursor f15a7348db feat(traffic-flow): enhance country handling in flow analytics
Docker images / prepare-release (push) Successful in 9s
Docker images / backend-test (push) Successful in 2m19s
Docker images / frontend-image (push) Successful in 3m26s
Docker images / updater-image (push) Successful in 47s
Docker images / backend-image (push) Successful in 2m23s
Docker images / notify-webhook (push) Skipped
Docker images / publish-release (push) Successful in 10s
- Introduced country-specific handling in traffic flow analytics, including new mappings for country nodes and edges.
- Added mock data for countries and their respective service paths to improve visualization in the Network Map.
- Updated the `countryName` function to utilize `Intl.DisplayNames` for better localization of country names.
- Enhanced tests to validate country handling and ensure accurate representation in flow analytics.

Co-authored-by: Cursor <[email protected]>
2026-09-12 11:38:37 +07:00
DenozordecandCursor f3c846201c feat(traffic-flow): enhance Instagram service handling and classification
Docker images / prepare-release (push) Successful in 9s
Docker images / backend-test (push) Successful in 2m34s
Docker images / frontend-image (push) Successful in 3m20s
Docker images / updater-image (push) Successful in 43s
Docker images / backend-image (push) Successful in 2m25s
Docker images / notify-webhook (push) Skipped
Docker images / publish-release (push) Successful in 13s
- Added support for Instagram in traffic flow resolution and classification, including new test cases to validate its behavior.
- Introduced `INSTAGRAM` brand handling in the traffic flow logic, ensuring accurate service identification for Instagram-related traffic.
- Updated existing tests to cover various scenarios involving Instagram, including IP resolution and service categorization.
- Enhanced the service brand icon component to include an Instagram icon for better visual representation.

Co-authored-by: Cursor <[email protected]>
2026-09-12 11:20:16 +07:00
DenozordecandCursor ee9804f1bb feat(traffic-flow): enhance flow analytics and IP handling
Docker images / prepare-release (push) Successful in 8s
Docker images / backend-test (push) Successful in 2m29s
Docker images / frontend-image (push) Successful in 3m20s
Docker images / updater-image (push) Successful in 51s
Docker images / backend-image (push) Successful in 2m40s
Docker images / notify-webhook (push) Skipped
Docker images / publish-release (push) Successful in 13s
- Introduced `canonicalIp` function to standardize IP address formats across the application, improving consistency in flow processing.
- Updated traffic flow analytics to utilize new endpoint resolution logic, enhancing accuracy in traffic classification.
- Enhanced tests for traffic flow analytics and IP handling, ensuring comprehensive coverage for new functionalities.
- Improved traffic flow destination resolution with additional test cases for various internet services, including Google and Fastly.

Co-authored-by: Cursor <[email protected]>
2026-09-12 01:51:23 +07:00
DenozordecandCursor 4ce6169d14 feat(traffic-flow): enhance traffic classification and map display
Docker images / prepare-release (push) Successful in 12s
Docker images / backend-test (push) Successful in 2m16s
Docker images / frontend-image (push) Successful in 2m42s
Docker images / updater-image (push) Successful in 43s
Docker images / backend-image (push) Successful in 2m30s
Docker images / notify-webhook (push) Skipped
Docker images / publish-release (push) Successful in 13s
- Added a new state variable `mapAsnLoaded` to track ASN loading status in the NetworkMapPage.
- Updated the page to conditionally display a message when ASN data is not loaded, improving user feedback.
- Enhanced traffic classification tests to include additional scenarios for Google and YouTube services, ensuring comprehensive coverage.
- Introduced a new function `pickMapInternetDest` to improve destination selection logic in traffic flow processing.
- Updated the traffic flow destination classification to handle new conditions and improve accuracy.

Co-authored-by: Cursor <[email protected]>
2026-09-12 01:07:26 +07:00
31 changed files with 1787 additions and 276 deletions
+462 -26
View File
@@ -27,7 +27,9 @@ import {
findServerByGreRemote,
greSourceWanIndexOnMap,
greTunnelProbe,
placeCountryServiceNodes,
placeServiceNodes,
SERVICE_COL_W,
type GreMapEdge,
type WanJhEdge,
} from "@/lib/network-map-layout"
@@ -55,8 +57,9 @@ import {
matchNetflowForWan,
type MatchedNetflowHop,
} from "@/lib/map-netflow-hops"
import type { FlowMapHop, FlowMapHopsDto, FlowMapService, FlowMapServiceEdge, FlowMapServicePath } from "@mmapp/contracts/traffic-flow"
import type { FlowMapCountryServiceGroup, FlowMapHop, FlowMapHopsDto, FlowMapService, FlowMapServiceEdge, FlowMapServicePath } from "@mmapp/contracts/traffic-flow"
import { ServiceBrandIcon } from "@/components/network-map/service-brand-icon"
import { CountryFlagSvg } from "@/components/network-map/country-flag-svg"
import { Button } from "@/components/ui/button"
import { StatusBadge } from "@/components/status-badge"
import { StatusDot } from "@/components/status-dot"
@@ -69,7 +72,7 @@ import {
import { cn } from "@/lib/utils"
import { formatServicePathLabel, formatServicePathTitle } from "@/lib/format-service-path-label"
import Link from "next/link"
import { Flag } from "@/components/flag"
import { Flag, countryName } from "@/components/flag"
// ─── Resource metrics (для мини-блока справа; числа детерминированы по id узла) ─
@@ -285,6 +288,12 @@ const MOCK_MAP_SERVICES: FlowMapService[] = [
{ id: "svc:aws", label: "AWS", category: "CDN", bytes: 9_000_000, bps: 3_600_000, share: 0.09 },
]
const MOCK_MAP_COUNTRIES: FlowMapService[] = [
{ id: "cc:us", label: "US", category: "Страна", bytes: 22_000_000, bps: 8_800_000, share: 0.38 },
{ id: "cc:nl", label: "NL", category: "Страна", bytes: 14_000_000, bps: 5_600_000, share: 0.31 },
{ id: "cc:de", label: "DE", category: "Страна", bytes: 9_000_000, bps: 3_600_000, share: 0.21 },
]
const MOCK_MAP_SERVICE_EDGES: FlowMapServiceEdge[] = [
{ fromId: "srv2", toId: "svc:google", bytes: 14_000_000, bps: 5_600_000, bpsFwd: 4_200_000, bpsRev: 1_400_000, clientName: "Alice", clients: [{ id: "u1", name: "Alice" }] },
{ fromId: "srv3", toId: "svc:google", bytes: 8_000_000, bps: 3_200_000, bpsFwd: 2_400_000, bpsRev: 800_000, clientName: "Bob", clients: [{ id: "u2", name: "Bob" }] },
@@ -293,6 +302,14 @@ const MOCK_MAP_SERVICE_EDGES: FlowMapServiceEdge[] = [
{ fromId: "srv3", toId: "svc:aws", bytes: 9_000_000, bps: 3_600_000, bpsFwd: 2_700_000, bpsRev: 900_000, clientName: "Bob", clients: [{ id: "u2", name: "Bob" }] },
]
const MOCK_MAP_COUNTRY_EDGES: FlowMapServiceEdge[] = [
{ fromId: "srv2", toId: "cc:us", bytes: 14_000_000, bps: 5_600_000, bpsFwd: 4_200_000, bpsRev: 1_400_000, clientName: "Alice", clients: [{ id: "u1", name: "Alice" }] },
{ fromId: "srv3", toId: "cc:us", bytes: 8_000_000, bps: 3_200_000, bpsFwd: 2_400_000, bpsRev: 800_000, clientName: "Bob", clients: [{ id: "u2", name: "Bob" }] },
{ fromId: "srv2", toId: "cc:nl", bytes: 9_000_000, bps: 3_600_000, bpsFwd: 2_800_000, bpsRev: 800_000, clientName: "Alice", clients: [{ id: "u1", name: "Alice" }] },
{ fromId: "srv3", toId: "cc:nl", bytes: 5_000_000, bps: 2_000_000, bpsFwd: 1_500_000, bpsRev: 500_000, clientName: "Bob", clients: [{ id: "u2", name: "Bob" }] },
{ fromId: "srv3", toId: "cc:de", bytes: 9_000_000, bps: 3_600_000, bpsFwd: 2_700_000, bpsRev: 900_000, clientName: "Bob", clients: [{ id: "u2", name: "Bob" }] },
]
const MOCK_MAP_SERVICE_PATHS: FlowMapServicePath[] = [
{ clientId: "u1", clientName: "Alice", viaId: "srv1", viaName: "mt-msk-core-01", enId: "srv2", enName: "mt-spb-edge-01", serviceId: "svc:google", bytes: 14_000_000, bps: 5_600_000 },
{ clientId: "u2", clientName: "Bob", viaId: "srv1", viaName: "mt-msk-core-01", enId: "srv3", enName: "mt-fra-edge-01", serviceId: "svc:google", bytes: 8_000_000, bps: 3_200_000 },
@@ -301,6 +318,56 @@ const MOCK_MAP_SERVICE_PATHS: FlowMapServicePath[] = [
{ clientId: "u2", clientName: "Bob", viaId: "srv1", viaName: "mt-msk-core-01", enId: "srv3", enName: "mt-fra-edge-01", serviceId: "svc:aws", bytes: 9_000_000, bps: 3_600_000 },
]
const MOCK_MAP_COUNTRY_PATHS: FlowMapServicePath[] = [
{ clientId: "u1", clientName: "Alice", viaId: "srv1", viaName: "mt-msk-core-01", enId: "srv2", enName: "mt-spb-edge-01", serviceId: "cc:us", bytes: 14_000_000, bps: 5_600_000 },
{ clientId: "u2", clientName: "Bob", viaId: "srv1", viaName: "mt-msk-core-01", enId: "srv3", enName: "mt-fra-edge-01", serviceId: "cc:us", bytes: 8_000_000, bps: 3_200_000 },
{ clientId: "u1", clientName: "Alice", viaId: "srv1", viaName: "mt-msk-core-01", enId: "srv2", enName: "mt-spb-edge-01", serviceId: "cc:nl", bytes: 9_000_000, bps: 3_600_000 },
{ clientId: "u2", clientName: "Bob", viaId: "srv1", viaName: "mt-msk-core-01", enId: "srv3", enName: "mt-fra-edge-01", serviceId: "cc:nl", bytes: 5_000_000, bps: 2_000_000 },
{ clientId: "u2", clientName: "Bob", viaId: "srv1", viaName: "mt-msk-core-01", enId: "srv3", enName: "mt-fra-edge-01", serviceId: "cc:de", bytes: 9_000_000, bps: 3_600_000 },
]
/** Доли — от байтов страны cc:us (22M из моков выше). */
const MOCK_COUNTRY_SERVICE_GROUPS: FlowMapCountryServiceGroup[] = [
{
countryId: "cc:us",
services: [
{ id: "cc:us|svc:google", label: "Google", category: "Веб", bytes: 12_000_000, bps: 4_800_000, share: 12 / 22 },
{ id: "cc:us|svc:cloudflare", label: "Cloudflare", category: "CDN", bytes: 7_000_000, bps: 2_800_000, share: 7 / 22 },
{ id: "cc:us|svc:aws", label: "AWS", category: "CDN", bytes: 3_000_000, bps: 1_200_000, share: 3 / 22 },
],
edges: [
{ fromId: "cc:us", toId: "cc:us|svc:google", bytes: 12_000_000, bps: 4_800_000, bpsFwd: 9_000_000, bpsRev: 3_000_000, clientName: "Alice", clients: [{ id: "u1", name: "Alice" }] },
{ fromId: "cc:us", toId: "cc:us|svc:cloudflare", bytes: 7_000_000, bps: 2_800_000, bpsFwd: 5_250_000, bpsRev: 1_750_000, clientName: "Alice", clients: [{ id: "u1", name: "Alice" }] },
{ fromId: "cc:us", toId: "cc:us|svc:aws", bytes: 3_000_000, bps: 1_200_000, bpsFwd: 2_250_000, bpsRev: 750_000, clientName: "Bob", clients: [{ id: "u2", name: "Bob" }] },
],
paths: [
{ clientId: "u1", clientName: "Alice", viaId: "srv1", viaName: "mt-msk-core-01", enId: "srv2", enName: "mt-spb-edge-01", serviceId: "cc:us|svc:google", bytes: 8_000_000, bps: 3_200_000 },
{ clientId: "u2", clientName: "Bob", viaId: "srv1", viaName: "mt-msk-core-01", enId: "srv3", enName: "mt-fra-edge-01", serviceId: "cc:us|svc:google", bytes: 4_000_000, bps: 1_600_000 },
{ clientId: "u1", clientName: "Alice", viaId: "srv1", viaName: "mt-msk-core-01", enId: "srv2", enName: "mt-spb-edge-01", serviceId: "cc:us|svc:cloudflare", bytes: 7_000_000, bps: 2_800_000 },
{ clientId: "u2", clientName: "Bob", viaId: "srv1", viaName: "mt-msk-core-01", enId: "srv3", enName: "mt-fra-edge-01", serviceId: "cc:us|svc:aws", bytes: 3_000_000, bps: 1_200_000 },
],
},
]
const DEST_MODE_KEY = "mm-network-map-dest-mode"
type DestMode = "services" | "countries"
function readDestMode(): DestMode {
if (typeof window === "undefined") return "services"
try {
return sessionStorage.getItem(DEST_MODE_KEY) === "countries" ? "countries" : "services"
} catch {
return "services"
}
}
function destDisplayLabel(node: FlowMapService | undefined, mode: DestMode, fallback = ""): string {
if (!node) return fallback
if (mode !== "countries") return node.label
if (node.id === "cc:other" || node.label === "Прочее") return "Прочее"
return countryName(node.label)
}
function servicePathKey(p: Pick<FlowMapServicePath, "clientId" | "viaId" | "enId" | "serviceId">): string {
return `${p.clientId}|${p.viaId}|${p.enId}|${p.serviceId}`
}
@@ -738,6 +805,10 @@ function ServiceNode({
isSel,
isVis,
isDragged,
destMode,
iso,
dim,
shareLabel,
onClick,
onMouseDown,
}: {
@@ -748,20 +819,27 @@ function ServiceNode({
isSel: boolean
isVis: boolean
isDragged: boolean
destMode: DestMode
iso?: string
/** Приглушение узла при раскрытии другой страны (остаётся на холсте). */
dim?: boolean
/** Подпись доли в tooltip: у вложенных сервисов — доля страны, не окна. */
shareLabel?: string
onClick: () => void
onMouseDown: (e: React.MouseEvent) => void
}) {
const bw = MAP_SERVICE_NODE_W
const bh = MAP_SERVICE_NODE_H
const flagIso = destMode === "countries" && iso && iso !== "Прочее" ? iso : ""
return (
<g
transform={`translate(${x},${y})`}
style={{ cursor: isDragged ? "grabbing" : "grab", transition: isDragged ? "none" : "opacity 0.25s" }}
opacity={isVis ? 1 : 0.08}
opacity={isVis ? (dim ? 0.35 : 1) : 0.08}
onMouseDown={(e) => { e.stopPropagation(); onMouseDown(e) }}
onClick={(e) => { e.stopPropagation(); onClick() }}
>
<title>{`${label} · ${serviceSharePct(share)} payload окна`}</title>
<title>{`${label} · ${serviceSharePct(share)} ${shareLabel ?? "payload окна"}`}</title>
{isSel && (
<rect
x={-bw / 2 - 6}
@@ -786,7 +864,9 @@ function ServiceNode({
strokeWidth={isSel ? 2.2 : 1.4}
/>
<g transform="translate(-11,-24)" pointerEvents="none">
<ServiceBrandIcon label={label} size={22} />
{flagIso
? <CountryFlagSvg iso={flagIso} size={22} />
: <ServiceBrandIcon label={destMode === "countries" ? "Прочее" : label} size={22} />}
</g>
<text textAnchor="middle" y="14" fontSize="8.5" fontWeight="700" fill="#e0f2fe" fontFamily="ui-monospace,monospace">
{label}
@@ -804,6 +884,7 @@ function ServicePathList({
services,
highlight,
viaMode,
destMode,
onToggle,
}: {
paths: FlowMapServicePath[]
@@ -811,6 +892,7 @@ function ServicePathList({
services: FlowMapService[]
highlight: { viaId: string; enId: string; serviceId: string } | null
viaMode: "via" | "service"
destMode: DestMode
onToggle: (p: FlowMapServicePath) => void
}) {
if (paths.length === 0) {
@@ -823,13 +905,14 @@ function ServicePathList({
const via = servers.find((s) => s.id === p.viaId)
const en = servers.find((s) => s.id === p.enId)
const svc = services.find((s) => s.id === p.serviceId)
const destLabel = destDisplayLabel(svc, destMode, p.serviceId)
const label = formatServicePathLabel(p, viaMode, {
viaName: via?.name,
viaSite: via?.site,
enName: en?.name,
serviceLabel: svc?.label,
serviceLabel: destLabel,
})
const title = formatServicePathTitle(label, svc?.label ?? p.serviceId)
const title = formatServicePathTitle(label, destLabel)
const active = Boolean(
highlight
&& highlight.viaId === p.viaId
@@ -1099,10 +1182,16 @@ export default function NetworkMapPage() {
const [mapServices, setMapServices] = useState<FlowMapService[]>([])
const [mapServiceEdges, setMapServiceEdges] = useState<FlowMapServiceEdge[]>([])
const [mapServicePaths, setMapServicePaths] = useState<FlowMapServicePath[]>([])
const [mapCountries, setMapCountries] = useState<FlowMapService[]>([])
const [mapCountryEdges, setMapCountryEdges] = useState<FlowMapServiceEdge[]>([])
const [mapCountryPaths, setMapCountryPaths] = useState<FlowMapServicePath[]>([])
const [mapCountryServiceGroups, setMapCountryServiceGroups] = useState<FlowMapCountryServiceGroup[]>([])
const [mapSharePct, setMapSharePct] = useState(5)
const [mapNamedBytes, setMapNamedBytes] = useState(0)
const [mapTotalBytes, setMapTotalBytes] = useState(0)
const [mapWindowSec, setMapWindowSec] = useState(300)
const [mapAsnLoaded, setMapAsnLoaded] = useState(true)
const [mapCountryLoaded, setMapCountryLoaded] = useState(true)
/** FQDN из GRE outer → IPv4 (ответ POST /api/network/resolve-hosts), для матчинга с WAN. */
const [greResolvedIpv4ByHost, setGreResolvedIpv4ByHost] = useState<Record<string, string>>({})
const [dataError, setDataError] = useState<string | null>(null)
@@ -1198,9 +1287,14 @@ export default function NetworkMapPage() {
setMapServices(MOCK_MAP_SERVICES)
setMapServiceEdges(MOCK_MAP_SERVICE_EDGES)
setMapServicePaths(MOCK_MAP_SERVICE_PATHS)
setMapCountries(MOCK_MAP_COUNTRIES)
setMapCountryEdges(MOCK_MAP_COUNTRY_EDGES)
setMapCountryPaths(MOCK_MAP_COUNTRY_PATHS)
setMapCountryServiceGroups(MOCK_COUNTRY_SERVICE_GROUPS)
setMapSharePct(5)
setMapNamedBytes(0)
setMapTotalBytes(0)
setMapCountryLoaded(true)
setDataError(null)
})
return
@@ -1229,8 +1323,40 @@ export default function NetworkMapPage() {
// ── Interaction ─────────────────────────────────────────────────────────────
const [selected, setSelected] = useState<Server | null>(null)
const [selectedService, setSelectedService] = useState<FlowMapService | null>(null)
/** Раскрытая страна (режим «Страны»): справа столбец её сервисов. Не персистится. */
const [expandedCountryId, setExpandedCountryId] = useState<string | null>(null)
const [destMode, setDestModeState] = useState<DestMode>("services")
useEffect(() => {
queueMicrotask(() => setDestModeState(readDestMode()))
}, [])
function setDestMode(mode: DestMode) {
setDestModeState(mode)
setSelectedService(null)
setExpandedCountryId(null)
setHighlightedPath(null)
try { sessionStorage.setItem(DEST_MODE_KEY, mode) } catch { /* private mode */ }
}
const expandedCountryGroup = useMemo(
() => destMode === "countries" && expandedCountryId
? mapCountryServiceGroups.find((g) => g.countryId === expandedCountryId) ?? null
: null,
[destMode, expandedCountryId, mapCountryServiceGroups],
)
const nestedServices = useMemo(() => expandedCountryGroup?.services ?? [], [expandedCountryGroup])
const liveSelectedService = selectedService
? (mapServices.find((s) => s.id === selectedService.id) ?? selectedService)
? (
(destMode === "countries" ? mapCountries : mapServices)
.find((s) => s.id === selectedService.id)
?? nestedServices.find((s) => s.id === selectedService.id)
?? selectedService
)
: null
const selectedNestedService = liveSelectedService
&& nestedServices.some((s) => s.id === liveSelectedService.id)
? liveSelectedService
: null
const expandedCountry = expandedCountryId
? mapCountries.find((c) => c.id === expandedCountryId) ?? null
: null
const [highlightedPath, setHighlightedPath] = useState<{ viaId: string; enId: string; serviceId: string } | null>(null)
const [selWanIdx, setSelWanIdx] = useState<number | null>(null)
@@ -1282,7 +1408,12 @@ export default function NetworkMapPage() {
setMapServices(MOCK_MAP_SERVICES)
setMapServiceEdges(MOCK_MAP_SERVICE_EDGES)
setMapServicePaths(MOCK_MAP_SERVICE_PATHS)
setMapCountries(MOCK_MAP_COUNTRIES)
setMapCountryEdges(MOCK_MAP_COUNTRY_EDGES)
setMapCountryPaths(MOCK_MAP_COUNTRY_PATHS)
setMapCountryServiceGroups(MOCK_COUNTRY_SERVICE_GROUPS)
setMapSharePct(5)
setMapCountryLoaded(true)
})
return
}
@@ -1292,6 +1423,10 @@ export default function NetworkMapPage() {
setMapServices([])
setMapServiceEdges([])
setMapServicePaths([])
setMapCountries([])
setMapCountryEdges([])
setMapCountryPaths([])
setMapCountryServiceGroups([])
})
return
}
@@ -1307,9 +1442,15 @@ export default function NetworkMapPage() {
setMapServices(res.services ?? [])
setMapServiceEdges(res.serviceEdges ?? [])
setMapServicePaths(res.servicePaths ?? [])
setMapCountries(res.countries ?? [])
setMapCountryEdges(res.countryEdges ?? [])
setMapCountryPaths(res.countryPaths ?? [])
setMapCountryServiceGroups(res.countryServiceGroups ?? [])
if (res.mapServiceMinSharePct != null) setMapSharePct(res.mapServiceMinSharePct)
setMapNamedBytes(res.namedBytes ?? 0)
setMapTotalBytes(res.totalBytes ?? 0)
if (res.asnLoaded != null) setMapAsnLoaded(res.asnLoaded)
if (res.countryLoaded != null) setMapCountryLoaded(res.countryLoaded)
if (res.windowSec) setMapWindowSec(res.windowSec)
})
.catch((err: unknown) => {
@@ -1513,9 +1654,13 @@ export default function NetworkMapPage() {
return m
}, [homeRouters, wanJhEdges, mapHops, showNetflow])
const visibleMapServices = showServices ? mapServices : []
const destNodes = destMode === "countries" ? mapCountries : mapServices
const destEdges = destMode === "countries" ? mapCountryEdges : mapServiceEdges
const destPaths = destMode === "countries" ? mapCountryPaths : mapServicePaths
const visibleMapServices = showServices ? destNodes : []
const visibleServiceEdges = showServices
? drawableServiceEdges(visibleMapServices, mapServiceEdges, mapServers, greEdges, nodePosById)
? drawableServiceEdges(visibleMapServices, destEdges, mapServers, greEdges, nodePosById)
: []
const nodes = mapServers
@@ -1537,8 +1682,24 @@ export default function NetworkMapPage() {
.map((s) => nodePosById[s.id])
.filter((p): p is { x: number; y: number } => Boolean(p)),
)
// Раскрытая страна: колонка стран уходит влево, правый x занимает столбец её сервисов.
const countryColShift = expandedCountryGroup ? SERVICE_COL_W : 0
const autoDestPos = countryColShift
? Object.fromEntries(
Object.entries(autoServicePos).map(([id, p]) => [id, { x: p.x - countryColShift, y: p.y }]),
)
: autoServicePos
const servicePosById = Object.fromEntries(
visibleMapServices.map((s) => [s.id, servicePositions[s.id] ?? autoServicePos[s.id]!]),
visibleMapServices.map((s) => [s.id, servicePositions[s.id] ?? autoDestPos[s.id]!]),
)
const autoNestedPos = placeCountryServiceNodes(
nestedServices.map((s) => s.id),
expandedCountryId ? servicePosById[expandedCountryId] ?? autoDestPos[expandedCountryId] : undefined,
)
const nestedPosById = Object.fromEntries(
nestedServices
.map((s) => [s.id, servicePositions[s.id] ?? autoNestedPos[s.id]] as const)
.filter((entry): entry is readonly [string, { x: number; y: number }] => Boolean(entry[1])),
)
// ── Refs ─────────────────────────────────────────────────────────────────────
@@ -1623,6 +1784,7 @@ export default function NetworkMapPage() {
setSelectedGreEdge(null)
setSelectedService(null)
setHighlightedPath(null)
setExpandedCountryId(null)
}
if (e.key === "=" || e.key === "+") applyZoomCenter(1.25)
if (e.key === "-") applyZoomCenter(1 / 1.25)
@@ -1720,6 +1882,7 @@ export default function NetworkMapPage() {
setSelectedGreEdge(null)
setSelectedService(null)
setHighlightedPath(null)
setExpandedCountryId(null)
}
}
@@ -1754,6 +1917,34 @@ export default function NetworkMapPage() {
}
// ── Side panel ────────────────────────────────────────────────────────────
useEffect(() => {
if (
expandedCountryId
&& (!mapCountries.some((s) => s.id === expandedCountryId)
|| !mapCountryServiceGroups.some((g) => g.countryId === expandedCountryId))
) {
queueMicrotask(() => setExpandedCountryId(null))
}
}, [mapCountries, mapCountryServiceGroups, expandedCountryId])
useEffect(() => {
if (!selectedService) return
const stillVisible =
destNodes.some((s) => s.id === selectedService.id)
|| nestedServices.some((s) => s.id === selectedService.id)
if (!stillVisible) {
queueMicrotask(() => {
setSelectedService(null)
setHighlightedPath(null)
})
}
}, [destMode, destNodes, nestedServices, selectedService])
useEffect(() => {
if (!showServices) queueMicrotask(() => setExpandedCountryId(null))
}, [showServices])
// Сдвиг колонки стран меняет систему координат: сбрасываем drag-овчины сервисов.
useEffect(() => {
queueMicrotask(() => setServicePositions({}))
}, [expandedCountryId])
function selectServer(s: Server) {
setSelectedGreEdge(null)
setSelectedService(null)
@@ -1768,6 +1959,10 @@ export default function NetworkMapPage() {
setSelWanIdx(null)
setHighlightedPath(null)
setSelectedService((prev: FlowMapService | null) => prev?.id === svc.id ? null : svc)
// Клик по стране в режиме «Страны» раскрывает столбец её сервисов; повторный — сворачивает.
if (destMode === "countries" && svc.id.startsWith("cc:") && !svc.id.includes("|")) {
setExpandedCountryId((prev) => (prev === svc.id ? null : svc.id))
}
}
function selectWan(s: Server, wanIdx: number) {
setSelectedGreEdge(null)
@@ -1917,6 +2112,27 @@ export default function NetworkMapPage() {
)}
</div>
{/* Dest overlay: services vs countries */}
<div className="flex items-center gap-0.5 rounded-md border border-border bg-muted/40 p-0.5">
{([
{ value: "services" as const, label: "Сервисы" },
{ value: "countries" as const, label: "Страны" },
]).map((b) => (
<button
key={b.value}
onClick={() => setDestMode(b.value)}
className={cn(
"px-2.5 py-1 text-xs rounded transition-colors whitespace-nowrap",
destMode === b.value
? "bg-background text-foreground shadow-sm"
: "text-muted-foreground hover:text-foreground",
)}
>
{b.label}
</button>
))}
</div>
{/* Layers dropdown */}
<div className="relative">
<button
@@ -1935,7 +2151,7 @@ export default function NetworkMapPage() {
{([
{ key: "showPingBadges", label: "Ping-значки", val: showPingBadges, set: setShowPingBadges, hint: "P" },
{ key: "showNetflow", label: "NetFlow", val: showNetflow, set: setShowNetflow, hint: "" },
{ key: "showServices", label: "Сервисы", val: showServices, set: setShowServices, hint: "" },
{ key: "showServices", label: "Назначения", val: showServices, set: setShowServices, hint: "" },
{ key: "showAnimDots", label: "Анимация трафика", val: showAnimDots, set: setShowAnimDots, hint: "" },
{ key: "showMinimap", label: "Минимап", val: showMinimap, set: setShowMinimap, hint: "M" },
{ key: "showHints", label: "Горячие клавиши", val: showHints, set: setShowHints, hint: "" },
@@ -1962,9 +2178,14 @@ export default function NetworkMapPage() {
))}
<p className="px-3 pt-1.5 pb-1 text-[10px] text-muted-foreground leading-snug">
{mapSharePct > 0
? `Порог доли сервиса${mapSharePct}% · Настройки → NetFlow`
: "Порог доли выключен (все бренды, макс. 20) · Настройки → NetFlow"}
? `Порог доли ≥ ${mapSharePct}% · Настройки → NetFlow`
: "Порог доли выключен (все узлы, макс. 20) · Настройки → NetFlow"}
</p>
{destMode === "countries" && !mapCountryLoaded && (
<p className="px-3 pb-1 text-[10px] text-amber-500 leading-snug">
GeoIP Country не загружен, страны из RIPE-кэша
</p>
)}
{(Object.keys(nodePositions).length > 0 || Object.keys(satPositions).length > 0 || Object.keys(servicePositions).length > 0) && (
<div className="border-t border-border/50 mt-1 pt-1">
<button
@@ -2281,6 +2502,32 @@ export default function NetworkMapPage() {
{visibleMapServices.map((svc) => {
const pos = servicePosById[svc.id]
if (!pos) return null
return (
<ServiceNode
key={svc.id}
label={destDisplayLabel(svc, destMode)}
share={svc.share}
x={pos.x}
y={pos.y}
isSel={selectedService?.id === svc.id}
isVis
dim={Boolean(expandedCountryGroup) && svc.id !== expandedCountryId}
isDragged={draggedSvcId === svc.id}
destMode={destMode}
iso={svc.label}
onMouseDown={(e) => onServiceMouseDown(e, svc.id, pos.x, pos.y)}
onClick={() => {
if (suppressClickRef.current) { suppressClickRef.current = false; return }
selectService(svc)
}}
/>
)
})}
{/* ── Сервисы раскрытой страны (второй столбец справа) ── */}
{expandedCountryGroup && nestedServices.map((svc) => {
const pos = nestedPosById[svc.id]
if (!pos) return null
return (
<ServiceNode
key={svc.id}
@@ -2291,6 +2538,8 @@ export default function NetworkMapPage() {
isSel={selectedService?.id === svc.id}
isVis
isDragged={draggedSvcId === svc.id}
destMode="services"
shareLabel="трафика страны"
onMouseDown={(e) => onServiceMouseDown(e, svc.id, pos.x, pos.y)}
onClick={() => {
if (suppressClickRef.current) { suppressClickRef.current = false; return }
@@ -2330,15 +2579,16 @@ export default function NetworkMapPage() {
&& highlightedPath.serviceId === edge.toId,
)
const pathDim = Boolean(highlightedPath) && !pathHit
const hl = pathHit || (!highlightedPath && (selectedService?.id === edge.toId || selected?.id === edge.fromId))
const hl = pathHit || (!highlightedPath && (selectedService?.id === edge.toId || selected?.id === edge.fromId || expandedCountryId === edge.toId))
const countryDim = !hl && Boolean(expandedCountryGroup) && edge.toId !== expandedCountryId
const svc = visibleMapServices.find((s) => s.id === edge.toId)
const enName = mapServers.find((s) => s.id === edge.fromId)?.name ?? edge.fromId
const clientLabel = (edge.clients?.map((c) => c.name).filter(Boolean).join(", ") || edge.clientName || "—")
const pathTitle = `${clientLabel}${enName}${svc?.label ?? edge.toId}`
const pathTitle = `${clientLabel}${enName}${destDisplayLabel(svc, destMode, edge.toId)}`
return (
<g
key={`${edge.fromId}|${edge.toId}`}
opacity={pathDim ? 0.12 : hl ? 1 : 0.72}
opacity={pathDim ? 0.12 : hl ? 1 : countryDim ? 0.35 : 0.72}
style={{ transition: "opacity 0.3s" }}
>
<title>{pathTitle}</title>
@@ -2384,6 +2634,81 @@ export default function NetworkMapPage() {
)
})}
{/* ── Раскрытая страна → её сервисы ── */}
{expandedCountryGroup?.edges.map((edge) => {
const from = servicePosById[edge.fromId]
const to = nestedPosById[edge.toId]
if (!from || !to) return null
const hop: MatchedNetflowHop = {
bytes: edge.bytes,
bps: edge.bps,
bpsFwd: edge.bpsFwd,
bpsRev: edge.bpsRev,
}
const clipped = clipSegmentCircleToRect(
from.x,
from.y,
MAP_SERVICE_NODE_W / 2,
to.x,
to.y,
MAP_SERVICE_NODE_W / 2,
MAP_SERVICE_NODE_H / 2,
)
const { mx, my } = edgeBadgePosition(clipped.x1, clipped.y1, clipped.x2, clipped.y2, 0.55, 16)
const hl = selectedService?.id === edge.toId
const svc = nestedServices.find((s) => s.id === edge.toId)
const country = mapCountries.find((s) => s.id === edge.fromId)
const clientLabel = (edge.clients?.map((c) => c.name).filter(Boolean).join(", ") || edge.clientName || "—")
const pathTitle = `${clientLabel}${destDisplayLabel(country, "countries", edge.fromId)}${edge.toId.split("|")[1] ?? edge.toId}`
return (
<g
key={`${edge.fromId}|${edge.toId}`}
opacity={hl ? 1 : 0.72}
style={{ transition: "opacity 0.3s" }}
>
<title>{pathTitle}</title>
<line
x1={clipped.x1} y1={clipped.y1} x2={clipped.x2} y2={clipped.y2}
stroke="#22d3ee"
strokeWidth={hopHasRate(hop) ? 2 : 1.3}
strokeDasharray="4 4"
opacity="0.9"
pointerEvents="none"
/>
<line
x1={clipped.x1} y1={clipped.y1} x2={clipped.x2} y2={clipped.y2}
stroke="#00000000"
strokeWidth={14}
strokeLinecap="round"
style={{ cursor: "pointer" }}
onPointerDown={(ev) => { ev.stopPropagation() }}
onClick={(ev) => {
ev.stopPropagation()
if (svc) selectService(svc)
}}
/>
{showAnimDots && hopHasRate(hop) && (
<circle r="3" fill="#67e8f9" opacity="0.85" pointerEvents="none">
<animateMotion dur="2.6s" repeatCount="indefinite"
path={`M ${clipped.x1} ${clipped.y1} L ${clipped.x2} ${clipped.y2}`} />
</circle>
)}
{hopHasRate(hop) && (
<NetflowRateBadge
mx={mx}
my={my}
hop={hop}
onOpen={(ev) => {
ev.stopPropagation()
const hit = nestedServices.find((s) => s.id === edge.toId)
if (hit) selectService(hit)
}}
/>
)}
</g>
)
})}
{/* ── Hover tooltip ── */}
{hoveredNode && !isDragging && (
<SvgTooltip n={hoveredNode} />
@@ -2422,7 +2747,9 @@ export default function NetworkMapPage() {
<g transform="translate(10, 164)">
<rect width="14" height="14" rx="4" fill="#08202c" stroke="#22d3ee" strokeWidth="1.2" />
<text x="22" y="11" fontSize="8.5" fill="#cbd5e1" fontFamily="system-ui">Сервис</text>
<text x="22" y="11" fontSize="8.5" fill="#cbd5e1" fontFamily="system-ui">
{destMode === "countries" ? "Страна" : "Сервис"}
</text>
</g>
<line x1="10" y1="186" x2="130" y2="186" stroke="rgba(255,255,255,0.07)" strokeWidth="1" />
@@ -2486,7 +2813,7 @@ export default function NetworkMapPage() {
satPos={effectiveSatPos}
wanJhEdges={visibleWanJhEdges}
homeRouters={homeRouters}
servicePos={servicePosById}
servicePos={{ ...servicePosById, ...nestedPosById }}
onClose={() => setShowMinimap(false)}
onPan={(x, y) => setPan({ x, y })}
/>
@@ -2721,16 +3048,87 @@ export default function NetworkMapPage() {
})()}
</div>
</>
) : selectedNestedService ? (
<>
<div className="flex items-start gap-2 px-4 py-3 border-b">
<div className="mt-0.5">
<ServiceBrandIcon label={selectedNestedService.label} size={22} />
</div>
<div className="flex-1 min-w-0">
<p className="font-mono font-semibold text-sm truncate">
{selectedNestedService.label}
</p>
<p className="text-xs text-muted-foreground mt-0.5">
{`Сервис в ${destDisplayLabel(expandedCountry ?? undefined, "countries")} · ${selectedNestedService.category}`}
</p>
</div>
<button
type="button"
onClick={() => setSelectedService(null)}
className="text-muted-foreground hover:text-foreground transition-colors"
>
<XIcon className="size-4" />
</button>
</div>
<div className="flex-1 overflow-y-auto px-4 py-4 flex flex-col gap-4">
<div className="flex flex-col gap-0">
<div className="flex items-center justify-between py-2 border-b border-border/50">
<span className="text-xs text-muted-foreground">Доля страны</span>
<span className="text-xs font-mono font-medium text-cyan-400">{serviceSharePct(selectedNestedService.share)}</span>
</div>
{mapTotalBytes > 0 && (
<div className="flex items-center justify-between py-2 border-b border-border/50">
<span className="text-xs text-muted-foreground">Доля окна</span>
<span className="text-xs font-mono font-medium text-cyan-400">
{serviceSharePct(selectedNestedService.bytes / mapTotalBytes)}
</span>
</div>
)}
<div className="flex items-center justify-between py-2 border-b border-border/50">
<span className="text-xs text-muted-foreground">Скорость</span>
<span className="text-xs font-mono font-medium">
{formatNetflowRate({
bytes: selectedNestedService.bytes,
bps: selectedNestedService.bps,
bpsFwd: selectedNestedService.bps,
bpsRev: 0,
})}
</span>
</div>
</div>
<div>
<p className="text-xs font-semibold text-muted-foreground uppercase tracking-wider mb-2">Выход</p>
<ServicePathList
paths={(expandedCountryGroup?.paths ?? [])
.filter((p) => p.serviceId === selectedNestedService.id)
.slice()
.sort((a, b) => b.bps - a.bps)}
servers={mapServers}
services={nestedServices}
highlight={highlightedPath}
viaMode="via"
destMode="services"
onToggle={togglePathHighlight}
/>
</div>
</div>
</>
) : liveSelectedService ? (
<>
<div className="flex items-start gap-2 px-4 py-3 border-b">
<div className="mt-0.5">
<ServiceBrandIcon label={liveSelectedService.label} size={22} />
{destMode === "countries" && liveSelectedService.label !== "Прочее" && liveSelectedService.id !== "cc:other"
? <Flag code={liveSelectedService.label} size={22} />
: <ServiceBrandIcon label={destMode === "countries" ? "Прочее" : liveSelectedService.label} size={22} />}
</div>
<div className="flex-1 min-w-0">
<p className="font-mono font-semibold text-sm truncate">{liveSelectedService.label}</p>
<p className="font-mono font-semibold text-sm truncate">
{destDisplayLabel(liveSelectedService, destMode)}
</p>
<p className="text-xs text-muted-foreground mt-0.5">
Конечный сервис · {liveSelectedService.category}
{destMode === "countries"
? `Конечная страна${liveSelectedService.label !== "Прочее" ? ` · ${liveSelectedService.label}` : ""}`
: `Конечный сервис · ${liveSelectedService.category}`}
</p>
</div>
<button
@@ -2767,6 +3165,18 @@ export default function NetworkMapPage() {
</span>
</div>
)}
{!mapAsnLoaded && destMode === "services" && (
<div className="flex items-center justify-between py-2 border-b border-border/50">
<span className="text-xs text-muted-foreground">GeoLite2 ASN</span>
<span className="text-xs font-mono font-medium text-amber-500">не загружена</span>
</div>
)}
{destMode === "countries" && !mapCountryLoaded && (
<div className="flex items-center justify-between py-2 border-b border-border/50">
<span className="text-xs text-muted-foreground">GeoIP Country</span>
<span className="text-xs font-mono font-medium text-amber-500">RIPE-кэш</span>
</div>
)}
<div className="flex items-center justify-between py-2 border-b border-border/50">
<span className="text-xs text-muted-foreground">Скорость</span>
<span className="text-xs font-mono font-medium">
@@ -2779,12 +3189,36 @@ export default function NetworkMapPage() {
</span>
</div>
</div>
{expandedCountryGroup && liveSelectedService.id === expandedCountryGroup.countryId && (
<div>
<p className="text-xs font-semibold text-muted-foreground uppercase tracking-wider mb-2">Сервисы в стране</p>
<div className="flex flex-col gap-1">
{expandedCountryGroup.services.map((svc) => (
<button
key={svc.id}
type="button"
onClick={() => selectService(svc)}
className={cn(
"flex items-center justify-between gap-2 rounded-md px-2 py-1.5 text-left text-xs transition-colors",
selectedService?.id === svc.id ? "bg-cyan-500/15 ring-1 ring-cyan-500/40" : "hover:bg-muted/50",
)}
>
<span className="flex items-center gap-1.5 min-w-0">
<ServiceBrandIcon label={svc.label} size={14} />
<span className="font-mono truncate">{svc.label}</span>
</span>
<span className="font-mono text-cyan-400 tabular-nums shrink-0">{serviceSharePct(svc.share)}</span>
</button>
))}
</div>
</div>
)}
<div>
<p className="text-xs font-semibold text-muted-foreground uppercase tracking-wider mb-2">Выход</p>
<div className="flex flex-col gap-3">
{visibleServiceEdges.filter((e) => e.toId === liveSelectedService.id).map((e) => {
const src = mapServers.find((s) => s.id === e.fromId)
const enPaths = mapServicePaths
const enPaths = destPaths
.filter((p) => p.serviceId === liveSelectedService.id && p.enId === e.fromId)
.slice()
.sort((a, b) => b.bps - a.bps)
@@ -2799,9 +3233,10 @@ export default function NetworkMapPage() {
<ServicePathList
paths={enPaths}
servers={mapServers}
services={mapServices}
services={destNodes}
highlight={highlightedPath}
viaMode="via"
destMode={destMode}
onToggle={togglePathHighlight}
/>
</div>
@@ -3040,7 +3475,7 @@ export default function NetworkMapPage() {
<div>
<p className="text-xs font-semibold text-muted-foreground uppercase tracking-wider mb-2">Пути</p>
<ServicePathList
paths={mapServicePaths
paths={destPaths
.filter((p) => (
selected.type === "exit-node"
? p.enId === selected.id
@@ -3049,9 +3484,10 @@ export default function NetworkMapPage() {
.slice()
.sort((a, b) => b.bps - a.bps)}
servers={mapServers}
services={mapServices}
services={destNodes}
highlight={highlightedPath}
viaMode="service"
destMode={destMode}
onToggle={togglePathHighlight}
/>
</div>
@@ -1,5 +1,5 @@
import assert from "node:assert/strict"
import { getStatistics, getStatisticsPivot, parseStatisticsPeriod, pivotDimsConflict } from "./statistics-aggregate.js"
import { getStatistics, getStatisticsPivot, normalizeFactService, parseStatisticsPeriod, pivotDimsConflict } from "./statistics-aggregate.js"
import { rememberServerIfaces, resetIfaceCacheForTests } from "./traffic-flow-ifindex.js"
import { setRefreshIfacesForTests } from "./traffic-flow-ifaces.js"
import { invalidateFlowCatalogCache } from "./traffic-flow-topology.js"
@@ -24,6 +24,43 @@ import { STATISTICS_UNBOUND_USER_ID } from "@mmapp/contracts/statistics"
assert.equal(pivotDimsConflict("country", "service"), false)
}
{
const nowMs = Date.parse("2026-09-12T12:00:00Z")
// «Сегодня»: окно до сейчас, не до конца суток — иначе avgBps размывается будущими часами.
const today = parseStatisticsPeriod("2026-09-12", "2026-09-12", nowMs)
assert.ok(today)
assert.equal(today.windowSec, 12 * 3600)
assert.equal(today.grain, "hour")
assert.equal(today.toDayExclusive, "2026-09-13", "дневные факты текущего дня не теряем")
// Прошлые периоды не клампятся.
const past = parseStatisticsPeriod("2026-09-10", "2026-09-10", nowMs)
assert.ok(past)
assert.equal(past.windowSec, 86_400)
// ISO-диапазон ровно 24 часа.
const iso24 = parseStatisticsPeriod("2026-09-11T12:00:00Z", "2026-09-12T12:00:00Z", nowMs)
assert.ok(iso24)
assert.equal(iso24.windowSec, 86_400)
assert.equal(iso24.grain, "hour")
// «to» далеко в будущем клампится к сейчас.
const futureTo = parseStatisticsPeriod("2026-09-11", "2026-09-20", nowMs)
assert.ok(futureTo)
assert.equal(futureTo.windowSec, 86_400 + 12 * 3600)
// Полностью будущий диапазон невалиден.
assert.equal(parseStatisticsPeriod("2026-09-13", "2026-09-14", nowMs), null)
}
{
// Таксономия сервисов как на карте: skip-список сворачивается в «Прочее».
assert.equal(normalizeFactService("Google"), "Google")
assert.equal(normalizeFactService("DNS"), "Прочее")
assert.equal(normalizeFactService("SSH"), "Прочее")
assert.equal(normalizeFactService("BGP"), "Прочее")
assert.equal(normalizeFactService("WireGuard"), "Прочее")
assert.equal(normalizeFactService("GRE"), "Прочее")
assert.equal(normalizeFactService("Прочее"), "Прочее")
assert.equal(normalizeFactService(""), "Прочее")
}
if (!(await withPgOrSkip())) {
console.log("statistics-aggregate.test.ts: skip")
process.exit(0)
+26 -7
View File
@@ -27,6 +27,13 @@ import {
wanIfaceLabel,
} from "./traffic-flow-facts-filter.js"
import { getServerCatalog, loadFlowTopology, type FlowTopology } from "./traffic-flow-topology.js"
import { OTHER_SERVICE, isNamedInternetService } from "./traffic-flow-brands.js"
/** Так же, как на карте сети: DNS/SSH/BGP/туннели и пустые метки — не отдельные сервисы, а «Прочее». */
export function normalizeFactService(label: string): string {
const s = String(label ?? "").trim()
return isNamedInternetService(s, "") ? s : OTHER_SERVICE
}
const TOP_N = 200
const HOUR_WINDOW_MS = 48 * 3600_000
@@ -58,7 +65,7 @@ function addUtcDays(day: string, n: number): string {
}
/** Parse from/to. Date-only `to` is inclusive (end of that UTC day). */
export function parseStatisticsPeriod(fromRaw: string, toRaw: string): ParsedPeriod | null {
export function parseStatisticsPeriod(fromRaw: string, toRaw: string, nowMs: number = Date.now()): ParsedPeriod | null {
const from = Date.parse(fromRaw.includes("T") ? fromRaw : `${fromRaw}T00:00:00Z`)
const toHasTime = toRaw.includes("T")
const to = Date.parse(toHasTime ? toRaw : `${toRaw}T00:00:00Z`)
@@ -75,6 +82,9 @@ export function parseStatisticsPeriod(fromRaw: string, toRaw: string): ParsedPer
toDayExclusive = addUtcDays(toUtcDay(toDate), 1)
toDate = new Date(`${toDayExclusive}T00:00:00Z`)
}
// Конец периода в будущем (например, «сегодня»): окно длится только до сейчас,
// иначе avgBps размывается ещё не наступившими часами суток.
if (toDate.getTime() > nowMs) toDate = new Date(nowMs)
if (toDate.getTime() <= from) return null
const windowSec = Math.max(1, Math.round((toDate.getTime() - from) / 1000))
const grain: "hour" | "day" = toDate.getTime() - from <= HOUR_WINDOW_MS ? "hour" : "day"
@@ -553,12 +563,14 @@ export async function getStatistics(query: StatisticsQuery): Promise<StatisticsD
period.windowSec,
)
const services = toBreakdown(
serviceRows.map((r) => ({
id: r.id,
label: r.id,
bytes: Number(r.bytes) || 0,
packets: Number(r.packets) || 0,
})),
Object.entries(serviceRows.reduce<Record<string, { bytes: number; packets: number }>>((acc, r) => {
const key = normalizeFactService(r.id)
const prev = acc[key] ?? { bytes: 0, packets: 0 }
prev.bytes += Number(r.bytes) || 0
prev.packets += Number(r.packets) || 0
acc[key] = prev
return acc
}, {})).map(([label, v]) => ({ id: label, label, bytes: v.bytes, packets: v.packets })),
bytes,
period.windowSec,
)
@@ -702,6 +714,13 @@ export async function getStatisticsPivot(query: StatisticsPivotQuery): Promise<S
GROUP BY 1, 2
`, [...join.params, ...where.params])
if (query.row === "service" || query.col === "service") {
for (const r of raw) {
if (query.row === "service") r.row_id = normalizeFactService(String(r.row_id ?? ""))
if (query.col === "service") r.col_id = normalizeFactService(String(r.col_id ?? ""))
}
}
if (query.row === "iface" || query.col === "iface") {
const ifaceServerIds: number[] = []
for (const r of raw) {
@@ -416,14 +416,44 @@ try {
resetIfaceCacheForTests()
resetRipeCacheForTests()
disableRipeEnqueueForTests()
seedRipeCacheForTests({
prefix: "74.125.0.0/16",
asn: 15169,
country: "US",
lat: null,
lng: null,
holder: "GOOGLE",
ok: true,
fetchedAt: Date.now(),
})
seedRipeCacheForTests({
prefix: "104.18.0.0/16",
asn: 13335,
country: "US",
lat: null,
lng: null,
holder: "CLOUDFLARENET",
ok: true,
fetchedAt: Date.now(),
})
seedRipeCacheForTests({
prefix: "146.75.0.0/16",
asn: 54113,
country: "US",
lat: null,
lng: null,
holder: "FASTLY",
ok: true,
fetchedAt: Date.now(),
})
rememberServerIfaces(7, [{ ".id": "*2", name: "ether1" }])
ingestParsedFlowsForServerForTests(7, [
{
src: "173.194.151.65",
dst: "10.200.100.53",
proto: 6,
src: "74.125.104.196/32",
dst: "10.200.100.53/32",
proto: 17,
srcPort: 443,
dstPort: 57182,
dstPort: 62598,
bytes: 12_000,
packets: 10,
inIface: "2",
@@ -440,15 +470,40 @@ try {
inIface: "2",
outIface: "2",
},
{
src: "146.75.118.132/32",
dst: "10.200.100.53/32",
proto: 6,
srcPort: 80,
dstPort: 35026,
bytes: 4_000,
packets: 5,
inIface: "2",
outIface: "2",
},
])
try {
const rev = await buildFlowAnalytics({ minutes: 5, serverId: 7 })
const google = rev.conversationsList.find((r) => r.src === "173.194.151.65")
const google = rev.conversationsList.find((r) => r.src === "74.125.104.196")
const cf = rev.conversationsList.find((r) => r.src === "104.18.35.51")
const fastly = rev.conversationsList.find((r) => r.src === "146.75.118.132")
assert.equal(google?.service, "YouTube")
assert.equal(google?.category, "Видео / стриминг")
assert.equal(google?.internetPeer, "74.125.104.196")
assert.equal(google?.internetPeerPort, 443)
assert.equal(google?.clientIp, "10.200.100.53")
assert.equal(google?.direction, "to_client")
assert.equal(google?.dstAsn, 15169)
assert.equal(google?.dstCountry, "US")
assert.ok(!String(google?.src).includes("/"), "DTO src без /32")
assert.equal(cf?.service, "Cloudflare")
assert.equal(cf?.category, "CDN")
assert.equal(fastly?.service, "Fastly")
assert.equal(fastly?.dstAsn, 54113)
assert.equal(fastly?.dstCountry, "US")
assert.ok(rev.asns?.some((r) => r.id === "54113"))
assert.ok(rev.countries?.some((r) => r.id === "US"))
assert.ok(!rev.services?.every((s) => s.label === "Прочее"), "сервисы не схлопнуты в Прочее")
} finally {
resetFlowRingsForTests()
resetIfaceCacheForTests()
@@ -265,8 +265,9 @@ async function buildFlowAnalyticsUncached(q: FlowAnalyticsQuery): Promise<FlowAn
natSrcPort: r.natSrcPort,
natDstPort: r.natDstPort,
})
const ep = destMeta.endpoints
if (destMeta.dest) peers.add(destMeta.dest)
const app = applicationName(r.proto, r.dstPort, r.srcPort)
const app = applicationName(r.proto, ep.peerPort || r.dstPort, ep.otherPort || r.srcPort)
const ripe = destMeta.ripe
const classified = destMeta.classified
bump(applications, app, r.bytes, r.packets)
@@ -312,8 +313,8 @@ async function buildFlowAnalyticsUncached(q: FlowAnalyticsQuery): Promise<FlowAn
conv.set(ckey, {
serverId: String(r.serverId),
serverName: nameById.get(r.serverId) ?? String(r.serverId),
src: r.src,
dst: r.dst,
src: ep.packetSrc || r.src,
dst: ep.packetDst || r.dst,
proto: r.proto,
protoName: protoName(r.proto),
srcPort: r.srcPort,
@@ -332,6 +333,10 @@ async function buildFlowAnalyticsUncached(q: FlowAnalyticsQuery): Promise<FlowAn
dstAsn: ripe?.asn || undefined,
clientId: client?.userId,
clientName: client?.name,
clientIp: ep.clientIp || undefined,
internetPeer: ep.internetPeer || undefined,
internetPeerPort: ep.peerPort || undefined,
direction: ep.direction,
enId: en ? String(en.id) : undefined,
enName: en?.name,
plane,
@@ -8,6 +8,8 @@ import {
OTHER_SERVICE,
isNamedInternetService,
mapServiceNodeId,
mapCountryNodeId,
mapCountryServiceNodeId,
resolveFlowBrand,
resolveRipeCountry,
} from "./traffic-flow-brands.js"
@@ -44,6 +46,13 @@ assert.equal(isNamedInternetService("DNS", "DNS"), false)
assert.equal(mapServiceNodeId("AWS"), "svc:aws")
assert.equal(mapServiceNodeId("Cloudflare"), "svc:cloudflare")
assert.equal(mapServiceNodeId("Прочее"), "svc:other")
assert.equal(mapCountryNodeId("US"), "cc:us")
assert.equal(mapCountryNodeId("nl"), "cc:nl")
assert.equal(mapCountryNodeId(""), "cc:other")
assert.equal(mapCountryNodeId("Прочее"), "cc:other")
assert.equal(mapCountryNodeId("EU"), "cc:other")
assert.equal(mapCountryServiceNodeId("cc:us", "svc:google"), "cc:us|svc:google")
assert.equal(mapCountryServiceNodeId("cc:other", "svc:other"), "cc:other|svc:other")
assert.equal(brandByAsn(714)?.service, "Apple")
assert.equal(brandByAsn(714)?.category, "CDN")
@@ -79,6 +88,33 @@ assert.equal(resolveFlowBrand("64.233.161.1", 0, "", 17, 443, 50000)?.service, "
assert.equal(resolveFlowBrand("64.233.161.1", 0, "", 6, 80, 50000)?.service, "Google")
assert.equal(resolveFlowBrand("2001:4860:4860::8888", 15169, "GOOGLE", 17, 53, 53000)?.service, "Google")
assert.equal(resolveFlowBrand("2001:4860:4860::8888", 15169, "GOOGLE", 17, 443, 50000)?.service, "YouTube")
assert.equal(brandByAsn(32934)?.service, "Meta")
assert.equal(lookupBrand("157.240.12.52", 0)?.service, "Meta")
assert.equal(lookupBrand("57.144.22.192", 0)?.service, "Meta")
assert.equal(brandByHolder("Instagram LLC")?.service, "Instagram")
assert.equal(mapServiceNodeId("Instagram"), "svc:instagram")
assert.equal(isNamedInternetService("Instagram", "Видео / стриминг"), true)
assert.equal(
resolveFlowBrand("157.240.12.52", 32934, "FACEBOOK", 6, 443, 51234)?.service,
"Instagram",
"HTTPS на Meta front → Instagram, как YouTube на Google",
)
assert.equal(
resolveFlowBrand("57.144.22.192", 0, "", 17, 443, 50000)?.service,
"Instagram",
"cdninstagram CIDR :443 без ASN → Instagram",
)
assert.equal(
resolveFlowBrand("157.240.12.52", 32934, "FACEBOOK", 6, 80, 50000)?.service,
"Meta",
":80 на Meta остаётся Meta",
)
assert.equal(
resolveFlowBrand("157.240.1.1", 54115, "WHATSAPP", 6, 443, 1)?.service,
"Meta",
"AS54115 WhatsApp не становится Instagram",
)
assert.equal(resolveRipeCountry("", 9059, ""), "IE")
assert.equal(resolveRipeCountry("", 24940, ""), "DE")
+48 -6
View File
@@ -39,6 +39,7 @@ const TIMEWEB: BrandHit = { service: "Timeweb", ...CDN }
const BEGET: BrandHit = { service: "Beget", ...CDN }
const DDOS_GUARD: BrandHit = { service: "DDoS-Guard", ...CDN }
const META: BrandHit = { service: "Meta", ...CDN }
const INSTAGRAM: BrandHit = { service: "Instagram", ...VIDEO }
const GOOGLE: BrandHit = { service: "Google", ...WEB }
const GITHUB: BrandHit = { service: "GitHub", ...WEB }
@@ -184,10 +185,27 @@ const CIDR_BRANDS: Array<{ cidr: string; prefixLen: number; hit: BrandHit }> = [
{ cidr: "216.239.32.0/19", prefixLen: 19, hit: GOOGLE },
{ cidr: "208.65.152.0/22", prefixLen: 22, hit: YOUTUBE },
{ cidr: "208.117.224.0/19", prefixLen: 19, hit: YOUTUBE },
{ cidr: "31.13.64.0/18", prefixLen: 18, hit: META },
{ cidr: "57.141.0.0/16", prefixLen: 16, hit: META },
{ cidr: "57.142.0.0/15", prefixLen: 15, hit: META },
{ cidr: "57.144.0.0/14", prefixLen: 14, hit: META },
{ cidr: "57.148.0.0/15", prefixLen: 15, hit: META },
{ cidr: "66.220.144.0/20", prefixLen: 20, hit: META },
{ cidr: "69.63.176.0/20", prefixLen: 20, hit: META },
{ cidr: "69.171.224.0/19", prefixLen: 19, hit: META },
{ cidr: "74.119.76.0/22", prefixLen: 22, hit: META },
{ cidr: "129.134.0.0/16", prefixLen: 16, hit: META },
{ cidr: "157.240.0.0/16", prefixLen: 16, hit: META },
{ cidr: "173.252.64.0/18", prefixLen: 18, hit: META },
{ cidr: "179.60.192.0/22", prefixLen: 22, hit: META },
{ cidr: "185.60.216.0/22", prefixLen: 22, hit: META },
{ cidr: "199.201.64.0/22", prefixLen: 22, hit: META },
{ cidr: "204.15.20.0/22", prefixLen: 22, hit: META },
].sort((a, b) => b.prefixLen - a.prefixLen)
const HOLDER_BRANDS: Array<{ re: RegExp; hit: BrandHit }> = [
{ re: /youtube/i, hit: YOUTUBE },
{ re: /instagram/i, hit: INSTAGRAM },
{ re: /valve|\bsteam\b/i, hit: STEAM },
{ re: /blizzard|battle.?net/i, hit: BLIZZARD },
{ re: /openai/i, hit: CHATGPT },
@@ -205,6 +223,9 @@ const NON_ISO = new Set(["EU", "AP", "ZZ", "XX", "A1", "A2", "O1"])
const STEAM_ASN = 32590
const GOOGLE_FRONT_ASN = new Set([15169, 396982])
/** AS32934 / AS63293 — Meta front (Facebook + Instagram CDN). AS54115 — WhatsApp, не Instagram. */
const META_FRONT_ASN = new Set([32934, 63293])
const WHATSAPP_ASN = 54115
function isGooglePublicDns(ip: string): boolean {
return ipInCidrV4(ip, "8.8.8.0/24") || ipInCidrV4(ip, "8.8.4.0/24")
@@ -274,9 +295,19 @@ export function lookupBrand(ip: string, asn: number): BrandHit | null {
return brandByCidr(ip) || brandByAsn(asn)
}
function isGoogleFront(asn: number, cidrBrand: BrandHit | null, asnBrand: BrandHit | null): boolean {
return GOOGLE_FRONT_ASN.has(asn) || cidrBrand?.service === "Google" || asnBrand?.service === "Google"
}
function isInstagramFront(asn: number, cidrBrand: BrandHit | null, asnBrand: BrandHit | null): boolean {
if (asn === WHATSAPP_ASN) return false
return META_FRONT_ASN.has(asn) || cidrBrand?.service === "Meta" || asnBrand?.service === "Meta"
}
/**
* Cloudflare CIDR бьёт holder (витрина на CF не становится Steam).
* Holder (YouTube и др.) бьёт остальные CIDR/ASN.
* Holder (YouTube / Instagram и др.) бьёт остальные CIDR/ASN.
* HTTPS/QUIC на Google front → YouTube (кроме 8.8.8.8); на Meta front → Instagram (кроме WhatsApp ASN).
* Порты Steam — только AS32590 и не выше Cloudflare CIDR.
*/
export function resolveFlowBrand(
@@ -292,13 +323,12 @@ export function resolveFlowBrand(
const holderBrand = brandByHolder(holder)
if (holderBrand) return holderBrand
const asnBrand = brandByAsn(asn)
if (
!isGooglePublicDns(ip)
&& isHttpsOrQuic(proto, dstPort, srcPort)
&& (GOOGLE_FRONT_ASN.has(asn) || cidrBrand?.service === "Google" || asnBrand?.service === "Google")
) {
if (!isGooglePublicDns(ip) && isHttpsOrQuic(proto, dstPort, srcPort) && isGoogleFront(asn, cidrBrand, asnBrand)) {
return YOUTUBE
}
if (isHttpsOrQuic(proto, dstPort, srcPort) && isInstagramFront(asn, cidrBrand, asnBrand)) {
return INSTAGRAM
}
const fromLookup = cidrBrand || asnBrand
if (fromLookup) return fromLookup
if (asn === STEAM_ASN && isSteamGamePort(proto, dstPort, srcPort)) return STEAM
@@ -334,3 +364,15 @@ export function mapServiceNodeId(label: string): string {
.replace(/^-+|-+$/g, "")
return `svc:${slug || "unknown"}`
}
/** `US` → `cc:us`; неизвестная / пустая → `cc:other`. */
export function mapCountryNodeId(code: string): string {
const iso = normalizeIsoCountry(code)
if (!iso) return "cc:other"
return `cc:${iso.toLowerCase()}`
}
/** id сервиса внутри страны: `cc:us|svc:google` — Google в US и NL не смешиваются в одном payload. */
export function mapCountryServiceNodeId(countryId: string, serviceId: string): string {
return `${countryId}|${serviceId}`
}
@@ -132,6 +132,73 @@ const greIgnore = classifyFlowDst("8.8.8.8", 47, 0, 0, {
fetchedAt: Date.now(),
}, { ignoreTunnelProto: true })
assert.equal(greIgnore.service, "Google")
const dnsGoogle = classifyFlowDst("8.8.8.8", 17, 53, 53000, {
prefix: "8.8.8.0/24",
asn: 15169,
country: "US",
lat: null,
lng: null,
holder: "GOOGLE",
ok: true,
fetchedAt: Date.now(),
})
assert.equal(dnsGoogle.service, "Google")
assert.notEqual(dnsGoogle.service, "Прочее")
const ipv6Yt = classifyFlowDst("2001:4860:4860::8888", 17, 443, 50000, {
prefix: "2001:4860:4860::8888/128",
asn: 15169,
country: "US",
lat: null,
lng: null,
holder: "GOOGLE",
ok: true,
fetchedAt: Date.now(),
})
assert.equal(ipv6Yt.service, "YouTube")
const instagram = classifyFlowDst("157.240.12.52", 6, 443, 62598, {
prefix: "157.240.0.0/16",
asn: 32934,
country: "US",
lat: null,
lng: null,
holder: "FACEBOOK",
ok: true,
fetchedAt: Date.now(),
})
assert.equal(instagram.service, "Instagram")
assert.equal(instagram.category, "Видео / стриминг")
assert.notEqual(instagram.service, "Meta")
const metaHttp = classifyFlowDst("157.240.12.52", 6, 80, 50000, {
prefix: "157.240.0.0/16",
asn: 32934,
country: "US",
lat: null,
lng: null,
holder: "FACEBOOK",
ok: true,
fetchedAt: Date.now(),
})
assert.equal(metaHttp.service, "Meta")
assert.equal(metaHttp.category, "CDN")
const igHolder = classifyFlowDst("203.0.113.80", 6, 443, 1, {
prefix: "203.0.113.0/24",
asn: 64503,
country: "US",
lat: null,
lng: null,
holder: "Instagram LLC",
ok: true,
fetchedAt: Date.now(),
})
assert.equal(igHolder.service, "Instagram")
const igCidr = classifyFlowDst("57.144.22.192", 17, 443, 50000, null)
assert.equal(igCidr.service, "Instagram")
assert.equal(igCidr.category, "Видео / стриминг")
const esp = classifyFlowDst("198.51.100.1", 50, 0, 0, null)
assert.equal(esp.category, "Туннель")
assert.equal(applicationName(17, 443, 50000), "QUIC")
@@ -48,7 +48,7 @@ export function seedFlowCatalogForTests(input: {
export function categoryFromPurpose(purpose: string, proto: number, dstPort: number, srcPort: number): string {
const p = purpose.toLowerCase()
if (/gaming|steam|epic|riot|playstation|roblox|ubisoft/.test(p)) return "Игры"
if (/streaming|youtube|netflix|twitch|video|spotify/.test(p)) return "Видео / стриминг"
if (/streaming|youtube|netflix|twitch|video|spotify|instagram/.test(p)) return "Видео / стриминг"
if (/cdn|cloudflare|akamai|fastly|hetzner|ovh|apple/.test(p)) return "CDN"
if (/voip|discord|zoom/.test(p)) return "Голос"
if (/openai|chatgpt|\bai\b/.test(p)) return "ИИ"
+101 -1
View File
@@ -4,7 +4,7 @@ import {
resetEngineForTests,
} from "./traffic-flow-engine.js"
import { factsSnapshotForTests } from "./traffic-flow-facts.js"
import { classifyInternetBrand, mapInternetBrand } from "./traffic-flow-dest.js"
import { classifyInternetBrand, mapInternetBrand, resolveInternetDest } from "./traffic-flow-dest.js"
import { disableCatalogFetchForTests, resetFlowCatalogForTests } from "./traffic-flow-classify.js"
import {
disableRipeEnqueueForTests,
@@ -149,6 +149,106 @@ assert.equal(mapInternetBrand("8.8.8.8", 6, 443, 51234, {
fetchedAt: Date.now(),
}).service, "Google")
assert.equal(classifyInternetBrand("8.8.8.8", 17, 53, 53000, {
prefix: "8.8.8.0/24",
asn: 15169,
country: "US",
lat: null,
lng: null,
holder: "GOOGLE",
ok: true,
fetchedAt: Date.now(),
})?.service, "Google")
assert.equal(mapInternetBrand("8.8.8.8", 17, 53, 53000, {
prefix: "8.8.8.0/24",
asn: 15169,
country: "US",
lat: null,
lng: null,
holder: "GOOGLE",
ok: true,
fetchedAt: Date.now(),
}).service, "Google")
assert.equal(mapInternetBrand("2001:4860:4860::8888", 17, 443, 50000, {
prefix: "2001:4860:4860::8888/128",
asn: 15169,
country: "US",
lat: null,
lng: null,
holder: "GOOGLE",
ok: true,
fetchedAt: Date.now(),
}).service, "YouTube")
assert.equal(mapInternetBrand("64.233.161.1", 17, 443, 50000, null).service, "YouTube")
assert.equal(mapInternetBrand("142.250.1.10", 6, 443, 1, null).service, "YouTube")
{
const meta = resolveInternetDest({
src: "74.125.104.196/32",
dst: "10.200.100.53/32",
proto: 17,
srcPort: 443,
dstPort: 62598,
serverId: 1,
inIface: "gre-client",
topo,
})
assert.equal(meta.dest, "74.125.104.196")
assert.equal(meta.classified.service, "YouTube")
assert.notEqual(meta.classified.service, "Прочее")
assert.equal(meta.endpoints.direction, "to_client")
assert.equal(meta.endpoints.clientIp, "10.200.100.53")
assert.equal(meta.asn, 0)
}
{
seedRipeCacheForTests({
prefix: "146.75.0.0/16",
asn: 54113,
country: "US",
lat: null,
lng: null,
holder: "FASTLY",
ok: true,
fetchedAt: Date.now(),
})
seedRipeCacheForTests({
prefix: "3.174.0.0/16",
asn: 16509,
country: "US",
lat: null,
lng: null,
holder: "AMAZON-AES",
ok: true,
fetchedAt: Date.now(),
})
const fastly = resolveInternetDest({
src: "146.75.118.132/32",
dst: "10.200.100.53/32",
proto: 6,
srcPort: 80,
dstPort: 35026,
serverId: 1,
inIface: "gre-client",
topo,
})
assert.equal(fastly.classified.service, "Fastly")
assert.equal(fastly.asn, 54113)
assert.equal(fastly.country, "US")
const aws = resolveInternetDest({
src: "3.174.2.35/32",
dst: "10.200.100.53/32",
proto: 6,
srcPort: 443,
dstPort: 43726,
serverId: 1,
inIface: "gre-client",
topo,
})
assert.equal(aws.classified.service, "AWS")
assert.equal(aws.asn, 16509)
}
resetEngineForTests()
seedFlowTopologyForTests(null)
resetRipeCacheForTests()
+49 -30
View File
@@ -3,7 +3,11 @@ import { isIsoCountry, isNamedInternetService, OTHER_SERVICE, resolveFlowBrand }
import { classifyFlowDst, type FlowClassification } from "./traffic-flow-classify.js"
import { resolveFlowIp } from "./traffic-flow-geoip.js"
import { canonicalFactIface } from "./traffic-flow-ifindex.js"
import { pickInternetDest, type InternetDestCtx } from "./traffic-flow-ip.js"
import {
resolveFlowEndpoints,
type FlowEndpoints,
type InternetDestCtx,
} from "./traffic-flow-ip.js"
import type { FlowIpMeta } from "./traffic-flow-ripe.js"
import {
flowOursHosts,
@@ -17,6 +21,7 @@ export interface InternetDestMeta {
classified: FlowClassification
country: string
asn: number
endpoints: FlowEndpoints
}
export function destCtxForIface(
@@ -41,7 +46,14 @@ export function destCtxForIface(
}
}
/** Бренд интернет-dest как на карте: GRE/ESP/WG — транспорт, не сервис. */
const OTHER_BRAND: FlowClassification = { service: OTHER_SERVICE, category: OTHER_SERVICE }
function isTunnelProto(proto: number, dstPort: number, srcPort: number): boolean {
if (proto === 47 || proto === 50) return true
return applicationName(proto, dstPort, srcPort) === "WireGuard"
}
/** Бренд интернет-dest: ASN/CIDR до skip DNS. GRE/ESP/WG — не сервис. */
export function classifyInternetBrand(
dst: string,
proto: number,
@@ -49,17 +61,15 @@ export function classifyInternetBrand(
srcPort: number,
ripe: FlowIpMeta | null,
): FlowClassification | null {
if (proto === 47 || proto === 50) return null
const app = applicationName(proto, dstPort, srcPort)
if (app === "WireGuard" || app === "DNS" || app === "SSH" || app === "BGP") return null
if (isTunnelProto(proto, dstPort, srcPort)) return null
const brand = resolveFlowBrand(dst, ripe?.asn ?? 0, ripe?.holder ?? "", proto, dstPort, srcPort)
if (!brand || !isNamedInternetService(brand.service, brand.category)) return null
return brand
if (brand && isNamedInternetService(brand.service, brand.category)) return brand
const app = applicationName(proto, dstPort, srcPort)
if (app === "DNS" || app === "SSH" || app === "BGP") return null
return null
}
const OTHER_BRAND: FlowClassification = { service: OTHER_SERVICE, category: OTHER_SERVICE }
/** Бренд для карты: именованный сервис или «Прочее» (GRE/ESP не сервис). */
/** Тот же классификатор, что аналитика (GeoLite2 ASN + catalog). Туннель → Прочее. */
export function mapInternetBrand(
dst: string,
proto: number,
@@ -67,8 +77,10 @@ export function mapInternetBrand(
srcPort: number,
ripe: FlowIpMeta | null,
): FlowClassification {
if (proto === 47 || proto === 50) return OTHER_BRAND
return classifyInternetBrand(dst, proto, dstPort, srcPort, ripe) ?? OTHER_BRAND
if (isTunnelProto(proto, dstPort, srcPort)) return OTHER_BRAND
const classified = classifyFlowDst(dst, proto, dstPort, srcPort, ripe, { ignoreTunnelProto: true })
if (isNamedInternetService(classified.service, classified.category)) return classified
return OTHER_BRAND
}
export function resolveInternetDest(opts: {
@@ -85,28 +97,35 @@ export function resolveInternetDest(opts: {
natSrcPort?: number
natDstPort?: number
}): InternetDestMeta {
const dest = pickInternetDest(
opts.src,
opts.dst,
opts.srcPort,
opts.dstPort,
destCtxForIface(opts.topo, opts.serverId, opts.inIface, {
natSrc: opts.natSrc,
natDst: opts.natDst,
natSrcPort: opts.natSrcPort,
natDstPort: opts.natDstPort,
}),
)
const ripe = dest ? resolveFlowIp(dest) : null
const classified = dest
? classifyFlowDst(dest, opts.proto, opts.dstPort, opts.srcPort, ripe, { ignoreTunnelProto: true })
: classifyFlowDst(opts.dst, opts.proto, opts.dstPort, opts.srcPort, ripe)
const ctx = destCtxForIface(opts.topo, opts.serverId, opts.inIface, {
natSrc: opts.natSrc,
natDst: opts.natDst,
natSrcPort: opts.natSrcPort,
natDstPort: opts.natDstPort,
})
const endpoints = resolveFlowEndpoints({
src: opts.src,
dst: opts.dst,
srcPort: opts.srcPort,
dstPort: opts.dstPort,
ctx,
})
const dest = endpoints.internetPeer
if (!dest) {
return { dest: "", ripe: null, classified, country: "", asn: 0 }
return { dest: "", ripe: null, classified: OTHER_BRAND, country: "", asn: 0, endpoints }
}
const ripe = resolveFlowIp(dest)
const classified = classifyFlowDst(
dest,
opts.proto,
endpoints.peerPort,
endpoints.otherPort,
ripe,
{ ignoreTunnelProto: true },
)
const country = ripe?.ok && isIsoCountry(ripe.country)
? ripe.country
: (ripe?.ok ? "" : "unknown")
const asn = ripe?.ok && ripe.asn ? ripe.asn : 0
return { dest, ripe, classified, country, asn }
return { dest, ripe, classified, country, asn, endpoints }
}
+7 -6
View File
@@ -9,6 +9,7 @@ import { invalidateTrafficFlowSettingsCache } from "./traffic-flow-settings.js"
import { maybeRefreshIfaces } from "./traffic-flow-ifaces.js"
import { canonicalFactIface } from "./traffic-flow-ifindex.js"
import { shouldWriteFlowFact } from "./traffic-flow-facts-filter.js"
import { canonicalIp } from "./traffic-flow-ip.js"
import { resolveInternetDest } from "./traffic-flow-dest.js"
import {
getServerCatalog,
@@ -68,12 +69,12 @@ export interface PendingFlowRow {
}
function inetOrNull(value: string | null | undefined): string | null {
const s = String(value ?? "").trim()
const s = canonicalIp(value)
return s.length > 0 ? s : null
}
export function isValidFlowInet(value: string): boolean {
const s = value.trim()
const s = canonicalIp(value)
if (!s) return false
const v4 = /^(\d{1,3})\.(\d{1,3})\.(\d{1,3})\.(\d{1,3})$/.exec(s)
if (v4) {
@@ -100,16 +101,16 @@ function clampPort(n: number): number {
}
function sanitizeNatIp(value: string | null | undefined): string {
const s = String(value ?? "").trim()
const s = canonicalIp(value)
if (!s || s === "0.0.0.0") return ""
return isValidFlowInet(s) ? s : ""
}
function sanitizeFlowRow(r: PendingFlowRow): PendingFlowRow | null {
const src = (r.src || "").trim() || "0.0.0.0"
const dst = (r.dst || "").trim() || "0.0.0.0"
const src = canonicalIp(r.src) || "0.0.0.0"
const dst = canonicalIp(r.dst) || "0.0.0.0"
if (!isValidFlowInet(src) || !isValidFlowInet(dst)) return null
const next = inetOrNull(r.nextHop)
const next = inetOrNull(canonicalIp(r.nextHop))
return {
...r,
src,
@@ -21,6 +21,7 @@ import {
minuteDimsSnapshotForTests,
} from "./traffic-flow-engine.js"
import { classifyFlowDst } from "./traffic-flow-classify.js"
import { mapInternetBrand } from "./traffic-flow-dest.js"
import { disableGeoipDbForTests } from "./geoip-settings.js"
import {
collectGeoipUpdateOnce,
@@ -89,6 +90,7 @@ setGeoipReadersForTests({
const hit = resolveFlowIp("8.8.8.8")
assert.equal(hit?.country, "US")
assert.equal(hit?.asn, 15169)
assert.equal(resolveFlowIp("8.8.8.8/32")?.asn, 15169, "GeoIP по inet::text /32")
assert.equal(hit?.holder, "GOOGLE")
assert.equal(hit?.ok, true)
@@ -102,6 +104,21 @@ assert.equal(lookupGeoip("6.6.6.6")?.country, "US")
const classified = classifyFlowDst("8.8.8.8", 6, 443, 51504, hit)
assert.equal(classified.service, "Google")
setGeoipReadersForTests({
country: fakeCountryReader({ "8.8.8.8": "US", "2001:4860:4860::8888": "US" }),
asn: fakeAsnReader({
"8.8.8.8": { asn: 15169, org: "GOOGLE" },
"2001:4860:4860::8888": { asn: 15169, org: "GOOGLE" },
"64.233.161.1": { asn: 15169, org: "GOOGLE" },
}),
})
const v6meta = resolveFlowIp("2001:4860:4860::8888")
assert.equal(v6meta?.asn, 15169)
assert.equal(mapInternetBrand("2001:4860:4860::8888", 17, 443, 50000, v6meta).service, "YouTube")
assert.notEqual(mapInternetBrand("2001:4860:4860::8888", 17, 443, 50000, v6meta).service, "Прочее")
const cidrYt = mapInternetBrand("64.233.161.1", 17, 443, 50000, resolveFlowIp("64.233.161.1"))
assert.equal(cidrYt.service, "YouTube")
// ── движок: dims country/asn наполняются из geoip-ридеров ────────────────────
resetEngineForTests()
ingestParsedFlowsForServerForTests(1, [{
+2 -2
View File
@@ -1,7 +1,7 @@
import { existsSync } from "node:fs"
import path from "node:path"
import { open, type AsnResponse, type CountryResponse, type Reader } from "maxmind"
import { isNonPublicIp } from "./traffic-flow-ip.js"
import { canonicalIp, isNonPublicIp } from "./traffic-flow-ip.js"
import { isIsoCountry, resolveRipeCountry } from "./traffic-flow-brands.js"
import { lookupRipeCached, type FlowIpMeta } from "./traffic-flow-ripe.js"
@@ -117,7 +117,7 @@ function safeAsn(reader: Reader<AsnResponse>, ip: string): { asn: number; holder
* (ok=true когда есть страна или ASN; null данных нет, пусть пробует RIPE).
*/
export function lookupGeoip(ip: string): FlowIpMeta | null {
const trimmed = String(ip ?? "").trim()
const trimmed = canonicalIp(ip)
if (!trimmed) return null
if (isNonPublicIp(trimmed)) return negativeMeta(trimmed)
const { country: countryReader, asn: asnReader } = readers
@@ -72,6 +72,7 @@ setPendingCapForTests(null)
assert.equal(isValidFlowInet("10.0.0.1"), true)
assert.equal(isValidFlowInet("8.8.8.8"), true)
assert.equal(isValidFlowInet("8.8.8.8/32"), true)
assert.equal(isValidFlowInet("0:0:0:0:0:0:0:1"), true)
assert.equal(isValidFlowInet("not-an-ip"), false)
assert.equal(isValidFlowInet("999.1.1.1"), false)
+9 -8
View File
@@ -29,6 +29,7 @@ import {
} from "./traffic-flow-settings.js"
import { applicationName } from "./traffic-flow-apps.js"
import { resolveIfaceName } from "./traffic-flow-ifaces.js"
import { canonicalIp } from "./traffic-flow-ip.js"
import { getServerCatalog } from "./traffic-flow-topology.js"
export type { PendingFlowRow }
@@ -357,10 +358,10 @@ export async function listStoredFlowRows(sinceIso: string): Promise<PendingFlowR
const keep = Math.max(20, settings.topN)
const stored = await dbAll<StoredBucketRow>(`
SELECT
server_id, bucket_at, src::text AS src, dst::text AS dst, proto,
server_id, bucket_at, COALESCE(host(src), '') AS src, COALESCE(host(dst), '') AS dst, proto,
src_port, dst_port, bytes, packets, in_iface, out_iface,
next_hop::text AS next_hop, flow_start_ms, flow_end_ms,
nat_src::text AS nat_src, nat_dst::text AS nat_dst, nat_src_port, nat_dst_port
COALESCE(host(next_hop), '') AS next_hop, flow_start_ms, flow_end_ms,
COALESCE(host(nat_src), '') AS nat_src, COALESCE(host(nat_dst), '') AS nat_dst, nat_src_port, nat_dst_port
FROM (
SELECT fb.*,
ROW_NUMBER() OVER (
@@ -376,8 +377,8 @@ export async function listStoredFlowRows(sinceIso: string): Promise<PendingFlowR
mergeInto(merged, {
serverId: Number(r.server_id),
bucketAt: isoBucketAt(r.bucket_at),
src: r.src,
dst: r.dst,
src: canonicalIp(r.src),
dst: canonicalIp(r.dst),
proto: Number(r.proto) || 0,
srcPort: Number(r.src_port) || 0,
dstPort: Number(r.dst_port) || 0,
@@ -385,11 +386,11 @@ export async function listStoredFlowRows(sinceIso: string): Promise<PendingFlowR
packets: Number(r.packets) || 0,
inIface: r.in_iface ?? "",
outIface: r.out_iface ?? "",
nextHop: r.next_hop ?? "",
nextHop: canonicalIp(r.next_hop ?? ""),
flowStartMs: Number(r.flow_start_ms) || 0,
flowEndMs: Number(r.flow_end_ms) || 0,
natSrc: r.nat_src ?? "",
natDst: r.nat_dst ?? "",
natSrc: canonicalIp(r.nat_src ?? ""),
natDst: canonicalIp(r.nat_dst ?? ""),
natSrcPort: Number(r.nat_src_port) || 0,
natDstPort: Number(r.nat_dst_port) || 0,
})
+77 -1
View File
@@ -1,14 +1,33 @@
import assert from "node:assert/strict"
import { isNonPublicIp, pickInternetDest, pickInternetPeer } from "./traffic-flow-ip.js"
import {
canonicalIp,
isNonPublicIp,
pickInternetDest,
pickInternetPeer,
pickMapInternetDest,
resolveFlowEndpoints,
} from "./traffic-flow-ip.js"
assert.equal(canonicalIp("74.125.104.196/32"), "74.125.104.196")
assert.equal(canonicalIp("10.200.100.53/32"), "10.200.100.53")
assert.equal(canonicalIp("::ffff:8.8.8.8"), "8.8.8.8")
assert.equal(canonicalIp("2001:4860:4860::8888/128"), "2001:4860:4860::8888")
assert.equal(isNonPublicIp("10.200.100.53"), true)
assert.equal(isNonPublicIp("10.200.100.53/32"), true)
assert.equal(isNonPublicIp("173.194.151.65"), false)
assert.equal(isNonPublicIp("74.125.104.196/32"), false, "PG inet::text не делает Google приватным")
assert.equal(
pickInternetPeer("173.194.151.65", "10.200.100.53", 443, 57182),
"173.194.151.65",
"reverse IPFIX: Google:443 → RFC1918",
)
assert.equal(
pickInternetPeer("74.125.104.196/32", "10.200.100.53/32", 443, 62598),
"74.125.104.196",
"inet::text /32 reverse Google",
)
assert.equal(
pickInternetPeer("10.200.100.53", "104.18.35.51", 53880, 443),
"104.18.35.51",
@@ -78,5 +97,62 @@ assert.equal(
"",
"NAT 0.0.0.0 не dest",
)
assert.equal(
pickMapInternetDest("10.200.100.53", "10.200.100.1", 53880, 443, {
...client,
natDst: "8.8.8.8",
natDstPort: 443,
}),
"8.8.8.8",
"карта: RFC1918 + NAT Google",
)
assert.equal(
pickMapInternetDest("10.200.100.53", "10.200.100.1", 53880, 443, client),
"",
"карта: RFC1918 без NAT → Прочее",
)
assert.equal(
pickInternetDest("173.194.151.65", "10.200.100.53", 12345, 57182, client),
"173.194.151.65",
"реверс googlevideo не :443 — публичный src",
)
assert.equal(
pickMapInternetDest("173.194.151.65", "10.200.100.53", 12345, 57182, client),
"173.194.151.65",
"карта: реверс googlevideo не :443 — всё равно публичный src",
)
assert.equal(
pickMapInternetDest("203.0.113.10", "198.51.100.1", 0, 0, { ours }),
"",
"карта: JH ours → EN ours всё ещё не dest",
)
{
const ep = resolveFlowEndpoints({
src: "74.125.104.196/32",
dst: "10.200.100.53/32",
srcPort: 443,
dstPort: 62598,
})
assert.equal(ep.internetPeer, "74.125.104.196")
assert.equal(ep.peerPort, 443)
assert.equal(ep.clientIp, "10.200.100.53")
assert.equal(ep.direction, "to_client")
assert.equal(ep.packetSrc, "74.125.104.196")
assert.equal(ep.packetDst, "10.200.100.53")
}
{
const ep = resolveFlowEndpoints({
src: "10.200.100.53",
dst: "104.18.35.51",
srcPort: 53880,
dstPort: 443,
})
assert.equal(ep.internetPeer, "104.18.35.51")
assert.equal(ep.peerPort, 443)
assert.equal(ep.clientIp, "10.200.100.53")
assert.equal(ep.direction, "from_client")
}
console.log("traffic-flow-ip.test.ts: ok")
+121 -11
View File
@@ -1,7 +1,25 @@
/** IPv4 helpers for RIPEstat prefix cache and EvoBGP CIDR match. */
/**
* Host-семантика PostgreSQL `host(inet)`: снимает `/32` `/128`, `::ffff:`.
* IPFIX 5-tuple не меняем только канонический вид адреса.
*/
export function canonicalIp(raw: string | undefined | null): string {
let t = String(raw ?? "").trim()
if (!t) return ""
const zone = t.indexOf("%")
if (zone >= 0) t = t.slice(0, zone)
if (t.toLowerCase().startsWith("::ffff:")) t = t.slice(7)
const slash = t.lastIndexOf("/")
if (slash >= 0) {
const plen = t.slice(slash + 1)
if (/^\d+$/.test(plen)) t = t.slice(0, slash)
}
return t.trim()
}
export function ipv4ToInt(ip: string): number | null {
const parts = String(ip ?? "").trim().split(".")
const parts = canonicalIp(ip).split(".")
if (parts.length !== 4) return null
let n = 0
for (const p of parts) {
@@ -26,12 +44,12 @@ export function parseCidrV4(cidr: string): { net: number; mask: number; prefixLe
export function ipInCidrV4(ip: string, cidr: string): boolean {
const addr = ipv4ToInt(ip)
const parsed = parseCidrV4(cidr)
if (addr == null || !parsed) return false
if (addr == null || parsed == null) return false
return ((addr & parsed.mask) >>> 0) === parsed.net
}
export function isNonPublicIp(ip: string): boolean {
const trimmed = String(ip ?? "").trim()
const trimmed = canonicalIp(ip)
if (!trimmed) return true
if (trimmed.includes(":")) {
const lower = trimmed.toLowerCase()
@@ -56,14 +74,14 @@ export function isNonPublicIp(ip: string): boolean {
const PEER_WELL_KNOWN_PORTS = new Set([80, 443, 53, 853])
export function isUnspecifiedIp(ip: string): boolean {
const t = String(ip ?? "").trim()
const t = canonicalIp(ip)
if (!t) return true
const lower = t.toLowerCase()
return t === "0.0.0.0" || lower === "::" || lower === "::0"
}
function usableIp(ip: string | undefined): string {
const t = String(ip ?? "").trim()
const t = canonicalIp(ip)
return isUnspecifiedIp(t) ? "" : t
}
@@ -81,13 +99,22 @@ export interface InternetDestCtx {
}
export function isLocalIp(ip: string, ours?: ReadonlySet<string>): boolean {
if (isUnspecifiedIp(ip) || isNonPublicIp(ip)) return true
return Boolean(ours?.has(String(ip ?? "").trim()))
const host = canonicalIp(ip)
if (isUnspecifiedIp(host) || isNonPublicIp(host)) return true
if (!ours || ours.size === 0) return false
if (ours.has(host) || ours.has(String(ip ?? "").trim())) return true
for (const o of ours) {
if (canonicalIp(o) === host) return true
}
return false
}
/**
* Интернет-назначение потока для ASN/страны/сервиса.
* Пустая строка dest нет (не GeoIP IP клиента / GRE-пира).
*
* boundClient: download CDNoverlay берём публичный src даже без :80/:443
* (googlevideo). Client-ISP overlay:well-known не dest (ASN клиента).
*/
export function pickInternetDest(
srcRaw: string,
@@ -112,8 +139,8 @@ export function pickInternetDest(
if (srcIp) {
const srcWk = PEER_WELL_KNOWN_PORTS.has(srcPortEff)
const dstWk = PEER_WELL_KNOWN_PORTS.has(dstPortEff)
if (srcWk && !dstWk) return srcIp
return ""
if (!srcWk && dstWk) return ""
return srcIp
}
return ""
}
@@ -132,10 +159,93 @@ export function pickInternetDest(
return dst
}
/**
* Dest для карты = тот же internet peer, что аналитика/факты.
* (исторически отдельный fallback без well-known trap теперь в pickInternetDest.)
*/
export function pickMapInternetDest(
srcRaw: string,
dstRaw: string,
srcPort: number,
dstPort: number,
ctx?: InternetDestCtx,
): string {
return pickInternetDest(srcRaw, dstRaw, srcPort, dstPort, ctx)
}
/**
* Интернет-сторона потока без топологии: у IPFIX сервис часто в src (Google:443 RFC1918).
* Для куба статистики используйте pickInternetDest.
*/
export function pickInternetPeer(src: string, dst: string, srcPort: number, dstPort: number): string {
return pickInternetDest(src, dst, srcPort, dstPort) || dst
return pickInternetDest(src, dst, srcPort, dstPort) || ""
}
export type FlowDirection = "to_client" | "from_client" | "transit"
export interface FlowEndpoints {
packetSrc: string
packetDst: string
internetPeer: string
peerPort: number
otherPort: number
clientIp: string
direction: FlowDirection
}
function sameHost(a: string, b: string | undefined): boolean {
const x = canonicalIp(a)
const y = canonicalIp(b)
return Boolean(x) && x === y
}
/** Роли концов IPFIX-пакета. 5-tuple не переворачивается. */
export function resolveFlowEndpoints(opts: {
src: string
dst: string
srcPort: number
dstPort: number
ctx?: InternetDestCtx
}): FlowEndpoints {
const packetSrc = usableIp(opts.src)
const packetDst = usableIp(opts.dst)
const internetPeer = pickInternetDest(opts.src, opts.dst, opts.srcPort, opts.dstPort, opts.ctx)
const ours = opts.ctx?.ours
const srcLocal = Boolean(packetSrc) && isLocalIp(packetSrc, ours)
const dstLocal = Boolean(packetDst) && isLocalIp(packetDst, ours)
let peerPort = 0
let otherPort = 0
if (internetPeer) {
if (sameHost(internetPeer, packetSrc) || sameHost(internetPeer, opts.ctx?.natSrc)) {
peerPort = opts.srcPort
otherPort = opts.dstPort
} else if (sameHost(internetPeer, packetDst) || sameHost(internetPeer, opts.ctx?.natDst)) {
peerPort = opts.dstPort
otherPort = opts.srcPort
} else {
peerPort = opts.ctx?.natDstPort || opts.dstPort
otherPort = opts.srcPort
}
}
let clientIp = ""
if (srcLocal && !dstLocal) clientIp = packetSrc
else if (dstLocal && !srcLocal) clientIp = packetDst
else if (srcLocal) clientIp = packetSrc
else if (dstLocal) clientIp = packetDst
let direction: FlowDirection = "transit"
if (internetPeer && clientIp) {
direction = sameHost(internetPeer, packetSrc) ? "to_client" : "from_client"
}
return {
packetSrc,
packetDst,
internetPeer,
peerPort,
otherPort,
clientIp,
direction,
}
}
@@ -6,6 +6,8 @@ import {
} from "./traffic-flow-ingest.js"
import {
buildFlowMapHops,
MAP_COUNTRY_SERVICE_MIN_NODES,
MAP_COUNTRY_SERVICE_NODE_CAP,
MAP_SERVICE_MIN_NODES,
MAP_SERVICE_NODE_CAP,
pickMapServices,
@@ -306,6 +308,15 @@ try {
assert.ok(googleEdge)
assert.equal(googleEdge.clientName, "Alice")
assert.equal((six.serviceEdges ?? []).reduce((n, e) => n + e.bytes, 0), 10_000)
const us = six.countries?.find((s) => s.id === "cc:us")
assert.ok(us, "Google ripe country US")
assert.equal(us.label, "US")
const usEdge = six.countryEdges?.find((e) => e.toId === "cc:us" && e.fromId === "9")
assert.ok(usEdge)
assert.ok(!(six.countryEdges ?? []).some((e) => e.toId.startsWith("svc:")), "страны не смешиваются с svc:*")
const usPath = six.countryPaths?.find((p) => p.serviceId === "cc:us" && p.enId === "9")
assert.ok(usPath)
assert.equal(usPath.clientName, "Alice")
} finally {
resetFlowRingsForTests()
resetIfaceCacheForTests()
@@ -810,4 +821,180 @@ try {
resetFlowCatalogForTests()
}
resetFlowRingsForTests()
resetIfaceCacheForTests()
resetRipeCacheForTests()
disableRipeEnqueueForTests()
seedFlowTopologyForTests(topo)
rememberServerIfaces(7, [
{ ".id": "*2", name: "gre-client" },
{ ".id": "*3", name: "gre-jh-en" },
])
googleRipe()
seedRipeCacheForTests({
prefix: "185.45.12.0/24",
asn: 13335,
country: "NL",
lat: 52.3,
lng: 4.9,
holder: "CLOUDFLARENET, NL",
ok: true,
fetchedAt: Date.now(),
})
ingestParsedFlowsForServerForTests(7, [
payloadFlow("8.8.8.8", 5000),
payloadFlow("185.45.12.10", 5000),
])
try {
resetFlowMapHopsCacheForTests()
const split = await buildFlowMapHops({ minutes: 5, minSharePct: 5 })
const us = split.countries?.find((s) => s.id === "cc:us")
const nl = split.countries?.find((s) => s.id === "cc:nl")
assert.ok(us, "US из ripe Google")
assert.ok(nl, "NL из ripe Cloudflare")
assert.equal(us.bytes, 5000)
assert.equal(nl.bytes, 5000)
assert.ok(split.countryEdges?.some((e) => e.toId === "cc:us" && e.fromId === "9"))
assert.ok(split.countryEdges?.some((e) => e.toId === "cc:nl" && e.fromId === "9"))
assert.ok(!(split.countryEdges ?? []).some((e) => e.toId.startsWith("svc:")))
assert.ok(split.serviceEdges?.some((e) => e.toId === "svc:google"))
assert.ok(!(split.serviceEdges ?? []).some((e) => e.toId.startsWith("cc:")))
assert.ok(split.countryPaths?.some((p) => p.serviceId === "cc:nl" && p.enId === "9"))
} finally {
seedFlowTopologyForTests(null)
resetFlowRingsForTests()
resetIfaceCacheForTests()
resetRipeCacheForTests()
resetFlowCatalogForTests()
}
resetFlowRingsForTests()
resetIfaceCacheForTests()
resetRipeCacheForTests()
disableRipeEnqueueForTests()
seedFlowTopologyForTests(topo)
rememberServerIfaces(7, [
{ ".id": "*2", name: "gre-client" },
{ ".id": "*3", name: "gre-jh-en" },
])
googleRipe()
ingestParsedFlowsForServerForTests(7, [
payloadFlow("8.8.8.8", 600),
payloadFlow("203.0.113.50", 9400),
])
try {
resetFlowMapHopsCacheForTests()
const nested = await buildFlowMapHops({ minutes: 5, minSharePct: 5 })
const usGroup = nested.countryServiceGroups?.find((g) => g.countryId === "cc:us")
assert.ok(usGroup, "группа сервисов cc:us")
const nestedGoogle = usGroup.services.find((s) => s.id === "cc:us|svc:google")
assert.ok(nestedGoogle, "cc:us|svc:google в группе")
assert.equal(nestedGoogle.label, "Google")
assert.ok(Math.abs(nestedGoogle.share - 1) < 0.01, "доля от байтов страны (600/600), не окна")
const nestedEdge = usGroup.edges.find((e) => e.toId === "cc:us|svc:google")
assert.ok(nestedEdge)
assert.equal(nestedEdge.fromId, "cc:us", "ребро вложенного слоя: страна → сервис")
assert.equal(nestedEdge.bytes, 600)
assert.ok(!usGroup.edges.some((e) => e.fromId.startsWith("svc:")), "fromId вложенных рёбер не svc:*")
const nestedPath = usGroup.paths.find((p) => p.serviceId === "cc:us|svc:google")
assert.ok(nestedPath)
assert.equal(nestedPath.enId, "9", "путь держит реальный EN для подсветки HR→JH→EN")
assert.equal(nestedPath.clientName, "Alice")
const otherGroup = nested.countryServiceGroups?.find((g) => g.countryId === "cc:other")
assert.ok(otherGroup, "группа cc:other (Прочее-страна)")
assert.ok(otherGroup.services.some((s) => s.id === "cc:other|svc:other"))
} finally {
seedFlowTopologyForTests(null)
resetFlowRingsForTests()
resetIfaceCacheForTests()
resetRipeCacheForTests()
resetFlowCatalogForTests()
}
resetFlowRingsForTests()
resetIfaceCacheForTests()
resetRipeCacheForTests()
disableRipeEnqueueForTests()
seedFlowTopologyForTests(topo)
rememberServerIfaces(7, [
{ ".id": "*2", name: "gre-client" },
{ ".id": "*3", name: "gre-jh-en" },
])
googleRipe()
seedRipeCacheForTests({
prefix: "185.45.12.0/24",
asn: 13335,
country: "NL",
lat: 52.3,
lng: 4.9,
holder: "CLOUDFLARENET, NL",
ok: true,
fetchedAt: Date.now(),
})
ingestParsedFlowsForServerForTests(7, [
payloadFlow("8.8.8.8", 5000),
payloadFlow("185.45.12.10", 5000),
])
try {
resetFlowMapHopsCacheForTests()
const nestedSplit = await buildFlowMapHops({ minutes: 5, minSharePct: 5 })
const usNested = nestedSplit.countryServiceGroups?.find((g) => g.countryId === "cc:us")
const nlNested = nestedSplit.countryServiceGroups?.find((g) => g.countryId === "cc:nl")
assert.ok(usNested && nlNested, "группы у обеих стран")
assert.ok(usNested.services.every((s) => s.id.startsWith("cc:us|")), "сервисы US только cc:us|*")
assert.ok(nlNested.services.every((s) => s.id.startsWith("cc:nl|")), "сервисы NL только cc:nl|*")
assert.ok(usNested.services.some((s) => s.id === "cc:us|svc:google"), "Google в US")
assert.ok(nlNested.services.some((s) => s.id === "cc:nl|svc:cloudflare"), "Cloudflare в NL")
assert.ok(!nlNested.services.some((s) => s.id === "cc:us|svc:google"), "Google US не утек в NL")
assert.ok(usNested.edges.every((e) => e.fromId === "cc:us"))
assert.ok(nlNested.edges.every((e) => e.fromId === "cc:nl"))
} finally {
seedFlowTopologyForTests(null)
resetFlowRingsForTests()
resetIfaceCacheForTests()
resetRipeCacheForTests()
resetFlowCatalogForTests()
}
resetFlowRingsForTests()
resetIfaceCacheForTests()
resetRipeCacheForTests()
disableRipeEnqueueForTests()
seedFlowTopologyForTests(topo)
rememberServerIfaces(7, [
{ ".id": "*2", name: "gre-client" },
{ ".id": "*3", name: "gre-jh-en" },
])
googleRipe()
for (const b of smallBrands) seedRipeAsn(b.ip, b.asn, b.holder)
ingestParsedFlowsForServerForTests(7, [
payloadFlow("8.8.8.8", 5000),
...smallBrands.map((b) => payloadFlow(b.ip, b.bytes)),
])
try {
resetFlowMapHopsCacheForTests()
const nestedTop = await buildFlowMapHops({ minutes: 5, minSharePct: 5 })
const usTop = nestedTop.countryServiceGroups?.find((g) => g.countryId === "cc:us")
assert.ok(usTop)
assert.equal(
usTop.services.length,
MAP_COUNTRY_SERVICE_MIN_NODES,
"мелкий хвост держится минимумом узлов внутри страны",
)
assert.ok(usTop.services.some((s) => s.id === "cc:us|svc:google"))
resetFlowMapHopsCacheForTests()
const nestedAll = await buildFlowMapHops({ minutes: 5, minSharePct: 0 })
const usAll = nestedAll.countryServiceGroups?.find((g) => g.countryId === "cc:us")
assert.ok(usAll)
assert.equal(usAll.services.length, MAP_COUNTRY_SERVICE_NODE_CAP, "cap вложенного слоя = 8")
assert.ok(!usAll.services.some((s) => s.id === "cc:us|svc:epic"), "ранг 9+ скрыт")
assert.ok(!usAll.services.some((s) => s.id === "cc:us|svc:riot"))
} finally {
seedFlowTopologyForTests(null)
resetFlowRingsForTests()
resetIfaceCacheForTests()
resetRipeCacheForTests()
resetFlowCatalogForTests()
}
console.log("traffic-flow-map-hops.test.ts: ok")
+271 -131
View File
@@ -1,16 +1,18 @@
import { eq } from "drizzle-orm"
import type { FlowMapHop, FlowMapHopsDto, FlowMapService, FlowMapServiceEdge, FlowMapServicePath } from "@mmapp/contracts/traffic-flow"
import type { FlowMapCountryServiceGroup, FlowMapHop, FlowMapHopsDto, FlowMapService, FlowMapServiceEdge, FlowMapServicePath } from "@mmapp/contracts/traffic-flow"
import { db } from "../db/index.js"
import { userInterfaceBindings } from "../db/schema.js"
import { flowRowMatchesFilter } from "./traffic-flow-apps.js"
import { OTHER_SERVICE, mapServiceNodeId } from "./traffic-flow-brands.js"
import { OTHER_SERVICE, isNamedInternetService, mapCountryNodeId, mapCountryServiceNodeId, mapServiceNodeId, resolveRipeCountry } from "./traffic-flow-brands.js"
import type { FlowIpMeta } from "./traffic-flow-ripe.js"
import { refreshFlowCatalogInBackground } from "./traffic-flow-classify.js"
import { dedupFlowRowsAcrossExporters, dedupFlowRowsMaxBytes } from "./traffic-flow-dedup.js"
import { getFlowListenerState, listFlowRowsForWindow } from "./traffic-flow-ingest.js"
import { resolveIfaceName } from "./traffic-flow-ifaces.js"
import { classifyFlowPlane, shouldKeepPlane } from "./traffic-flow-planes.js"
import { destCtxForIface, mapInternetBrand } from "./traffic-flow-dest.js"
import { pickInternetDest } from "./traffic-flow-ip.js"
import { resolveFlowIp } from "./traffic-flow-geoip.js"
import { resolveFlowEndpoints } from "./traffic-flow-ip.js"
import { geoipReadersStatus, resolveFlowIp } from "./traffic-flow-geoip.js"
import { getTrafficFlowSettingsRow } from "./traffic-flow-settings.js"
import { loadFlowTopology, resolveClient, resolveEn, getServerCatalog, type FlowTopology } from "./traffic-flow-topology.js"
import { flowDataEpoch } from "./traffic-flow-engine.js"
@@ -19,6 +21,11 @@ export const DEFAULT_MAP_SERVICE_MIN_SHARE_PCT = 5
export const MAP_SERVICE_NODE_CAP = 20
/** Минимум узлов-брендов на карте, даже если доля ниже порога. */
export const MAP_SERVICE_MIN_NODES = 8
/** Cap сервисов внутри раскрытой страны (база доли — байты страны, не окна). */
export const MAP_COUNTRY_SERVICE_NODE_CAP = 8
/** Минимум узлов-сервисов внутри страны, даже если доля ниже порога. */
export const MAP_COUNTRY_SERVICE_MIN_NODES = 4
export const MAP_COUNTRY_CATEGORY = "Страна"
const HOPS_CACHE_TTL_MS = 2000
export interface FlowMapHopsQuery {
@@ -63,6 +70,32 @@ interface DstAcc {
fromBytes: Map<string, FromAcc>
}
interface DestTotal {
label: string
category: string
bytes: number
}
interface DestEdgeAcc {
fromId: string
toId: string
bytes: number
bytesFwd: number
bytesRev: number
clients: Map<string, string>
}
interface DestPathAcc {
clientId: string
clientName: string
viaId: string
viaName: string
enId: string
enName: string
serviceId: string
bytes: number
}
function bumpClient(clients: Map<string, ClientAcc>, bytes: number, client: { userId: string; name: string } | null): void {
const id = client?.userId || "—"
const name = client?.name || "—"
@@ -99,12 +132,194 @@ export function clampMapServiceMinSharePct(n: unknown): number {
}
/** Доля от payload окна; порог ИЛИ топ-N, затем cap. */
export function pickMapServices(ranked: FlowMapService[], minSharePct: number): FlowMapService[] {
if (minSharePct <= 0) return ranked.slice(0, MAP_SERVICE_NODE_CAP)
export function pickMapServices(
ranked: FlowMapService[],
minSharePct: number,
cap: number = MAP_SERVICE_NODE_CAP,
minNodes: number = MAP_SERVICE_MIN_NODES,
): FlowMapService[] {
if (minSharePct <= 0) return ranked.slice(0, cap)
const minShare = minSharePct / 100
return ranked
.filter((s, i) => s.share >= minShare || i < MAP_SERVICE_MIN_NODES)
.slice(0, MAP_SERVICE_NODE_CAP)
.filter((s, i) => s.share >= minShare || i < minNodes)
.slice(0, cap)
}
function bumpDestTotal(totals: Map<string, DestTotal>, id: string, label: string, category: string, bytes: number): void {
const prev = totals.get(id)
if (prev) {
prev.bytes += bytes
return
}
totals.set(id, { label, category, bytes })
}
function bumpDestFrom(
edges: Map<string, DestEdgeAcc>,
paths: Map<string, DestPathAcc>,
toId: string,
from: FromAcc,
exporterId: string,
fromId: string,
enName: string,
viaName: string,
/** fromId ребра, если отличается от EN (вложенный слой: страна → сервис). */
edgeFromId: string = fromId,
): void {
const edgeKey = `${edgeFromId}|${toId}`
const prevEdge = edges.get(edgeKey)
const namedClients = new Map<string, string>()
for (const [id, c] of from.clients) {
if (id !== "—") namedClients.set(id, c.name)
}
if (prevEdge) {
prevEdge.bytes += from.bytes
prevEdge.bytesFwd += from.bytes
for (const [id, name] of namedClients) prevEdge.clients.set(id, name)
} else {
edges.set(edgeKey, {
fromId: edgeFromId,
toId,
bytes: from.bytes,
bytesFwd: from.bytes,
bytesRev: 0,
clients: namedClients,
})
}
for (const [clientId, c] of from.clients) {
const pathKey = `${clientId}|${fromId}|${toId}`
const prevPath = paths.get(pathKey)
if (prevPath) {
prevPath.bytes += c.bytes
if (exporterId !== fromId && prevPath.viaId === fromId) {
prevPath.viaId = exporterId
prevPath.viaName = viaName
}
if (prevPath.clientName === "—" && c.name !== "—") prevPath.clientName = c.name
} else {
paths.set(pathKey, {
clientId,
clientName: c.name,
viaId: exporterId,
viaName,
enId: fromId,
enName,
serviceId: toId,
bytes: c.bytes,
})
}
}
}
function finalizeDestLayer(
totals: Map<string, DestTotal>,
edges: Map<string, DestEdgeAcc>,
paths: Map<string, DestPathAcc>,
windowSec: number,
minSharePct: number,
shareBase: number,
cap: number = MAP_SERVICE_NODE_CAP,
minNodes: number = MAP_SERVICE_MIN_NODES,
): { nodes: FlowMapService[]; edges: FlowMapServiceEdge[]; paths: FlowMapServicePath[] } {
const nodes = pickMapServices(
[...totals.entries()]
.map(([id, s]) => ({
id,
label: s.label,
category: s.category,
bytes: s.bytes,
bps: (s.bytes * 8) / windowSec,
share: shareBase > 0 ? s.bytes / shareBase : 0,
}))
.sort((a, b) => b.bytes - a.bytes),
minSharePct,
cap,
minNodes,
)
const keep = new Set(nodes.map((s) => s.id))
const outEdges: FlowMapServiceEdge[] = [...edges.values()]
.filter((e) => keep.has(e.toId))
.map((e) => {
const clients = [...e.clients.entries()].map(([id, name]) => ({ id, name }))
const first = clients[0]
return {
fromId: e.fromId,
toId: e.toId,
bytes: e.bytes,
bps: (e.bytes * 8) / windowSec,
bpsFwd: (e.bytesFwd * 8) / windowSec,
bpsRev: (e.bytesRev * 8) / windowSec,
...(first ? { clientId: first.id, clientName: first.name } : {}),
...(clients.length ? { clients } : {}),
}
})
.sort((a, b) => b.bytes - a.bytes)
const outPaths: FlowMapServicePath[] = [...paths.values()]
.filter((p) => keep.has(p.serviceId))
.map((p) => ({
clientId: p.clientId,
clientName: p.clientName,
viaId: p.viaId,
viaName: p.viaName,
enId: p.enId,
enName: p.enName,
serviceId: p.serviceId,
bytes: p.bytes,
bps: (p.bytes * 8) / windowSec,
}))
.sort((a, b) => b.bps - a.bps)
return { nodes, edges: outEdges, paths: outPaths }
}
/** Группы сервисов по странам: только страны, прошедшие отбор слоя стран; доля — от байтов страны, не окна. */
function buildCountryServiceGroups(
nestedTotals: Map<string, DestTotal>,
nestedEdges: Map<string, DestEdgeAcc>,
nestedPaths: Map<string, DestPathAcc>,
countryNodes: FlowMapService[],
windowSec: number,
minSharePct: number,
): FlowMapCountryServiceGroup[] {
const groups: FlowMapCountryServiceGroup[] = []
for (const country of countryNodes) {
const prefix = `${country.id}|`
const totals = new Map<string, DestTotal>()
const edges = new Map<string, DestEdgeAcc>()
const paths = new Map<string, DestPathAcc>()
for (const [id, t] of nestedTotals) {
if (id.startsWith(prefix)) totals.set(id, t)
}
if (totals.size === 0) continue
for (const [key, e] of nestedEdges) {
if (e.toId.startsWith(prefix)) edges.set(key, e)
}
for (const [key, p] of nestedPaths) {
if (p.serviceId.startsWith(prefix)) paths.set(key, p)
}
const out = finalizeDestLayer(
totals,
edges,
paths,
windowSec,
minSharePct,
country.bytes,
MAP_COUNTRY_SERVICE_NODE_CAP,
MAP_COUNTRY_SERVICE_MIN_NODES,
)
groups.push({ countryId: country.id, services: out.nodes, edges: out.edges, paths: out.paths })
}
return groups
}
function countryDestFromRipe(ripe: FlowIpMeta | null, destKey: string): { id: string; label: string; category: string } {
if (!destKey || destKey === "__other__" || !ripe?.ok) {
return { id: mapCountryNodeId(""), label: OTHER_SERVICE, category: MAP_COUNTRY_CATEGORY }
}
const iso = resolveRipeCountry(ripe.country, ripe.asn, ripe.holder)
if (!iso) {
return { id: mapCountryNodeId(""), label: OTHER_SERVICE, category: MAP_COUNTRY_CATEGORY }
}
return { id: mapCountryNodeId(iso), label: iso, category: MAP_COUNTRY_CATEGORY }
}
function hopsQueryKey(q: FlowMapHopsQuery, minSharePct: number): string {
@@ -192,6 +407,7 @@ async function resolveMinSharePct(q: FlowMapHopsQuery): Promise<number> {
}
async function buildFlowMapHopsUncached(q: FlowMapHopsQuery, minSharePct: number): Promise<FlowMapHopsDto> {
refreshFlowCatalogInBackground()
const windowSec = Math.max(60, q.minutes * 60)
const raw = await listFlowRowsForWindow(q.minutes)
const allow = q.userId ? await userIfaceAllow(q.userId) : null
@@ -334,18 +550,19 @@ async function buildFlowMapHopsUncached(q: FlowMapHopsQuery, minSharePct: number
const inName = resolveIfaceName(r.serverId, r.inIface).name
const outName = resolveIfaceName(r.serverId, r.outIface).name
totalBytes += r.bytes
const dest = pickInternetDest(
r.src,
r.dst,
r.srcPort,
r.dstPort,
destCtxForIface(topo, r.serverId, inName, {
const ep = resolveFlowEndpoints({
src: r.src,
dst: r.dst,
srcPort: r.srcPort,
dstPort: r.dstPort,
ctx: destCtxForIface(topo, r.serverId, inName, {
natSrc: r.natSrc,
natDst: r.natDst,
natSrcPort: r.natSrcPort,
natDstPort: r.natDstPort,
}),
)
})
const dest = ep.internetPeer
const destKey = dest || "__other__"
const client = resolveMapClient(topo, r.serverId, inName, outName)
const prevDst = dstAcc.get(destKey)
@@ -356,8 +573,8 @@ async function buildFlowMapHopsUncached(q: FlowMapHopsQuery, minSharePct: number
const acc: DstAcc = {
bytes: r.bytes,
proto: r.proto,
dstPort: r.dstPort,
srcPort: r.srcPort,
dstPort: ep.peerPort || r.dstPort,
srcPort: ep.otherPort || r.srcPort,
fromBytes: new Map(),
}
bumpFrom(acc, String(r.serverId), r.bytes, client)
@@ -365,25 +582,15 @@ async function buildFlowMapHopsUncached(q: FlowMapHopsQuery, minSharePct: number
}
}
const svcTotals = new Map<string, { label: string; category: string; bytes: number }>()
const svcEdges = new Map<string, {
fromId: string
toId: string
bytes: number
bytesFwd: number
bytesRev: number
clients: Map<string, string>
}>()
const svcPaths = new Map<string, {
clientId: string
clientName: string
viaId: string
viaName: string
enId: string
enName: string
serviceId: string
bytes: number
}>()
const svcTotals = new Map<string, DestTotal>()
const svcEdges = new Map<string, DestEdgeAcc>()
const svcPaths = new Map<string, DestPathAcc>()
const ccTotals = new Map<string, DestTotal>()
const ccEdges = new Map<string, DestEdgeAcc>()
const ccPaths = new Map<string, DestPathAcc>()
const nestedTotals = new Map<string, DestTotal>()
const nestedEdges = new Map<string, DestEdgeAcc>()
const nestedPaths = new Map<string, DestPathAcc>()
for (const h of hops.values()) {
if (h.kind !== "gre" || !h.toId) continue
@@ -420,114 +627,41 @@ async function buildFlowMapHopsUncached(q: FlowMapHopsQuery, minSharePct: number
const classified = dst && dst !== "__other__"
? mapInternetBrand(dst, acc.proto, acc.dstPort, acc.srcPort, ripe)
: { service: OTHER_SERVICE, category: OTHER_SERVICE }
const toId = mapServiceNodeId(classified.service)
const prevSvc = svcTotals.get(toId)
if (prevSvc) prevSvc.bytes += acc.bytes
else svcTotals.set(toId, { label: classified.service, category: classified.category, bytes: acc.bytes })
const svcId = mapServiceNodeId(classified.service)
bumpDestTotal(svcTotals, svcId, classified.service, classified.category, acc.bytes)
const country = countryDestFromRipe(ripe, dst)
bumpDestTotal(ccTotals, country.id, country.label, country.category, acc.bytes)
const nestedId = mapCountryServiceNodeId(country.id, svcId)
bumpDestTotal(nestedTotals, nestedId, classified.service, classified.category, acc.bytes)
for (const [exporterId, from] of acc.fromBytes) {
const fromId = anchorEnId(exporterId)
if (!fromId) continue
const edgeKey = `${fromId}|${toId}`
const prevEdge = svcEdges.get(edgeKey)
const namedClients = new Map<string, string>()
for (const [id, c] of from.clients) {
if (id !== "—") namedClients.set(id, c.name)
}
if (prevEdge) {
prevEdge.bytes += from.bytes
prevEdge.bytesFwd += from.bytes
for (const [id, name] of namedClients) prevEdge.clients.set(id, name)
} else {
svcEdges.set(edgeKey, {
fromId,
toId,
bytes: from.bytes,
bytesFwd: from.bytes,
bytesRev: 0,
clients: namedClients,
})
}
const enName = nodeName(fromId)
const viaName = nodeName(exporterId)
for (const [clientId, c] of from.clients) {
const pathKey = `${clientId}|${fromId}|${toId}`
const prevPath = svcPaths.get(pathKey)
if (prevPath) {
prevPath.bytes += c.bytes
if (exporterId !== fromId && prevPath.viaId === fromId) {
prevPath.viaId = exporterId
prevPath.viaName = viaName
}
if (prevPath.clientName === "—" && c.name !== "—") prevPath.clientName = c.name
} else {
svcPaths.set(pathKey, {
clientId,
clientName: c.name,
viaId: exporterId,
viaName,
enId: fromId,
enName,
serviceId: toId,
bytes: c.bytes,
})
}
}
bumpDestFrom(svcEdges, svcPaths, svcId, from, exporterId, fromId, enName, viaName)
bumpDestFrom(ccEdges, ccPaths, country.id, from, exporterId, fromId, enName, viaName)
bumpDestFrom(nestedEdges, nestedPaths, nestedId, from, exporterId, fromId, enName, viaName, country.id)
}
}
const namedBytes = [...svcTotals.values()]
.filter((s) => s.label !== OTHER_SERVICE)
.filter((s) => isNamedInternetService(s.label, s.category))
.reduce((n, s) => n + s.bytes, 0)
const unclassifiedBytes = Math.max(0, totalBytes - namedBytes)
const shareBase = totalBytes > 0 ? totalBytes : namedBytes
const services = pickMapServices(
[...svcTotals.entries()]
.map(([id, s]) => ({
id,
label: s.label,
category: s.category,
bytes: s.bytes,
bps: (s.bytes * 8) / windowSec,
share: shareBase > 0 ? s.bytes / shareBase : 0,
}))
.sort((a, b) => b.bytes - a.bytes),
const servicesOut = finalizeDestLayer(svcTotals, svcEdges, svcPaths, windowSec, minSharePct, shareBase)
const countriesOut = finalizeDestLayer(ccTotals, ccEdges, ccPaths, windowSec, minSharePct, shareBase)
const countryServiceGroups = buildCountryServiceGroups(
nestedTotals,
nestedEdges,
nestedPaths,
countriesOut.nodes,
windowSec,
minSharePct,
)
const keepSvc = new Set(services.map((s) => s.id))
const serviceEdges: FlowMapServiceEdge[] = [...svcEdges.values()]
.filter((e) => keepSvc.has(e.toId))
.map((e) => {
const clients = [...e.clients.entries()].map(([id, name]) => ({ id, name }))
const first = clients[0]
return {
fromId: e.fromId,
toId: e.toId,
bytes: e.bytes,
bps: (e.bytes * 8) / windowSec,
bpsFwd: (e.bytesFwd * 8) / windowSec,
bpsRev: (e.bytesRev * 8) / windowSec,
...(first ? { clientId: first.id, clientName: first.name } : {}),
...(clients.length ? { clients } : {}),
}
})
.sort((a, b) => b.bytes - a.bytes)
const servicePaths: FlowMapServicePath[] = [...svcPaths.values()]
.filter((p) => keepSvc.has(p.serviceId))
.map((p) => ({
clientId: p.clientId,
clientName: p.clientName,
viaId: p.viaId,
viaName: p.viaName,
enId: p.enId,
enName: p.enName,
serviceId: p.serviceId,
bytes: p.bytes,
bps: (p.bytes * 8) / windowSec,
}))
.sort((a, b) => b.bps - a.bps)
const listener = getFlowListenerState()
const geo = geoipReadersStatus()
return {
hops: [...hops.values()]
.map((a) => toHop(a, windowSec))
@@ -538,9 +672,15 @@ async function buildFlowMapHopsUncached(q: FlowMapHopsQuery, minSharePct: number
totalBytes,
namedBytes,
unclassifiedBytes,
services,
serviceEdges,
servicePaths,
asnLoaded: geo.asnLoaded,
countryLoaded: geo.countryLoaded,
services: servicesOut.nodes,
serviceEdges: servicesOut.edges,
servicePaths: servicesOut.paths,
countries: countriesOut.nodes,
countryEdges: countriesOut.edges,
countryPaths: countriesOut.paths,
countryServiceGroups,
mapServiceMinSharePct: minSharePct,
dedupApplied: wantDedup,
excludeMeshApplied: excludeMesh,
+7 -3
View File
@@ -1,3 +1,5 @@
import { canonicalIp } from "./traffic-flow-ip.js"
export interface ParsedFlow {
src: string
dst: string
@@ -44,11 +46,13 @@ export function normalizeParsedFlow(flow: ParsedFlowInput): ParsedFlow {
return {
...emptyParsedFlow(),
...flow,
nextHop: flow.nextHop ?? "",
src: canonicalIp(flow.src) || flow.src || "",
dst: canonicalIp(flow.dst) || flow.dst || "",
nextHop: canonicalIp(flow.nextHop ?? ""),
flowStartMs: flow.flowStartMs ?? 0,
flowEndMs: flow.flowEndMs ?? 0,
natSrc: flow.natSrc ?? "",
natDst: flow.natDst ?? "",
natSrc: canonicalIp(flow.natSrc ?? ""),
natDst: canonicalIp(flow.natDst ?? ""),
natSrcPort: flow.natSrcPort ?? 0,
natDstPort: flow.natDstPort ?? 0,
inIface: flow.inIface ?? "",
+2 -2
View File
@@ -1,5 +1,5 @@
import { dbAll, dbQuery } from "../db/index.js"
import { ipv4ToInt, isNonPublicIp, parseCidrV4 } from "./traffic-flow-ip.js"
import { canonicalIp, ipv4ToInt, isNonPublicIp, parseCidrV4 } from "./traffic-flow-ip.js"
import { resolveRipeCountry } from "./traffic-flow-brands.js"
export interface FlowIpMeta {
@@ -264,7 +264,7 @@ function negative(prefix: string): FlowIpMeta {
export function lookupRipeCached(ip: string): FlowIpMeta | null {
loadSqlite()
const trimmed = String(ip ?? "").trim()
const trimmed = canonicalIp(ip)
lastCandidateCount = 0
if (!trimmed) return null
if (isNonPublicIp(trimmed)) {
@@ -2,6 +2,7 @@ import { db, dbAll } from "../db/index.js"
import { parseJsonArray } from "../db/json.js"
import { appUsers, servers, userInterfaceBindings } from "../db/schema.js"
import { mapRosInterfaceType, parseRawInterfaces } from "../modules/users/iface-type.js"
import { canonicalIp } from "./traffic-flow-ip.js"
import type { PlaneTopology } from "./traffic-flow-planes.js"
export interface FlowClientBinding {
@@ -176,10 +177,12 @@ export function flowOursHosts(topo: FlowTopology | null | undefined): Set<string
const ours = new Set<string>()
if (!topo) return ours
for (const h of topo.enHosts) {
if (h) ours.add(h)
const ip = canonicalIp(h)
if (ip) ours.add(ip)
}
for (const h of topo.jhHosts) {
if (h) ours.add(h)
const ip = canonicalIp(h)
if (ip) ours.add(ip)
}
return ours
}
@@ -3,6 +3,7 @@
import { useMemo } from "react"
import { type ColumnDef, getCoreRowModel, useReactTable } from "@tanstack/react-table"
import type { FlowTalkerDto } from "@mmapp/contracts/traffic-flow"
import { ArrowDownIcon, ArrowUpIcon, MinusIcon } from "lucide-react"
import { DataGridShell } from "@/components/data-grids/shared/data-grid-shell"
import {
DATA_GRID_CELL_PAD,
@@ -19,6 +20,40 @@ function formatBytes(n: number): string {
return `${n} Б`
}
function formatEndpoint(ip: string | undefined, port: number | undefined): string {
const host = String(ip ?? "").trim()
if (!host) return "—"
return port ? `${host}:${port}` : host
}
function packetTuple(row: FlowTalkerDto): string {
const src = formatEndpoint(row.src, row.srcPort)
const dst = formatEndpoint(row.dst, row.dstPort)
return `${src}${dst}`
}
function FlowDirectionMark({ direction }: { direction: FlowTalkerDto["direction"] }) {
if (direction === "to_client") {
return (
<span className="inline-flex text-muted-foreground" title="Download: интернет → клиент">
<ArrowDownIcon className="size-3" />
</span>
)
}
if (direction === "from_client") {
return (
<span className="inline-flex text-muted-foreground" title="Upload: клиент → интернет">
<ArrowUpIcon className="size-3" />
</span>
)
}
return (
<span className="inline-flex text-muted-foreground" title="Транзит">
<MinusIcon className="size-3" />
</span>
)
}
function TrafficFlowsDataGrid({
rows,
emptyHint,
@@ -30,9 +65,16 @@ function TrafficFlowsDataGrid({
() => [
{
id: "client",
accessorFn: (r) => r.clientName ?? "",
accessorFn: (r) => r.clientName || r.clientIp || "",
header: () => <span className="text-xs font-medium text-muted-foreground">Клиент</span>,
cell: ({ row }) => <span className="text-xs">{row.original.clientName || "—"}</span>,
cell: ({ row }) => (
<span className="flex min-w-0 flex-col gap-0.5">
<span className="text-xs truncate">{row.original.clientName || "—"}</span>
{row.original.clientIp ? (
<span className="font-mono text-[10px] text-muted-foreground truncate">{row.original.clientIp}</span>
) : null}
</span>
),
meta: { headerClassName: DATA_GRID_CELL_PAD_FIRST, cellClassName: DATA_GRID_CELL_PAD_FIRST },
},
{
@@ -43,27 +85,26 @@ function TrafficFlowsDataGrid({
meta: { headerClassName: DATA_GRID_CELL_PAD, cellClassName: DATA_GRID_CELL_PAD },
},
{
id: "src",
accessorKey: "src",
header: () => <span className="text-xs font-medium text-muted-foreground">Src</span>,
cell: ({ row }) => (
<span className="font-mono text-xs">
{row.original.src}
{row.original.srcPort ? `:${row.original.srcPort}` : ""}
</span>
),
id: "direction",
accessorFn: (r) => r.direction ?? "",
header: () => <span className="text-xs font-medium text-muted-foreground">Напр.</span>,
cell: ({ row }) => <FlowDirectionMark direction={row.original.direction} />,
meta: { headerClassName: DATA_GRID_CELL_PAD, cellClassName: DATA_GRID_CELL_PAD },
},
{
id: "dst",
accessorKey: "dst",
header: () => <span className="text-xs font-medium text-muted-foreground">Dst</span>,
cell: ({ row }) => (
<span className="font-mono text-xs">
{row.original.dst}
{row.original.dstPort ? `:${row.original.dstPort}` : ""}
</span>
),
id: "internet",
accessorFn: (r) => r.internetPeer ?? r.dst,
header: () => <span className="text-xs font-medium text-muted-foreground">Интернет</span>,
cell: ({ row }) => {
const r = row.original
const label = formatEndpoint(r.internetPeer, r.internetPeerPort)
const tuple = packetTuple(r)
return (
<span className="font-mono text-xs truncate max-w-[220px]" title={tuple}>
{label}
</span>
)
},
meta: { headerClassName: DATA_GRID_CELL_PAD, cellClassName: DATA_GRID_CELL_PAD },
},
{
+29 -2
View File
@@ -20,9 +20,29 @@ const COUNTRY_NAMES: Record<string, string> = {
HK: "Гонконг",
}
/** Country name in Russian (fallback to code) */
let regionNames: Intl.DisplayNames | null | undefined
function regionDisplayName(iso: string): string | undefined {
try {
if (regionNames === undefined) {
regionNames = typeof Intl !== "undefined" && "DisplayNames" in Intl
? new Intl.DisplayNames(["ru"], { type: "region" })
: null
}
return regionNames?.of(iso) ?? undefined
} catch {
return undefined
}
}
/** Country name in Russian (fallback to ISO code) */
export function countryName(code: string): string {
return COUNTRY_NAMES[code.toUpperCase()] ?? code
const iso = code.toUpperCase()
if (!iso) return code
if (COUNTRY_NAMES[iso]) return COUNTRY_NAMES[iso]
const intl = regionDisplayName(iso)
if (intl && intl !== iso) return intl
return iso
}
interface FlagProps {
@@ -39,6 +59,13 @@ function nearestCdnSize(px: number): number {
return CDN_SIZES.find(s => s >= px) ?? CDN_SIZES[CDN_SIZES.length - 1]
}
/** CDN URL for SVG `<image href>` (flagcdn widths only). */
export function flagCdnUrl(code: string, size = 40): string | null {
const lower = code.toLowerCase()
if (!/^[a-z]{2}$/.test(lower)) return null
return `https://flagcdn.com/w${nearestCdnSize(size)}/${lower}.png`
}
/**
* Renders a flag <img> for a given ISO 3166-1 alpha-2 country code.
* Source: https://flagcdn.com — free CDN, no API key needed.
@@ -0,0 +1,17 @@
"use client"
import { flagCdnUrl } from "@/components/flag"
export function CountryFlagSvg({ iso, size = 22 }: { iso: string; size?: number }) {
const url = flagCdnUrl(iso, size)
if (!url) return null
const h = Math.round(size * 0.75)
return (
<image
href={url}
width={size}
height={h}
preserveAspectRatio="xMidYMid meet"
/>
)
}
@@ -74,6 +74,14 @@ export function ServiceBrandIcon({ label, size = 22 }: { label: string; size?: n
<path d="M10.2 9.2v5.6L15.6 12Z" fill="#fff" />
</BrandSvg>
)
case "instagram":
return (
<BrandSvg size={size}>
<rect x="3" y="3" width="18" height="18" rx="5" fill="#E4405F" />
<circle cx="12" cy="12.2" r="4.1" fill="none" stroke="#fff" strokeWidth="1.8" />
<circle cx="16.3" cy="7.7" r="1.15" fill="#fff" />
</BrandSvg>
)
case "netflix":
return (
<BrandSvg size={size}>
+29 -7
View File
@@ -48,13 +48,24 @@ export function parseYmd(s: string): Date | null {
return Number.isNaN(d.getTime()) ? null : d
}
/** Значение периода — дата (YYYY-MM-DD) или полный ISO-таймстамп (пресет «24 ч»). */
export function parseRangeDate(s: string): Date | null {
if (s.includes("T")) {
const d = new Date(s)
return Number.isNaN(d.getTime()) ? null : d
}
return parseYmd(s)
}
export function rangeForPreset(preset: PeriodPreset, now = new Date()): DateRangeYmd {
const to = formatYmd(now)
if (preset === "today") return { from: to, to }
if (preset === "24h") {
const from = new Date(now)
from.setDate(from.getDate() - 1)
return { from: formatYmd(from), to }
// Последние 24 часа от сейчас: полный ISO, иначе date-only «вчера…сегодня» даёт до ~48 ч.
return {
from: new Date(now.getTime() - 24 * 3600_000).toISOString(),
to: now.toISOString(),
}
}
if (preset === "7d") {
const from = new Date(now)
@@ -110,15 +121,20 @@ export function rangeToSelector(range: DateRangeYmd): DateSelectorValue {
return {
period: "day",
operator: "between",
startDate: parseYmd(range.from) ?? undefined,
endDate: parseYmd(range.to) ?? undefined,
startDate: parseRangeDate(range.from) ?? undefined,
endDate: parseRangeDate(range.to) ?? undefined,
}
}
function formatRangeLabel(range: DateRangeYmd): string {
const from = parseYmd(range.from)
const to = parseYmd(range.to)
const from = parseRangeDate(range.from)
const to = parseRangeDate(range.to)
if (!from || !to) return "Период"
if (range.from.includes("T") || range.to.includes("T")) {
return formatYmd(from) === formatYmd(to)
? `${format(from, "d MMM yyyy HH:mm", { locale: ru })} ${format(to, "HH:mm", { locale: ru })}`
: `${format(from, "d MMM HH:mm", { locale: ru })} ${format(to, "d MMM HH:mm", { locale: ru })}`
}
if (range.from === range.to) return format(from, "d MMM yyyy", { locale: ru })
return `${format(from, "d MMM", { locale: ru })} ${format(to, "d MMM yyyy", { locale: ru })}`
}
@@ -145,6 +161,12 @@ export function PeriodSelector({
)
const activePreset = PRESETS.find((p) => {
// «24 ч» хранится ISO-таймстампами: сверяем разбором (длительность окна), а не строками.
if (p.id === "24h") {
const f = parseRangeDate(range.from)
const t = parseRangeDate(range.to)
return Boolean(f && t && Math.abs((t.getTime() - f.getTime()) - 24 * 3600_000) < 60_000)
}
const r = rangeForPreset(p.id)
return r.from === range.from && r.to === range.to
})?.id
+21
View File
@@ -39,6 +39,7 @@ const W = 1240
const H = 580
const MARGIN = 72
const SERVICE_COL_W = 150
export { SERVICE_COL_W }
/** Карточка конечного сервиса на карте (центр = позиция узла). */
export const MAP_SERVICE_NODE_W = 86
@@ -454,6 +455,26 @@ export function placeServiceNodes(
return out
}
/** Столбец сервисов раскрытой страны: тот же правый x, по вертикали вокруг Y страны. */
export function placeCountryServiceNodes(
serviceIds: string[],
parentPos: { x: number; y: number } | undefined,
): Record<string, { x: number; y: number }> {
const out: Record<string, { x: number; y: number }> = {}
if (serviceIds.length === 0 || !parentPos) return out
const minY = MARGIN + 70
const maxY = H - 72
const x = W - MARGIN - SERVICE_COL_W / 2
const n = serviceIds.length
const gap = Math.min(80, (maxY - minY) / Math.max(1, n))
const span = gap * (n - 1)
const start = clamp(parentPos.y - span / 2, minY, maxY - span)
serviceIds.forEach((id, i) => {
out[id] = { x, y: n === 1 ? clamp(parentPos.y, minY, maxY) : start + i * gap }
})
return out
}
/**
* Суммарная задержка «дом JH» в миллисекундах: те же поля `Server.latency`, что показываются в разделе Серверы.
* Отдельного ICMP по ребру нет это не замер линии, а сумма каталожных latency концов.
+19
View File
@@ -93,6 +93,10 @@ export const flowTalkerDtoSchema = z.object({
dstAsn: z.number().int().optional(),
clientId: z.string().optional(),
clientName: z.string().optional(),
clientIp: z.string().optional(),
internetPeer: z.string().optional(),
internetPeerPort: z.number().int().optional(),
direction: z.enum(["to_client", "from_client", "transit"]).optional(),
enId: z.string().optional(),
enName: z.string().optional(),
plane: z.string().optional(),
@@ -312,6 +316,14 @@ export const flowMapServicePathDtoSchema = z.object({
bps: z.number().nonnegative(),
})
/** Сервисы внутри одной страны: id вида `cc:us|svc:google`, рёбра от `countryId`. */
export const flowMapCountryServiceGroupDtoSchema = z.object({
countryId: z.string(),
services: z.array(flowMapServiceDtoSchema),
edges: z.array(flowMapServiceEdgeDtoSchema),
paths: z.array(flowMapServicePathDtoSchema),
})
export const flowMapHopsDtoSchema = z.object({
hops: z.array(flowMapHopDtoSchema),
live: z.boolean(),
@@ -320,9 +332,15 @@ export const flowMapHopsDtoSchema = z.object({
totalBytes: z.number().nonnegative().optional(),
namedBytes: z.number().nonnegative().optional(),
unclassifiedBytes: z.number().nonnegative().optional(),
asnLoaded: z.boolean().optional(),
countryLoaded: z.boolean().optional(),
services: z.array(flowMapServiceDtoSchema).optional(),
serviceEdges: z.array(flowMapServiceEdgeDtoSchema).optional(),
servicePaths: z.array(flowMapServicePathDtoSchema).optional(),
countries: z.array(flowMapServiceDtoSchema).optional(),
countryEdges: z.array(flowMapServiceEdgeDtoSchema).optional(),
countryPaths: z.array(flowMapServicePathDtoSchema).optional(),
countryServiceGroups: z.array(flowMapCountryServiceGroupDtoSchema).optional(),
mapServiceMinSharePct: z.number().min(0).max(100).optional(),
dedupApplied: z.boolean(),
excludeMeshApplied: z.boolean(),
@@ -347,4 +365,5 @@ export type FlowMapHop = z.infer<typeof flowMapHopDtoSchema>
export type FlowMapService = z.infer<typeof flowMapServiceDtoSchema>
export type FlowMapServiceEdge = z.infer<typeof flowMapServiceEdgeDtoSchema>
export type FlowMapServicePath = z.infer<typeof flowMapServicePathDtoSchema>
export type FlowMapCountryServiceGroup = z.infer<typeof flowMapCountryServiceGroupDtoSchema>
export type FlowMapHopsDto = z.infer<typeof flowMapHopsDtoSchema>
+1 -1
View File
File diff suppressed because one or more lines are too long