Compare commits

...
37 Commits
Author SHA1 Message Date
DenozordecandCursor a3502b9755 fix(ipsec): убрать неподдерживаемый параметр при подписи сертификата
Docker images / prepare-release (push) Successful in 6s
Docker images / backend-test (push) Successful in 1m37s
Docker images / frontend-image (push) Successful in 2m17s
Docker images / updater-image (push) Successful in 38s
Docker images / backend-image (push) Successful in 2m20s
Docker images / notify-webhook (push) Skipped
Docker images / publish-release (push) Successful in 9s
Co-authored-by: Cursor <[email protected]>
2026-09-12 22:21:19 +07:00
DenozordecandCursor 0e1524c600 fix(ipsec): клиенты через общий IKEv2-peer и отдельный клиентский сертификат
Docker images / prepare-release (push) Successful in 7s
Docker images / backend-test (push) Successful in 1m51s
Docker images / frontend-image (push) Successful in 2m54s
Docker images / updater-image (push) Successful in 40s
Docker images / backend-image (push) Successful in 2m11s
Docker images / notify-webhook (push) Skipped
Docker images / publish-release (push) Successful in 10s
Co-authored-by: Cursor <[email protected]>
2026-09-12 22:06:35 +07:00
DenozordecandCursor 11ae7593f4 fix(ipsec): убрать неподдерживаемый параметр при добавлении сертификата
Co-authored-by: Cursor <[email protected]>
2026-09-12 22:06:27 +07:00
Denozordec 3eb75ea0b8 feat(ipsec): add certificate export functionality by name
Docker images / prepare-release (push) Successful in 12s
Docker images / backend-test (push) Successful in 2m41s
Docker images / frontend-image (push) Successful in 2m57s
Docker images / updater-image (push) Successful in 48s
Docker images / backend-image (push) Successful in 2m48s
Docker images / notify-webhook (push) Skipped
Docker images / publish-release (push) Successful in 13s
- Implemented a new backend route for exporting existing client certificates in .p12 format by name.
- Enhanced the frontend to support exporting certificates directly from the IPsec server grid.
- Updated the IPsec page to manage certificate states and handle exports effectively.
- Introduced new utility functions for certificate handling and improved data structures to accommodate the changes.
- Added tests to ensure the reliability of the new certificate export feature.
2026-09-12 21:39:14 +07:00
Denozordec 7365d8d8fb feat(ipsec): enhance IPsec management with peer and certificate handling
Docker images / prepare-release (push) Successful in 8s
Docker images / backend-test (push) Successful in 2m39s
Docker images / frontend-image (push) Successful in 2m56s
Docker images / updater-image (push) Successful in 44s
Docker images / backend-image (push) Successful in 2m35s
Docker images / notify-webhook (push) Skipped
Docker images / publish-release (push) Successful in 15s
- Added support for managing IPsec peers, including editing and deleting functionality.
- Introduced a new UI component for displaying peers and their details within the IPsec server grid.
- Updated the IPsec user creation form to allow binding identities to specific peers.
- Enhanced backend routes and services to handle peer patching and deletion requests.
- Improved data structures to accommodate multiple peers and certificates for each server.
- Added tests to ensure proper functionality of new peer management features.
2026-09-12 21:04:49 +07:00
Denozordec 7755d77340 feat(ipsec): управление IKEv2/IPsec VPN и клиентами из одного окна
Docker images / prepare-release (push) Successful in 15s
Docker images / backend-test (push) Successful in 2m32s
Docker images / frontend-image (push) Successful in 4m19s
Docker images / updater-image (push) Successful in 50s
Docker images / backend-image (push) Successful in 2m40s
Docker images / notify-webhook (push) Skipped
Docker images / publish-release (push) Successful in 8s
- мастер инициализации сервера: CA и серверный сертификаты, peer/profile/proposal, пул, mode-config, policy-template, managed NAT masquerade
- клиенты по сертификату (RSA) и PSK: статический IP или из пула, онлайн-статус по active-peers
- скачивание .p12 и strongSwan .sswan с инструкцией, перекачка с новой passphrase
- история изменений (config_revisions, секция ipsec) и restore только managed-объектов
- привязка IPsec-клиентов к пользователям приложения по Common Name
- страница /ipsec с KPI и вкладками Клиенты/Сервер/CLI, сайдбар, command palette
2026-09-12 20:20:56 +07:00
Denozordec 564aae21f0 fix(statistics): выровнять трафик-данные с картой сети
Docker images / prepare-release (push) Successful in 10s
Docker images / backend-test (push) Successful in 2m40s
Docker images / frontend-image (push) Successful in 3m28s
Docker images / updater-image (push) Successful in 50s
Docker images / backend-image (push) Successful in 3m5s
Docker images / notify-webhook (push) Skipped
Docker images / publish-release (push) Successful in 10s
- Окно периода клампится к текущему моменту: «Сегодня»/«7 д» больше не делят байты на ещё не наступившие часы суток (avgBps/bps были занижены).
- Пресет «24 ч» шлёт точные ISO-границы «сейчас−24ч … сейчас» вместо дат «вчера…сегодня» (~48 ч); подпись диапазона и подсветка пресета сверяются по длительности окна.
- Таксономия сервисов как на карте: DNS/SSH/BGP/WireGuard/GRE и пустые метки сворачиваются в «Прочее» (normalizeFactService) в разбивке сервисов и pivot country×service.
- Тесты: кламп окна, ISO-24ч, будущее «to», невалидный диапазон, нормализация меток.
2026-09-12 13:22:06 +07:00
Denozordec 0fdd2fc1e1 feat(traffic-flow): раскрытие сервисов страны на карте сети
Docker images / prepare-release (push) Successful in 8s
Docker images / backend-test (push) Successful in 2m11s
Docker images / frontend-image (push) Successful in 3m14s
Docker images / updater-image (push) Successful in 43s
Docker images / backend-image (push) Successful in 2m44s
Docker images / notify-webhook (push) Skipped
Docker images / publish-release (push) Successful in 10s
- Контракт: countryServiceGroups (группа сервисов страны) в flowMapHopsDto, тип FlowMapCountryServiceGroup.
- Бэкенд: вложенная агрегация brand×ISO в том же проходе hops; рёбра страна→сервис, пути с реальным EN; cap 8 / min 4, доля от байтов страны.
- Хелпер mapCountryServiceNodeId: id вида cc:us|svc:google, сервисы разных стран не смешиваются.
- Карта сети: клик по стране сдвигает колонку стран и раскрывает столбец её сервисов; остальные страны приглушены; панели страны и вложенного сервиса.
- Тесты: группы US/US+NL, рёбра от страны, cap/min вложенного слоя; mock countryServiceGroups для демо-режима.
2026-09-12 12:47:30 +07:00
DenozordecandCursor f15a7348db feat(traffic-flow): enhance country handling in flow analytics
Docker images / prepare-release (push) Successful in 9s
Docker images / backend-test (push) Successful in 2m19s
Docker images / frontend-image (push) Successful in 3m26s
Docker images / updater-image (push) Successful in 47s
Docker images / backend-image (push) Successful in 2m23s
Docker images / notify-webhook (push) Skipped
Docker images / publish-release (push) Successful in 10s
- Introduced country-specific handling in traffic flow analytics, including new mappings for country nodes and edges.
- Added mock data for countries and their respective service paths to improve visualization in the Network Map.
- Updated the `countryName` function to utilize `Intl.DisplayNames` for better localization of country names.
- Enhanced tests to validate country handling and ensure accurate representation in flow analytics.

Co-authored-by: Cursor <[email protected]>
2026-09-12 11:38:37 +07:00
DenozordecandCursor f3c846201c feat(traffic-flow): enhance Instagram service handling and classification
Docker images / prepare-release (push) Successful in 9s
Docker images / backend-test (push) Successful in 2m34s
Docker images / frontend-image (push) Successful in 3m20s
Docker images / updater-image (push) Successful in 43s
Docker images / backend-image (push) Successful in 2m25s
Docker images / notify-webhook (push) Skipped
Docker images / publish-release (push) Successful in 13s
- Added support for Instagram in traffic flow resolution and classification, including new test cases to validate its behavior.
- Introduced `INSTAGRAM` brand handling in the traffic flow logic, ensuring accurate service identification for Instagram-related traffic.
- Updated existing tests to cover various scenarios involving Instagram, including IP resolution and service categorization.
- Enhanced the service brand icon component to include an Instagram icon for better visual representation.

Co-authored-by: Cursor <[email protected]>
2026-09-12 11:20:16 +07:00
DenozordecandCursor ee9804f1bb feat(traffic-flow): enhance flow analytics and IP handling
Docker images / prepare-release (push) Successful in 8s
Docker images / backend-test (push) Successful in 2m29s
Docker images / frontend-image (push) Successful in 3m20s
Docker images / updater-image (push) Successful in 51s
Docker images / backend-image (push) Successful in 2m40s
Docker images / notify-webhook (push) Skipped
Docker images / publish-release (push) Successful in 13s
- Introduced `canonicalIp` function to standardize IP address formats across the application, improving consistency in flow processing.
- Updated traffic flow analytics to utilize new endpoint resolution logic, enhancing accuracy in traffic classification.
- Enhanced tests for traffic flow analytics and IP handling, ensuring comprehensive coverage for new functionalities.
- Improved traffic flow destination resolution with additional test cases for various internet services, including Google and Fastly.

Co-authored-by: Cursor <[email protected]>
2026-09-12 01:51:23 +07:00
DenozordecandCursor 4ce6169d14 feat(traffic-flow): enhance traffic classification and map display
Docker images / prepare-release (push) Successful in 12s
Docker images / backend-test (push) Successful in 2m16s
Docker images / frontend-image (push) Successful in 2m42s
Docker images / updater-image (push) Successful in 43s
Docker images / backend-image (push) Successful in 2m30s
Docker images / notify-webhook (push) Skipped
Docker images / publish-release (push) Successful in 13s
- Added a new state variable `mapAsnLoaded` to track ASN loading status in the NetworkMapPage.
- Updated the page to conditionally display a message when ASN data is not loaded, improving user feedback.
- Enhanced traffic classification tests to include additional scenarios for Google and YouTube services, ensuring comprehensive coverage.
- Introduced a new function `pickMapInternetDest` to improve destination selection logic in traffic flow processing.
- Updated the traffic flow destination classification to handle new conditions and improve accuracy.

Co-authored-by: Cursor <[email protected]>
2026-09-12 01:07:26 +07:00
DenozordecandCursor 60a0970d73 feat(traffic-flow): enhance traffic flow analytics and overlay configuration
Docker images / prepare-release (push) Successful in 12s
Docker images / backend-test (push) Successful in 2m42s
Docker images / frontend-image (push) Successful in 3m22s
Docker images / updater-image (push) Successful in 46s
Docker images / backend-image (push) Successful in 2m55s
Docker images / notify-webhook (push) Skipped
Docker images / publish-release (push) Successful in 12s
- Updated the traffic flow analytics to include new state variables for named bytes, total bytes, and window seconds, improving data granularity.
- Modified the NetworkMapPage to display classified traffic data, including a new section for showing classified and total bytes.
- Enhanced the applyFlowOverlay function to support a new option for disabling GRE fast path, allowing for more flexible traffic flow configurations.
- Added a toggle in the FlowOverlaySheet component to enable or disable the GRE fast path setting, improving user control over traffic processing.
- Updated tests to validate the new functionalities and ensure accurate traffic flow analytics.

Co-authored-by: Cursor <[email protected]>
2026-09-12 00:03:13 +07:00
DenozordecandCursor fc29dcede7 feat(traffic-flow): add NAT fields to flow processing and analytics
Docker images / prepare-release (push) Successful in 10s
Docker images / backend-test (push) Successful in 2m22s
Docker images / frontend-image (push) Successful in 3m30s
Docker images / updater-image (push) Successful in 45s
Docker images / backend-image (push) Successful in 2m51s
Docker images / notify-webhook (push) Skipped
Docker images / publish-release (push) Successful in 9s
- Introduced new fields for NAT source and destination IPs, as well as their respective ports, in the flow data model.
- Updated database schema and migration scripts to accommodate the new NAT fields in the `flow_buckets` table.
- Enhanced flow analytics and processing functions to utilize the new NAT fields, improving accuracy in traffic flow analysis.
- Added tests to validate the handling of NAT data in various scenarios, ensuring robustness in flow processing.

Co-authored-by: Cursor <[email protected]>
2026-09-11 23:01:48 +07:00
DenozordecandCursor 5f774ce26e feat(gre): enhance GRE tunnel status handling and IPsec configuration
Docker images / prepare-release (push) Successful in 10s
Docker images / backend-test (push) Successful in 2m18s
Docker images / frontend-image (push) Successful in 3m1s
Docker images / updater-image (push) Successful in 40s
Docker images / backend-image (push) Successful in 2m31s
Docker images / notify-webhook (push) Skipped
Docker images / publish-release (push) Successful in 10s
- Introduced a new `greStatusMeta` function to streamline status retrieval and default handling.
- Updated GRE tunnel command generation to improve IPsec configuration handling, including optional fields for encryption and authentication algorithms.
- Enhanced the GRE tunnels data grid to display IPsec settings more clearly, including handling cases where certain parameters may be undefined.
- Refactored status display logic in the GRE page to utilize the new status meta function for consistency.

Co-authored-by: Cursor <[email protected]>
2026-09-11 15:39:32 +07:00
DenozordecandCursor 25b82997b6 feat(config-revisions): implement configuration history for Firewall, GRE, and WireGuard
Docker images / prepare-release (push) Successful in 12s
Docker images / backend-test (push) Successful in 2m20s
Docker images / frontend-image (push) Successful in 2m51s
Docker images / updater-image (push) Successful in 44s
Docker images / backend-image (push) Successful in 2m49s
Docker images / notify-webhook (push) Skipped
Docker images / publish-release (push) Successful in 12s
- Added configuration history management for Firewall, GRE, and WireGuard pages, enabling users to view and restore previous configurations.
- Introduced new components for displaying configuration history and integrated them into the respective pages.
- Enhanced API routes to support fetching and restoring configuration revisions, ensuring data consistency across the application.
- Updated state management to handle loading and restoring states effectively, improving user experience during data operations.
- Enhanced tests to cover new functionalities and ensure reliability.

Co-authored-by: Cursor <[email protected]>
2026-09-11 14:03:00 +07:00
DenozordecandCursor b4a3c3a925 feat(filters, recursive-routes): enhance configuration history and live data handling
- Introduced configuration history management in Filters and Recursive Routes pages, allowing users to view and restore previous configurations.
- Updated state management to handle live data loading and error states more effectively, improving user experience during data fetching.
- Added new components for displaying configuration history and integrated them into existing pages.
- Enhanced API interactions to support fetching and applying configuration revisions, ensuring data consistency across the application.
- Updated tests to cover new functionalities and ensure reliability.

Co-authored-by: Cursor <[email protected]>
2026-09-11 12:15:36 +07:00
DenozordecandCursor 9c0ee7940e feat(bgp, vxlan, ospf): enhance server data handling and introduce new routes
Docker images / prepare-release (push) Successful in 7s
Docker images / backend-test (push) Successful in 1m40s
Docker images / frontend-image (push) Successful in 2m50s
Docker images / updater-image (push) Successful in 42s
Docker images / backend-image (push) Successful in 2m40s
Docker images / notify-webhook (push) Skipped
Docker images / publish-release (push) Successful in 10s
- Added support for fetching and displaying server data in BGP, VXLAN, and OSPF pages, improving the overall user experience.
- Introduced new backend routes for OSPF and VXLAN, allowing for better data management and retrieval.
- Implemented mapping functions for backend server data to frontend types, ensuring consistency across components.
- Enhanced the sidebar to display counts for BGP sessions, VXLAN tunnels, and containers, providing users with quick insights into their network status.
- Updated tests to cover new functionalities and ensure reliability.

Co-authored-by: Cursor <[email protected]>
2026-09-11 11:18:04 +07:00
DenozordecandCursor 5750590b68 feat(traffic-flow): add rebuild facts endpoint and enhance traffic flow analytics
Docker images / prepare-release (push) Successful in 9s
Docker images / backend-test (push) Successful in 2m6s
Docker images / frontend-image (push) Successful in 3m15s
Docker images / updater-image (push) Successful in 48s
Docker images / backend-image (push) Successful in 2m50s
Docker images / notify-webhook (push) Skipped
Docker images / publish-release (push) Successful in 9s
- Introduced a new endpoint `/traffic/flow/rebuild-facts` to rebuild flow facts from buckets, improving data accuracy and management.
- Updated traffic flow analytics to utilize the new `resolveInternetDest` function for better destination resolution.
- Enhanced tests for traffic flow IP handling and added new utility functions for managing internet destinations.

Co-authored-by: Cursor <[email protected]>
2026-09-11 10:32:51 +07:00
DenozordecandCursor 3c42c114f5 refactor(statistics): improve component structure and enhance state management
Docker images / prepare-release (push) Successful in 11s
Docker images / backend-test (push) Successful in 2m17s
Docker images / frontend-image (push) Successful in 4m11s
Docker images / updater-image (push) Successful in 48s
Docker images / backend-image (push) Successful in 2m18s
Docker images / notify-webhook (push) Skipped
Docker images / publish-release (push) Successful in 8s
- Introduced Suspense for lazy loading in StatisticsPage to optimize rendering.
- Refactored StatisticsPage to separate inner logic into StatisticsPageInner for better readability.
- Updated PeriodSelector to utilize useCallback for handling state changes, improving performance and clarity.

Co-authored-by: Cursor <[email protected]>
2026-09-11 09:38:31 +07:00
DenozordecandCursor 5aef419582 fix(statistics): считать уникальный payload без дублей hops
Docker images / prepare-release (push) Successful in 8s
Docker images / backend-test (push) Successful in 2m10s
Docker images / frontend-image (push) Successful in 3m13s
Docker images / updater-image (push) Successful in 46s
Docker images / backend-image (push) Successful in 2m52s
Docker images / notify-webhook (push) Skipped
Docker images / publish-release (push) Successful in 10s
Co-authored-by: Cursor <[email protected]>
2026-09-11 01:27:40 +07:00
DenozordecandCursor 0c0dfa1df7 fix(statistics): не двоить overlay и транзит в кубе
Docker images / prepare-release (push) Successful in 8s
Docker images / backend-test (push) Successful in 2m18s
Docker images / frontend-image (push) Successful in 3m25s
Docker images / updater-image (push) Successful in 48s
Docker images / backend-image (push) Successful in 3m9s
Docker images / notify-webhook (push) Skipped
Docker images / publish-release (push) Successful in 14s
Co-authored-by: Cursor <[email protected]>
2026-09-11 00:46:25 +07:00
DenozordecandCursor c162a41bc0 fix(network-map): подписать unbound-пути парой серверов
Docker images / prepare-release (push) Successful in 7s
Docker images / backend-test (push) Successful in 2m22s
Docker images / frontend-image (push) Successful in 3m29s
Docker images / updater-image (push) Successful in 52s
Docker images / backend-image (push) Successful in 2m41s
Docker images / notify-webhook (push) Skipped
Docker images / publish-release (push) Successful in 12s
Co-authored-by: Cursor <[email protected]>
2026-09-10 22:19:05 +07:00
DenozordecandCursor 97e43b2335 feat(statistics): enhance interface handling and data aggregation
Docker images / prepare-release (push) Successful in 9s
Docker images / backend-test (push) Successful in 2m29s
Docker images / frontend-image (push) Successful in 3m24s
Docker images / updater-image (push) Successful in 44s
Docker images / backend-image (push) Successful in 2m39s
Docker images / notify-webhook (push) Skipped
Docker images / publish-release (push) Successful in 15s
Updated the statistics aggregation service to improve interface resolution and data handling. Introduced new functions for managing interface aliases and collapsing server interface rows, ensuring accurate data representation. Enhanced test coverage for interface resolution and added checks for new functionality.

- Implemented `factIfaceAliases` and `collapseServerIfaceRows` for better interface data management.
- Updated `resolveIfaceName` to handle additional cases for interface indexing.
- Enhanced tests for interface resolution and aggregation logic.

Co-authored-by: Cursor <[email protected]>
2026-09-10 21:49:50 +07:00
DenozordecandCursor db21e1217c feat(network-map): enhance service selection logic and UI updates
Docker images / prepare-release (push) Successful in 9s
Docker images / backend-test (push) Successful in 2m19s
Docker images / frontend-image (push) Successful in 2m52s
Docker images / updater-image (push) Successful in 46s
Docker images / backend-image (push) Successful in 2m20s
Docker images / notify-webhook (push) Skipped
Docker images / publish-release (push) Successful in 11s
Refactor the service selection mechanism to utilize a live reference for selected services, ensuring accurate display of service details. Update UI components to reflect changes in service selection, including dynamic updates for service metrics and paths. Additionally, introduce new utility functions for deduplicating flow rows across exporters in the backend, improving data handling in traffic flow analysis.

- Updated service selection logic in NetworkMapPage component.
- Enhanced UI to display live service data.
- Added deduplication functions in traffic-flow-dedup module for improved data integrity.

Co-authored-by: Cursor <[email protected]>
2026-09-10 19:54:16 +07:00
DenozordecandCursor 5bb9066be8 feat(statistics): добавить BI-разрез и сводную матрицу
Docker images / prepare-release (push) Successful in 10s
Docker images / backend-test (push) Successful in 2m23s
Docker images / frontend-image (push) Successful in 3m30s
Docker images / updater-image (push) Successful in 52s
Docker images / backend-image (push) Successful in 3m8s
Docker images / notify-webhook (push) Skipped
Docker images / publish-release (push) Successful in 14s
Сопоставить клиентов с ifIndex как на карте трафика, чтобы KPI пользователей не обнулялся. На экране — разрез остальных измерений и сводная матрица.

Co-authored-by: Cursor <[email protected]>
2026-09-10 16:07:25 +07:00
DenozordecandCursor b1fd259f10 fix(statistics): убрать параметры хранения с родительских партиций
Docker images / prepare-release (push) Successful in 9s
Docker images / backend-test (push) Successful in 2m25s
Docker images / frontend-image (push) Successful in 3m26s
Docker images / updater-image (push) Successful in 45s
Docker images / backend-image (push) Successful in 2m32s
Docker images / notify-webhook (push) Skipped
Docker images / publish-release (push) Successful in 13s
PostgreSQL 42809: FILLFACTOR и autovacuum нельзя задавать на partitioned parent — только на листьях.

Co-authored-by: Cursor <[email protected]>
2026-09-10 13:36:35 +07:00
DenozordecandCursor 3687bb8fa2 feat(statistics): добавить раздел статистики трафика
Docker images / prepare-release (push) Successful in 10s
Docker images / backend-test (push) Successful in 2m19s
Docker images / frontend-image (push) Successful in 4m47s
Docker images / updater-image (push) Successful in 48s
Docker images / backend-image (push) Successful in 2m53s
Docker images / notify-webhook (push) Skipped
Docker images / publish-release (push) Successful in 13s
Куб IPFIX час+день с AND-слайсами и экраном отчётности /statistics, живой /traffic не меняем.

Co-authored-by: Cursor <[email protected]>
2026-09-10 11:57:47 +07:00
Denozordec a80caf5676 feat(netflow): локальные GeoLite2-базы для стран и ASN потоков
Docker images / prepare-release (push) Successful in 13s
Docker images / backend-test (push) Successful in 2m13s
Docker images / frontend-image (push) Successful in 4m31s
Docker images / updater-image (push) Successful in 48s
Docker images / backend-image (push) Successful in 3m1s
Docker images / notify-webhook (push) Skipped
Docker images / publish-release (push) Successful in 15s
Lookup страны/ASN при ingest теперь идёт сначала по локальным mmdb
MaxMind GeoLite2 (Country + ASN, зеркало P3TERX, без ключей), с
мгновенным синхронным ответом для IPv4/IPv6; RIPEstat остаётся
fallback до первой загрузки баз и при промахе.

- backend/src/services/traffic-flow-geoip.ts: ридеры maxmind, фасад
  resolveFlowIp (geoip-first → RIPE-кэш), статус ридеров
- geoip-update-collector.ts: conditional GET по ETag, валидация пробоем
  8.8.8.8 (US / AS15169), атомарная подмена с .prev-откатом
- джоба планировщика geoip_update (по умолчанию раз в 7 дней),
  настройки geoip_settings + миграция 0004, API /api/geoip
  (GET/PUT/update), контракты @mmapp/contracts/geoip
- engine/analytics/map-hops переведены на resolveFlowIp
- секция «GeoIP-базы» в настройках NetFlow, метки джобы на странице
  сбора данных, тесты test:geoip
2026-09-09 23:33:31 +07:00
DenozordecandCursor d39e3454aa feat(network-map): показывать ASN-бренды по доле среди сервисов
Docker images / prepare-release (push) Successful in 11s
Docker images / backend-test (push) Successful in 2m5s
Docker images / frontend-image (push) Successful in 3m15s
Docker images / updater-image (push) Successful in 48s
Docker images / backend-image (push) Successful in 2m44s
Docker images / notify-webhook (push) Skipped
Docker images / publish-release (push) Successful in 15s
Долю считать среди именованных брендов, а не от всего окна. На карте всегда оставлять топ-8 и добавить иконки новых брендов.

Co-authored-by: Cursor <[email protected]>
2026-09-09 16:53:06 +07:00
DenozordecandCursor a8f2055c77 feat(netflow): расширить каталог ASN популярных сервисов
Docker images / prepare-release (push) Successful in 11s
Docker images / backend-test (push) Successful in 2m5s
Docker images / frontend-image (push) Successful in 3m14s
Docker images / updater-image (push) Successful in 46s
Docker images / backend-image (push) Successful in 4m2s
Docker images / notify-webhook (push) Skipped
Docker images / publish-release (push) Successful in 11s
В аналитике потока видны сети со своим AS (Apple, Steam, VK, Epic и другие), а не только Cloudflare/AWS. Holder и порты Steam не перебивают Cloudflare CIDR.

Co-authored-by: Cursor <[email protected]>
2026-09-09 13:18:58 +07:00
DenozordecandCursor cdeb97d841 feat(certificates): добавить отключение автообновления на странице сертификатов
Docker images / prepare-release (push) Successful in 10s
Docker images / backend-test (push) Successful in 2m7s
Docker images / frontend-image (push) Successful in 3m19s
Docker images / updater-image (push) Successful in 49s
Docker images / backend-image (push) Successful in 2m45s
Docker images / notify-webhook (push) Skipped
Docker images / publish-release (push) Successful in 15s
Co-authored-by: Cursor <[email protected]>
2026-09-08 16:21:50 +07:00
DenozordecandCursor 36c5305db7 feat(dashboard): собрать ops-дашборд на ReUI Frame с живыми данными
Docker images / prepare-release (push) Successful in 10s
Docker images / backend-test (push) Successful in 2m13s
Docker images / frontend-image (push) Successful in 3m14s
Docker images / updater-image (push) Successful in 47s
Docker images / backend-image (push) Successful in 2m44s
Docker images / notify-webhook (push) Skipped
Docker images / publish-release (push) Successful in 14s
Co-authored-by: Cursor <[email protected]>
2026-09-08 15:13:27 +07:00
Denozordec 5f2b4e2d40 feat(backups): replace button with checkbox for server selection in backup settings
Docker images / prepare-release (push) Successful in 11s
Docker images / backend-test (push) Successful in 2m4s
Docker images / frontend-image (push) Successful in 3m10s
Docker images / updater-image (push) Successful in 42s
Docker images / backend-image (push) Successful in 2m29s
Docker images / notify-webhook (push) Skipped
Docker images / publish-release (push) Successful in 10s
Updated the BackupCreateSheet and BackupsSettings components to use checkboxes for selecting servers instead of buttons. This improves user interaction by allowing for easier selection and toggling of server states. The Checkbox component is now integrated for better accessibility and functionality.
2026-09-08 14:47:44 +07:00
DenozordecandCursor cff26813b9 feat(backups): добавить S3-хранилище и обновить экран бэкапов
Docker images / prepare-release (push) Successful in 9s
Docker images / backend-test (push) Successful in 2m18s
Docker images / frontend-image (push) Successful in 4m37s
Docker images / updater-image (push) Successful in 46s
Docker images / backend-image (push) Successful in 3m6s
Docker images / notify-webhook (push) Skipped
Docker images / publish-release (push) Successful in 16s
Сохранять снимки в S3-compatible бакет, скачивать и удалять их вместе с локальными файлами.
Привести /backups к DNA /servers: Frame, KPI, фильтры и реальное восстановление.

Co-authored-by: Cursor <[email protected]>
2026-09-08 14:06:27 +07:00
DenozordecandCursor 39c8ec4a02 fix(netflow): починить live-аналитику и снизить запись в PostgreSQL
Docker images / prepare-release (push) Successful in 11s
Docker images / backend-test (push) Successful in 2m9s
Docker images / frontend-image (push) Successful in 3m20s
Docker images / updater-image (push) Successful in 49s
Docker images / backend-image (push) Successful in 2m44s
Docker images / notify-webhook (push) Skipped
Docker images / publish-release (push) Successful in 10s
Коллектор снова отдаёт назначения в живом потоке. Ошибки записи видны в статусе, лишние перезаписи минутных агрегатов убраны.

Co-authored-by: Cursor <[email protected]>
2026-09-08 12:46:37 +07:00
DenozordecandCursor c6c859a495 perf(db): сжать схему PostgreSQL 18 и повторно загрузить SQLite
Docker images / prepare-release (push) Successful in 10s
Docker images / backend-test (push) Successful in 2m2s
Docker images / frontend-image (push) Successful in 3m23s
Docker images / updater-image (push) Successful in 44s
Docker images / backend-image (push) Successful in 2m47s
Docker images / notify-webhook (push) Skipped
Docker images / publish-release (push) Successful in 11s
При первом рестарте wipe всех таблиц и импорт из SQLite в компактную схему. Retention через DROP PARTITION, lz4 и AIO worker.

Co-authored-by: Cursor <[email protected]>
2026-09-08 10:54:36 +07:00
220 changed files with 26802 additions and 3049 deletions
@@ -0,0 +1,63 @@
# Локальные GeoLite2-базы (Country + ASN) для потоков по странам и ASN
## Контекст
Сейчас страна и ASN для netflow-потоков резолвятся через внешний RIPEstat API (`backend/src/services/traffic-flow-ripe.ts`): лимит 30 новых префиксов/мин, очередь на 90, кэш в PG `flow_ip_meta`. Новые IP «дозревают» с задержкой, IPv6 не покрывается (кэш индексируется только по IPv4). Локальные mmdb-базы дают мгновенный синхронный lookup всех IP без внешних вызовов.
Решения (подтверждены):
- Источник — **MaxMind GeoLite2 через P3TERX-зеркало**: `https://github.com/P3TERX/GeoLite.mmdb/raw/download/GeoLite2-Country.mmdb` и `.../GeoLite2-ASN.mmdb`. Без регистрации, ключей и tar-распаковки. Точность по стране у GeoLite2 и IPinfo паритетная (<1% ошибок у обоих, arXiv 2026); выбран P3TERX за надёжность зеркала (5.2k звёзд) и преемственность: текущий RIPE-путь и так читает GeoLite (`maxmind-geo-lite`), история в кэше остаётся консистентной.
- **RIPEstat остаётся fallback** (до первой загрузки баз / если lookup не дал результата).
- City-базу не качаем (lat/lng фронтенд не использует).
## Изменения
### 1. Зависимость
- `npm install -w mikrotik-manager-backend maxmind` — sync-чтение mmdb, встроенные TS-типы, без транзитивных зависимостей, Node 22 ок.
### 2. Новый сервис `backend/src/services/traffic-flow-geoip.ts`
(по конвенциям окружения traffic-flow-*: контракт → маршрут → сервис, без БД-логики в маршрутах)
- Каталог: `backend/storage/geoip/` (конвенция `storage/backups`), файлы `GeoLite2-Country.mmdb`, `GeoLite2-ASN.mmdb`.
- `initGeoip()` — открыть ридеры best-effort при старте (из `index.ts` рядом с `startTrafficFlowListener`), независимо от настроек автообновления: файлы есть — работают.
- `lookupGeoip(ip): FlowIpMeta | null` — синхронно: `country.iso_code` (fallback `registered_country.iso_code`) с валидацией `isIsoCountry`, ASN = `autonomous_system_number`, holder = `autonomous_system_organization`; приватные IP → negative-запись как в RIPE (`isNonPublicIp`); в PG не пишем (lookup и так быстрый). IPv6 поддержан ридером.
- `resolveFlowIp(ip)` — фасад: `lookupGeoip(ip) ?? lookupRipeCached(ip)`; главный экспорт для потребителей.
- `geoipStatus()` — loaded, даты сборки баз (метаданные mmdb). Тест-хук `setGeoipReadersForTests`. Смена ридеров после обновления — атомарная замена ссылок.
### 3. Коллектор `backend/src/services/geoip-update-collector.ts`
`collectGeoipUpdateOnce()` по образцу `certificate-renew-collector.ts`:
1. Conditional GET с ETag/If-None-Match из настроек → 304 = skip (фолбэк-сравнение: размер/содержимое).
2. Скачивание в `*.tmp` через глобальный `fetch` + AbortController с таймаутом (внешний HTTP из service-слоя — по правилу fastify-backend-drizzle).
3. Валидация: открыть ридер из tmp-файла, пробой 8.8.8.8 (страна US, ASN 15169).
4. `fs.rename` атомарная подмена, старый файл → `*.prev` (откат, если новый ридер не открылся).
5. Перезагрузка ридеров, статус в настройках; snapshot для `scheduler_runs` (checked/downloaded/skipped/bytes/error).
### 4. Планировщик (`backend/src/services/scheduler.ts`)
- `JOB_KEYS` += `geoip_update`; case в `runSchedulerJobBody`; блок в `refreshScheduler()` по образцу `certificates_renew`: интервал `Math.max(6ч, updateIntervalSec*1000)`, по умолчанию 7 дней (upstream обновляется еженедельно) + немедленный первый запуск при включённой настройке.
### 5. Схема и миграция
- `backend/src/db/schema.ts`: singleton `geoip_settings``enabled` (default true), `updateIntervalSec` (default 604800), `lastCheckAt`, `lastSuccessAt`, `lastError`, `countryBuildAt`, `asnBuildAt`, `etagsJson` (jsonb), `createdAt/updatedAt`.
- Миграция: `npm run db:generate` → файл в `backend/drizzle/`.
### 6. API + контракты
- `packages/contracts/src/geoip.ts`: zod-схемы настроек/статуса (все входы — Zod, по правилам проекта).
- Новый `backend/src/routes/geoip.ts`, регистрация в `index.ts` с prefix `/api`:
- `GET /api/geoip` — настройки + статус (ready, даты сборки, последняя проверка/ошибка);
- `PUT /api/geoip` — сохранить настройки, затем `refreshScheduler()`;
- `POST /api/geoip/update` — запустить загрузку сейчас (409, если уже идёт; флаг-гард как в коллекторах).
### 7. Интеграция в пайплайн (geoip-first, RIPE-fallback)
- `traffic-flow-engine.ts` (`queueParsedFlows`, ~строка 336): `lookupRipeCached``resolveFlowIp`. Логика misses не меняется: при готовом mmdb публичные IP (v4+v6) резолвятся сразу, очередь RIPE пустеет; до скачивания баз — прежнее поведение.
- Остальные вызовы `lookupRipeCached``resolveFlowIp` (grep: как минимум `traffic-flow-analytics.ts` ~258271).
- `classifyFlowDst`/бренды не трогаем: holder из mmdb (org name) встаёт в существующие `HOLDER_BRANDS`-регулярки как есть.
### 8. Frontend (по next-shadcn-production / ui-guardian: только переиспользование)
- Секция «GeoIP-базы (GeoLite2)» внутри существующей `components/traffic/netflow-settings-panel.tsx`: статус (готово/не скачано, даты сборки Country/ASN, последняя проверка, ошибка), тумблер автообновления, интервал, кнопка «Обновить сейчас» с индикатором. Только уже используемые в панели примитивы (Switch/Button/поля) — никаких новых визуальных паттернов и Card-shell. API-клиент через существующие http-хелперы.
### 9. Хаускипинг, тесты, проверки
- `backend/.gitignore`: `storage/geoip/`.
- Тесты `backend/src/services/traffic-flow-geoip.test.ts` + скрипт `test:geoip` (по образцу `test:traffic-flow`): приоритет фасада (geoip hit → RIPE не зовётся; miss → fallback), negative на приватных IP, фильтрация EU/ZZ через `isIsoCountry`, коллектор с мокнутым fetch (304-skip, битый файл → подмены нет, `.prev` сохранён), dims по стране/ASN с засеянным ридером.
- Проверки после реализации (обязательно по правилам): типы/сборка бэка (`npm run build -w mikrotik-manager-backend`), типы фронта при правке UI (`npx tsc --noEmit`), `npm run test:geoip` и `test:traffic-flow`; предупреждения не игнорировать.
- Коммит: `feat(netflow): <subject по-русски>` — новая пользовательская фича (мгновенные страна/ASN в потоках), по commit-messages-ru.
- README: короткий раздел о GeoIP; примечание, что в Docker `storage/geoip` ephemeral без тома — базы перекачаются после пересоздания контейнера (~17 МБ); при желании смонтировать volume.
## Что это даёт
Страна и ASN появляются у потока мгновенно при ingest (включая IPv6), без ограничения скорости RIPE; dims `country`/`asn` в `flow_daily_dims`, аналитика (карта, топы, monthly) становятся полными сразу. Внешняя зависимость от stat.ripe.net остаётся только как fallback до первой загрузки баз.
@@ -0,0 +1,46 @@
# IKEv2/IPsec VPN: клиенты, IP-адреса и сертификаты — по образцу WireGuard
## Архитектура (как у WireGuard)
Роутер — источник истины: live-чтение через существующий REST-клиент (`mikrotik.ts`), мутации прямыми REST-вызовами, история через `config_revisions` (новая секция `"ipsec"`; `section` — TEXT без CHECK → **SQL-миграция не нужна**). Новых таблиц нет. Объекты, созданные мастером/при создании клиентов, помечаются managed-комментарием `mm-ipsec` (по образцу managed-markers).
## Модель RouterOS (что создаём/читаем)
Мастер инициализации сервера (per router): CA-сертификат (`/certificate add` + `sign`, key-usage=key-cert-sign,crl-sign), серверный сертификат (CN=адрес/домен, SAN, tls-server), `/ip/ipsec/peer` (passive, exchange-mode=ike2, certificate=серверный cert), `/ip/ipsec/profile`+`proposal` (aes-256/sha256/modp2048, pfs), `/ip pool` + `/ip/ipsec/mode-config` «VPN» (address-pool, dns), policy-template, и опциональное (default on, чекбокс) managed srcnat masquerade-правило «интернет клиентам».
Клиент = `/ip/ipsec/identity`: **Сертификат** (`/certificate add` CN=<имя> → `sign` ca=<CA> → `export-certificate type=pkcs12` → файл `.p12` скачивается с роутера → identity match-by=certificate) или **PSK** (auth-method=pre-shared-key, remote-id+secret). IP: «из пула» (общий mode-config) или статический (персональный `mc-<user>` mode-config с `address=x.x.x.x/32`). Онлайн — `/ip/ipsec/active-peers`.
Клиентские загрузки: `.p12` (cert+key+CA, passphrase) + strongSwan `.sswan` (генерация на бэкенде) + текстовая инструкция. Перекачка `.p12` — повторный export с новой passphrase (ключ остаётся на роутере).
## Бэкенд
1. **`mikrotik.ts`**: `downloadFile(name): Promise<Buffer>` — бинарно-безопасный GET `/rest/file/<name>` (с fallback `flash/<name>`, как у `uploadTextFile`; текущий конвейер парсит JSON как utf8 — нужен Buffer-режим); хелперы `signCertificate` (с поллингом готовности, sign небыстрый), `exportCertificatePkcs12(name, passphrase)`.
2. **`services/ipsec-config.ts`** (чистые, тестируемые): генератор `.sswan` и инструкции, поиск свободного IP в пуле, naming-конвенции (`ipsec-ca`, `ipsec-server`, `mc-<user>`, `ipsec-user-<name>`).
3. **`services/ipsec-ca.ts`**: `ensureCa/ensureServerCert/issueClientCert/exportClientP12` — add → sign (poll) → export → download.
4. **`services/ipsec-ros.ts`**: put/patch/delete для `/ip/ipsec/{peer,identity,mode-config,profile,proposal,policy}`, `/ip/pool`, NAT managed-правило.
5. **`services/ipsec-live.ts`**: `fetchIpsecState(server)` — параллельные GET peer/identity/mode-config/pool/active-peers/certificate (+какие из них наши по маркеру/имени) → DTO; `listIpsec()` fan-out с failures; `countIpsecClients()` для сайдбара.
6. **`entity-snapshots.ts`**: `canonicalIpsecSnapshot/parseIpsecSnapshot/planIpsecRestore` (секреты через `isHiddenSecret`); `CONFIG_SECTIONS` += `"ipsec"`; enum в `schema.ts` (миграции нет).
7. **`routes/ipsec.ts`**: `GET /ipsec` (+observed revision), `POST /ipsec/server/init`, `DELETE /ipsec/server/:serverId` (только managed-объекты), `POST /ipsec/users` (ответ включает одноразовый p12 base64), `PATCH /ipsec/users/:serverId/:rosId` (имя/статический IP/psk), `DELETE /ipsec/users/...` (identity + mc + опционально клиентский cert), `POST /ipsec/users/:serverId/:rosId/cert` (перекачка p12), `GET /ipsec/revisions` + `POST /ipsec/revisions/:id/restore` — всё по образцу `routes/wireguard.ts` (`captureAndAppendRevision`). Регистрация в `index.ts`; `/api/ipsec` в обе группы network-правил `permissions.ts`.
8. **Модуль Пользователи**: `InterfaceType` += `"ipsec"` (iface-type.ts, schema.ts enum — TEXT, без миграции); каталог интерфейсов дополняется IPsec-клиентами (identity name + CN в peerPublicKey/peerName) — привязка app-пользователя к VPN-клиенту.
## Контракт
`packages/contracts/src/ipsec.ts` (+`package.json` exports, `index.ts`): `ipsecPeerDto/identityDto/modeConfigDto/poolDto/activePeerDto/certInfoDto/serverSummaryDto`, list-response с failures, `initRequest` (CN/SAN, пул, DNS, NAT-чекбокс), `userCreateRequest` (имя, auth: certificate|psk, psk?, ip: static|pool, passphrase, daysValid), `certDownloadResponse {filename, contentB64, mime}`.
## Фронт
`app/(main)/ipsec/page.tsx` — «одно окно» по образцу WG-страницы: ServerRail + KPI (серверы IKEv2 / клиенты / онлайн / CA) + Tabs **«Клиенты»** (grid: имя, сервер, аутентификация, IP, онлайн, действия — скачать .p12/.sswan, изменить, удалить; Sheet создания клиента), **«Сервер»** (peer/pool/mode-config/certs + мастер инициализации Stepper + статус NAT-правила), **«CLI»** (шпаргалка). История/restore — `ConfigHistorySheet`. Компоненты `components/ipsec/*` на существующих DataGridShell/form-kit/Sheet; скачивание бинарного p12 — Blob из base64 (по образцу `downloadText`), `.sswan`/инструкция — через `CodeExportSheet`. Сайдбар «IPsec / IKEv2» в «Управление» + бейдж (`sidebar-counts.ts` + mock в `sidebar-badges.ts`) + command-palette. Mock-данные в `lib/data.ts` для демо-режима.
## Тесты и проверка
- `ipsec-config.test.ts` (чистые функции: sswan, свободный IP, naming); `entity-snapshots.test.ts``planIpsecRestore` + `opsTouchOnly(["/ip/ipsec", ...])`.
- `npm run build -w @mmapp/contracts`; `tsc` backend/root; `npm --prefix backend run test:config-sync && test:wireguard` + новые; eslint без новых ошибок.
- Демо-режим: визуальная проверка страницы в браузере (как в прошлый раз).
## Риски (проверить на живом роутере при внедрении)
- REST-скачивание файла `GET /rest/file/<name>` — еслиRouterOS отдаёт только метаданные, fallback: чтение `contents` маленьких PEM-файлов или сборка p12 из PEM-частей.
- Точные имена полей mode-config (`address` vs `address-pool`) сверить с живым GET и адаптировать.
- `/certificate/sign` — поллинг готовности с таймаутом ~30–60 с.
Объём большой; коммит/пуш — по готовности, отдельным `feat(ipsec): …`.
+8
View File
@@ -190,6 +190,14 @@ npm run build -w @mmapp/contracts
npm --prefix backend run db:migrate-from-sqlite
```
### GeoIP-базы GeoLite2 (страны и ASN для NetFlow)
Backend держит локальные mmdb-базы MaxMind GeoLite2 (Country + ASN) в `backend/storage/geoip/` и скачивает их с зеркала [P3TERX/GeoLite.mmdb](https://github.com/P3TERX/GeoLite.mmdb) — без регистрации и ключей. Lookup страны/ASN потока при ingest становится мгновенным (включая IPv6) и не упирается в лимиты RIPEstat; пока базы не скачаны или lookup промахнулся, работает прежний RIPE-fallback.
Управление — секция «GeoIP-базы (GeoLite2)» в настройках NetFlow (страница «Сбор данных»): автообновление (по умолчанию проверка раз в 7 дней, upstream обновляется еженедельно), статус сборки баз и кнопка «Обновить сейчас». Джоба планировщика — `geoip_update`. Атрибуция: данные MaxMind GeoLite2, CC BY-SA 4.0.
Примечание для Docker: каталог `storage/geoip` внутри контейнера ephemeral — без смонтированного volume базы (~17 МБ) перекачаются после пересоздания контейнера. Каталог переопределяется переменной `GEOIP_DIR`.
## CI/CD (Gitea Actions)
Файл: `.gitea/workflows/docker.yml` (имя workflow: **Docker images**).
File diff suppressed because it is too large Load Diff
+130 -42
View File
@@ -1,6 +1,6 @@
"use client"
import { Fragment, useState, useMemo, useEffect } from "react"
import { useState, useMemo, useEffect } from "react"
import { PageHeader } from "@/components/page-header"
import { DataPageToolbar } from "@/components/data-page-toolbar"
import { BgpSessionsDataGrid } from "@/components/data-grids/bgp-sessions-data-grid"
@@ -25,6 +25,9 @@ import {
} from "lucide-react"
import { useDataSource } from "@/lib/data-source"
import { requestJson } from "@/shared/api/http-client"
import { servers as mockServers, type Server } from "@/lib/data"
import { ServerRailLayout, ServerRailMobileButton } from "@/components/server-rail-layout"
import { ALL_SERVERS_ID, type ServerTileItem } from "@/components/server-tile-rail"
// ─── types ────────────────────────────────────────────────────────────────────
@@ -233,6 +236,37 @@ interface BackendBgpSession {
capabilities: string[]; lastError: string | null
}
interface BackendServer {
id: number
name: string
host: string
type?: Server["type"]
site?: string
country: string
asn?: string
enabled: boolean
status?: Server["status"]
latency?: number | null
}
function mapBackendServer(s: BackendServer): Server {
return {
id: String(s.id),
name: s.name || s.host,
host: s.host,
model: "—",
os: "—",
site: s.site ?? "",
country: s.country || "UN",
asn: s.asn ?? "",
type: s.type ?? "exit-node",
enabled: s.enabled,
status: s.status ?? "online",
latency: s.latency ?? null,
sessions: 0,
}
}
function backendToFrontend(b: BackendBgpSession): BgpSession {
return {
id: `${b.serverId}-${b.id}`,
@@ -623,27 +657,40 @@ const TABS: Array<{ id: BgpTab; label: string; icon: React.ReactNode }> = [
export default function BgpPage() {
const [activeTab, setActiveTab] = useState<BgpTab>("sessions")
const [selectedServerId, setSelectedServerId] = useState(ALL_SERVERS_ID)
const { mode, backendUrl } = useDataSource()
const isLive = mode === "live"
const [liveSessions, setLiveSessions] = useState<BgpSession[]>([])
const [liveServers, setLiveServers] = useState<Server[]>([])
const [loading, setLoading] = useState(false)
const [fetchedAt, setFetchedAt] = useState<Date | null>(null)
const [liveError, setLiveError] = useState<string | null>(null)
const [fetchTick, setFetchTick] = useState(0)
useEffect(() => {
if (!isLive) return
if (!isLive) {
queueMicrotask(() => {
setLiveSessions([])
setLiveServers([])
setLiveError(null)
})
return
}
let cancelled = false
queueMicrotask(() => {
if (cancelled) return
setLoading(true)
setLiveError(null)
void requestJson<BackendBgpSession[]>(backendUrl, "/api/bgp/sessions")
.then(data => {
void Promise.all([
requestJson<BackendBgpSession[]>(backendUrl, "/api/bgp/sessions"),
requestJson<BackendServer[]>(backendUrl, "/api/servers"),
])
.then(([data, servers]) => {
if (cancelled) return
setLiveSessions(data.map(backendToFrontend))
setLiveServers(servers.filter((s) => s.enabled).map(mapBackendServer))
setFetchedAt(new Date())
setLoading(false)
})
@@ -656,50 +703,87 @@ export default function BgpPage() {
return () => { cancelled = true }
}, [isLive, backendUrl, fetchTick])
// Use live or mock data for all tabs and KPI
const sessions = isLive ? liveSessions : SESSIONS
const allSessions = isLive ? liveSessions : SESSIONS
const displayServers = isLive ? liveServers : mockServers.filter((s) => s.enabled)
const effectiveServerId =
selectedServerId === ALL_SERVERS_ID || displayServers.some((s) => s.id === selectedServerId)
? selectedServerId
: ALL_SERVERS_ID
const sessions = useMemo(() => {
if (effectiveServerId === ALL_SERVERS_ID) return allSessions
return allSessions.filter((s) => s.serverId === effectiveServerId)
}, [allSessions, effectiveServerId])
const railItems = useMemo<ServerTileItem[]>(() => {
const counts = new Map<string, number>()
for (const s of allSessions) {
counts.set(s.serverId, (counts.get(s.serverId) ?? 0) + 1)
}
return displayServers.map((s) => ({
id: s.id,
name: s.name,
host: s.host,
site: s.site,
country: s.country,
status: s.status,
type: s.type,
count: counts.get(s.id) ?? 0,
enabled: s.enabled,
title: [s.name, s.host, s.asn].filter(Boolean).join(" · "),
}))
}, [displayServers, allSessions])
const established = sessions.filter(s => s.state === "Established").length
const notEstab = sessions.length - established
const totalRx = sessions.reduce((a, s) => a + s.prefixesRx, 0)
const serverCount = useMemo(
() => new Set(liveSessions.map(s => s.serverId)).size,
[liveSessions],
() => new Set(sessions.map(s => s.serverId)).size,
[sessions],
)
return (
<div className="flex flex-col h-full">
<PageHeader
crumbs={[{ label: "Управление" }, { label: "BGP" }]}
actions={
<>
<Button variant="outline" size="sm" onClick={() => setFetchTick(t => t + 1)}>
<RefreshCwIcon className={cn("size-4", loading && "animate-spin")} />
Обновить
</Button>
<Button variant="outline" size="sm"><DownloadIcon className="size-4" />Экспорт</Button>
</>
}
/>
{/* tab bar */}
<div className="border-b bg-background shrink-0">
<div className="flex items-center px-6">
{TABS.map(t => (
<button key={t.id} onClick={() => setActiveTab(t.id)}
className={cn(
"flex items-center gap-2 px-4 py-3 text-sm font-medium border-b-2 transition-colors -mb-px",
activeTab === t.id
? "border-primary text-foreground"
: "border-transparent text-muted-foreground hover:text-foreground hover:border-border",
)}>
{t.icon}{t.label}
</button>
))}
<ServerRailLayout
items={railItems}
selectedId={effectiveServerId}
onSelect={setSelectedServerId}
showAll
allCount={displayServers.length}
loading={isLive && loading && displayServers.length === 0}
header={
<PageHeader
crumbs={[{ label: "Управление" }, { label: "BGP" }]}
actions={
<>
<ServerRailMobileButton />
<Button variant="outline" size="sm" onClick={() => setFetchTick(t => t + 1)}>
<RefreshCwIcon className={cn("size-4", loading && "animate-spin")} />
Обновить
</Button>
<Button variant="outline" size="sm"><DownloadIcon className="size-4" />Экспорт</Button>
</>
}
/>
}
banner={
<div className="border-b bg-background shrink-0">
<div className="flex items-center px-6">
{TABS.map(t => (
<button key={t.id} onClick={() => setActiveTab(t.id)}
className={cn(
"flex items-center gap-2 px-4 py-3 text-sm font-medium border-b-2 transition-colors -mb-px",
activeTab === t.id
? "border-primary text-foreground"
: "border-transparent text-muted-foreground hover:text-foreground hover:border-border",
)}>
{t.icon}{t.label}
</button>
))}
</div>
</div>
</div>
<div className="flex-1 overflow-y-auto p-6">
}
>
<div className="flex flex-col gap-5">
{/* data source banner */}
@@ -729,11 +813,16 @@ export default function BgpPage() {
<AlertDescription className="text-xs">Ошибка загрузки: {liveError}</AlertDescription>
</Alert>
)}
{isLive && !loading && liveSessions.length === 0 && !liveError && fetchedAt && (
{isLive && !loading && allSessions.length === 0 && !liveError && fetchedAt && (
<div className="rounded-md border border-border bg-muted/30 px-4 py-6 text-center text-sm text-muted-foreground">
BGP не настроен ни на одном сервере
</div>
)}
{isLive && !loading && allSessions.length > 0 && sessions.length === 0 && !liveError && (
<div className="rounded-md border border-border bg-muted/30 px-4 py-6 text-center text-sm text-muted-foreground">
На выбранном сервере нет BGP-сессий
</div>
)}
{mode === "mock" && (
<span className="inline-flex w-fit items-center gap-1.5 rounded-full border border-border bg-muted/40 px-2.5 py-0.5 text-[11px] font-medium text-muted-foreground">
Моковые данные
@@ -794,7 +883,6 @@ export default function BgpPage() {
{activeTab === "analytics" && <AnalyticsTab sessions={sessions} />}
</div>
</div>
</div>
</ServerRailLayout>
)
}
+4 -1
View File
@@ -11,6 +11,7 @@ import { OpsPanel } from "@/components/ops-panel"
import { DataPageCard } from "@/components/data-page-card"
import { DataPageToolbar } from "@/components/data-page-toolbar"
import { CertificatesDataGrid } from "@/components/data-grids/certificates-data-grid"
import { CertificateRenewSettingsPanel } from "@/components/certificates/certificate-renew-settings"
import { KpiStatGrid } from "@/components/reui-kit/kpi-stat-grid"
import { ServerRailLayout, ServerRailMobileButton } from "@/components/server-rail-layout"
import { ALL_SERVERS_ID, type ServerTileItem } from "@/components/server-tile-rail"
@@ -243,7 +244,7 @@ function CertPartReference() {
return (
<OpsPanel
title="RouterOS 7 · /certificate — справка CLI"
description="RouterOS 7.22+ · публичные LE для Cloudflare через backend DNS-01, не через /certificate add-acme на устройстве."
description="RouterOS 7 умеет обновлять Let's Encrypt сам. Этот CLI — справка; автообновление MM включается панелью выше."
contentClassName="px-5 py-4"
>
<div className="grid grid-cols-1 sm:grid-cols-3 gap-4 text-xs font-mono">
@@ -715,6 +716,8 @@ export default function CertificatesPage() {
<CertPartKpi displayCerts={scopedCerts} expiring={expiring} expired={expired} />
<CertificateRenewSettingsPanel backendUrl={backendUrl} liveReady={liveReady} />
{liveReady && (
<CertPartAcmeSettings
acmeDirectoryUrl={acmeDirectoryUrl}
+253 -70
View File
@@ -1,15 +1,17 @@
"use client"
import { useMemo, useState } from "react"
import { useCallback, useEffect, useMemo, useState } from "react"
import { PageHeader } from "@/components/page-header"
import { routerContainers, servers } from "@/lib/data"
import type { RouterContainer } from "@/lib/data"
import { routerContainers as mockContainers, servers as mockServers } from "@/lib/data"
import type { RouterContainer, Server } from "@/lib/data"
import { Flag } from "@/components/flag"
import { Frame, FramePanel } from "@/components/reui/frame"
import { KpiStatGrid } from "@/components/reui-kit/kpi-stat-grid"
import { OpsPanel } from "@/components/ops-panel"
import { Button } from "@/components/ui/button"
import { Alert, AlertDescription } from "@/components/ui/alert"
import { cn } from "@/lib/utils"
import { toast } from "sonner"
import {
DropdownMenu, DropdownMenuTrigger, DropdownMenuContent,
DropdownMenuItem, DropdownMenuSeparator,
@@ -18,18 +20,47 @@ import {
BoxIcon, PlayIcon, StopCircleIcon, SearchIcon,
MoreHorizontalIcon, Trash2Icon, PencilIcon, PowerIcon,
CodeXmlIcon, ActivityIcon, ServerIcon,
TerminalIcon, AlertCircleIcon,
TerminalIcon, AlertCircleIcon, RefreshCwIcon,
} from "lucide-react"
import {
Sheet, SheetContent, SheetHeader, SheetTitle,
SheetDescription, SheetFooter, SheetClose,
} from "@/components/ui/sheet"
import { CodeExportSheet } from "@/components/reui-kit/code-export-sheet"
import { useDataSource } from "@/lib/data-source"
import { requestJson } from "@/shared/api/http-client"
import { ServerRailLayout, ServerRailMobileButton } from "@/components/server-rail-layout"
import { ALL_SERVERS_ID, type ServerTileItem } from "@/components/server-tile-rail"
// ─── helpers ──────────────────────────────────────────────────────────────────
interface BackendServer {
id: number
name: string
host: string
type?: Server["type"]
site?: string
country: string
asn?: string
enabled: boolean
status?: Server["status"]
latency?: number | null
}
function serverFor(id: string) {
return servers.find((s) => s.id === id)
interface ContainersApiResponse {
containers: RouterContainer[]
}
function mapBackendServer(s: BackendServer): Server {
return {
id: String(s.id),
name: s.name || s.host,
host: s.host,
model: "—",
os: "—",
site: s.site ?? "",
country: s.country || "UN",
asn: s.asn ?? "",
type: s.type ?? "exit-node",
enabled: s.enabled,
status: s.status ?? "online",
latency: s.latency ?? null,
sessions: 0,
}
}
function statusConfig(status: RouterContainer["status"]) {
@@ -52,10 +83,8 @@ function statusConfig(status: RouterContainer["status"]) {
}[status]
}
// ─── RSC generator ────────────────────────────────────────────────────────────
function generateContainerRsc(c: RouterContainer): string {
const srv = serverFor(c.serverId)
function generateContainerRsc(c: RouterContainer, serverById: Record<string, Server>): string {
const srv = serverById[c.serverId]
const lines: string[] = []
lines.push(`# RouterOS Container — ${c.name}`)
if (srv) lines.push(`# Сервер: ${srv.name} (${srv.host})`)
@@ -63,13 +92,11 @@ function generateContainerRsc(c: RouterContainer): string {
lines.push(`# RouterOS 7.4+ · /container`)
lines.push(``)
// interface
for (const iface of c.interfaces) {
lines.push(`/interface/veth/add name=${iface} address=172.17.0.2/24 gateway=172.17.0.1`)
}
lines.push(``)
// envs
if (c.envs.length > 0) {
lines.push(`/container/envs/add name=${c.name}-envs \\`)
for (const { key, value } of c.envs) {
@@ -78,7 +105,6 @@ function generateContainerRsc(c: RouterContainer): string {
lines.push(``)
}
// mounts
for (const m of c.mounts) {
lines.push(`/container/mounts/add name=${c.name}-mount-${m.dst.replace(/\//g, "-").slice(1)} \\`)
if (m.src) lines.push(` src=${m.src} \\`)
@@ -86,7 +112,6 @@ function generateContainerRsc(c: RouterContainer): string {
lines.push(``)
}
// container
lines.push(`/container/add \\`)
lines.push(` remote-image=${c.image}:${c.tag} \\`)
lines.push(` interface=${c.interfaces[0] ?? "veth-container"} \\`)
@@ -100,12 +125,18 @@ function generateContainerRsc(c: RouterContainer): string {
return lines.join("\n")
}
// ─── Export Sheet ─────────────────────────────────────────────────────────────
function ExportSheet({ open, container, onClose }: {
open: boolean; container: RouterContainer | null; onClose: () => void
function ExportSheet({
open, container, onClose, serverById,
}: {
open: boolean
container: RouterContainer | null
onClose: () => void
serverById: Record<string, Server>
}) {
const code = useMemo(() => container ? generateContainerRsc(container) : "", [container])
const code = useMemo(
() => (container ? generateContainerRsc(container, serverById) : ""),
[container, serverById],
)
return (
<CodeExportSheet
@@ -125,17 +156,29 @@ function ExportSheet({ open, container, onClose }: {
)
}
// ─── Container card ───────────────────────────────────────────────────────────
function ContainerCard({
container,
server,
live,
busy,
onExport,
onStart,
onStop,
onRestart,
onRemove,
}: {
container: RouterContainer
server?: Server
live: boolean
busy: boolean
onExport: () => void
onStart: () => void
onStop: () => void
onRestart: () => void
onRemove: () => void
}) {
const srv = serverFor(container.serverId)
const cfg = statusConfig(container.status)
const canMutate = live && Boolean(container.rosId)
return (
<Frame dense className="w-full overflow-hidden">
@@ -150,29 +193,36 @@ function ContainerCard({
</div>
<DropdownMenu>
<DropdownMenuTrigger render={
<Button variant="ghost" size="icon" className="size-7 shrink-0">
<Button variant="ghost" size="icon" className="size-7 shrink-0" disabled={busy}>
<MoreHorizontalIcon className="size-4" />
</Button>
} />
<DropdownMenuContent side="bottom" align="end">
{container.status === "running" ? (
<DropdownMenuItem><StopCircleIcon className="size-4 text-amber-500" />Остановить</DropdownMenuItem>
<DropdownMenuItem disabled={!canMutate} onClick={onStop}>
<StopCircleIcon className="size-4 text-amber-500" />Остановить
</DropdownMenuItem>
) : (
<DropdownMenuItem><PlayIcon className="size-4 text-emerald-500" />Запустить</DropdownMenuItem>
<DropdownMenuItem disabled={!canMutate} onClick={onStart}>
<PlayIcon className="size-4 text-emerald-500" />Запустить
</DropdownMenuItem>
)}
<DropdownMenuItem><TerminalIcon className="size-4" />Логи</DropdownMenuItem>
<DropdownMenuItem><PencilIcon className="size-4" />Редактировать</DropdownMenuItem>
<DropdownMenuItem disabled><TerminalIcon className="size-4" />Логи</DropdownMenuItem>
<DropdownMenuItem disabled><PencilIcon className="size-4" />Редактировать</DropdownMenuItem>
<DropdownMenuItem onClick={onExport}><CodeXmlIcon className="size-4" />Экспорт .rsc</DropdownMenuItem>
<DropdownMenuSeparator />
<DropdownMenuItem><PowerIcon className="size-4" />Перезапустить</DropdownMenuItem>
<DropdownMenuItem disabled={!canMutate} onClick={onRestart}>
<PowerIcon className="size-4" />Перезапустить
</DropdownMenuItem>
<DropdownMenuSeparator />
<DropdownMenuItem variant="destructive"><Trash2Icon className="size-4" />Удалить</DropdownMenuItem>
<DropdownMenuItem variant="destructive" disabled={!canMutate} onClick={onRemove}>
<Trash2Icon className="size-4" />Удалить
</DropdownMenuItem>
</DropdownMenuContent>
</DropdownMenu>
</div>
<div className="px-4 py-3 flex flex-col gap-3">
{/* image */}
<div className="flex items-center gap-2">
<BoxIcon className="size-3.5 text-muted-foreground shrink-0" />
<span className="font-mono text-xs text-foreground/80">
@@ -180,17 +230,15 @@ function ContainerCard({
</span>
</div>
{/* server */}
{srv && (
{server && (
<div className="flex items-center gap-2 text-xs text-muted-foreground">
<ServerIcon className="size-3.5 shrink-0" />
<Flag code={srv.country} size={12} />
<span className="font-mono">{srv.name}</span>
<Flag code={server.country} size={12} />
<span className="font-mono">{server.name}</span>
</div>
)}
{/* uptime + stats */}
{container.status === "running" && (
{container.status === "running" && (container.uptime || container.cpu !== undefined || container.memMb !== undefined) && (
<div className="flex items-center gap-4 text-xs text-muted-foreground border-t pt-2.5">
{container.uptime && (
<div className="flex items-center gap-1">
@@ -216,7 +264,6 @@ function ContainerCard({
</div>
)}
{/* interfaces */}
{container.interfaces.length > 0 && (
<div className="flex flex-wrap gap-1">
{container.interfaces.map((i) => (
@@ -227,7 +274,6 @@ function ContainerCard({
</div>
)}
{/* mounts */}
{container.mounts.length > 0 && (
<div className="flex flex-col gap-1">
{container.mounts.map((m, idx) => (
@@ -249,50 +295,183 @@ function ContainerCard({
)
}
// ════════════════════════════════════════════════════════════════════════════
export default function ContainersPage() {
const [search, setSearch] = useState("")
const [statusFilter, setStatusFilter] = useState<RouterContainer["status"] | "all">("all")
const { mode, backendUrl } = useDataSource()
const isLive = mode === "live"
const [search, setSearch] = useState("")
const [statusFilter, setStatusFilter] = useState<RouterContainer["status"] | "all">("all")
const [exportContainer, setExportContainer] = useState<RouterContainer | null>(null)
const [selectedServerId, setSelectedServerId] = useState(ALL_SERVERS_ID)
const [liveContainers, setLiveContainers] = useState<RouterContainer[]>([])
const [liveServers, setLiveServers] = useState<Server[]>([])
const [loading, setLoading] = useState(false)
const [busyId, setBusyId] = useState<string | null>(null)
const [liveError, setLiveError] = useState<string | null>(null)
const loadLive = useCallback(async () => {
if (!isLive) return
setLoading(true)
setLiveError(null)
try {
const [cRes, sRes] = await Promise.all([
requestJson<ContainersApiResponse>(backendUrl, "/api/containers"),
requestJson<BackendServer[]>(backendUrl, "/api/servers"),
])
setLiveContainers(cRes.containers ?? [])
setLiveServers(sRes.filter((s) => s.enabled).map(mapBackendServer))
} catch (e) {
setLiveError(e instanceof Error ? e.message : "Ошибка загрузки")
setLiveContainers([])
} finally {
setLoading(false)
}
}, [isLive, backendUrl])
useEffect(() => {
if (!isLive) {
queueMicrotask(() => {
setLiveContainers([])
setLiveServers([])
setLiveError(null)
})
return
}
queueMicrotask(() => {
void loadLive()
})
}, [isLive, loadLive])
const displayContainers = isLive ? liveContainers : mockContainers
const displayServers = isLive ? liveServers : mockServers.filter((s) => s.enabled)
const effectiveServerId =
selectedServerId === ALL_SERVERS_ID || displayServers.some((s) => s.id === selectedServerId)
? selectedServerId
: ALL_SERVERS_ID
const scoped = useMemo(() => {
if (effectiveServerId === ALL_SERVERS_ID) return displayContainers
return displayContainers.filter((c) => c.serverId === effectiveServerId)
}, [displayContainers, effectiveServerId])
const serverById = useMemo(
() => Object.fromEntries(displayServers.map((s) => [s.id, s])),
[displayServers],
)
const railItems = useMemo<ServerTileItem[]>(() => (
displayServers.map((s) => ({
id: s.id,
name: s.name,
host: s.host,
site: s.site,
country: s.country,
status: s.status,
type: s.type,
enabled: s.enabled,
meta: String(displayContainers.filter((c) => c.serverId === s.id).length),
}))
), [displayServers, displayContainers])
const filtered = useMemo(() => {
return routerContainers.filter((c) => {
return scoped.filter((c) => {
if (statusFilter !== "all" && c.status !== statusFilter) return false
if (!search) return true
const q = search.toLowerCase()
return (
c.name.toLowerCase().includes(q) ||
c.image.toLowerCase().includes(q) ||
(serverFor(c.serverId)?.name.toLowerCase().includes(q) ?? false)
(serverById[c.serverId]?.name.toLowerCase().includes(q) ?? false)
)
})
}, [search, statusFilter])
}, [search, statusFilter, scoped, serverById])
const running = routerContainers.filter((c) => c.status === "running").length
const stopped = routerContainers.filter((c) => c.status === "stopped").length
const errors = routerContainers.filter((c) => c.status === "error").length
const running = scoped.filter((c) => c.status === "running").length
const stopped = scoped.filter((c) => c.status === "stopped").length
const errors = scoped.filter((c) => c.status === "error").length
async function mutate(c: RouterContainer, action: "start" | "stop" | "restart" | "remove") {
if (!isLive || !c.rosId) {
toast.info("Действие доступно только в live-режиме")
return
}
if (action === "remove" && !window.confirm(`Удалить контейнер ${c.name}?`)) return
setBusyId(c.id)
try {
await requestJson(backendUrl, `/api/servers/${c.serverId}/containers/${action}`, {
method: "POST",
body: JSON.stringify({ rosId: c.rosId }),
})
const labels = { start: "запущен", stop: "остановлен", restart: "перезапущен", remove: "удалён" }
toast.success(`${c.name}: ${labels[action]}`)
await loadLive()
} catch (e) {
toast.error(e instanceof Error ? e.message : "Ошибка RouterOS")
} finally {
setBusyId(null)
}
}
return (
<div className="flex flex-col h-full">
<PageHeader
crumbs={[{ label: "Управление" }, { label: "Контейнеры" }]}
actions={
<Button size="sm">
<BoxIcon className="size-4" />Новый контейнер
</Button>
}
/>
<div className="flex-1 overflow-y-auto p-6">
<>
<ServerRailLayout
items={railItems}
selectedId={effectiveServerId}
onSelect={setSelectedServerId}
showAll
allCount={displayServers.length}
loading={isLive && loading && displayServers.length === 0}
header={
<PageHeader
crumbs={[{ label: "Управление" }, { label: "Контейнеры" }]}
actions={
<>
<ServerRailMobileButton />
<Button
variant="outline"
size="sm"
onClick={() => { void loadLive() }}
disabled={!isLive || loading}
>
<RefreshCwIcon className={cn("size-4", loading && "animate-spin")} />
Обновить
</Button>
<Button size="sm">
<BoxIcon className="size-4" />Новый контейнер
</Button>
</>
}
/>
}
>
<div className="flex flex-col gap-5">
{isLive && liveError && (
<Alert variant="warning" className="py-2">
<AlertCircleIcon />
<AlertDescription className="text-xs">Ошибка загрузки: {liveError}</AlertDescription>
</Alert>
)}
{isLive && !loading && displayContainers.length === 0 && !liveError && (
<div className="rounded-md border border-border bg-muted/30 px-4 py-6 text-center text-sm text-muted-foreground">
Контейнеры не найдены. Нужен пакет container (RouterOS 7.4+).
</div>
)}
{mode === "mock" && (
<span className="inline-flex w-fit items-center gap-1.5 rounded-full border border-border bg-muted/40 px-2.5 py-0.5 text-[11px] font-medium text-muted-foreground">
Моковые данные
</span>
)}
<KpiStatGrid
aria-label="Сводка контейнеров"
items={[
{
id: "all",
label: "Всего",
value: routerContainers.length,
value: scoped.length,
icon: <BoxIcon className="size-4" />,
iconClassName: "text-muted-foreground",
},
@@ -321,7 +500,6 @@ export default function ContainersPage() {
]}
/>
{/* Info banner */}
<div className="flex items-start gap-3 rounded-lg bg-violet-500/5 border border-violet-500/20 px-4 py-3 text-sm">
<BoxIcon className="size-5 text-violet-500 shrink-0 mt-0.5" />
<div>
@@ -333,7 +511,6 @@ export default function ContainersPage() {
</div>
</div>
{/* Toolbar */}
<div className="flex items-center gap-3 flex-wrap">
<div className="flex items-center gap-2 h-8 px-3 border border-input rounded-md bg-background min-w-[240px]">
<SearchIcon className="size-3.5 text-muted-foreground shrink-0" />
@@ -363,7 +540,6 @@ export default function ContainersPage() {
<span className="text-sm text-muted-foreground ml-auto">{filtered.length} контейнеров</span>
</div>
{/* Grid */}
{filtered.length === 0 ? (
<div className="flex flex-col items-center justify-center py-16 text-center text-muted-foreground">
<BoxIcon className="size-10 mb-3 opacity-20" />
@@ -376,13 +552,19 @@ export default function ContainersPage() {
<ContainerCard
key={c.id}
container={c}
server={serverById[c.serverId]}
live={isLive}
busy={busyId === c.id}
onExport={() => setExportContainer(c)}
onStart={() => { void mutate(c, "start") }}
onStop={() => { void mutate(c, "stop") }}
onRestart={() => { void mutate(c, "restart") }}
onRemove={() => { void mutate(c, "remove") }}
/>
))}
</div>
)}
{/* RouterOS reference */}
<OpsPanel title="RouterOS 7.4+ · /container — быстрые команды" contentClassName="px-5 py-4">
<div className="grid grid-cols-1 sm:grid-cols-3 gap-4 text-xs font-mono">
{[
@@ -442,13 +624,14 @@ export default function ContainersPage() {
</OpsPanel>
</div>
</div>
</ServerRailLayout>
<ExportSheet
open={!!exportContainer}
container={exportContainer}
onClose={() => setExportContainer(null)}
serverById={serverById}
/>
</div>
</>
)
}
File diff suppressed because it is too large Load Diff
+40
View File
@@ -34,6 +34,7 @@ import {
type InternetPathRunSnapshot,
type CertificatesRenewRunSnapshot,
type BackupsRunSnapshot,
type GeoipUpdateRunSnapshot,
type PingRunSnapshot,
type ResourcesRunSnapshot,
type SchedulerRunSnapshot,
@@ -340,6 +341,41 @@ function SnapshotTables({ snap }: { snap: SchedulerRunSnapshot }) {
</div>
)
}
if (snap.job === "geoip_update") {
const g = snap as GeoipUpdateRunSnapshot
return (
<div className="flex flex-col gap-3">
{g.skipped ? (
<p className="text-xs text-amber-600 dark:text-amber-400">Прогон пропущен: обновление уже выполнялось или задача отключена.</p>
) : null}
<dl className="grid grid-cols-2 gap-3 text-xs sm:grid-cols-4">
<div>
<dt className="text-muted-foreground">Баз проверено</dt>
<dd className="font-mono font-medium">{g.checked}</dd>
</div>
<div>
<dt className="text-muted-foreground">Скачано</dt>
<dd className="font-mono font-medium">{g.downloaded}</dd>
</div>
<div>
<dt className="text-muted-foreground">Без изменений</dt>
<dd className="font-mono font-medium">{g.skippedUnchanged}</dd>
</div>
<div>
<dt className="text-muted-foreground">Объём, МБ</dt>
<dd className="font-mono font-medium">{(g.bytes / 1024 / 1024).toFixed(1)}</dd>
</div>
</dl>
{g.errors.length ? (
<div className="rounded-md border border-amber-500/40 bg-amber-500/10 px-3 py-2 text-xs text-amber-800 dark:text-amber-200 flex flex-col gap-1">
{g.errors.map((e, i) => (
<p key={i} className="break-words">{e}</p>
))}
</div>
) : null}
</div>
)
}
if (snap.job === "alert_engine") {
const a = snap as AlertEngineRunSnapshot
return (
@@ -1127,7 +1163,11 @@ export default function DataCollectionPage() {
onChange={(e) => setRenewBeforeDaysDraft(e.target.value)}
className="h-8 text-sm"
inputMode="numeric"
disabled={!draftCertRenewEnabled}
/>
<p className="text-[11px] text-muted-foreground mt-1">
Вкл/выкл автообновления MM также на странице «Сертификаты». Не включайте вместе со встроенным ACME RouterOS.
</p>
</div>
</div>
<p className="text-[11px] text-muted-foreground leading-snug">
+172 -138
View File
@@ -27,7 +27,7 @@ import {
StarIcon, ArrowUpDownIcon, ArrowUpIcon, ArrowDownIcon,
FileCodeIcon, CopyIcon, NetworkIcon, TagIcon,
ArrowRightIcon, AlertTriangleIcon, LoaderCircleIcon, RouteIcon,
CheckCircle2Icon, XCircleIcon, CircleDashedIcon, RefreshCwIcon,
RefreshCwIcon, HistoryIcon,
} from "lucide-react"
import {
Sheet, SheetContent, SheetHeader, SheetTitle, SheetDescription, SheetFooter,
@@ -36,13 +36,13 @@ import { Tabs, TabsContent, TabsList, TabsTrigger } from "@/components/ui/tabs"
import { Tooltip, TooltipContent, TooltipTrigger } from "@/components/ui/tooltip"
import { toast } from "sonner"
import { CodeExportSheet } from "@/components/reui-kit/code-export-sheet"
import { ConfigHistorySheet } from "@/components/config-history-sheet"
import type { ConfigRevisionDto } from "@/lib/config-revisions"
import { ServerRailLayout, ServerRailMobileButton } from "@/components/server-rail-layout"
import { type ServerTileItem } from "@/components/server-tile-rail"
// ── helpers ────────────────────────────────────────────────────────────────────
type FilterRouterSyncStatus = "synced" | "drift" | "missing"
function newId() { return `r${Date.now()}-${Math.random().toString(36).slice(2, 6)}` }
function innerIpToGateway(ip: string) { return ip.split("/")[0] }
@@ -1239,6 +1239,9 @@ interface BackendServer {
interface LiveFiltersResponse {
rulesets: ServerFilterRuleset[]
greTunnels: GreTunnel[]
live?: boolean
stale?: boolean
error?: string
}
function buildRulesets(serverList: Server[], sourceRulesets: ServerFilterRuleset[]): ServerFilterRuleset[] {
@@ -1325,18 +1328,14 @@ export default function FiltersPage() {
const [sheetMode, setSheetMode] = useState<"create" | "edit">("create")
const [sheetInitial, setSheetInitial]= useState<RuleForm>(emptyForm())
const [editingId, setEditingId] = useState<string | null>(null)
const [previewOpen, setPreviewOpen] = useState(false)
const [copyOpen, setCopyOpen] = useState(false)
const [syncBusy, setSyncBusy] = useState<"from" | "to" | null>(null)
const [routerCompare, setRouterCompare] = useState<{
serverId: string
byCommunity: Record<string, FilterRouterSyncStatus>
} | null>(null)
const [routerCompareLoading, setRouterCompareLoading] = useState(false)
useEffect(() => {
setRouterCompare(null)
}, [selectedServerId])
const [previewOpen, setPreviewOpen] = useState(false)
const [copyOpen, setCopyOpen] = useState(false)
const [applyBusy, setApplyBusy] = useState(false)
const [liveStale, setLiveStale] = useState(false)
const [historyOpen, setHistoryOpen] = useState(false)
const [historyLoading, setHistoryLoading] = useState(false)
const [historyRestoring, setHistoryRestoring] = useState(false)
const [revisions, setRevisions] = useState<ConfigRevisionDto[]>([])
useEffect(() => {
if (!isLive) {
@@ -1347,6 +1346,7 @@ export default function FiltersPage() {
setRulesets(buildRulesets(servers, serverFilterRulesets))
setSelectedServerId(servers[0]?.id ?? "")
setLiveLoadState("idle")
setLiveStale(false)
})
return
}
@@ -1390,18 +1390,43 @@ export default function FiltersPage() {
})
}, [isLive, apiFetch])
const loadLiveRules = useCallback(async (serverId: string) => {
if (!isLive || !serverId) return
try {
const fresh = await apiFetch<LiveFiltersResponse>(
`/api/filters/rules?serverId=${encodeURIComponent(serverId)}`,
)
const liveRules = fresh.rulesets.find((r) => r.serverId === serverId)?.rules ?? fresh.rulesets[0]?.rules ?? []
setRulesets((prev) => {
const has = prev.some((rs) => rs.serverId === serverId)
if (!has) return [...prev, { serverId, rules: liveRules }]
return prev.map((rs) => rs.serverId === serverId ? { ...rs, rules: liveRules } : rs)
})
setLiveStale(Boolean(fresh.stale))
if (fresh.greTunnels?.length) {
setGreByServer((prev) => ({ ...prev, [serverId]: fresh.greTunnels }))
}
} catch (err) {
setLiveStale(true)
toast.error("Не удалось прочитать правила с роутера", { description: String(err) })
}
}, [isLive, apiFetch])
useEffect(() => {
if (!isLive || !selectedServerId) return
if (!isLive || !selectedServerId || liveLoadState !== "idle") return
if (!liveServers.some((s) => s.id === selectedServerId)) return
void Promise.all([
loadLiveRules(selectedServerId),
ensureGreTunnels(selectedServerId),
ensureRecursiveRoutes(selectedServerId),
])
}, [isLive, selectedServerId, ensureGreTunnels, ensureRecursiveRoutes])
}, [isLive, selectedServerId, liveLoadState, liveServers, ensureGreTunnels, ensureRecursiveRoutes, loadLiveRules])
const allServers = isLive ? liveServers : servers
const allTunnels = isLive ? (greByServer[selectedServerId] ?? []) : greTunnels
const allServers = !isLive || liveLoadState === "error" ? servers : liveServers
const allTunnels = !isLive || liveLoadState === "error" ? greTunnels : (greByServer[selectedServerId] ?? [])
const selectedServer = allServers.find(s => s.id === selectedServerId) ?? allServers[0]
const totalRules = rulesets.reduce((s, r) => s + r.rules.length, 0)
const mutationsLocked = isLive && (applyBusy || liveStale || liveLoadState === "error")
const filterRailItems = useMemo<ServerTileItem[]>(() => (
allServers.map((s) => ({
@@ -1427,21 +1452,6 @@ export default function FiltersPage() {
[rulesets, selectedServerId],
)
const fetchRouterCompare = useCallback(async () => {
if (!isLive || !selectedServerId) return
setRouterCompareLoading(true)
try {
const d = await apiFetch<{ byCommunity: Record<string, FilterRouterSyncStatus> }>(
`/api/filters/router-compare?serverId=${encodeURIComponent(selectedServerId)}`,
)
setRouterCompare({ serverId: selectedServerId, byCommunity: d.byCommunity })
} catch {
setRouterCompare(null)
} finally {
setRouterCompareLoading(false)
}
}, [isLive, selectedServerId, apiFetch])
const filteredRules = useMemo(() => {
const q = search.toLowerCase()
if (!q) return currentRules
@@ -1453,68 +1463,94 @@ export default function FiltersPage() {
)
}, [currentRules, search, communityNameMap])
const updateRules = useCallback((serverId: string, updater: (rules: FilterRule[]) => FilterRule[]) => {
setRouterCompare(rc => (rc && rc.serverId === serverId ? null : rc))
setRulesets(prev => {
const next = prev.map(rs =>
rs.serverId === serverId ? { ...rs, rules: updater(rs.rules) } : rs
)
if (isLive) {
void apiFetch<{ ok: boolean }>("/api/filters/rules", {
method: "PUT",
body: JSON.stringify({ rulesets: next }),
}).catch(() => {})
}
return next
})
}, [isLive, apiFetch])
const syncFromRouter = useCallback(async () => {
if (!isLive || syncBusy) return
setSyncBusy("from")
const applyRules = useCallback(async (
serverId: string,
nextRules: FilterRule[],
source: "apply" | "copy" = "apply",
) => {
const prev = rulesets
setRulesets((p) => p.map((rs) => rs.serverId === serverId ? { ...rs, rules: nextRules } : rs))
if (!isLive) return
if (liveStale) {
setRulesets(prev)
toast.error("Роутер недоступен — изменения заблокированы")
return
}
setApplyBusy(true)
try {
await apiFetch<{ ok: boolean }>("/api/filters/sync/from-router", {
method: "POST",
body: JSON.stringify({ serverId: selectedServerId }),
const res = await apiFetch<{ ok: boolean; rules?: FilterRule[] }>("/api/filters/rules", {
method: "PUT",
body: JSON.stringify({ serverId, rules: nextRules, source }),
})
const fresh = await apiFetch<LiveFiltersResponse>("/api/filters/rules")
setRulesets(buildRulesets(allServers, fresh.rulesets))
if (res.rules) {
setRulesets((p) => p.map((rs) => rs.serverId === serverId ? { ...rs, rules: res.rules ?? nextRules } : rs))
}
toast.success("Правила применены на роутер")
} catch (err) {
setRulesets(prev)
toast.error("Не удалось применить правила на роутер", { description: String(err) })
} finally {
setApplyBusy(false)
}
}, [isLive, apiFetch, rulesets, liveStale])
const refreshFromRouter = useCallback(async () => {
if (!isLive || !selectedServerId || applyBusy) return
setApplyBusy(true)
try {
await loadLiveRules(selectedServerId)
await Promise.all([
ensureGreTunnels(selectedServerId),
ensureRecursiveRoutes(selectedServerId),
])
await fetchRouterCompare()
} finally {
setSyncBusy(null)
setApplyBusy(false)
}
}, [isLive, syncBusy, apiFetch, allServers, selectedServerId, ensureGreTunnels, ensureRecursiveRoutes, fetchRouterCompare])
}, [isLive, selectedServerId, applyBusy, loadLiveRules, ensureGreTunnels, ensureRecursiveRoutes])
const syncToRouter = useCallback(async () => {
if (!isLive || syncBusy || !selectedServerId) return
setSyncBusy("to")
const loadRevisions = useCallback(async () => {
if (!isLive || !selectedServerId) return
setHistoryLoading(true)
try {
const res = await apiFetch<{
ok: boolean
updatedServers: number
pushedRules: number
errors?: Array<{ serverId: number; error: string }>
}>("/api/filters/sync/to-router", {
method: "POST",
body: JSON.stringify({ serverId: selectedServerId }),
})
if (res.ok) {
toast.success(`Загружено правил на роутер: ${res.pushedRules}`)
} else {
const detail = res.errors?.[0]?.error ?? "неизвестная ошибка"
toast.error("Не удалось загрузить правила на роутер", { description: detail })
}
await fetchRouterCompare()
const res = await apiFetch<{ revisions: ConfigRevisionDto[] }>(
`/api/filters/revisions?serverId=${encodeURIComponent(selectedServerId)}`,
)
setRevisions(res.revisions)
} catch (err) {
toast.error("Не удалось загрузить правила на роутер", { description: String(err) })
toast.error("Не удалось загрузить историю", { description: String(err) })
setRevisions([])
} finally {
setSyncBusy(null)
setHistoryLoading(false)
}
}, [isLive, syncBusy, selectedServerId, apiFetch, fetchRouterCompare])
}, [isLive, selectedServerId, apiFetch])
const restoreRevision = useCallback(async (id: string) => {
if (!isLive || !selectedServerId) return
setHistoryRestoring(true)
try {
const res = await apiFetch<{ ok: boolean; rules?: FilterRule[] }>(
`/api/filters/revisions/${encodeURIComponent(id)}/restore`,
{ method: "POST", body: JSON.stringify({ serverId: selectedServerId }) },
)
if (res.rules) {
setRulesets((p) => p.map((rs) => rs.serverId === selectedServerId ? { ...rs, rules: res.rules ?? [] } : rs))
} else {
await loadLiveRules(selectedServerId)
}
setLiveStale(false)
toast.success("Версия применена на роутер")
await loadRevisions()
} catch (err) {
toast.error("Не удалось откатить", { description: String(err) })
} finally {
setHistoryRestoring(false)
}
}, [isLive, selectedServerId, apiFetch, loadLiveRules, loadRevisions])
const updateRules = useCallback((serverId: string, updater: (rules: FilterRule[]) => FilterRule[]) => {
const current = rulesets.find((rs) => rs.serverId === serverId)?.rules ?? []
void applyRules(serverId, updater(current))
}, [rulesets, applyRules])
const openCreate = () => {
setSheetInitial(emptyForm()); setSheetMode("create"); setEditingId(null); setSheetOpen(true)
@@ -1532,6 +1568,7 @@ export default function FiltersPage() {
}
const handleSave = (form: RuleForm) => {
if (mutationsLocked) return
const { gatewayKind: _gk, ...payload } = form
if (sheetMode === "create") {
updateRules(selectedServerId, rules => [
@@ -1549,34 +1586,30 @@ export default function FiltersPage() {
setSheetOpen(false)
}
const handleDelete = (id: string) => updateRules(selectedServerId, rules => rules.filter(r => r.id !== id))
const handleDelete = (id: string) => {
if (mutationsLocked) return
updateRules(selectedServerId, rules => rules.filter(r => r.id !== id))
}
const handleCopyRules = useCallback((targetServerId: string, rules: FilterRule[], mode: CopyMode) => {
updateRules(targetServerId, existing =>
mode === "replace" ? rules : [...existing, ...rules]
)
}, [updateRules])
const existing = rulesets.find((rs) => rs.serverId === targetServerId)?.rules ?? []
const next = mode === "replace" ? rules : [...existing, ...rules]
void applyRules(targetServerId, next, "copy")
}, [rulesets, applyRules])
const handleMoveUp = (index: number) => {
if (index === 0) return
if (mutationsLocked || index === 0) return
updateRules(selectedServerId, rules => {
const n = [...rules]; [n[index - 1], n[index]] = [n[index], n[index - 1]]; return n
})
}
const handleMoveDown = (index: number) => {
if (mutationsLocked) return
updateRules(selectedServerId, rules => {
if (index >= rules.length - 1) return rules
const n = [...rules]; [n[index], n[index + 1]] = [n[index + 1], n[index]]; return n
})
}
if (!selectedServer) {
return (
<div className="flex h-full items-center justify-center text-sm text-muted-foreground">
Нет доступных серверов
</div>
)
}
if (isLive && liveLoadState === "loading") {
return (
<div className="flex h-full flex-col items-center justify-center gap-3 text-sm text-muted-foreground">
@@ -1586,6 +1619,14 @@ export default function FiltersPage() {
)
}
if (!selectedServer) {
return (
<div className="flex h-full items-center justify-center text-sm text-muted-foreground">
Нет доступных серверов
</div>
)
}
return (
<>
<ServerRailLayout
@@ -1604,35 +1645,25 @@ export default function FiltersPage() {
<Button
variant="outline"
size="sm"
onClick={syncFromRouter}
disabled={syncBusy !== null}
title="Синхронизация Router → БД"
onClick={() => void refreshFromRouter()}
disabled={applyBusy}
title="Прочитать актуальные правила с роутера"
>
{syncBusy === "from" ? "Синк Router → DB…" : "Router → DB"}
<RefreshCwIcon className={cn("size-4", applyBusy && "animate-spin")} />
Обновить
</Button>
<Button
variant="outline"
size="sm"
onClick={syncToRouter}
disabled={syncBusy !== null}
title="Синхронизация БД → Router"
onClick={() => {
setHistoryOpen(true)
void loadRevisions()
}}
disabled={applyBusy}
title="История версий и откат на CHR"
>
{syncBusy === "to" ? "Синк DB → Router…" : "DB → Router"}
</Button>
<Button
variant="outline"
size="sm"
onClick={() => void fetchRouterCompare()}
disabled={syncBusy !== null || routerCompareLoading}
title="Сравнить правила в БД с цепочкой bgp-in на MikroTik"
className="gap-1.5"
>
{routerCompareLoading ? (
<LoaderCircleIcon className="size-4 animate-spin" />
) : (
<RefreshCwIcon className="size-4" />
)}
Сверить
<HistoryIcon className="size-4" />
История
</Button>
</>
)}
@@ -1642,12 +1673,12 @@ export default function FiltersPage() {
<Button
variant="outline" size="sm"
onClick={() => setCopyOpen(true)}
disabled={currentRules.length === 0}
disabled={currentRules.length === 0 || mutationsLocked}
title="Копировать правила на другой сервер"
>
<CopyIcon className="size-4" />Копировать
</Button>
<Button size="sm" onClick={openCreate}>
<Button size="sm" onClick={openCreate} disabled={mutationsLocked}>
<PlusIcon className="size-4" />Новое правило
</Button>
</>
@@ -1662,6 +1693,12 @@ export default function FiltersPage() {
Бекенд недоступен показаны демо-данные из lib/data. Проверьте URL бекенда в настройках.
</div>
)}
{isLive && liveStale && liveLoadState !== "error" && (
<div className="shrink-0 border-b border-amber-500/30 bg-amber-500/10 px-6 py-2.5 text-xs text-amber-700 dark:text-amber-400 flex items-center gap-2">
<AlertTriangleIcon className="size-3.5 shrink-0" />
Роутер недоступен показан кэш. Изменения заблокированы, пока не удастся прочитать CHR.
</div>
)}
<div className="border-b px-4 py-3 flex items-center gap-3 flex-wrap shrink-0 md:px-6">
<div className="relative min-w-[200px] max-w-xs flex-1">
<SearchIcon className="absolute left-2.5 top-1/2 -translate-y-1/2 size-3.5 text-muted-foreground pointer-events-none" />
@@ -1745,15 +1782,7 @@ export default function FiltersPage() {
)}>{selectedServer.latency}мс</span>
)}
<div className="ml-auto flex items-center gap-2 text-xs text-muted-foreground flex-wrap justify-end">
{isLive && routerCompare?.serverId === selectedServerId && currentRules.length > 0 && (
<span className="font-mono tabular-nums">
роутер:{" "}
<span className="text-emerald-600 dark:text-emerald-500">
{Object.values(routerCompare.byCommunity).filter(s => s === "synced").length}
</span>
/{currentRules.length} совпало
</span>
)}
{applyBusy && <span>Применение на роутер</span>}
<span>{currentRules.length} правил</span>
</div>
</div>
@@ -1790,12 +1819,6 @@ export default function FiltersPage() {
serversList={allServers}
communityNameMap={communityNameMap}
recursiveRoutes={recRoutesByServer[selectedServerId] ?? []}
routerSyncByCommunity={
!isLive || !routerCompare || routerCompare.serverId !== selectedServerId
? null
: routerCompare.byCommunity
}
isLive={isLive}
enableSorting={!!search}
onEdit={openEdit}
onDelete={handleDelete}
@@ -1805,7 +1828,7 @@ export default function FiltersPage() {
)}
{/* add rule shortcut */}
<button onClick={openCreate}
<button onClick={openCreate} disabled={mutationsLocked}
className="w-full flex items-center gap-2 px-5 py-2 text-xs text-muted-foreground hover:text-foreground hover:bg-muted/20 transition-colors border-t">
<PlusIcon className="size-3.5" />
Добавить правило для {selectedServer.name}
@@ -1854,6 +1877,17 @@ export default function FiltersPage() {
recRoutesByServer={recRoutesByServer}
ensureRecursiveFor={ensureRecursiveRoutes}
/>
<ConfigHistorySheet
open={historyOpen}
onOpenChange={setHistoryOpen}
title="История фильтров"
itemLabel="правил"
revisions={revisions}
loading={historyLoading}
restoring={historyRestoring}
onRestore={restoreRevision}
/>
</>
)
}
+67 -1
View File
@@ -49,12 +49,14 @@ import {
PowerIcon, CheckCircleIcon,
PlayIcon, SquareIcon, RotateCcwIcon, ZapIcon,
CheckCircle2Icon, XCircleIcon, MinusCircleIcon, SkipForwardIcon,
SlidersHorizontalIcon, RefreshCwIcon,
SlidersHorizontalIcon, RefreshCwIcon, HistoryIcon,
} from "lucide-react"
import { CodeExportSheet } from "@/components/reui-kit/code-export-sheet"
import { ServerRailLayout, ServerRailMobileButton } from "@/components/server-rail-layout"
import { ALL_SERVERS_ID, type ServerTileItem } from "@/components/server-tile-rail"
import { toast } from "sonner"
import { ConfigHistorySheet } from "@/components/config-history-sheet"
import type { ConfigRevisionDto } from "@/lib/config-revisions"
// ─── Types ────────────────────────────────────────────────────────────────────
@@ -1947,6 +1949,10 @@ function FirewallPageInner() {
const [exportOpen, setExportOpen] = useState(false)
const [editingAddr, setEditingAddr] = useState<Partial<AddressListEntry> | null>(null)
const [addrSheetOpen, setAddrSheetOpen] = useState(false)
const [historyOpen, setHistoryOpen] = useState(false)
const [revisions, setRevisions] = useState<ConfigRevisionDto[]>([])
const [historyLoading, setHistoryLoading] = useState(false)
const [historyRestoring, setHistoryRestoring] = useState(false)
const loadLive = useCallback(async () => {
if (!isLive) return
@@ -1970,6 +1976,42 @@ function FirewallPageInner() {
}
}, [isLive, apiFetch])
const historyServerId = selectedServerId === ALL_SERVERS_ID ? null : selectedServerId
const loadRevisions = useCallback(async () => {
if (!isLive || !historyServerId) return
setHistoryLoading(true)
try {
const res = await apiFetch<{ revisions: ConfigRevisionDto[] }>(
`/api/firewall/revisions?serverId=${encodeURIComponent(historyServerId)}`,
)
setRevisions(res.revisions)
} catch (err) {
toast.error("Не удалось загрузить историю", { description: String(err) })
setRevisions([])
} finally {
setHistoryLoading(false)
}
}, [isLive, historyServerId, apiFetch])
const restoreRevision = useCallback(async (id: string) => {
if (!isLive || !historyServerId) return
setHistoryRestoring(true)
try {
await apiFetch(
`/api/firewall/revisions/${encodeURIComponent(id)}/restore`,
{ method: "POST", body: JSON.stringify({ serverId: historyServerId }) },
)
toast.success("Версия применена на роутер")
await loadLive()
await loadRevisions()
} catch (err) {
toast.error("Не удалось откатить", { description: String(err) })
} finally {
setHistoryRestoring(false)
}
}, [isLive, historyServerId, apiFetch, loadLive, loadRevisions])
useEffect(() => {
if (!isLive) {
queueMicrotask(() => {
@@ -2373,6 +2415,19 @@ function FirewallPageInner() {
<RefreshCwIcon className={cn("size-4", dataLoading && "animate-spin")} />
Обновить
</Button>
<Button
variant="outline"
size="sm"
onClick={() => {
setHistoryOpen(true)
void loadRevisions()
}}
disabled={!isLive || !historyServerId || dataLoading}
title={!historyServerId ? "Выберите сервер, чтобы смотреть историю" : "История версий и откат на CHR"}
>
<HistoryIcon className="size-4" />
История
</Button>
<Button variant="outline" size="sm" onClick={() => setExportOpen(true)}>
<CodeXmlIcon className="size-4" />Экспорт .rsc
</Button>
@@ -2586,6 +2641,17 @@ function FirewallPageInner() {
onClose={() => setExportOpen(false)}
rules={familyRules}
/>
<ConfigHistorySheet
open={historyOpen}
onOpenChange={setHistoryOpen}
title="История Firewall"
itemLabel="объектов"
revisions={revisions}
loading={historyLoading}
restoring={historyRestoring}
onRestore={restoreRevision}
/>
</>
)
}
+289 -58
View File
@@ -13,11 +13,23 @@ import { useDataSource } from "@/lib/data-source"
import { requestJson } from "@/shared/api/http-client"
import { cn } from "@/lib/utils"
import { toast } from "sonner"
import { Frame, FramePanel } from "@/components/reui/frame"
import { KpiStatGrid } from "@/components/reui-kit/kpi-stat-grid"
import { OpsPanel } from "@/components/ops-panel"
import { Button } from "@/components/ui/button"
import { Input } from "@/components/ui/input"
import { ConfigHistorySheet } from "@/components/config-history-sheet"
import type { ConfigRevisionDto } from "@/lib/config-revisions"
import {
AlertDialog,
AlertDialogAction,
AlertDialogCancel,
AlertDialogContent,
AlertDialogDescription,
AlertDialogFooter,
AlertDialogHeader,
AlertDialogMedia,
AlertDialogTitle,
} from "@/components/ui/alert-dialog"
import {
Sheet, SheetContent, SheetHeader, SheetTitle,
SheetDescription, SheetFooter, SheetClose,
@@ -31,7 +43,7 @@ import {
LockIcon, LockOpenIcon, ShieldCheckIcon, NetworkIcon,
EyeIcon, EyeOffIcon, ChevronDownIcon, ChevronRightIcon,
CodeXmlIcon, PencilIcon, PowerIcon, Trash2Icon,
DatabaseIcon,
DatabaseIcon, HistoryIcon, TriangleAlertIcon,
} from "lucide-react"
import { CodeExportSheet } from "@/components/reui-kit/code-export-sheet"
import { ServerRailLayout, ServerRailMobileButton } from "@/components/server-rail-layout"
@@ -56,6 +68,10 @@ const STATUS_MAP: Record<GreStatus, { label: string; dot: string }> = {
down: { label: "Down", dot: "bg-red-500" },
}
function greStatusMeta(status: GreStatus | undefined) {
return STATUS_MAP[status ?? "degraded"] ?? STATUS_MAP.degraded
}
// ─── RouterOS code generator ─────────────────────────────────────────────────
function generateRosCommands(t: GreTunnel, serverById: Record<string, Server>): string {
@@ -91,10 +107,10 @@ function generateRosCommands(t: GreTunnel, serverById: Record<string, Server>):
lines.push(` address=${t.localInnerIp} \\`)
lines.push(` interface=${t.name}`)
// IPsec manual equivalent
if (t.ipsec) {
const ikeMode = t.ipsec.ikeVersion === "ikev2" ? "ike2" : "ike1"
const pfsGroup = t.ipsec.pfs ? t.ipsec.dhGroup : "none"
// IPsec: live CHR имеет только ipsec-secret; proposal — у моков/формы
if (t.ipsec?.encAlg && t.ipsec.authAlg) {
const ikeMode = t.ipsec.ikeVersion === "ikev1" ? "ike1" : "ike2"
const pfsGroup = t.ipsec.pfs ? (t.ipsec.dhGroup ?? "none") : "none"
lines.push("")
lines.push("# ── IPsec (авто через ipsec-secret; ручной эквивалент) ───────")
@@ -111,13 +127,16 @@ function generateRosCommands(t: GreTunnel, serverById: Record<string, Server>):
lines.push(` enc-algorithms=${ENC_ROS[t.ipsec.encAlg]} \\`)
lines.push(` auth-algorithms=${AUTH_ROS[t.ipsec.authAlg]} \\`)
lines.push(` pfs-group=${pfsGroup} \\`)
lines.push(` lifetime=${t.ipsec.lifetime}`)
lines.push(` lifetime=${t.ipsec.lifetime ?? "1d"}`)
lines.push("")
lines.push(`/ip ipsec policy add \\`)
lines.push(` src-address=${t.localAddress !== "0.0.0.0" ? t.localAddress + "/32" : "0.0.0.0/0"} \\`)
lines.push(` dst-address=${t.remoteAddress}/32 \\`)
lines.push(` proposal=${t.name} \\`)
lines.push(` tunnel=yes`)
} else if (t.ipsec) {
lines.push("")
lines.push("# IPsec: peer/policy создаёт RouterOS по ipsec-secret")
}
return lines.join("\n")
@@ -126,7 +145,7 @@ function generateRosCommands(t: GreTunnel, serverById: Record<string, Server>):
// ─── small ui helpers ────────────────────────────────────────────────────────
function TunnelStatus({ status }: { status: GreStatus }) {
const s = STATUS_MAP[status]
const s = greStatusMeta(status)
return (
<span className="inline-flex items-center gap-1.5 text-sm">
<span className={`size-1.5 rounded-full ${s.dot}`} />
@@ -164,6 +183,7 @@ interface BackendServer {
interface GreTunnelsApiResponse {
tunnels: GreTunnel[]
failures?: Array<{ serverId: string; serverName?: string; error: string }>
}
function makeApiFetch(backendUrl: string) {
@@ -237,7 +257,6 @@ export default function GrePage() {
const [liveTunnels, setLiveTunnels] = useState<GreTunnel[]>([])
const [dataLoading, setDataLoading] = useState(false)
const [dataError, setDataError] = useState<string | null>(null)
const [syncJhBusy, setSyncJhBusy] = useState(false)
const [pageTab, setPageTab] = useState<PageTab>("tunnels")
const [tabFilter, setTabFilter] = useState<TabFilter>("all")
@@ -245,6 +264,15 @@ export default function GrePage() {
const [selectedServerId, setSelectedServerId] = useState(ALL_SERVERS_ID)
const [tunnelOpen, setTunnelOpen] = useState(false)
const [tunnelMode, setTunnelMode] = useState<"create" | "edit">("create")
const [editingTunnel, setEditingTunnel] = useState<GreTunnel | null>(null)
const [pendingDelete, setPendingDelete] = useState<GreTunnel | null>(null)
const [mutateBusy, setMutateBusy] = useState(false)
const [liveStale, setLiveStale] = useState(false)
const [historyOpen, setHistoryOpen] = useState(false)
const [revisions, setRevisions] = useState<ConfigRevisionDto[]>([])
const [historyLoading, setHistoryLoading] = useState(false)
const [historyRestoring, setHistoryRestoring] = useState(false)
const [poolOpen, setPoolOpen] = useState(false)
const [codePreviewTunnel, setCodePreviewTunnel] = useState<GreTunnel | null>(null)
@@ -261,14 +289,19 @@ export default function GrePage() {
try {
const [backendServers, greRes] = await Promise.all([
apiFetch<BackendServer[]>("/api/servers"),
apiFetch<GreTunnelsApiResponse>("/api/filters/gre-tunnels"),
apiFetch<GreTunnelsApiResponse>("/api/gre/tunnels"),
])
setLiveServers(backendServers.map(mapBackendToServer))
setLiveTunnels(greRes.tunnels)
setLiveStale(false)
if (greRes.failures?.length) {
toast.warning(
`Не удалось опросить: ${greRes.failures.map((f) => f.serverName ?? f.serverId).join(", ")}`,
)
}
} catch (e) {
setDataError(e instanceof Error ? e.message : "Ошибка загрузки")
setLiveServers([])
setLiveTunnels([])
setLiveStale(true)
} finally {
setDataLoading(false)
}
@@ -280,6 +313,7 @@ export default function GrePage() {
setLiveServers([])
setLiveTunnels([])
setDataError(null)
setLiveStale(false)
})
return
}
@@ -323,39 +357,186 @@ export default function GrePage() {
[displayPools],
)
const syncJhToDb = useCallback(async () => {
if (!isLive || syncJhBusy) return
const jh = displayServers.filter((s) => s.type === "jump-host" && s.enabled)
if (jh.length === 0) {
toast.info("Нет включённых Jump Host в списке серверов")
const historyServerId = selectedServerId === ALL_SERVERS_ID ? null : selectedServerId
const mutationsLocked = isLive && (mutateBusy || liveStale || historyRestoring)
const loadRevisions = useCallback(async () => {
if (!isLive || !historyServerId) return
setHistoryLoading(true)
try {
const res = await apiFetch<{ revisions: ConfigRevisionDto[] }>(
`/api/gre/revisions?serverId=${encodeURIComponent(historyServerId)}`,
)
setRevisions(res.revisions)
} catch (err) {
toast.error("Не удалось загрузить историю", { description: String(err) })
setRevisions([])
} finally {
setHistoryLoading(false)
}
}, [isLive, historyServerId, apiFetch])
const restoreRevision = useCallback(async (id: string) => {
if (!isLive || !historyServerId) return
setHistoryRestoring(true)
try {
await apiFetch(
`/api/gre/revisions/${encodeURIComponent(id)}/restore`,
{ method: "POST", body: JSON.stringify({ serverId: historyServerId }) },
)
toast.success("Версия применена на роутер")
await loadLive()
await loadRevisions()
} catch (err) {
toast.error("Не удалось откатить", { description: String(err) })
} finally {
setHistoryRestoring(false)
}
}, [isLive, historyServerId, apiFetch, loadLive, loadRevisions])
function tunnelWriteBody(form: typeof defaultTunnelForm) {
return {
serverId: form.serverId,
name: form.name.trim(),
localAddress: form.localAddress.trim() || undefined,
remoteAddress: form.remoteAddress.trim(),
localInnerIp: form.localInnerIp.trim() || undefined,
remoteInnerIp: form.remoteInnerIp.trim() || undefined,
comment: form.comment || undefined,
enabled: form.enabled,
mtu: form.mtu,
keepaliveInterval: form.keepaliveInterval,
keepaliveRetries: form.keepaliveRetries,
dscp: form.dscp,
clampTcpMss: form.clampTcpMss,
allowFastPath: form.allowFastPath,
ipsecSecret: form.ipsecEnabled ? form.ipsecSecret : undefined,
}
}
async function submitTunnel() {
if (!isLive) {
toast.info("Создание на роутер доступно только в live-режиме")
return
}
setSyncJhBusy(true)
const errors: string[] = []
try {
for (const s of jh) {
try {
await apiFetch<{ ok: boolean }>("/api/filters/sync/from-router", {
method: "POST",
body: JSON.stringify({ serverId: s.id }),
})
} catch (e) {
errors.push(`${s.name}: ${e instanceof Error ? e.message : "ошибка"}`)
}
}
const fresh = await apiFetch<GreTunnelsApiResponse>("/api/filters/gre-tunnels")
setLiveTunnels(fresh.tunnels)
if (errors.length) {
toast.warning(`Синхронизировано JH: ${jh.length - errors.length}/${jh.length}. Ошибки: ${errors.join("; ")}`)
} else {
toast.success(`Правила с ${jh.length} JH записаны в БД, список GRE обновлён.`)
}
} catch (e) {
toast.error(e instanceof Error ? e.message : "Ошибка после синхронизации")
} finally {
setSyncJhBusy(false)
if (liveStale) {
toast.error("Роутер недоступен — изменения заблокированы")
return
}
}, [isLive, syncJhBusy, apiFetch, displayServers])
if (!tForm.name.trim() || !tForm.serverId || !tForm.remoteAddress.trim()) {
toast.error("Заполните имя, сервер и удалённый адрес")
return
}
if (tForm.ipsecEnabled && tForm.ipsecSecret.trim().length < 8) {
toast.error("Для IPsec нужен PSK не короче 8 символов")
return
}
setMutateBusy(true)
try {
if (tunnelMode === "edit" && editingTunnel) {
await apiFetch("/api/gre/tunnels", {
method: "PATCH",
body: JSON.stringify({
...tunnelWriteBody(tForm),
rosId: editingTunnel.id,
name: editingTunnel.name,
}),
})
toast.success(`Туннель ${tForm.name} обновлён`)
} else {
await apiFetch("/api/gre/tunnels", {
method: "POST",
body: JSON.stringify(tunnelWriteBody(tForm)),
})
toast.success(`Туннель ${tForm.name} создан`)
}
setTunnelOpen(false)
setEditingTunnel(null)
await loadLive()
} catch (err) {
toast.error("Не удалось сохранить туннель", { description: String(err) })
} finally {
setMutateBusy(false)
}
}
async function toggleTunnel(t: GreTunnel) {
if (!isLive || mutationsLocked) return
setMutateBusy(true)
try {
await apiFetch("/api/gre/tunnels", {
method: "PATCH",
body: JSON.stringify({
serverId: t.serverId,
rosId: t.id,
name: t.name,
enabled: !t.enabled,
remoteAddress: t.remoteAddress,
}),
})
toast.success(t.enabled ? `Выключен ${t.name}` : `Включён ${t.name}`)
await loadLive()
} catch (err) {
toast.error("Не удалось изменить туннель", { description: String(err) })
} finally {
setMutateBusy(false)
}
}
async function confirmDeleteTunnel() {
const t = pendingDelete
if (!t || !isLive) return
setMutateBusy(true)
try {
await apiFetch("/api/gre/tunnels", {
method: "DELETE",
body: JSON.stringify({ serverId: t.serverId, rosId: t.id, name: t.name }),
})
toast.success(`Удалён ${t.name}`)
setPendingDelete(null)
await loadLive()
} catch (err) {
toast.error("Не удалось удалить туннель", { description: String(err) })
} finally {
setMutateBusy(false)
}
}
function openCreateTunnel() {
setTunnelMode("create")
setEditingTunnel(null)
setTForm({
...defaultTunnelForm,
serverId: selectedServerId === ALL_SERVERS_ID ? "" : selectedServerId,
})
setTunnelOpen(true)
}
function openEditTunnel(t: GreTunnel) {
setTunnelMode("edit")
setEditingTunnel(t)
setTForm({
...defaultTunnelForm,
name: t.name,
serverId: t.serverId,
localAddress: t.localAddress === "0.0.0.0" ? "" : t.localAddress,
remoteAddress: t.remoteAddress,
poolId: t.poolId === "live" ? "" : t.poolId,
localInnerIp: t.localInnerIp,
remoteInnerIp: t.remoteInnerIp,
comment: t.comment,
enabled: t.enabled,
ipsecEnabled: !!t.ipsec,
ipsecSecret: t.ipsec?.secret ?? "",
mtu: t.mtu,
keepaliveInterval: t.keepaliveInterval,
keepaliveRetries: t.keepaliveRetries,
dscp: String(t.dscp),
clampTcpMss: t.clampTcpMss,
allowFastPath: t.allowFastPath,
})
setTunnelOpen(true)
}
useEffect(() => {
if (dataError) toast.error(dataError)
@@ -403,6 +584,14 @@ export default function GrePage() {
showAll
allCount={displayServers.length}
loading={isLive && dataLoading && displayServers.length === 0}
banner={
isLive && liveStale ? (
<div className="shrink-0 border-b border-amber-500/30 bg-amber-500/10 px-6 py-2.5 text-xs text-amber-700 dark:text-amber-400 flex items-center gap-2">
<TriangleAlertIcon className="size-3.5 shrink-0" />
Роутер недоступен показан кэш. Изменения заблокированы, пока не удастся прочитать CHR.
</div>
) : null
}
header={
<PageHeader
crumbs={[{ label: "Управление" }, { label: "GRE-туннели" }]}
@@ -422,14 +611,17 @@ export default function GrePage() {
<Button
variant="outline"
size="sm"
onClick={() => { void syncJhToDb() }}
disabled={!isLive || syncJhBusy || dataLoading}
title="Загрузить правила фильтрации с каждого Jump Host в БД и обновить опрос GRE"
onClick={() => {
setHistoryOpen(true)
void loadRevisions()
}}
disabled={!isLive || !historyServerId || dataLoading}
title={!historyServerId ? "Выберите сервер, чтобы смотреть историю" : "История версий и откат на CHR"}
>
<DatabaseIcon className={cn("size-4", syncJhBusy && "animate-pulse")} />
JH БД
<HistoryIcon className="size-4" />
История
</Button>
<Button size="sm" onClick={() => { setTForm(defaultTunnelForm); setTunnelOpen(true) }}>
<Button size="sm" onClick={openCreateTunnel} disabled={mutateBusy}>
<PlusIcon className="size-4" />Добавить туннель
</Button>
</>
@@ -521,6 +713,10 @@ export default function GrePage() {
servers={displayServers}
pools={displayPools}
onCodePreview={setCodePreviewTunnel}
onEdit={openEditTunnel}
onToggle={(t) => { void toggleTunnel(t) }}
onDelete={setPendingDelete}
mutationsLocked={mutationsLocked}
/>
</DataPageCard>
)}
@@ -548,7 +744,7 @@ export default function GrePage() {
<div className="flex flex-wrap gap-2">
{poolTunnels.map((t, tunnelIndex) => (
<div key={`${t.id}:${t.serverId}:${t.name}:${tunnelIndex}`} className="flex items-center gap-2 border border-border rounded-md px-3 py-1.5 bg-muted/30 text-xs">
<span className={`size-1.5 rounded-full ${STATUS_MAP[t.status].dot}`} />
<span className={`size-1.5 rounded-full ${greStatusMeta(t.status).dot}`} />
<span className="font-mono font-medium">{t.name}</span>
<span className="text-muted-foreground">{t.localInnerIp} {t.remoteInnerIp}</span>
{t.ipsec && <LockIcon className="size-3 text-emerald-400" />}
@@ -609,8 +805,8 @@ export default function GrePage() {
codePreviewTunnel ? (
<div className="flex flex-wrap gap-3 text-xs shrink-0">
<span className="flex items-center gap-1.5">
<span className={`size-1.5 rounded-full ${STATUS_MAP[codePreviewTunnel.status].dot}`} />
{STATUS_MAP[codePreviewTunnel.status].label}
<span className={`size-1.5 rounded-full ${greStatusMeta(codePreviewTunnel.status).dot}`} />
{greStatusMeta(codePreviewTunnel.status).label}
</span>
<span className="text-muted-foreground">·</span>
<span>{serverById[codePreviewTunnel.serverId]?.name}</span>
@@ -625,7 +821,7 @@ export default function GrePage() {
<span className="text-muted-foreground">·</span>
<span className="flex items-center gap-1 text-success">
<LockIcon className="size-3" />
IPsec {IKE_LABELS[codePreviewTunnel.ipsec.ikeVersion]}
IPsec {codePreviewTunnel.ipsec.ikeVersion ? IKE_LABELS[codePreviewTunnel.ipsec.ikeVersion] : "PSK"}
</span>
</>
) : null}
@@ -638,18 +834,18 @@ export default function GrePage() {
<Sheet open={tunnelOpen} onOpenChange={setTunnelOpen}>
<SheetContent side="right" className="w-full sm:max-w-lg flex flex-col gap-0 p-0">
<SheetHeader className="px-6 pt-6 pb-4 border-b shrink-0">
<SheetTitle>Новый GRE-туннель</SheetTitle>
<SheetDescription>RouterOS 7.20+ · /interface gre add</SheetDescription>
<SheetTitle>{tunnelMode === "edit" ? "Редактировать GRE-туннель" : "Новый GRE-туннель"}</SheetTitle>
<SheetDescription>RouterOS 7.20+ · /interface gre {tunnelMode === "edit" ? "set" : "add"}</SheetDescription>
</SheetHeader>
<div className="flex-1 overflow-y-auto px-6 py-5 flex flex-col gap-5">
<div className="flex flex-col gap-4">
<SectionTitle>Основные</SectionTitle>
<FormField label="Имя интерфейса" required hint="Только латиница, цифры и дефис, например gre-msk-spb">
<Input className="font-mono" placeholder="gre-msk-spb" value={tForm.name} onChange={(e) => setT("name", e.target.value)} />
<Input className="font-mono" placeholder="gre-msk-spb" value={tForm.name} disabled={tunnelMode === "edit"} onChange={(e) => setT("name", e.target.value)} />
</FormField>
<FormField label="Сервер (MikroTik)" required>
<select value={tForm.serverId} onChange={(e) => setT("serverId", e.target.value)}
<select value={tForm.serverId} onChange={(e) => setT("serverId", e.target.value)} disabled={tunnelMode === "edit"}
className="h-8 w-full rounded-lg border border-input bg-background px-2.5 text-sm text-foreground outline-none focus-visible:border-ring focus-visible:ring-3 focus-visible:ring-ring/50">
<option value="" disabled>Выбрать сервер</option>
{displayServers.map((s) => <option key={s.id} value={s.id}>{s.name} ({s.site})</option>)}
@@ -676,7 +872,7 @@ export default function GrePage() {
<div className="flex flex-col gap-4">
<SectionTitle>Внутренний IP</SectionTitle>
<FormField label="IP-пул" required hint="Из какого пула выделяется /30-блок">
<FormField label="IP-пул" hint="Необязательно — внутренний IP можно указать вручную">
<select value={tForm.poolId} onChange={(e) => setT("poolId", e.target.value)}
className="h-8 w-full rounded-lg border border-input bg-background px-2.5 text-sm text-foreground outline-none focus-visible:border-ring focus-visible:ring-3 focus-visible:ring-ring/50">
<option value="" disabled>Выбрать пул</option>
@@ -798,7 +994,9 @@ export default function GrePage() {
<SheetFooter className="px-6 py-4 border-t shrink-0 flex-row gap-2">
<SheetClose render={<Button variant="outline" className="flex-1" />}>Отмена</SheetClose>
<Button className="flex-1" onClick={() => setTunnelOpen(false)}>Создать туннель</Button>
<Button className="flex-1" onClick={() => void submitTunnel()} disabled={mutateBusy}>
{tunnelMode === "edit" ? "Сохранить" : "Создать туннель"}
</Button>
</SheetFooter>
</SheetContent>
</Sheet>
@@ -845,6 +1043,39 @@ export default function GrePage() {
</SheetFooter>
</SheetContent>
</Sheet>
<ConfigHistorySheet
open={historyOpen}
onOpenChange={setHistoryOpen}
title="История GRE"
itemLabel="туннелей"
revisions={revisions}
loading={historyLoading}
restoring={historyRestoring}
onRestore={restoreRevision}
/>
<AlertDialog open={!!pendingDelete} onOpenChange={(v) => { if (!v) setPendingDelete(null) }}>
<AlertDialogContent size="default">
<AlertDialogHeader>
<AlertDialogMedia className="bg-destructive/10 text-destructive">
<Trash2Icon />
</AlertDialogMedia>
<AlertDialogTitle>Удалить GRE-туннель?</AlertDialogTitle>
<AlertDialogDescription>
{pendingDelete
? `${pendingDelete.name} на сервере ${serverById[pendingDelete.serverId]?.name ?? pendingDelete.serverId}. Будут удалены интерфейс и связанный /ip/address.`
: null}
</AlertDialogDescription>
</AlertDialogHeader>
<AlertDialogFooter>
<AlertDialogCancel onClick={() => setPendingDelete(null)}>Отмена</AlertDialogCancel>
<AlertDialogAction variant="destructive" onClick={() => void confirmDeleteTunnel()}>
Удалить
</AlertDialogAction>
</AlertDialogFooter>
</AlertDialogContent>
</AlertDialog>
</>
)
}
+896
View File
@@ -0,0 +1,896 @@
"use client"
import { useCallback, useEffect, useMemo, useState } from "react"
import { PageHeader } from "@/components/page-header"
import { servers as mockServers } from "@/lib/data"
import type { Server } from "@/lib/data"
import { DataPageCard } from "@/components/data-page-card"
import { DataPageToolbar } from "@/components/data-page-toolbar"
import { IpsecUsersGrid } from "@/components/ipsec/ipsec-users-grid"
import { IpsecServerGrid } from "@/components/ipsec/ipsec-server-grid"
import { IpsecUserSheet, type IpsecUserFormState } from "@/components/ipsec/ipsec-user-sheet"
import { IpsecInitSheet, type IpsecInitFormState } from "@/components/ipsec/ipsec-init-sheet"
import { IpsecCertSheet } from "@/components/ipsec/ipsec-cert-sheet"
import { IpsecPeerSheet, type IpsecPeerFormState } from "@/components/ipsec/ipsec-peer-sheet"
import { Button } from "@/components/ui/button"
import {
Alert,
AlertDescription,
AlertTitle,
} from "@/components/reui/alert"
import { KpiStatGrid } from "@/components/reui-kit/kpi-stat-grid"
import { OpsPanel } from "@/components/ops-panel"
import { Tabs, TabsContent, TabsList, TabsTrigger } from "@/components/ui/tabs"
import {
AlertDialog,
AlertDialogAction,
AlertDialogCancel,
AlertDialogContent,
AlertDialogDescription,
AlertDialogFooter,
AlertDialogHeader,
AlertDialogMedia,
AlertDialogTitle,
} from "@/components/ui/alert-dialog"
import { useDataSource } from "@/lib/data-source"
import { ApiClientError, requestJson } from "@/shared/api/http-client"
import {
createIpsecUser,
deleteIpsecCert,
deleteIpsecPeer,
deleteIpsecServer,
deleteIpsecUser,
exportIpsecCertByName,
exportIpsecUserCert,
initIpsecServer,
listIpsec,
patchIpsecPeer,
patchIpsecUser,
restoreIpsecRevision,
} from "@/shared/api/ipsec"
import type {
IpsecCertBundle,
IpsecCertInfoDto,
IpsecClientDto,
IpsecPeerDto,
IpsecServerSummaryDto,
} from "@mmapp/contracts/ipsec"
import { ServerRailLayout, ServerRailMobileButton } from "@/components/server-rail-layout"
import {
ALL_SERVERS_ID,
type ServerTileItem,
} from "@/components/server-tile-rail"
import { ConfigHistorySheet } from "@/components/config-history-sheet"
import type { ConfigRevisionDto } from "@/lib/config-revisions"
import { toast } from "sonner"
import { findFreePoolIp } from "@/lib/ipsec-client"
import {
ShieldCheckIcon, PlusIcon, KeyRoundIcon,
UsersIcon, ActivityIcon, RefreshCwIcon, InfoIcon,
Trash2Icon, CodeXmlIcon, AlertCircleIcon, HistoryIcon,
} from "lucide-react"
type IpsecWorkspaceTab = "clients" | "server" | "cli"
type IpsecFailure = { serverId: string; serverName?: string; error: string }
type PendingDelete =
| { kind: "user"; client: IpsecClientDto }
| { kind: "server"; summary: IpsecServerSummaryDto }
| { kind: "peer"; serverId: string; peer: IpsecPeerDto }
| { kind: "cert"; serverId: string; cert: IpsecCertInfoDto }
const MOCK_IPSEC_PEER: IpsecPeerDto = {
id: "srv2:pp1",
rosId: "*PP1",
serverId: "srv2",
serverName: "mt-spb-edge-01",
name: "ipsec-vpn",
address: "0.0.0.0/0",
exchangeMode: "ike2",
passive: true,
certificate: "ipsec-server",
profile: "ipsec-vpn",
disabled: false,
managed: true,
}
const MOCK_IPSEC_SERVERS: IpsecServerSummaryDto[] = [
{
serverId: "srv2",
serverName: "mt-spb-edge-01",
initialized: true,
ike2Ready: true,
serverEndpoint: "vpn.example.com",
pool: { id: "srv2:p1", rosId: "*P1", serverId: "srv2", name: "ipsec-vpn", ranges: "10.77.0.2-10.77.0.254", managed: true },
sharedModeConfig: { id: "srv2:m1", rosId: "*M1", serverId: "srv2", name: "ipsec-vpn", addressPool: "ipsec-vpn", staticDns: "10.77.0.1", managed: true },
peer: MOCK_IPSEC_PEER,
peers: [MOCK_IPSEC_PEER],
caCert: { name: "ipsec-ca", commonName: "MikrotikManager IPsec CA", keySize: "4096", expiresAt: "2036-09-01", trusted: true, hasPrivateKey: true, role: "ca", managed: true },
serverCert: { name: "ipsec-server", commonName: "vpn.example.com", keySize: "2048", expiresAt: "2031-09-01", trusted: true, hasPrivateKey: true, role: "server", managed: true },
natRuleManaged: true,
clientsTotal: 3,
clientsOnline: 1,
certs: [],
},
]
const MOCK_IPSEC_CLIENTS: IpsecClientDto[] = [
{
id: "srv2:*I1", rosId: "*I1", serverId: "srv2", serverName: "mt-spb-edge-01",
name: "alice", authMethod: "certificate", kind: "cert", certificateName: "ipsec-user-alice", certName: "ipsec-user-alice", signedBy: "MyCA", commonName: "alice",
staticIp: "10.77.0.10", modeConfigName: "mc-ipsec-alice", peerName: "ipsec-vpn",
online: true, activeAddress: "10.100.1.7", activeSince: "2h", disabled: false, managed: true,
},
{
id: "srv2:*I2", rosId: "*I2", serverId: "srv2", serverName: "mt-spb-edge-01",
name: "bob", authMethod: "certificate", kind: "cert", certificateName: "ipsec-user-bob", certName: "ipsec-user-bob", signedBy: "MyCA", commonName: "bob",
peerName: "ipsec-vpn", online: false, disabled: false, managed: true,
},
{
id: "srv2:*I3", rosId: "*I3", serverId: "srv2", serverName: "mt-spb-edge-01",
name: "tablet-psk", authMethod: "pre-shared-key", kind: "psk", remoteId: "tablet",
peerName: "ipsec-vpn", online: false, disabled: false, managed: true,
},
]
interface BackendServer {
id: number
name: string
host: string
country: string
type?: Server["type"]
enabled: boolean
status?: Server["status"]
latency?: number | null
asn?: string
}
function mapBackendServer(s: BackendServer): Server {
return {
id: String(s.id),
name: s.name || s.host,
host: s.host,
model: "—",
os: "—",
site: "",
country: s.country || "UN",
asn: s.asn ?? "",
type: s.type ?? "exit-node",
enabled: s.enabled,
status: s.status ?? "online",
latency: s.latency ?? null,
sessions: 0,
}
}
export default function IpsecPage() {
const { mode, backendUrl } = useDataSource()
const isLive = mode === "live"
const [summaries, setSummaries] = useState<IpsecServerSummaryDto[]>([])
const [clients, setClients] = useState<IpsecClientDto[]>([])
const [liveServers, setLiveServers] = useState<Server[]>([])
const [failures, setFailures] = useState<IpsecFailure[]>([])
const [loading, setLoading] = useState(false)
const [busy, setBusy] = useState(false)
const [certBusy, setCertBusy] = useState(false)
const [selectedServerId, setSelectedServerId] = useState(ALL_SERVERS_ID)
const [workspaceTab, setWorkspaceTab] = useState<IpsecWorkspaceTab>("clients")
const [search, setSearch] = useState("")
const [initOpen, setInitOpen] = useState(false)
const [userOpen, setUserOpen] = useState(false)
const [editing, setEditing] = useState<IpsecClientDto | null>(null)
const [certOpen, setCertOpen] = useState(false)
const [certBundle, setCertBundle] = useState<IpsecCertBundle | null>(null)
const [certClient, setCertClient] = useState<IpsecClientDto | null>(null)
const [certByName, setCertByName] = useState<{ serverId: string; name: string } | null>(null)
const [peerOpen, setPeerOpen] = useState(false)
const [editingPeer, setEditingPeer] = useState<{ serverId: string; peer: IpsecPeerDto } | null>(null)
const [pendingDelete, setPendingDelete] = useState<PendingDelete | null>(null)
const [historyOpen, setHistoryOpen] = useState(false)
const [revisions, setRevisions] = useState<ConfigRevisionDto[]>([])
const [historyLoading, setHistoryLoading] = useState(false)
const [historyRestoring, setHistoryRestoring] = useState(false)
const loadLive = useCallback(async () => {
if (!isLive) return
setLoading(true)
try {
const [ipsec, servers] = await Promise.all([
listIpsec(backendUrl),
requestJson<BackendServer[]>(backendUrl, "/api/servers"),
])
setSummaries(ipsec.servers)
setClients(ipsec.clients)
setLiveServers(servers.filter((s) => s.enabled).map(mapBackendServer))
setFailures(ipsec.failures ?? [])
if (ipsec.failures?.length) {
toast.warning(
`Не удалось опросить: ${ipsec.failures.map((f) => f.serverName ?? f.serverId).join(", ")}`,
)
}
} catch (e) {
toast.error(e instanceof Error ? e.message : "Ошибка загрузки IPsec")
setSummaries([])
setClients([])
setFailures([])
} finally {
setLoading(false)
}
}, [isLive, backendUrl])
useEffect(() => {
if (!isLive) {
queueMicrotask(() => {
setSummaries([])
setClients([])
setLiveServers([])
setFailures([])
})
return
}
queueMicrotask(() => {
void loadLive()
})
}, [isLive, loadLive])
const displayServers = isLive ? liveServers : mockServers.filter((s) => s.enabled)
const displaySummaries = isLive ? summaries : MOCK_IPSEC_SERVERS
const displayClients = isLive ? clients : MOCK_IPSEC_CLIENTS
const effectiveServerId =
selectedServerId === ALL_SERVERS_ID || displayServers.some((s) => s.id === selectedServerId)
? selectedServerId
: ALL_SERVERS_ID
const historyServerId = effectiveServerId === ALL_SERVERS_ID ? null : effectiveServerId
const loadRevisions = useCallback(async () => {
if (!isLive || !historyServerId) return
setHistoryLoading(true)
try {
const res = await requestJson<{ revisions: ConfigRevisionDto[] }>(
backendUrl,
`/api/ipsec/revisions?serverId=${encodeURIComponent(historyServerId)}`,
)
setRevisions(res.revisions)
} catch (err) {
toast.error("Не удалось загрузить историю", { description: String(err) })
setRevisions([])
} finally {
setHistoryLoading(false)
}
}, [isLive, historyServerId, backendUrl])
const restoreRevision = useCallback(async (id: string) => {
if (!isLive || !historyServerId) return
setHistoryRestoring(true)
try {
await restoreIpsecRevision(backendUrl, id, historyServerId)
toast.success("Версия применена на роутер")
await loadLive()
await loadRevisions()
} catch (err) {
toast.error("Не удалось откатить", { description: String(err) })
} finally {
setHistoryRestoring(false)
}
}, [isLive, historyServerId, backendUrl, loadLive, loadRevisions])
const scopedClients = useMemo(() => {
if (effectiveServerId === ALL_SERVERS_ID) return displayClients
return displayClients.filter((c) => c.serverId === effectiveServerId)
}, [displayClients, effectiveServerId])
const scopedSummaries = useMemo(() => {
if (effectiveServerId === ALL_SERVERS_ID) return displaySummaries
return displaySummaries.filter((s) => s.serverId === effectiveServerId)
}, [displaySummaries, effectiveServerId])
const filteredClients = useMemo(() => {
if (!search) return scopedClients
const q = search.toLowerCase()
return scopedClients.filter(
(c) =>
c.name.toLowerCase().includes(q) ||
c.serverName.toLowerCase().includes(q) ||
(c.commonName ?? "").toLowerCase().includes(q) ||
(c.staticIp ?? "").includes(q),
)
}, [scopedClients, search])
const serversWithIpsec = scopedSummaries.filter((s) => s.peers.length > 0).length
const clientsOnline = scopedClients.filter((c) => c.online).length
const caOk = scopedSummaries.some((s) => s.caCert)
const compactServer = effectiveServerId !== ALL_SERVERS_ID
const sheetServerId = compactServer ? effectiveServerId : undefined
const certNames = useMemo(
() => Array.from(new Set(scopedSummaries.flatMap((s) => (s.certs ?? []).map((c) => c.name)))).sort(),
[scopedSummaries],
)
const peerOptions = useMemo(
() => scopedSummaries.flatMap((s) => s.peers.map((p) => ({ serverId: s.serverId, name: p.name, managed: p.managed }))),
[scopedSummaries],
)
const serverOptions = displayServers.map((s) => ({
id: s.id,
name: s.name,
host: s.host,
}))
/** Подсказка свободного IP из пула выбранного сервера. */
const freeIpHint = useMemo(() => {
const summary = scopedSummaries.find((s) => s.ike2Ready && s.pool)
if (!summary?.pool) return undefined
const taken = scopedClients.map((c) => c.staticIp).filter(Boolean) as string[]
return findFreePoolIp(summary.pool.ranges, taken) ?? undefined
}, [scopedSummaries, scopedClients])
const railItems = useMemo<ServerTileItem[]>(() => {
const counts = new Map<string, number>()
for (const c of displayClients) {
counts.set(c.serverId, (counts.get(c.serverId) ?? 0) + 1)
}
return displayServers.map((s) => ({
id: s.id,
name: s.name,
host: s.host,
site: s.site,
country: s.country,
status: s.status,
type: s.type,
count: counts.get(s.id) ?? 0,
enabled: s.enabled,
title: [s.name, s.host, s.asn].filter(Boolean).join(" · "),
}))
}, [displayServers, displayClients])
const handleInit = async (form: IpsecInitFormState) => {
setBusy(true)
try {
await initIpsecServer(backendUrl, {
serverId: form.serverId,
serverEndpoint: form.serverEndpoint.trim(),
poolCidr: form.poolCidr.trim(),
dns: form.dns.trim() || undefined,
caDaysValid: 3650,
serverDaysValid: 3650,
clientDaysValid: 1825,
createNatRule: form.createNatRule,
})
toast.success("IKEv2-сервер инициализирован")
setInitOpen(false)
await loadLive()
} catch (e) {
toast.error("Ошибка инициализации", {
description: e instanceof Error ? e.message : String(e),
})
} finally {
setBusy(false)
}
}
const handleCreateUser = async (form: IpsecUserFormState) => {
setBusy(true)
try {
const res = await createIpsecUser(backendUrl, {
serverId: form.serverId,
name: form.name.trim(),
peerName: form.peerName.trim() || undefined,
authMethod: form.authMethod,
psk: form.authMethod === "pre-shared-key" ? form.psk : undefined,
remoteId: form.authMethod === "pre-shared-key" ? form.remoteId.trim() || undefined : undefined,
staticIp: form.useStaticIp && form.staticIp.trim() ? form.staticIp.trim() : undefined,
passphrase: form.passphrase.trim() || undefined,
})
toast.success(`Клиент «${form.name.trim()}» создан`)
setUserOpen(false)
setEditing(null)
await loadLive()
if (res.bundle) {
setCertBundle(res.bundle)
setCertClient(res.client ?? null)
setCertByName(null)
setCertOpen(true)
}
} catch (e) {
toast.error("Ошибка создания клиента", {
description: e instanceof Error ? e.message : String(e),
})
} finally {
setBusy(false)
}
}
const handleEditUser = async (form: IpsecUserFormState) => {
if (!editing) return
setBusy(true)
try {
await patchIpsecUser(backendUrl, editing.serverId, editing.rosId, {
name: form.name.trim() !== editing.name ? form.name.trim() : undefined,
staticIp: form.useStaticIp && form.staticIp.trim() ? form.staticIp.trim() : null,
psk: form.authMethod === "pre-shared-key" && form.psk.trim() ? form.psk : undefined,
})
toast.success("Клиент обновлён")
setUserOpen(false)
setEditing(null)
await loadLive()
} catch (e) {
toast.error("Ошибка обновления", { description: e instanceof Error ? e.message : String(e) })
} finally {
setBusy(false)
}
}
const openCert = async (client: IpsecClientDto) => {
const byName = client.kind === "cert" && Boolean(client.certName)
setCertClient(byName ? null : client)
setCertByName(byName ? { serverId: client.serverId, name: client.certName! } : null)
setCertBundle(null)
setCertOpen(true)
setCertBusy(true)
try {
const passphrase = `mm-${Math.random().toString(36).slice(2, 10)}`
const bundle = byName
? await exportIpsecCertByName(backendUrl, client.serverId, client.certName!, passphrase)
: await exportIpsecUserCert(backendUrl, client.serverId, client.rosId, passphrase)
setCertBundle(bundle)
} catch (e) {
toast.error("Ошибка экспорта сертификата", {
description: e instanceof Error ? e.message : String(e),
})
setCertOpen(false)
} finally {
setCertBusy(false)
}
}
/** Экспорт .p12 существующего клиентского серта по имени (client1/anakondra и т.п.). */
const openCertByName = async (serverId: string, cert: IpsecCertInfoDto) => {
setCertByName({ serverId, name: cert.name })
setCertClient(null)
setCertBundle(null)
setCertOpen(true)
setCertBusy(true)
try {
const passphrase = `mm-${Math.random().toString(36).slice(2, 10)}`
const bundle = await exportIpsecCertByName(backendUrl, serverId, cert.name, passphrase)
setCertBundle(bundle)
} catch (e) {
toast.error("Ошибка экспорта сертификата", {
description: e instanceof Error ? e.message : String(e),
})
setCertOpen(false)
} finally {
setCertBusy(false)
}
}
const reexportCert = async (passphrase: string) => {
if (!certClient && !certByName) return
setCertBusy(true)
try {
const bundle = certByName
? await exportIpsecCertByName(backendUrl, certByName.serverId, certByName.name, passphrase)
: await exportIpsecUserCert(backendUrl, certClient!.serverId, certClient!.rosId, passphrase)
setCertBundle(bundle)
} catch (e) {
toast.error("Ошибка экспорта", { description: e instanceof Error ? e.message : String(e) })
} finally {
setCertBusy(false)
}
}
const confirmDelete = async () => {
if (!pendingDelete) return
const target = pendingDelete
setBusy(true)
try {
if (target.kind === "user") {
await deleteIpsecUser(backendUrl, target.client.serverId, target.client.rosId)
toast.success(`Клиент «${target.client.name}» удалён`)
} else if (target.kind === "server") {
await deleteIpsecServer(backendUrl, target.summary.serverId)
toast.success(`IKEv2-сервер на ${target.summary.serverName} удалён`)
} else if (target.kind === "peer") {
await deleteIpsecPeer(backendUrl, target.serverId, target.peer.rosId)
toast.success(`Peer «${target.peer.name}» удалён`)
} else {
await deleteIpsecCert(backendUrl, target.serverId, target.cert.name)
toast.success(`Сертификат «${target.cert.name}» удалён`)
}
await loadLive()
} catch (e) {
const forceDelete = async (fn: () => Promise<unknown>) => {
try {
await fn()
toast.success("Удалено")
await loadLive()
} catch (err) {
toast.error("Не удалось удалить", { description: err instanceof Error ? err.message : String(err) })
}
}
if (e instanceof ApiClientError && e.status === 409 && target.kind === "peer") {
toast.error("На peer ссылаются identity", {
description: e.message,
action: {
label: "Удалить принудительно",
onClick: () => { void forceDelete(() => deleteIpsecPeer(backendUrl, target.serverId, target.peer.rosId, { force: true })) },
},
})
} else if (e instanceof ApiClientError && e.status === 409 && target.kind === "cert") {
toast.error("Сертификат используется", {
description: e.message,
action: {
label: "Удалить принудительно",
onClick: () => { void forceDelete(() => deleteIpsecCert(backendUrl, target.serverId, target.cert.name, { force: true })) },
},
})
} else {
toast.error("Ошибка удаления", { description: e instanceof Error ? e.message : String(e) })
}
} finally {
setBusy(false)
setPendingDelete(null)
}
}
const openEditPeer = (serverId: string, peer: IpsecPeerDto) => {
setEditingPeer({ serverId, peer })
setPeerOpen(true)
}
const handleEditPeer = async (form: IpsecPeerFormState) => {
if (!editingPeer) return
setBusy(true)
try {
await patchIpsecPeer(backendUrl, editingPeer.serverId, editingPeer.peer.rosId, {
name: form.name.trim() !== editingPeer.peer.name ? form.name.trim() : undefined,
address: form.address.trim() || undefined,
exchangeMode: form.exchangeMode.trim() || undefined,
passive: form.passive,
certificate: form.certificate.trim() || undefined,
profile: form.profile.trim() || undefined,
disabled: form.disabled,
})
toast.success("Peer обновлён")
setPeerOpen(false)
setEditingPeer(null)
await loadLive()
} catch (e) {
toast.error("Ошибка обновления peer", { description: e instanceof Error ? e.message : String(e) })
} finally {
setBusy(false)
}
}
const deleteDialogTitle = (() => {
if (!pendingDelete) return ""
switch (pendingDelete.kind) {
case "server": return `Удалить IKEv2-сервер на ${pendingDelete.summary.serverName}?`
case "user": return `Удалить клиента «${pendingDelete.client.name}»?`
case "peer": return `Удалить peer «${pendingDelete.peer.name}»?`
case "cert": return `Удалить сертификат «${pendingDelete.cert.name}»?`
}
})()
const deleteDialogDescription = (() => {
if (!pendingDelete) return ""
switch (pendingDelete.kind) {
case "server":
return "Удалит managed-объекты (identity, mode-config, peer, пул, NAT) и сертификаты IKEv2 на этом роутере. Действие можно откатить через «Историю» (кроме сертификатов)."
case "user":
return "Удалит identity, персональный mode-config и клиентский сертификат на роутере."
case "peer":
return "Удалит peer на роутере. Если на него ссылаются identity — потребуется принудительное удаление."
case "cert":
return "Удалит сертификат с роутера. Если он используется peer или identity — потребуется принудительное удаление."
}
})()
return (
<>
<ServerRailLayout
items={railItems}
selectedId={effectiveServerId}
onSelect={setSelectedServerId}
showAll
allCount={displayClients.length}
loading={isLive && loading && displayServers.length === 0}
header={
<PageHeader
crumbs={[{ label: "Управление" }, { label: "IPsec / IKEv2" }]}
actions={
<>
<ServerRailMobileButton />
{isLive && (
<Button
size="sm"
variant="outline"
disabled={loading}
onClick={() => void loadLive()}
>
<RefreshCwIcon className={`size-4 ${loading ? "animate-spin" : ""}`} />
Обновить
</Button>
)}
{isLive && (
<Button
size="sm"
variant="outline"
disabled={loading || !historyServerId}
title={!historyServerId ? "Выберите сервер, чтобы смотреть историю" : "История версий и откат"}
onClick={() => {
setHistoryOpen(true)
void loadRevisions()
}}
>
<HistoryIcon className="size-4" />
История
</Button>
)}
{isLive ? (
<Button size="sm" onClick={() => setUserOpen(true)}>
<PlusIcon className="size-4" />
Новый клиент
</Button>
) : null}
</>
}
/>
}
>
<div className="flex flex-col gap-4">
<KpiStatGrid
aria-label="Сводка IPsec"
items={[
{
id: "servers",
label: "Серверов IKEv2",
value: serversWithIpsec,
icon: <ShieldCheckIcon className="size-4" />,
iconClassName: "text-muted-foreground",
},
{
id: "clients",
label: "Клиентов",
value: scopedClients.length,
icon: <UsersIcon className="size-4" />,
iconClassName: "text-info",
},
{
id: "online",
label: "Онлайн",
value: clientsOnline,
icon: <ActivityIcon className="size-4" />,
iconClassName: "text-success",
},
{
id: "ca",
label: "CA",
value: caOk ? "ок" : "—",
icon: <KeyRoundIcon className="size-4" />,
iconClassName: "text-primary",
},
]}
/>
{failures.length > 0 ? (
<Alert variant="destructive">
<AlertCircleIcon />
<AlertTitle>Не удалось опросить часть роутеров</AlertTitle>
<AlertDescription>
{failures.map((f) => f.serverName ?? f.serverId).join(", ")}
</AlertDescription>
</Alert>
) : null}
{!isLive ? (
<Alert variant="info">
<InfoIcon />
<AlertTitle>Mock-режим</AlertTitle>
<AlertDescription>
Переключитесь в live в настройках, чтобы управлять IKEv2 на MikroTik.
</AlertDescription>
</Alert>
) : null}
<Tabs
value={workspaceTab}
onValueChange={(v) => setWorkspaceTab(v as IpsecWorkspaceTab)}
className="gap-3"
>
<TabsList variant="line">
<TabsTrigger value="clients" className="gap-1.5">
<UsersIcon className="size-3.5" />
Клиенты
</TabsTrigger>
<TabsTrigger value="server" className="gap-1.5">
<ShieldCheckIcon className="size-3.5" />
Сервер
</TabsTrigger>
<TabsTrigger value="cli" className="gap-1.5">
<CodeXmlIcon className="size-3.5" />
CLI
</TabsTrigger>
</TabsList>
<TabsContent value="clients" className="mt-0 outline-none">
<DataPageCard>
<DataPageToolbar
search={search}
onSearchChange={setSearch}
searchPlaceholder="Поиск по имени / CN / IP…"
actions={
isLive ? (
<Button size="sm" onClick={() => { setEditing(null); setUserOpen(true) }}>
<PlusIcon className="size-4" />
Клиент
</Button>
) : undefined
}
/>
<IpsecUsersGrid
clients={filteredClients}
compactServer={compactServer}
onDownloadCert={isLive ? (row) => void openCert(row) : undefined}
onEdit={isLive ? (row) => { setEditing(row); setUserOpen(true) } : undefined}
onDelete={isLive ? (row) => setPendingDelete({ kind: "user", client: row }) : undefined}
/>
</DataPageCard>
</TabsContent>
<TabsContent value="server" className="mt-0 outline-none">
<IpsecServerGrid
servers={scopedSummaries}
onInit={isLive ? (s) => { setSelectedServerId(s.serverId); setInitOpen(true) } : undefined}
onRemove={isLive ? (s) => setPendingDelete({ kind: "server", summary: s }) : undefined}
onEditPeer={isLive ? openEditPeer : undefined}
onDeletePeer={isLive ? (serverId, peer) => setPendingDelete({ kind: "peer", serverId, peer }) : undefined}
onDeleteCert={isLive ? (serverId, cert) => setPendingDelete({ kind: "cert", serverId, cert }) : undefined}
onExportCert={isLive ? (serverId, cert) => void openCertByName(serverId, cert) : undefined}
/>
</TabsContent>
<TabsContent value="cli" className="mt-0 outline-none">
<OpsPanel title="RouterOS 7 · /ip ipsec — быстрые команды" contentClassName="px-5 py-4">
<div className="grid grid-cols-1 gap-4 font-mono text-xs sm:grid-cols-3">
{[
{
title: "CA и серверный серт.",
lines: [
"/certificate add name=ipsec-ca \\",
" common-name=\"MM IPsec CA\" \\",
" key-size=4096 \\",
" key-usage=key-cert-sign,crl-sign",
"/certificate sign ipsec-ca",
"/certificate add name=ipsec-server \\",
" common-name=vpn.example.com \\",
" subject-alt-name=DNS:vpn.example.com",
"/certificate sign ipsec-server ca=ipsec-ca",
],
},
{
title: "Peer + identity",
lines: [
"/ip ipsec peer add \\",
" name=ipsec-vpn address=0.0.0.0/0 \\",
" exchange-mode=ike2 passive=yes \\",
" certificate=ipsec-server send-cert=always",
"/ip ipsec identity add \\",
" peer=ipsec-vpn auth-method=rsa-key \\",
" certificate=ipsec-server \\",
" match-by=certificate \\",
" generate-policy=port-strict \\",
" mode-config=ipsec-vpn",
],
},
{
title: "Клиент: серт и .p12",
lines: [
"/certificate add name=ipsec-user-alice \\",
" common-name=alice key-size=2048 \\",
" key-usage=digital-signature,\\",
" key-encipherment,tls-client",
"/certificate sign ipsec-user-alice \\",
" ca=ipsec-ca",
"/certificate export-certificate \\",
" ipsec-user-alice type=pkcs12 \\",
" export-passphrase=*****",
"",
"# Онлайн-клиенты:",
"/ip ipsec active-peers print",
],
},
].map((b) => (
<div key={b.title}>
<p className="mb-1.5 font-sans text-[11px] font-semibold uppercase tracking-wide text-foreground/80">
{b.title}
</p>
<pre className="overflow-x-auto rounded-md bg-muted p-2.5 text-[11px] leading-relaxed text-muted-foreground">
{b.lines.join("\n")}
</pre>
</div>
))}
</div>
</OpsPanel>
</TabsContent>
</Tabs>
</div>
</ServerRailLayout>
<IpsecUserSheet
open={userOpen}
onOpenChange={(v) => { setUserOpen(v); if (!v) setEditing(null) }}
servers={serverOptions}
peers={peerOptions}
busy={busy}
defaultServerId={sheetServerId}
editing={editing}
freeIpHint={freeIpHint}
onSubmit={editing ? handleEditUser : handleCreateUser}
/>
<IpsecPeerSheet
open={peerOpen}
onOpenChange={(v) => { setPeerOpen(v); if (!v) setEditingPeer(null) }}
busy={busy}
editing={editingPeer?.peer ?? null}
certificates={certNames}
onSubmit={handleEditPeer}
/>
<IpsecInitSheet
open={initOpen}
onOpenChange={setInitOpen}
servers={serverOptions}
busy={busy}
defaultServerId={sheetServerId}
onSubmit={handleInit}
/>
<IpsecCertSheet
open={certOpen}
onOpenChange={(v) => { setCertOpen(v); if (!v) setCertByName(null) }}
bundle={certBundle}
busy={certBusy}
onReexport={reexportCert}
/>
<AlertDialog open={pendingDelete != null} onOpenChange={(v) => { if (!v) setPendingDelete(null) }}>
<AlertDialogContent size="default">
<AlertDialogHeader>
<AlertDialogMedia className="bg-destructive/10 text-destructive">
<Trash2Icon />
</AlertDialogMedia>
<AlertDialogTitle>{deleteDialogTitle}</AlertDialogTitle>
<AlertDialogDescription>{deleteDialogDescription}</AlertDialogDescription>
</AlertDialogHeader>
<AlertDialogFooter>
<AlertDialogCancel disabled={busy}>Отмена</AlertDialogCancel>
<AlertDialogAction
variant="destructive"
disabled={busy}
onClick={(e) => { e.preventDefault(); void confirmDelete() }}
>
<Trash2Icon className="size-4" />
Удалить
</AlertDialogAction>
</AlertDialogFooter>
</AlertDialogContent>
</AlertDialog>
<ConfigHistorySheet
open={historyOpen}
onOpenChange={setHistoryOpen}
revisions={revisions}
loading={historyLoading}
restoring={historyRestoring}
onRestore={(id) => void restoreRevision(id)}
title="История конфигураций IPsec"
itemLabel="конфигурация"
/>
</>
)
}
+530 -51
View File
@@ -27,7 +27,9 @@ import {
findServerByGreRemote,
greSourceWanIndexOnMap,
greTunnelProbe,
placeCountryServiceNodes,
placeServiceNodes,
SERVICE_COL_W,
type GreMapEdge,
type WanJhEdge,
} from "@/lib/network-map-layout"
@@ -55,8 +57,9 @@ import {
matchNetflowForWan,
type MatchedNetflowHop,
} from "@/lib/map-netflow-hops"
import type { FlowMapHop, FlowMapHopsDto, FlowMapService, FlowMapServiceEdge, FlowMapServicePath } from "@mmapp/contracts/traffic-flow"
import type { FlowMapCountryServiceGroup, FlowMapHop, FlowMapHopsDto, FlowMapService, FlowMapServiceEdge, FlowMapServicePath } from "@mmapp/contracts/traffic-flow"
import { ServiceBrandIcon } from "@/components/network-map/service-brand-icon"
import { CountryFlagSvg } from "@/components/network-map/country-flag-svg"
import { Button } from "@/components/ui/button"
import { StatusBadge } from "@/components/status-badge"
import { StatusDot } from "@/components/status-dot"
@@ -67,8 +70,9 @@ import {
CableIcon, CopyIcon, ActivityIcon, ExternalLinkIcon,
} from "lucide-react"
import { cn } from "@/lib/utils"
import { formatServicePathLabel, formatServicePathTitle } from "@/lib/format-service-path-label"
import Link from "next/link"
import { Flag } from "@/components/flag"
import { Flag, countryName } from "@/components/flag"
// ─── Resource metrics (для мини-блока справа; числа детерминированы по id узла) ─
@@ -284,6 +288,12 @@ const MOCK_MAP_SERVICES: FlowMapService[] = [
{ id: "svc:aws", label: "AWS", category: "CDN", bytes: 9_000_000, bps: 3_600_000, share: 0.09 },
]
const MOCK_MAP_COUNTRIES: FlowMapService[] = [
{ id: "cc:us", label: "US", category: "Страна", bytes: 22_000_000, bps: 8_800_000, share: 0.38 },
{ id: "cc:nl", label: "NL", category: "Страна", bytes: 14_000_000, bps: 5_600_000, share: 0.31 },
{ id: "cc:de", label: "DE", category: "Страна", bytes: 9_000_000, bps: 3_600_000, share: 0.21 },
]
const MOCK_MAP_SERVICE_EDGES: FlowMapServiceEdge[] = [
{ fromId: "srv2", toId: "svc:google", bytes: 14_000_000, bps: 5_600_000, bpsFwd: 4_200_000, bpsRev: 1_400_000, clientName: "Alice", clients: [{ id: "u1", name: "Alice" }] },
{ fromId: "srv3", toId: "svc:google", bytes: 8_000_000, bps: 3_200_000, bpsFwd: 2_400_000, bpsRev: 800_000, clientName: "Bob", clients: [{ id: "u2", name: "Bob" }] },
@@ -292,6 +302,14 @@ const MOCK_MAP_SERVICE_EDGES: FlowMapServiceEdge[] = [
{ fromId: "srv3", toId: "svc:aws", bytes: 9_000_000, bps: 3_600_000, bpsFwd: 2_700_000, bpsRev: 900_000, clientName: "Bob", clients: [{ id: "u2", name: "Bob" }] },
]
const MOCK_MAP_COUNTRY_EDGES: FlowMapServiceEdge[] = [
{ fromId: "srv2", toId: "cc:us", bytes: 14_000_000, bps: 5_600_000, bpsFwd: 4_200_000, bpsRev: 1_400_000, clientName: "Alice", clients: [{ id: "u1", name: "Alice" }] },
{ fromId: "srv3", toId: "cc:us", bytes: 8_000_000, bps: 3_200_000, bpsFwd: 2_400_000, bpsRev: 800_000, clientName: "Bob", clients: [{ id: "u2", name: "Bob" }] },
{ fromId: "srv2", toId: "cc:nl", bytes: 9_000_000, bps: 3_600_000, bpsFwd: 2_800_000, bpsRev: 800_000, clientName: "Alice", clients: [{ id: "u1", name: "Alice" }] },
{ fromId: "srv3", toId: "cc:nl", bytes: 5_000_000, bps: 2_000_000, bpsFwd: 1_500_000, bpsRev: 500_000, clientName: "Bob", clients: [{ id: "u2", name: "Bob" }] },
{ fromId: "srv3", toId: "cc:de", bytes: 9_000_000, bps: 3_600_000, bpsFwd: 2_700_000, bpsRev: 900_000, clientName: "Bob", clients: [{ id: "u2", name: "Bob" }] },
]
const MOCK_MAP_SERVICE_PATHS: FlowMapServicePath[] = [
{ clientId: "u1", clientName: "Alice", viaId: "srv1", viaName: "mt-msk-core-01", enId: "srv2", enName: "mt-spb-edge-01", serviceId: "svc:google", bytes: 14_000_000, bps: 5_600_000 },
{ clientId: "u2", clientName: "Bob", viaId: "srv1", viaName: "mt-msk-core-01", enId: "srv3", enName: "mt-fra-edge-01", serviceId: "svc:google", bytes: 8_000_000, bps: 3_200_000 },
@@ -300,6 +318,56 @@ const MOCK_MAP_SERVICE_PATHS: FlowMapServicePath[] = [
{ clientId: "u2", clientName: "Bob", viaId: "srv1", viaName: "mt-msk-core-01", enId: "srv3", enName: "mt-fra-edge-01", serviceId: "svc:aws", bytes: 9_000_000, bps: 3_600_000 },
]
const MOCK_MAP_COUNTRY_PATHS: FlowMapServicePath[] = [
{ clientId: "u1", clientName: "Alice", viaId: "srv1", viaName: "mt-msk-core-01", enId: "srv2", enName: "mt-spb-edge-01", serviceId: "cc:us", bytes: 14_000_000, bps: 5_600_000 },
{ clientId: "u2", clientName: "Bob", viaId: "srv1", viaName: "mt-msk-core-01", enId: "srv3", enName: "mt-fra-edge-01", serviceId: "cc:us", bytes: 8_000_000, bps: 3_200_000 },
{ clientId: "u1", clientName: "Alice", viaId: "srv1", viaName: "mt-msk-core-01", enId: "srv2", enName: "mt-spb-edge-01", serviceId: "cc:nl", bytes: 9_000_000, bps: 3_600_000 },
{ clientId: "u2", clientName: "Bob", viaId: "srv1", viaName: "mt-msk-core-01", enId: "srv3", enName: "mt-fra-edge-01", serviceId: "cc:nl", bytes: 5_000_000, bps: 2_000_000 },
{ clientId: "u2", clientName: "Bob", viaId: "srv1", viaName: "mt-msk-core-01", enId: "srv3", enName: "mt-fra-edge-01", serviceId: "cc:de", bytes: 9_000_000, bps: 3_600_000 },
]
/** Доли — от байтов страны cc:us (22M из моков выше). */
const MOCK_COUNTRY_SERVICE_GROUPS: FlowMapCountryServiceGroup[] = [
{
countryId: "cc:us",
services: [
{ id: "cc:us|svc:google", label: "Google", category: "Веб", bytes: 12_000_000, bps: 4_800_000, share: 12 / 22 },
{ id: "cc:us|svc:cloudflare", label: "Cloudflare", category: "CDN", bytes: 7_000_000, bps: 2_800_000, share: 7 / 22 },
{ id: "cc:us|svc:aws", label: "AWS", category: "CDN", bytes: 3_000_000, bps: 1_200_000, share: 3 / 22 },
],
edges: [
{ fromId: "cc:us", toId: "cc:us|svc:google", bytes: 12_000_000, bps: 4_800_000, bpsFwd: 9_000_000, bpsRev: 3_000_000, clientName: "Alice", clients: [{ id: "u1", name: "Alice" }] },
{ fromId: "cc:us", toId: "cc:us|svc:cloudflare", bytes: 7_000_000, bps: 2_800_000, bpsFwd: 5_250_000, bpsRev: 1_750_000, clientName: "Alice", clients: [{ id: "u1", name: "Alice" }] },
{ fromId: "cc:us", toId: "cc:us|svc:aws", bytes: 3_000_000, bps: 1_200_000, bpsFwd: 2_250_000, bpsRev: 750_000, clientName: "Bob", clients: [{ id: "u2", name: "Bob" }] },
],
paths: [
{ clientId: "u1", clientName: "Alice", viaId: "srv1", viaName: "mt-msk-core-01", enId: "srv2", enName: "mt-spb-edge-01", serviceId: "cc:us|svc:google", bytes: 8_000_000, bps: 3_200_000 },
{ clientId: "u2", clientName: "Bob", viaId: "srv1", viaName: "mt-msk-core-01", enId: "srv3", enName: "mt-fra-edge-01", serviceId: "cc:us|svc:google", bytes: 4_000_000, bps: 1_600_000 },
{ clientId: "u1", clientName: "Alice", viaId: "srv1", viaName: "mt-msk-core-01", enId: "srv2", enName: "mt-spb-edge-01", serviceId: "cc:us|svc:cloudflare", bytes: 7_000_000, bps: 2_800_000 },
{ clientId: "u2", clientName: "Bob", viaId: "srv1", viaName: "mt-msk-core-01", enId: "srv3", enName: "mt-fra-edge-01", serviceId: "cc:us|svc:aws", bytes: 3_000_000, bps: 1_200_000 },
],
},
]
const DEST_MODE_KEY = "mm-network-map-dest-mode"
type DestMode = "services" | "countries"
function readDestMode(): DestMode {
if (typeof window === "undefined") return "services"
try {
return sessionStorage.getItem(DEST_MODE_KEY) === "countries" ? "countries" : "services"
} catch {
return "services"
}
}
function destDisplayLabel(node: FlowMapService | undefined, mode: DestMode, fallback = ""): string {
if (!node) return fallback
if (mode !== "countries") return node.label
if (node.id === "cc:other" || node.label === "Прочее") return "Прочее"
return countryName(node.label)
}
function servicePathKey(p: Pick<FlowMapServicePath, "clientId" | "viaId" | "enId" | "serviceId">): string {
return `${p.clientId}|${p.viaId}|${p.enId}|${p.serviceId}`
}
@@ -737,6 +805,10 @@ function ServiceNode({
isSel,
isVis,
isDragged,
destMode,
iso,
dim,
shareLabel,
onClick,
onMouseDown,
}: {
@@ -747,20 +819,27 @@ function ServiceNode({
isSel: boolean
isVis: boolean
isDragged: boolean
destMode: DestMode
iso?: string
/** Приглушение узла при раскрытии другой страны (остаётся на холсте). */
dim?: boolean
/** Подпись доли в tooltip: у вложенных сервисов — доля страны, не окна. */
shareLabel?: string
onClick: () => void
onMouseDown: (e: React.MouseEvent) => void
}) {
const bw = MAP_SERVICE_NODE_W
const bh = MAP_SERVICE_NODE_H
const flagIso = destMode === "countries" && iso && iso !== "Прочее" ? iso : ""
return (
<g
transform={`translate(${x},${y})`}
style={{ cursor: isDragged ? "grabbing" : "grab", transition: isDragged ? "none" : "opacity 0.25s" }}
opacity={isVis ? 1 : 0.08}
opacity={isVis ? (dim ? 0.35 : 1) : 0.08}
onMouseDown={(e) => { e.stopPropagation(); onMouseDown(e) }}
onClick={(e) => { e.stopPropagation(); onClick() }}
>
<title>{`${label} · ${serviceSharePct(share)} трафика окна`}</title>
<title>{`${label} · ${serviceSharePct(share)} ${shareLabel ?? "payload окна"}`}</title>
{isSel && (
<rect
x={-bw / 2 - 6}
@@ -785,7 +864,9 @@ function ServiceNode({
strokeWidth={isSel ? 2.2 : 1.4}
/>
<g transform="translate(-11,-24)" pointerEvents="none">
<ServiceBrandIcon label={label} size={22} />
{flagIso
? <CountryFlagSvg iso={flagIso} size={22} />
: <ServiceBrandIcon label={destMode === "countries" ? "Прочее" : label} size={22} />}
</g>
<text textAnchor="middle" y="14" fontSize="8.5" fontWeight="700" fill="#e0f2fe" fontFamily="ui-monospace,monospace">
{label}
@@ -803,6 +884,7 @@ function ServicePathList({
services,
highlight,
viaMode,
destMode,
onToggle,
}: {
paths: FlowMapServicePath[]
@@ -810,6 +892,7 @@ function ServicePathList({
services: FlowMapService[]
highlight: { viaId: string; enId: string; serviceId: string } | null
viaMode: "via" | "service"
destMode: DestMode
onToggle: (p: FlowMapServicePath) => void
}) {
if (paths.length === 0) {
@@ -820,9 +903,16 @@ function ServicePathList({
{paths.map((p) => {
const rowKey = servicePathKey(p)
const via = servers.find((s) => s.id === p.viaId)
const viaLabel = via?.site || p.viaName
const en = servers.find((s) => s.id === p.enId)
const svc = services.find((s) => s.id === p.serviceId)
const mid = viaMode === "via" ? viaLabel : (svc?.label ?? p.serviceId)
const destLabel = destDisplayLabel(svc, destMode, p.serviceId)
const label = formatServicePathLabel(p, viaMode, {
viaName: via?.name,
viaSite: via?.site,
enName: en?.name,
serviceLabel: destLabel,
})
const title = formatServicePathTitle(label, destLabel)
const active = Boolean(
highlight
&& highlight.viaId === p.viaId
@@ -833,13 +923,14 @@ function ServicePathList({
<button
key={rowKey}
type="button"
title={title}
onClick={() => onToggle(p)}
className={cn(
"flex items-center justify-between gap-2 rounded-md px-2 py-1.5 text-left text-xs transition-colors",
active ? "bg-cyan-500/15 ring-1 ring-cyan-500/40" : "hover:bg-muted/50",
)}
>
<span className="font-mono truncate min-w-0">{p.clientName} · {mid}</span>
<span className="font-mono truncate min-w-0">{label}</span>
<span className="font-mono text-emerald-400 tabular-nums shrink-0">
{formatNetflowRate({ bytes: p.bytes, bps: p.bps, bpsFwd: p.bps, bpsRev: 0 })}
</span>
@@ -1091,7 +1182,16 @@ export default function NetworkMapPage() {
const [mapServices, setMapServices] = useState<FlowMapService[]>([])
const [mapServiceEdges, setMapServiceEdges] = useState<FlowMapServiceEdge[]>([])
const [mapServicePaths, setMapServicePaths] = useState<FlowMapServicePath[]>([])
const [mapCountries, setMapCountries] = useState<FlowMapService[]>([])
const [mapCountryEdges, setMapCountryEdges] = useState<FlowMapServiceEdge[]>([])
const [mapCountryPaths, setMapCountryPaths] = useState<FlowMapServicePath[]>([])
const [mapCountryServiceGroups, setMapCountryServiceGroups] = useState<FlowMapCountryServiceGroup[]>([])
const [mapSharePct, setMapSharePct] = useState(5)
const [mapNamedBytes, setMapNamedBytes] = useState(0)
const [mapTotalBytes, setMapTotalBytes] = useState(0)
const [mapWindowSec, setMapWindowSec] = useState(300)
const [mapAsnLoaded, setMapAsnLoaded] = useState(true)
const [mapCountryLoaded, setMapCountryLoaded] = useState(true)
/** FQDN из GRE outer → IPv4 (ответ POST /api/network/resolve-hosts), для матчинга с WAN. */
const [greResolvedIpv4ByHost, setGreResolvedIpv4ByHost] = useState<Record<string, string>>({})
const [dataError, setDataError] = useState<string | null>(null)
@@ -1187,7 +1287,14 @@ export default function NetworkMapPage() {
setMapServices(MOCK_MAP_SERVICES)
setMapServiceEdges(MOCK_MAP_SERVICE_EDGES)
setMapServicePaths(MOCK_MAP_SERVICE_PATHS)
setMapCountries(MOCK_MAP_COUNTRIES)
setMapCountryEdges(MOCK_MAP_COUNTRY_EDGES)
setMapCountryPaths(MOCK_MAP_COUNTRY_PATHS)
setMapCountryServiceGroups(MOCK_COUNTRY_SERVICE_GROUPS)
setMapSharePct(5)
setMapNamedBytes(0)
setMapTotalBytes(0)
setMapCountryLoaded(true)
setDataError(null)
})
return
@@ -1216,6 +1323,41 @@ export default function NetworkMapPage() {
// ── Interaction ─────────────────────────────────────────────────────────────
const [selected, setSelected] = useState<Server | null>(null)
const [selectedService, setSelectedService] = useState<FlowMapService | null>(null)
/** Раскрытая страна (режим «Страны»): справа столбец её сервисов. Не персистится. */
const [expandedCountryId, setExpandedCountryId] = useState<string | null>(null)
const [destMode, setDestModeState] = useState<DestMode>("services")
useEffect(() => {
queueMicrotask(() => setDestModeState(readDestMode()))
}, [])
function setDestMode(mode: DestMode) {
setDestModeState(mode)
setSelectedService(null)
setExpandedCountryId(null)
setHighlightedPath(null)
try { sessionStorage.setItem(DEST_MODE_KEY, mode) } catch { /* private mode */ }
}
const expandedCountryGroup = useMemo(
() => destMode === "countries" && expandedCountryId
? mapCountryServiceGroups.find((g) => g.countryId === expandedCountryId) ?? null
: null,
[destMode, expandedCountryId, mapCountryServiceGroups],
)
const nestedServices = useMemo(() => expandedCountryGroup?.services ?? [], [expandedCountryGroup])
const liveSelectedService = selectedService
? (
(destMode === "countries" ? mapCountries : mapServices)
.find((s) => s.id === selectedService.id)
?? nestedServices.find((s) => s.id === selectedService.id)
?? selectedService
)
: null
const selectedNestedService = liveSelectedService
&& nestedServices.some((s) => s.id === liveSelectedService.id)
? liveSelectedService
: null
const expandedCountry = expandedCountryId
? mapCountries.find((c) => c.id === expandedCountryId) ?? null
: null
const [highlightedPath, setHighlightedPath] = useState<{ viaId: string; enId: string; serviceId: string } | null>(null)
const [selWanIdx, setSelWanIdx] = useState<number | null>(null)
const [hoveredId, setHoveredId] = useState<string | null>(null)
@@ -1266,7 +1408,12 @@ export default function NetworkMapPage() {
setMapServices(MOCK_MAP_SERVICES)
setMapServiceEdges(MOCK_MAP_SERVICE_EDGES)
setMapServicePaths(MOCK_MAP_SERVICE_PATHS)
setMapCountries(MOCK_MAP_COUNTRIES)
setMapCountryEdges(MOCK_MAP_COUNTRY_EDGES)
setMapCountryPaths(MOCK_MAP_COUNTRY_PATHS)
setMapCountryServiceGroups(MOCK_COUNTRY_SERVICE_GROUPS)
setMapSharePct(5)
setMapCountryLoaded(true)
})
return
}
@@ -1276,6 +1423,10 @@ export default function NetworkMapPage() {
setMapServices([])
setMapServiceEdges([])
setMapServicePaths([])
setMapCountries([])
setMapCountryEdges([])
setMapCountryPaths([])
setMapCountryServiceGroups([])
})
return
}
@@ -1291,7 +1442,16 @@ export default function NetworkMapPage() {
setMapServices(res.services ?? [])
setMapServiceEdges(res.serviceEdges ?? [])
setMapServicePaths(res.servicePaths ?? [])
setMapCountries(res.countries ?? [])
setMapCountryEdges(res.countryEdges ?? [])
setMapCountryPaths(res.countryPaths ?? [])
setMapCountryServiceGroups(res.countryServiceGroups ?? [])
if (res.mapServiceMinSharePct != null) setMapSharePct(res.mapServiceMinSharePct)
setMapNamedBytes(res.namedBytes ?? 0)
setMapTotalBytes(res.totalBytes ?? 0)
if (res.asnLoaded != null) setMapAsnLoaded(res.asnLoaded)
if (res.countryLoaded != null) setMapCountryLoaded(res.countryLoaded)
if (res.windowSec) setMapWindowSec(res.windowSec)
})
.catch((err: unknown) => {
if (cancelled) return
@@ -1494,9 +1654,13 @@ export default function NetworkMapPage() {
return m
}, [homeRouters, wanJhEdges, mapHops, showNetflow])
const visibleMapServices = showServices ? mapServices : []
const destNodes = destMode === "countries" ? mapCountries : mapServices
const destEdges = destMode === "countries" ? mapCountryEdges : mapServiceEdges
const destPaths = destMode === "countries" ? mapCountryPaths : mapServicePaths
const visibleMapServices = showServices ? destNodes : []
const visibleServiceEdges = showServices
? drawableServiceEdges(visibleMapServices, mapServiceEdges, mapServers, greEdges, nodePosById)
? drawableServiceEdges(visibleMapServices, destEdges, mapServers, greEdges, nodePosById)
: []
const nodes = mapServers
@@ -1518,8 +1682,24 @@ export default function NetworkMapPage() {
.map((s) => nodePosById[s.id])
.filter((p): p is { x: number; y: number } => Boolean(p)),
)
// Раскрытая страна: колонка стран уходит влево, правый x занимает столбец её сервисов.
const countryColShift = expandedCountryGroup ? SERVICE_COL_W : 0
const autoDestPos = countryColShift
? Object.fromEntries(
Object.entries(autoServicePos).map(([id, p]) => [id, { x: p.x - countryColShift, y: p.y }]),
)
: autoServicePos
const servicePosById = Object.fromEntries(
visibleMapServices.map((s) => [s.id, servicePositions[s.id] ?? autoServicePos[s.id]!]),
visibleMapServices.map((s) => [s.id, servicePositions[s.id] ?? autoDestPos[s.id]!]),
)
const autoNestedPos = placeCountryServiceNodes(
nestedServices.map((s) => s.id),
expandedCountryId ? servicePosById[expandedCountryId] ?? autoDestPos[expandedCountryId] : undefined,
)
const nestedPosById = Object.fromEntries(
nestedServices
.map((s) => [s.id, servicePositions[s.id] ?? autoNestedPos[s.id]] as const)
.filter((entry): entry is readonly [string, { x: number; y: number }] => Boolean(entry[1])),
)
// ── Refs ─────────────────────────────────────────────────────────────────────
@@ -1604,6 +1784,7 @@ export default function NetworkMapPage() {
setSelectedGreEdge(null)
setSelectedService(null)
setHighlightedPath(null)
setExpandedCountryId(null)
}
if (e.key === "=" || e.key === "+") applyZoomCenter(1.25)
if (e.key === "-") applyZoomCenter(1 / 1.25)
@@ -1701,6 +1882,7 @@ export default function NetworkMapPage() {
setSelectedGreEdge(null)
setSelectedService(null)
setHighlightedPath(null)
setExpandedCountryId(null)
}
}
@@ -1735,6 +1917,34 @@ export default function NetworkMapPage() {
}
// ── Side panel ────────────────────────────────────────────────────────────
useEffect(() => {
if (
expandedCountryId
&& (!mapCountries.some((s) => s.id === expandedCountryId)
|| !mapCountryServiceGroups.some((g) => g.countryId === expandedCountryId))
) {
queueMicrotask(() => setExpandedCountryId(null))
}
}, [mapCountries, mapCountryServiceGroups, expandedCountryId])
useEffect(() => {
if (!selectedService) return
const stillVisible =
destNodes.some((s) => s.id === selectedService.id)
|| nestedServices.some((s) => s.id === selectedService.id)
if (!stillVisible) {
queueMicrotask(() => {
setSelectedService(null)
setHighlightedPath(null)
})
}
}, [destMode, destNodes, nestedServices, selectedService])
useEffect(() => {
if (!showServices) queueMicrotask(() => setExpandedCountryId(null))
}, [showServices])
// Сдвиг колонки стран меняет систему координат: сбрасываем drag-овчины сервисов.
useEffect(() => {
queueMicrotask(() => setServicePositions({}))
}, [expandedCountryId])
function selectServer(s: Server) {
setSelectedGreEdge(null)
setSelectedService(null)
@@ -1749,6 +1959,10 @@ export default function NetworkMapPage() {
setSelWanIdx(null)
setHighlightedPath(null)
setSelectedService((prev: FlowMapService | null) => prev?.id === svc.id ? null : svc)
// Клик по стране в режиме «Страны» раскрывает столбец её сервисов; повторный — сворачивает.
if (destMode === "countries" && svc.id.startsWith("cc:") && !svc.id.includes("|")) {
setExpandedCountryId((prev) => (prev === svc.id ? null : svc.id))
}
}
function selectWan(s: Server, wanIdx: number) {
setSelectedGreEdge(null)
@@ -1898,6 +2112,27 @@ export default function NetworkMapPage() {
)}
</div>
{/* Dest overlay: services vs countries */}
<div className="flex items-center gap-0.5 rounded-md border border-border bg-muted/40 p-0.5">
{([
{ value: "services" as const, label: "Сервисы" },
{ value: "countries" as const, label: "Страны" },
]).map((b) => (
<button
key={b.value}
onClick={() => setDestMode(b.value)}
className={cn(
"px-2.5 py-1 text-xs rounded transition-colors whitespace-nowrap",
destMode === b.value
? "bg-background text-foreground shadow-sm"
: "text-muted-foreground hover:text-foreground",
)}
>
{b.label}
</button>
))}
</div>
{/* Layers dropdown */}
<div className="relative">
<button
@@ -1916,7 +2151,7 @@ export default function NetworkMapPage() {
{([
{ key: "showPingBadges", label: "Ping-значки", val: showPingBadges, set: setShowPingBadges, hint: "P" },
{ key: "showNetflow", label: "NetFlow", val: showNetflow, set: setShowNetflow, hint: "" },
{ key: "showServices", label: "Сервисы", val: showServices, set: setShowServices, hint: "" },
{ key: "showServices", label: "Назначения", val: showServices, set: setShowServices, hint: "" },
{ key: "showAnimDots", label: "Анимация трафика", val: showAnimDots, set: setShowAnimDots, hint: "" },
{ key: "showMinimap", label: "Минимап", val: showMinimap, set: setShowMinimap, hint: "M" },
{ key: "showHints", label: "Горячие клавиши", val: showHints, set: setShowHints, hint: "" },
@@ -1943,9 +2178,14 @@ export default function NetworkMapPage() {
))}
<p className="px-3 pt-1.5 pb-1 text-[10px] text-muted-foreground leading-snug">
{mapSharePct > 0
? `Порог доли сервиса${mapSharePct}% · Настройки → NetFlow`
: "Порог доли выключен (все бренды, макс. 20) · Настройки → NetFlow"}
? `Порог доли ≥ ${mapSharePct}% · Настройки → NetFlow`
: "Порог доли выключен (все узлы, макс. 20) · Настройки → NetFlow"}
</p>
{destMode === "countries" && !mapCountryLoaded && (
<p className="px-3 pb-1 text-[10px] text-amber-500 leading-snug">
GeoIP Country не загружен, страны из RIPE-кэша
</p>
)}
{(Object.keys(nodePositions).length > 0 || Object.keys(satPositions).length > 0 || Object.keys(servicePositions).length > 0) && (
<div className="border-t border-border/50 mt-1 pt-1">
<button
@@ -2262,6 +2502,32 @@ export default function NetworkMapPage() {
{visibleMapServices.map((svc) => {
const pos = servicePosById[svc.id]
if (!pos) return null
return (
<ServiceNode
key={svc.id}
label={destDisplayLabel(svc, destMode)}
share={svc.share}
x={pos.x}
y={pos.y}
isSel={selectedService?.id === svc.id}
isVis
dim={Boolean(expandedCountryGroup) && svc.id !== expandedCountryId}
isDragged={draggedSvcId === svc.id}
destMode={destMode}
iso={svc.label}
onMouseDown={(e) => onServiceMouseDown(e, svc.id, pos.x, pos.y)}
onClick={() => {
if (suppressClickRef.current) { suppressClickRef.current = false; return }
selectService(svc)
}}
/>
)
})}
{/* ── Сервисы раскрытой страны (второй столбец справа) ── */}
{expandedCountryGroup && nestedServices.map((svc) => {
const pos = nestedPosById[svc.id]
if (!pos) return null
return (
<ServiceNode
key={svc.id}
@@ -2272,6 +2538,8 @@ export default function NetworkMapPage() {
isSel={selectedService?.id === svc.id}
isVis
isDragged={draggedSvcId === svc.id}
destMode="services"
shareLabel="трафика страны"
onMouseDown={(e) => onServiceMouseDown(e, svc.id, pos.x, pos.y)}
onClick={() => {
if (suppressClickRef.current) { suppressClickRef.current = false; return }
@@ -2311,15 +2579,16 @@ export default function NetworkMapPage() {
&& highlightedPath.serviceId === edge.toId,
)
const pathDim = Boolean(highlightedPath) && !pathHit
const hl = pathHit || (!highlightedPath && (selectedService?.id === edge.toId || selected?.id === edge.fromId))
const hl = pathHit || (!highlightedPath && (selectedService?.id === edge.toId || selected?.id === edge.fromId || expandedCountryId === edge.toId))
const countryDim = !hl && Boolean(expandedCountryGroup) && edge.toId !== expandedCountryId
const svc = visibleMapServices.find((s) => s.id === edge.toId)
const enName = mapServers.find((s) => s.id === edge.fromId)?.name ?? edge.fromId
const clientLabel = (edge.clients?.map((c) => c.name).filter(Boolean).join(", ") || edge.clientName || "—")
const pathTitle = `${clientLabel}${enName}${svc?.label ?? edge.toId}`
const pathTitle = `${clientLabel}${enName}${destDisplayLabel(svc, destMode, edge.toId)}`
return (
<g
key={`${edge.fromId}|${edge.toId}`}
opacity={pathDim ? 0.12 : hl ? 1 : 0.72}
opacity={pathDim ? 0.12 : hl ? 1 : countryDim ? 0.35 : 0.72}
style={{ transition: "opacity 0.3s" }}
>
<title>{pathTitle}</title>
@@ -2365,6 +2634,81 @@ export default function NetworkMapPage() {
)
})}
{/* ── Раскрытая страна → её сервисы ── */}
{expandedCountryGroup?.edges.map((edge) => {
const from = servicePosById[edge.fromId]
const to = nestedPosById[edge.toId]
if (!from || !to) return null
const hop: MatchedNetflowHop = {
bytes: edge.bytes,
bps: edge.bps,
bpsFwd: edge.bpsFwd,
bpsRev: edge.bpsRev,
}
const clipped = clipSegmentCircleToRect(
from.x,
from.y,
MAP_SERVICE_NODE_W / 2,
to.x,
to.y,
MAP_SERVICE_NODE_W / 2,
MAP_SERVICE_NODE_H / 2,
)
const { mx, my } = edgeBadgePosition(clipped.x1, clipped.y1, clipped.x2, clipped.y2, 0.55, 16)
const hl = selectedService?.id === edge.toId
const svc = nestedServices.find((s) => s.id === edge.toId)
const country = mapCountries.find((s) => s.id === edge.fromId)
const clientLabel = (edge.clients?.map((c) => c.name).filter(Boolean).join(", ") || edge.clientName || "—")
const pathTitle = `${clientLabel}${destDisplayLabel(country, "countries", edge.fromId)}${edge.toId.split("|")[1] ?? edge.toId}`
return (
<g
key={`${edge.fromId}|${edge.toId}`}
opacity={hl ? 1 : 0.72}
style={{ transition: "opacity 0.3s" }}
>
<title>{pathTitle}</title>
<line
x1={clipped.x1} y1={clipped.y1} x2={clipped.x2} y2={clipped.y2}
stroke="#22d3ee"
strokeWidth={hopHasRate(hop) ? 2 : 1.3}
strokeDasharray="4 4"
opacity="0.9"
pointerEvents="none"
/>
<line
x1={clipped.x1} y1={clipped.y1} x2={clipped.x2} y2={clipped.y2}
stroke="#00000000"
strokeWidth={14}
strokeLinecap="round"
style={{ cursor: "pointer" }}
onPointerDown={(ev) => { ev.stopPropagation() }}
onClick={(ev) => {
ev.stopPropagation()
if (svc) selectService(svc)
}}
/>
{showAnimDots && hopHasRate(hop) && (
<circle r="3" fill="#67e8f9" opacity="0.85" pointerEvents="none">
<animateMotion dur="2.6s" repeatCount="indefinite"
path={`M ${clipped.x1} ${clipped.y1} L ${clipped.x2} ${clipped.y2}`} />
</circle>
)}
{hopHasRate(hop) && (
<NetflowRateBadge
mx={mx}
my={my}
hop={hop}
onOpen={(ev) => {
ev.stopPropagation()
const hit = nestedServices.find((s) => s.id === edge.toId)
if (hit) selectService(hit)
}}
/>
)}
</g>
)
})}
{/* ── Hover tooltip ── */}
{hoveredNode && !isDragging && (
<SvgTooltip n={hoveredNode} />
@@ -2403,7 +2747,9 @@ export default function NetworkMapPage() {
<g transform="translate(10, 164)">
<rect width="14" height="14" rx="4" fill="#08202c" stroke="#22d3ee" strokeWidth="1.2" />
<text x="22" y="11" fontSize="8.5" fill="#cbd5e1" fontFamily="system-ui">Сервис</text>
<text x="22" y="11" fontSize="8.5" fill="#cbd5e1" fontFamily="system-ui">
{destMode === "countries" ? "Страна" : "Сервис"}
</text>
</g>
<line x1="10" y1="186" x2="130" y2="186" stroke="rgba(255,255,255,0.07)" strokeWidth="1" />
@@ -2467,7 +2813,7 @@ export default function NetworkMapPage() {
satPos={effectiveSatPos}
wanJhEdges={visibleWanJhEdges}
homeRouters={homeRouters}
servicePos={servicePosById}
servicePos={{ ...servicePosById, ...nestedPosById }}
onClose={() => setShowMinimap(false)}
onPan={(x, y) => setPan({ x, y })}
/>
@@ -2702,16 +3048,87 @@ export default function NetworkMapPage() {
})()}
</div>
</>
) : selectedService ? (
) : selectedNestedService ? (
<>
<div className="flex items-start gap-2 px-4 py-3 border-b">
<div className="mt-0.5">
<ServiceBrandIcon label={selectedService.label} size={22} />
<ServiceBrandIcon label={selectedNestedService.label} size={22} />
</div>
<div className="flex-1 min-w-0">
<p className="font-mono font-semibold text-sm truncate">{selectedService.label}</p>
<p className="font-mono font-semibold text-sm truncate">
{selectedNestedService.label}
</p>
<p className="text-xs text-muted-foreground mt-0.5">
Конечный сервис · {selectedService.category}
{`Сервис в ${destDisplayLabel(expandedCountry ?? undefined, "countries")} · ${selectedNestedService.category}`}
</p>
</div>
<button
type="button"
onClick={() => setSelectedService(null)}
className="text-muted-foreground hover:text-foreground transition-colors"
>
<XIcon className="size-4" />
</button>
</div>
<div className="flex-1 overflow-y-auto px-4 py-4 flex flex-col gap-4">
<div className="flex flex-col gap-0">
<div className="flex items-center justify-between py-2 border-b border-border/50">
<span className="text-xs text-muted-foreground">Доля страны</span>
<span className="text-xs font-mono font-medium text-cyan-400">{serviceSharePct(selectedNestedService.share)}</span>
</div>
{mapTotalBytes > 0 && (
<div className="flex items-center justify-between py-2 border-b border-border/50">
<span className="text-xs text-muted-foreground">Доля окна</span>
<span className="text-xs font-mono font-medium text-cyan-400">
{serviceSharePct(selectedNestedService.bytes / mapTotalBytes)}
</span>
</div>
)}
<div className="flex items-center justify-between py-2 border-b border-border/50">
<span className="text-xs text-muted-foreground">Скорость</span>
<span className="text-xs font-mono font-medium">
{formatNetflowRate({
bytes: selectedNestedService.bytes,
bps: selectedNestedService.bps,
bpsFwd: selectedNestedService.bps,
bpsRev: 0,
})}
</span>
</div>
</div>
<div>
<p className="text-xs font-semibold text-muted-foreground uppercase tracking-wider mb-2">Выход</p>
<ServicePathList
paths={(expandedCountryGroup?.paths ?? [])
.filter((p) => p.serviceId === selectedNestedService.id)
.slice()
.sort((a, b) => b.bps - a.bps)}
servers={mapServers}
services={nestedServices}
highlight={highlightedPath}
viaMode="via"
destMode="services"
onToggle={togglePathHighlight}
/>
</div>
</div>
</>
) : liveSelectedService ? (
<>
<div className="flex items-start gap-2 px-4 py-3 border-b">
<div className="mt-0.5">
{destMode === "countries" && liveSelectedService.label !== "Прочее" && liveSelectedService.id !== "cc:other"
? <Flag code={liveSelectedService.label} size={22} />
: <ServiceBrandIcon label={destMode === "countries" ? "Прочее" : liveSelectedService.label} size={22} />}
</div>
<div className="flex-1 min-w-0">
<p className="font-mono font-semibold text-sm truncate">
{destDisplayLabel(liveSelectedService, destMode)}
</p>
<p className="text-xs text-muted-foreground mt-0.5">
{destMode === "countries"
? `Конечная страна${liveSelectedService.label !== "Прочее" ? ` · ${liveSelectedService.label}` : ""}`
: `Конечный сервис · ${liveSelectedService.category}`}
</p>
</div>
<button
@@ -2726,50 +3143,107 @@ export default function NetworkMapPage() {
<div className="flex flex-col gap-0">
<div className="flex items-center justify-between py-2 border-b border-border/50">
<span className="text-xs text-muted-foreground">Доля окна</span>
<span className="text-xs font-mono font-medium text-cyan-400">{serviceSharePct(selectedService.share)}</span>
<span className="text-xs font-mono font-medium text-cyan-400">{serviceSharePct(liveSelectedService.share)}</span>
</div>
{mapTotalBytes > 0 && (
<div className="flex items-center justify-between py-2 border-b border-border/50">
<span className="text-xs text-muted-foreground">Классифицировано</span>
<span className="text-xs font-mono font-medium text-cyan-400">
{formatNetflowRate({
bytes: mapNamedBytes,
bps: (mapNamedBytes * 8) / Math.max(1, mapWindowSec),
bpsFwd: 0,
bpsRev: 0,
})}
{" из "}
{formatNetflowRate({
bytes: mapTotalBytes,
bps: (mapTotalBytes * 8) / Math.max(1, mapWindowSec),
bpsFwd: 0,
bpsRev: 0,
})}
</span>
</div>
)}
{!mapAsnLoaded && destMode === "services" && (
<div className="flex items-center justify-between py-2 border-b border-border/50">
<span className="text-xs text-muted-foreground">GeoLite2 ASN</span>
<span className="text-xs font-mono font-medium text-amber-500">не загружена</span>
</div>
)}
{destMode === "countries" && !mapCountryLoaded && (
<div className="flex items-center justify-between py-2 border-b border-border/50">
<span className="text-xs text-muted-foreground">GeoIP Country</span>
<span className="text-xs font-mono font-medium text-amber-500">RIPE-кэш</span>
</div>
)}
<div className="flex items-center justify-between py-2 border-b border-border/50">
<span className="text-xs text-muted-foreground">Скорость</span>
<span className="text-xs font-mono font-medium">
{formatNetflowRate({
bytes: selectedService.bytes,
bps: selectedService.bps,
bpsFwd: selectedService.bps,
bytes: liveSelectedService.bytes,
bps: liveSelectedService.bps,
bpsFwd: liveSelectedService.bps,
bpsRev: 0,
})}
</span>
</div>
</div>
{expandedCountryGroup && liveSelectedService.id === expandedCountryGroup.countryId && (
<div>
<p className="text-xs font-semibold text-muted-foreground uppercase tracking-wider mb-2">Сервисы в стране</p>
<div className="flex flex-col gap-1">
{expandedCountryGroup.services.map((svc) => (
<button
key={svc.id}
type="button"
onClick={() => selectService(svc)}
className={cn(
"flex items-center justify-between gap-2 rounded-md px-2 py-1.5 text-left text-xs transition-colors",
selectedService?.id === svc.id ? "bg-cyan-500/15 ring-1 ring-cyan-500/40" : "hover:bg-muted/50",
)}
>
<span className="flex items-center gap-1.5 min-w-0">
<ServiceBrandIcon label={svc.label} size={14} />
<span className="font-mono truncate">{svc.label}</span>
</span>
<span className="font-mono text-cyan-400 tabular-nums shrink-0">{serviceSharePct(svc.share)}</span>
</button>
))}
</div>
</div>
)}
<div>
<p className="text-xs font-semibold text-muted-foreground uppercase tracking-wider mb-2">Выход</p>
<div className="flex flex-col gap-1.5">
{visibleServiceEdges.filter((e) => e.toId === selectedService.id).map((e) => {
<div className="flex flex-col gap-3">
{visibleServiceEdges.filter((e) => e.toId === liveSelectedService.id).map((e) => {
const src = mapServers.find((s) => s.id === e.fromId)
const enPaths = destPaths
.filter((p) => p.serviceId === liveSelectedService.id && p.enId === e.fromId)
.slice()
.sort((a, b) => b.bps - a.bps)
return (
<div key={`${e.fromId}|${e.toId}`} className="flex items-center justify-between text-xs">
<span className="font-mono truncate">{src?.name ?? e.fromId}</span>
<span className="font-mono text-emerald-400 tabular-nums">
{formatNetflowRate({ bytes: e.bytes, bps: e.bps, bpsFwd: e.bpsFwd, bpsRev: e.bpsRev })}
</span>
<div key={`${e.fromId}|${e.toId}`} className="flex flex-col gap-1.5">
<div className="flex items-center justify-between text-xs">
<span className="font-mono truncate">{src?.name ?? e.fromId}</span>
<span className="font-mono text-emerald-400 tabular-nums">
{formatNetflowRate({ bytes: e.bytes, bps: e.bps, bpsFwd: e.bpsFwd, bpsRev: e.bpsRev })}
</span>
</div>
<ServicePathList
paths={enPaths}
servers={mapServers}
services={destNodes}
highlight={highlightedPath}
viaMode="via"
destMode={destMode}
onToggle={togglePathHighlight}
/>
</div>
)
})}
</div>
</div>
<div>
<p className="text-xs font-semibold text-muted-foreground uppercase tracking-wider mb-2">Пути</p>
<ServicePathList
paths={mapServicePaths
.filter((p) => p.serviceId === selectedService.id)
.slice()
.sort((a, b) => b.bps - a.bps)}
servers={mapServers}
services={mapServices}
highlight={highlightedPath}
viaMode="via"
onToggle={togglePathHighlight}
/>
</div>
</div>
</>
) : selected ? (
@@ -3001,14 +3475,19 @@ export default function NetworkMapPage() {
<div>
<p className="text-xs font-semibold text-muted-foreground uppercase tracking-wider mb-2">Пути</p>
<ServicePathList
paths={mapServicePaths
.filter((p) => p.viaId === selected.id || p.enId === selected.id)
paths={destPaths
.filter((p) => (
selected.type === "exit-node"
? p.enId === selected.id
: p.viaId === selected.id
))
.slice()
.sort((a, b) => b.bps - a.bps)}
servers={mapServers}
services={mapServices}
services={destNodes}
highlight={highlightedPath}
viaMode="service"
destMode={destMode}
onToggle={togglePathHighlight}
/>
</div>
+29 -3
View File
@@ -122,11 +122,17 @@ interface BackendBfdSession {
packetsRx: number; packetsTx: number; stateChanges: number
}
interface BackendOspfRoute {
id: string; serverId: number; serverName: string; serverSite: string
destination: string; type: OspfRoute["type"]; cost: number; nextHop: string; via: string; area: string
}
interface BackendOspfAll {
neighbors: BackendNeighbor[]
interfaces: BackendInterface[]
instances: BackendInstance[]
bfdSessions: BackendBfdSession[]
routes?: BackendOspfRoute[]
}
function isRefInterfaceName(name: string): boolean {
@@ -213,6 +219,22 @@ function backendToBfdSession(b: BackendBfdSession): BfdSession {
}
}
function backendToRoute(b: BackendOspfRoute): OspfRoute {
const allowed: OspfRoute["type"][] = ["O", "O IA", "O E1", "O E2"]
const type = allowed.includes(b.type) ? b.type : "O"
return {
id: `${b.serverId}-${b.id}`,
destination: b.destination,
type,
cost: b.cost,
nextHop: b.nextHop,
via: b.via,
serverId: String(b.serverId),
serverLabel: b.serverName,
area: b.area || "—",
}
}
// ─── mock data ────────────────────────────────────────────────────────────────
const COST_STEP = 10
@@ -1172,7 +1194,7 @@ export default function OspfPage() {
}, [isLive, backendUrl, fetchTick])
// Derive frontend types from backend data or use mocks
const { items, neighbors, graphNodes, graphEdges, routerIds, bfdSessions } = useMemo(() => {
const { items, neighbors, graphNodes, graphEdges, routerIds, bfdSessions, routes } = useMemo(() => {
if (isLive && liveData) {
// Build interface→cost map for neighbor cost lookup
const ifaceMap = new Map<string, number>()
@@ -1185,6 +1207,7 @@ export default function OspfPage() {
.filter((item) => !isRefInterfaceName(item.interfaceName))
const neighbors = liveData.neighbors.map(b => backendToNeighbor(b, ifaceMap))
const bfdSessions = (liveData.bfdSessions ?? []).map(backendToBfdSession)
const routes = (liveData.routes ?? []).map(backendToRoute)
// Build routerIds from instances
const routerIds: Record<string, string> = {}
@@ -1195,7 +1218,7 @@ export default function OspfPage() {
}
const { nodes: graphNodes, edges: graphEdges } = buildLiveGraph(neighbors)
return { items, neighbors, graphNodes, graphEdges, routerIds, bfdSessions }
return { items, neighbors, graphNodes, graphEdges, routerIds, bfdSessions, routes }
}
if (isLive) {
return {
@@ -1205,6 +1228,7 @@ export default function OspfPage() {
graphEdges: [],
routerIds: {},
bfdSessions: [],
routes: [],
}
}
return {
@@ -1214,6 +1238,7 @@ export default function OspfPage() {
graphEdges: MOCK_GRAPH_EDGES,
routerIds: MOCK_ROUTER_IDS,
bfdSessions: MOCK_BFD,
routes: MOCK_ROUTES,
}
}, [isLive, liveData])
@@ -1257,6 +1282,7 @@ export default function OspfPage() {
const displayItems = filterServerId === ALL_SERVERS_ID ? items : items.filter(i => i.routerKey === filterServerId)
const displayNeighbors = filterServerId === ALL_SERVERS_ID ? neighbors : neighbors.filter(n => n.localRouter === filterServerId)
const displayBfdSessions = filterServerId === ALL_SERVERS_ID ? bfdSessions : bfdSessions.filter(b => b.serverId === filterServerId)
const displayRoutes = filterServerId === ALL_SERVERS_ID ? routes : routes.filter(r => r.serverId === filterServerId)
const ospfRailItems = useMemo<ServerTileItem[]>(() => (
ospfServers.map((s) => {
@@ -1398,7 +1424,7 @@ export default function OspfPage() {
routerIds={routerIds}
/>
)}
{activeTab === "routes" && <RoutesTab routes={isLive ? [] : MOCK_ROUTES} />}
{activeTab === "routes" && <RoutesTab routes={displayRoutes} />}
{activeTab === "bfd" && <BfdTab sessions={displayBfdSessions} />}
</div>
+119 -58
View File
@@ -18,9 +18,12 @@ import { Flag } from "@/components/flag"
import { useDataSource } from "@/lib/data-source"
import { cn } from "@/lib/utils"
import { servers as mockServers, type Server } from "@/lib/data"
import { PlusIcon, SaveIcon, TrashIcon, SearchIcon, XIcon, PencilIcon, CheckIcon, AlertCircleIcon } from "lucide-react"
import { PlusIcon, TrashIcon, SearchIcon, XIcon, PencilIcon, CheckIcon, AlertCircleIcon, RefreshCwIcon, HistoryIcon, AlertTriangleIcon } from "lucide-react"
import { toast } from "sonner"
import { requestJson } from "@/shared/api/http-client"
import { ServerRailLayout, ServerRailMobileButton } from "@/components/server-rail-layout"
import { ConfigHistorySheet } from "@/components/config-history-sheet"
import type { ConfigRevisionDto } from "@/lib/config-revisions"
import { type ServerTileItem } from "@/components/server-tile-rail"
interface BackendServer {
@@ -364,7 +367,7 @@ export default function RecursiveRoutesPage() {
const [servers, setServers] = useState<Server[]>([])
const [selectedServerId, setSelectedServerId] = useState<string>("")
const [rows, setRows] = useState<RecursiveRouteRow[]>([])
const [busy, setBusy] = useState<"load" | "save" | "from" | "to" | null>(null)
const [busy, setBusy] = useState<"load" | "apply" | null>(null)
const [search, setSearch] = useState("")
const [sheetOpen, setSheetOpen] = useState(false)
const [sheetMode, setSheetMode] = useState<"create" | "edit">("create")
@@ -373,6 +376,11 @@ export default function RecursiveRoutesPage() {
const [gatewayOptions, setGatewayOptions] = useState<GatewayOption[]>([])
const [expandedGroupKey, setExpandedGroupKey] = useState<string | null>(null)
const [opError, setOpError] = useState<string | null>(null)
const [liveStale, setLiveStale] = useState(false)
const [historyOpen, setHistoryOpen] = useState(false)
const [historyLoading, setHistoryLoading] = useState(false)
const [historyRestoring, setHistoryRestoring] = useState(false)
const [revisions, setRevisions] = useState<ConfigRevisionDto[]>([])
/** В live не дергаем API с id мока (srv1…) пока не подтянули /api/servers */
const [liveServerListReady, setLiveServerListReady] = useState(false)
@@ -407,10 +415,13 @@ export default function RecursiveRoutesPage() {
setOpError(null)
setBusy("load")
try {
const res = await apiFetch<{ routes: RecursiveRouteRow[] }>(`/api/recursive-routes?serverId=${selectedServerId}`)
const res = await apiFetch<{ routes: RecursiveRouteRow[]; stale?: boolean }>(
`/api/recursive-routes?serverId=${selectedServerId}`,
)
setRows(res.routes)
setLiveStale(Boolean(res.stale))
} catch (e) {
setRows([])
setLiveStale(true)
setOpError(e instanceof Error ? e.message : "Не удалось загрузить маршруты")
} finally {
setBusy(null)
@@ -437,56 +448,65 @@ export default function RecursiveRoutesPage() {
void loadGateways()
}, [loadGateways])
const saveToDb = useCallback(async () => {
if (!isLive || !selectedServerId) return
const applyRoutes = useCallback(async (next: RecursiveRouteRow[]) => {
if (!isLive || !selectedServerId || liveStale) return
const prev = rows
setRows(next)
setOpError(null)
setBusy("save")
setBusy("apply")
try {
await apiFetch<{ ok: boolean }>("/api/recursive-routes", {
const res = await apiFetch<{ ok: boolean; routes?: RecursiveRouteRow[] }>("/api/recursive-routes", {
method: "PUT",
body: JSON.stringify({ serverId: selectedServerId, routes: rows }),
body: JSON.stringify({ serverId: selectedServerId, routes: next }),
})
await loadRoutes()
setRows(res.routes ?? next)
setLiveStale(false)
toast.success("Маршруты применены на роутер")
} catch (e) {
setOpError(e instanceof Error ? e.message : "Не удалось сохранить маршруты в БД")
setRows(prev)
const msg = e instanceof Error ? e.message : "Не удалось применить маршруты на роутер"
setOpError(msg)
toast.error(msg)
} finally {
setBusy(null)
}
}, [isLive, selectedServerId, rows, apiFetch, loadRoutes])
}, [isLive, selectedServerId, liveStale, rows, apiFetch])
const syncFromRouter = useCallback(async () => {
const loadRevisions = useCallback(async () => {
if (!isLive || !selectedServerId) return
setOpError(null)
setBusy("from")
setHistoryLoading(true)
try {
await apiFetch<{ ok: boolean }>("/api/recursive-routes/sync/from-router", {
method: "POST",
body: JSON.stringify({ serverId: selectedServerId }),
})
await loadRoutes()
const res = await apiFetch<{ revisions: ConfigRevisionDto[] }>(
`/api/recursive-routes/revisions?serverId=${encodeURIComponent(selectedServerId)}`,
)
setRevisions(res.revisions)
} catch (e) {
setOpError(e instanceof Error ? e.message : "Не удалось синхронизировать маршруты с роутера")
toast.error(e instanceof Error ? e.message : "Не удалось загрузить историю")
setRevisions([])
} finally {
setBusy(null)
}
}, [isLive, selectedServerId, apiFetch, loadRoutes])
const syncToRouter = useCallback(async () => {
if (!isLive || !selectedServerId) return
setOpError(null)
setBusy("to")
try {
await apiFetch<{ ok: boolean }>("/api/recursive-routes/sync/to-router", {
method: "POST",
body: JSON.stringify({ serverId: selectedServerId }),
})
} catch (e) {
setOpError(e instanceof Error ? e.message : "Не удалось применить маршруты на роутер")
} finally {
setBusy(null)
setHistoryLoading(false)
}
}, [isLive, selectedServerId, apiFetch])
const restoreRevision = useCallback(async (id: string) => {
if (!isLive || !selectedServerId) return
setHistoryRestoring(true)
try {
const res = await apiFetch<{ ok: boolean; routes?: RecursiveRouteRow[] }>(
`/api/recursive-routes/revisions/${encodeURIComponent(id)}/restore`,
{ method: "POST", body: JSON.stringify({ serverId: selectedServerId }) },
)
setRows(res.routes ?? [])
setLiveStale(false)
toast.success("Версия применена на роутер")
await loadRevisions()
} catch (e) {
toast.error(e instanceof Error ? e.message : "Не удалось откатить")
} finally {
setHistoryRestoring(false)
}
}, [isLive, selectedServerId, apiFetch, loadRevisions])
function groupKeyOf(row: RecursiveRouteRow): string {
return row.dstAddress.trim().toLowerCase()
}
@@ -529,22 +549,26 @@ export default function RecursiveRoutesPage() {
disabled: false,
country: ep.country || inferCountry(ep.gateway) || "",
})
let next: RecursiveRouteRow[]
if (sheetMode === "create") {
const base = `new-${Date.now()}-${Math.random().toString(36).slice(2, 6)}`
const expanded = v.endpoints.map((ep, i) => toRow(ep, `${base}-${i}`))
setRows(prev => [...prev, ...expanded])
next = [...rows, ...expanded]
} else if (editingGroupKey) {
setRows(prev => {
const kept = prev.filter(r => groupKeyOf(r) !== editingGroupKey)
const base = `edit-${Date.now()}-${Math.random().toString(36).slice(2, 6)}`
const expanded = v.endpoints.map((ep, i) => toRow(ep, `${base}-${i}`))
return [...kept, ...expanded]
})
const kept = rows.filter(r => groupKeyOf(r) !== editingGroupKey)
const base = `edit-${Date.now()}-${Math.random().toString(36).slice(2, 6)}`
const expanded = v.endpoints.map((ep, i) => toRow(ep, `${base}-${i}`))
next = [...kept, ...expanded]
} else {
setSheetOpen(false)
return
}
setSheetOpen(false)
void applyRoutes(next)
}
const currentServer = servers.find(s => s.id === selectedServerId)
const mutationsLocked = !isLive || busy !== null || liveStale
const rrRailItems = useMemo<ServerTileItem[]>(() => (
servers.map((s) => ({
id: s.id,
@@ -602,16 +626,33 @@ export default function RecursiveRoutesPage() {
actions={
<>
<ServerRailMobileButton />
<Button variant="outline" size="sm" onClick={syncFromRouter} disabled={!isLive || busy !== null}>
{busy === "from" ? "Синхронизация..." : "Router => DB"}
</Button>
<Button variant="outline" size="sm" onClick={syncToRouter} disabled={!isLive || busy !== null}>
{busy === "to" ? "Применение..." : "DB => Router"}
</Button>
<Button variant="outline" size="sm" onClick={saveToDb} disabled={!isLive || busy !== null}>
<SaveIcon className="size-4" />Сохранить в БД
</Button>
<Button size="sm" onClick={openCreate} disabled={!isLive || busy !== null}>
{isLive && (
<>
<Button
variant="outline"
size="sm"
onClick={() => void loadRoutes()}
disabled={busy !== null}
title="Прочитать маршруты с роутера"
>
<RefreshCwIcon className={cn("size-4", busy === "load" && "animate-spin")} />
Обновить
</Button>
<Button
variant="outline"
size="sm"
onClick={() => {
setHistoryOpen(true)
void loadRevisions()
}}
disabled={busy !== null}
>
<HistoryIcon className="size-4" />
История
</Button>
</>
)}
<Button size="sm" onClick={openCreate} disabled={mutationsLocked}>
<PlusIcon className="size-4" />Добавить
</Button>
</>
@@ -643,6 +684,12 @@ export default function RecursiveRoutesPage() {
{opError}
</div>
)}
{liveStale && (
<div className="w-full text-xs text-amber-700 dark:text-amber-400 bg-amber-500/10 border border-amber-500/20 rounded-md px-3 py-2 flex items-center gap-2">
<AlertTriangleIcon className="size-3.5 shrink-0" />
Роутер недоступен показан кэш. Изменения заблокированы.
</div>
)}
</div>
}
>
@@ -679,10 +726,13 @@ export default function RecursiveRoutesPage() {
expandedKey={expandedGroupKey}
onExpandedChange={setExpandedGroupKey}
onEdit={openEdit}
onDelete={(g) => setRows((prev) => prev.filter((r) => groupKeyOf(r) !== g.key))}
onDelete={(g) => {
if (mutationsLocked) return
void applyRoutes(rows.filter((r) => groupKeyOf(r) !== g.key))
}}
/>
<button onClick={openCreate}
className="w-full flex items-center gap-2 px-5 py-2 text-xs text-muted-foreground hover:text-foreground hover:bg-muted/20 transition-colors border-t">
<button onClick={openCreate} disabled={mutationsLocked}
className="w-full flex items-center gap-2 px-5 py-2 text-xs text-muted-foreground hover:text-foreground hover:bg-muted/20 transition-colors border-t disabled:opacity-50">
<PlusIcon className="size-3.5" />
Добавить маршрут
</button>
@@ -698,6 +748,17 @@ export default function RecursiveRoutesPage() {
onClose={() => setSheetOpen(false)}
gateways={gatewayOptions}
/>
<ConfigHistorySheet
open={historyOpen}
onOpenChange={setHistoryOpen}
title="История маршрутов"
itemLabel="маршрутов"
revisions={revisions}
loading={historyLoading}
restoring={historyRestoring}
onRestore={restoreRevision}
/>
</>
)
}
+558
View File
@@ -0,0 +1,558 @@
"use client"
import { Suspense, useCallback, useEffect, useMemo, useState } from "react"
import { useSearchParams } from "next/navigation"
import {
ActivityIcon,
DatabaseIcon,
GaugeIcon,
ServerIcon,
UsersIcon,
} from "lucide-react"
import { PageHeader } from "@/components/page-header"
import { KpiStatGrid } from "@/components/reui-kit/kpi-stat-grid"
import { DataPageCard } from "@/components/data-page-card"
import { DataPageToolbar } from "@/components/data-page-toolbar"
import { SegmentedControl } from "@/components/form-kit"
import { EmptyState } from "@/components/empty-state"
import { PeriodSelector, rangeForPreset, type DateRangeYmd } from "@/components/statistics/period-selector"
import { StatisticsVolumeChart } from "@/components/statistics/statistics-volume-chart"
import { DimensionSelect, PivotDimSelect } from "@/components/statistics/dimension-select"
import { SliceChips } from "@/components/statistics/slice-chips"
import { BreakdownDashboard } from "@/components/statistics/breakdown-dashboard"
import { StatisticsPivotGrid } from "@/components/statistics/statistics-pivot-grid"
import {
StatisticsBreakdownDataGrid,
type StatisticsSliceKind,
} from "@/components/data-grids/statistics-breakdown-data-grid"
import { Alert, AlertDescription, AlertTitle } from "@/components/reui/alert"
import type { Filter } from "@/components/reui/filters"
import { STATISTICS_FILTER_FIELDS } from "@/lib/data-filters/statistics-filter-fields"
import {
isStatisticsPivotDim,
isStatisticsSliceKind,
STATISTICS_DIMS,
} from "@/lib/statistics-dims"
import { useDataSource } from "@/lib/data-source"
import { fmtBps, formatBytes } from "@/lib/fmt-rate"
import {
getStatistics,
getStatisticsPivot,
STATISTICS_UNBOUND_USER_ID,
type StatisticsDto,
type StatisticsPivotDto,
type StatisticsQuery,
} from "@/shared/api/statistics"
import type { StatisticsBreakdownRow, StatisticsPivotDim } from "@mmapp/contracts/statistics"
/**
* BI-куб трафика: критерий остальные разрезы + pivot.
* Preview: https://reui.io/preview/base/dashboard-1 · https://reui.io/preview/base/stats-12
* · https://reui.io/preview/base/data-grid-filtering-2 · https://reui.io/preview/base/solution-analytics-8
* · https://reui.io/docs/components/base/frame · https://reui.io/docs/components/base/data-grid
*/
const EMPTY: StatisticsDto = {
from: "",
to: "",
grain: "day",
kpis: {
bytes: 0,
packets: 0,
avgBps: 0,
users: 0,
servers: 0,
ifaces: 0,
topCountry: "",
topService: "",
},
series: [],
users: [],
servers: [],
interfaces: [],
countries: [],
services: [],
asns: [],
}
const EMPTY_PIVOT: StatisticsPivotDto = {
rowDim: "country",
colDim: "service",
metric: "bytes",
columns: [],
rows: [],
otherBytes: 0,
}
interface CubeSlices {
country?: string
service?: string
asn?: string
serverId?: string
userId?: string
iface?: string
}
const SLICE_KEYS = ["country", "service", "asn", "serverId", "userId", "iface"] as const
function readRange(sp: URLSearchParams): DateRangeYmd {
const from = sp.get("from")
const to = sp.get("to")
if (from && to && from <= to) return { from, to }
return rangeForPreset("7d")
}
function readDim(sp: URLSearchParams): StatisticsSliceKind {
const t = sp.get("dim") ?? sp.get("tab")
return t && isStatisticsSliceKind(t) ? t : "users"
}
function readView(sp: URLSearchParams): "explore" | "pivot" {
return sp.get("view") === "pivot" ? "pivot" : "explore"
}
function readPlanes(sp: URLSearchParams): "unique" | "all" {
return sp.get("planes") === "all" ? "all" : "unique"
}
function readPivotDim(sp: URLSearchParams, key: string, fallback: StatisticsPivotDim): StatisticsPivotDim {
const v = sp.get(key)
return v && isStatisticsPivotDim(v) ? v : fallback
}
function readSlices(sp: URLSearchParams): CubeSlices {
const next: CubeSlices = {}
for (const key of SLICE_KEYS) {
const v = sp.get(key)?.trim()
if (v) next[key] = v
}
return next
}
function slicesToFilters(slices: CubeSlices): Filter[] {
return SLICE_KEYS.flatMap((key) => {
const val = slices[key]
if (!val) return []
return [{ id: key, field: key, operator: "is", values: [val] }]
})
}
function filtersToSlices(filters: Filter[]): CubeSlices {
const next: CubeSlices = {}
for (const f of filters) {
const raw = String(f.values[0] ?? "").trim()
if (!raw) continue
if (f.field === "country") next.country = raw.toUpperCase().slice(0, 2)
else if (f.field === "service") next.service = raw
else if (f.field === "asn") next.asn = raw.replace(/[^\d]/g, "")
else if (f.field === "serverId") next.serverId = raw
else if (f.field === "userId") next.userId = raw
else if (f.field === "iface") next.iface = raw
}
return next
}
function toQuery(range: DateRangeYmd, slices: CubeSlices, planes: "unique" | "all"): StatisticsQuery {
const serverId = slices.serverId ? Number(slices.serverId) : undefined
const asn = slices.asn != null && slices.asn !== "" ? Number(slices.asn) : undefined
return {
from: range.from,
to: range.to,
serverId: Number.isFinite(serverId) && (serverId ?? 0) > 0 ? serverId : undefined,
userId: slices.userId,
iface: slices.iface,
country: slices.country && slices.country.length === 2 ? slices.country : undefined,
service: slices.service,
asn: Number.isFinite(asn) ? asn : undefined,
planes,
}
}
function selectedIdForKind(kind: StatisticsSliceKind, slices: CubeSlices): string | undefined {
if (kind === "users") return slices.userId
if (kind === "servers") return slices.serverId
if (kind === "countries") return slices.country
if (kind === "services") return slices.service
if (kind === "asns") return slices.asn
if (kind === "interfaces" && slices.serverId && slices.iface) {
return `${slices.serverId}:${slices.iface}`
}
if (kind === "interfaces") return slices.iface
return undefined
}
function rowsForKind(data: StatisticsDto, kind: StatisticsSliceKind) {
if (kind === "users") return data.users
if (kind === "servers") return data.servers
if (kind === "interfaces") return data.interfaces
if (kind === "countries") return data.countries
if (kind === "services") return data.services
return data.asns
}
function hasAnySlice(slices: CubeSlices): boolean {
return SLICE_KEYS.some((k) => Boolean(slices[k]))
}
function hiddenKinds(slices: CubeSlices): Set<StatisticsSliceKind> {
const hidden = new Set<StatisticsSliceKind>()
if (slices.userId) hidden.add("users")
if (slices.serverId) hidden.add("servers")
if (slices.iface) hidden.add("interfaces")
if (slices.country) hidden.add("countries")
if (slices.service) hidden.add("services")
if (slices.asn) hidden.add("asns")
return hidden
}
function applyDimValue(slices: CubeSlices, kind: StatisticsSliceKind, rowId: string): CubeSlices {
const next: CubeSlices = { ...slices }
if (kind === "users") {
if (rowId === STATISTICS_UNBOUND_USER_ID) return next
if (next.userId === rowId) delete next.userId
else next.userId = rowId
} else if (kind === "servers") {
if (next.serverId === rowId) delete next.serverId
else next.serverId = rowId
} else if (kind === "countries") {
if (next.country === rowId) delete next.country
else next.country = rowId
} else if (kind === "services") {
if (next.service === rowId) delete next.service
else next.service = rowId
} else if (kind === "asns") {
if (next.asn === rowId) delete next.asn
else next.asn = rowId
} else {
const colon = rowId.indexOf(":")
const sid = colon >= 0 ? rowId.slice(0, colon) : undefined
const iface = colon >= 0 ? rowId.slice(colon + 1) : rowId
if (next.iface === iface && next.serverId === sid) {
delete next.iface
delete next.serverId
} else {
next.iface = iface
if (sid) next.serverId = sid
}
}
return next
}
function applyPivotDim(slices: CubeSlices, dim: StatisticsPivotDim, id: string): CubeSlices {
const kind = STATISTICS_DIMS.find((d) => d.pivot === dim)?.id ?? "users"
return applyDimValue(slices, kind, id)
}
function chipList(slices: CubeSlices): { key: string; label: string }[] {
const chips: { key: string; label: string }[] = []
if (slices.country) chips.push({ key: "country", label: `страна ${slices.country}` })
if (slices.service) chips.push({ key: "service", label: `сервис ${slices.service}` })
if (slices.asn) chips.push({ key: "asn", label: `ASN ${slices.asn}` })
if (slices.serverId) chips.push({ key: "serverId", label: `сервер ${slices.serverId}` })
if (slices.userId) chips.push({ key: "userId", label: `пользователь ${slices.userId}` })
if (slices.iface) chips.push({ key: "iface", label: `iface ${slices.iface}` })
return chips
}
function StatisticsPageInner() {
const searchParams = useSearchParams()
const { mode, backendUrl, prefsHydrated } = useDataSource()
const isLive = mode === "live"
const range = useMemo(() => readRange(searchParams), [searchParams])
const slices = useMemo(() => readSlices(searchParams), [searchParams])
const filters = useMemo(() => slicesToFilters(slices), [slices])
const dim = useMemo(() => readDim(searchParams), [searchParams])
const view = useMemo(() => readView(searchParams), [searchParams])
const planes = useMemo(() => readPlanes(searchParams), [searchParams])
const pivotRow = useMemo(() => readPivotDim(searchParams, "pivotRow", "country"), [searchParams])
const pivotCol = useMemo(() => readPivotDim(searchParams, "pivotCol", "service"), [searchParams])
const [data, setData] = useState<StatisticsDto>(EMPTY)
const [pivot, setPivot] = useState<StatisticsPivotDto>(EMPTY_PIVOT)
const [loading, setLoading] = useState(false)
const [error, setError] = useState<string | null>(null)
const replaceParams = useCallback(
(patch: Record<string, string | undefined>) => {
const sp = new URLSearchParams(searchParams.toString())
for (const [k, v] of Object.entries(patch)) {
if (v) sp.set(k, v)
else sp.delete(k)
}
const qs = sp.toString()
if (qs === searchParams.toString()) return
window.history.replaceState(null, "", qs ? `/statistics?${qs}` : "/statistics")
},
[searchParams],
)
const setRange = useCallback(
(next: DateRangeYmd) => {
replaceParams({ from: next.from, to: next.to })
},
[replaceParams],
)
const setSlices = useCallback(
(next: CubeSlices) => {
replaceParams({
country: next.country,
service: next.service,
asn: next.asn,
serverId: next.serverId,
userId: next.userId,
iface: next.iface,
})
},
[replaceParams],
)
useEffect(() => {
if (!prefsHydrated || !isLive) return
let cancelled = false
void (async () => {
setLoading(true)
setError(null)
try {
const query = toQuery(range, slices, planes)
const dto = await getStatistics(backendUrl, query)
if (!cancelled) setData(dto)
if (view === "pivot" && pivotRow !== pivotCol) {
const matrix = await getStatisticsPivot(backendUrl, {
...query,
row: pivotRow,
col: pivotCol,
metric: "bytes",
})
if (!cancelled) setPivot(matrix)
}
} catch (e: unknown) {
if (!cancelled) {
setData(EMPTY)
setPivot(EMPTY_PIVOT)
setError(e instanceof Error ? e.message : "Не удалось загрузить статистику")
}
} finally {
if (!cancelled) setLoading(false)
}
})()
return () => {
cancelled = true
}
}, [backendUrl, isLive, prefsHydrated, range, slices, view, pivotRow, pivotCol, planes])
const viewData = isLive ? data : EMPTY
const sliced = hasAnySlice(slices)
const emptyCube = !isLive || (!loading && viewData.kpis.bytes === 0 && viewData.interfaces.length === 0)
function handleRowClick(kind: StatisticsSliceKind, row: StatisticsBreakdownRow) {
if (kind === "users" && row.id === STATISTICS_UNBOUND_USER_ID) return
if (kind === "interfaces" && row.label.includes("· дубль")) return
setSlices(applyDimValue(slices, kind, row.id))
}
function handlePivotCell(rowId: string, colId: string) {
if (rowId === "__other__" || colId === "__other__") return
let next = applyPivotDim(slices, pivotRow, rowId)
next = applyPivotDim(next, pivotCol, colId)
replaceParams({
country: next.country,
service: next.service,
asn: next.asn,
serverId: next.serverId,
userId: next.userId,
iface: next.iface,
view: "explore",
})
}
const kpis = viewData.kpis
const chips = chipList(slices)
const countLabel =
view === "pivot"
? `${pivot.rows.length} × ${pivot.columns.length}`
: sliced
? `${STATISTICS_DIMS.filter((d) => !hiddenKinds(slices).has(d.id)).length} разрезов`
: `${rowsForKind(viewData, dim).length} строк`
return (
<div className="flex h-full flex-col">
<PageHeader
crumbs={[{ label: "Обзор", href: "/dashboard" }, { label: "Статистика" }]}
actions={<PeriodSelector range={range} onChange={setRange} />}
/>
<div className="flex flex-1 flex-col gap-4 overflow-y-auto px-4 py-4 md:gap-6 md:px-6 md:py-5">
{!isLive ? (
<Alert>
<AlertTitle>Живые данные выключены</AlertTitle>
<AlertDescription>
Куб статистики строится из IPFIX. Переключитесь на живой источник, чтобы увидеть отчёт.
</AlertDescription>
</Alert>
) : null}
{error ? (
<Alert variant="destructive">
<AlertTitle>Ошибка загрузки</AlertTitle>
<AlertDescription>{error}</AlertDescription>
</Alert>
) : null}
{!slices.serverId && isLive && !emptyCube ? (
<Alert>
<AlertTitle>Уникальный объём</AlertTitle>
<AlertDescription>
Объём трафик клиентов на GRE/WG, без повторного учёта JHEN и WAN.
</AlertDescription>
</Alert>
) : null}
<KpiStatGrid
aria-label="Сводка трафика"
isLoading={loading}
skeletonCount={5}
items={[
{
id: "bytes",
label: "Объём",
value: formatBytes(kpis.bytes),
hint: "GRE/WG клиентов, без hops",
icon: <DatabaseIcon />,
iconClassName: "text-muted-foreground",
},
{
id: "packets",
label: "Пакеты",
value: kpis.packets.toLocaleString("ru-RU"),
hint: kpis.topService ? `топ: ${kpis.topService}` : undefined,
icon: <ActivityIcon />,
iconClassName: "text-muted-foreground",
},
{
id: "bps",
label: "Средний bitrate",
value: fmtBps(kpis.avgBps),
icon: <GaugeIcon />,
iconClassName: "text-muted-foreground",
},
{
id: "users",
label: "Пользователи",
value: String(kpis.users),
icon: <UsersIcon />,
iconClassName: "text-muted-foreground",
},
{
id: "servers",
label: "Серверы",
value: String(kpis.servers),
hint: slices.serverId
? (kpis.ifaces ? `${kpis.ifaces} iface` : undefined)
: planes === "all"
? "WAN и дубли в списке"
: "без WAN и overlay",
icon: <ServerIcon />,
iconClassName: "text-muted-foreground",
},
]}
/>
<StatisticsVolumeChart series={viewData.series} grain={viewData.grain} />
<DataPageCard>
<DataPageToolbar
leading={
<div className="flex flex-wrap items-center gap-3">
<SegmentedControl
value={view}
onChange={(next) => replaceParams({ view: next === "pivot" ? "pivot" : "explore" })}
options={[
{ value: "explore", label: "Разрез" },
{ value: "pivot", label: "Сводка" },
]}
/>
<SegmentedControl
value={planes}
onChange={(next) => replaceParams({ planes: next === "all" ? "all" : undefined })}
options={[
{ value: "unique", label: "Уникальный" },
{ value: "all", label: "Все плоскости" },
]}
/>
{view === "explore" && !sliced ? (
<DimensionSelect
label="Критерий"
value={dim}
onChange={(next) => replaceParams({ dim: next })}
/>
) : null}
{view === "pivot" ? (
<>
<PivotDimSelect
label="Строки"
value={pivotRow}
exclude={pivotCol}
onChange={(next) => replaceParams({ pivotRow: next })}
/>
<PivotDimSelect
label="Колонки"
value={pivotCol}
exclude={pivotRow}
onChange={(next) => replaceParams({ pivotCol: next })}
/>
</>
) : null}
</div>
}
filters={filters}
onFiltersChange={(next) => setSlices(filtersToSlices(next))}
filterFields={STATISTICS_FILTER_FIELDS}
countLabel={countLabel}
/>
<SliceChips
chips={chips}
onRemove={(key) => {
const next = { ...slices }
delete next[key as keyof CubeSlices]
setSlices(next)
}}
/>
{emptyCube ? (
<EmptyState
title="Нет данных куба"
description="За выбранный период нет IPFIX-фактов. Куб заполняется с момента деплоя, без бэкфилла за год."
/>
) : view === "pivot" ? (
<StatisticsPivotGrid data={isLive ? pivot : EMPTY_PIVOT} onCellClick={handlePivotCell} isLoading={loading} />
) : sliced ? (
<BreakdownDashboard
data={viewData}
hidden={hiddenKinds(slices)}
selectedIdFor={(kind) => selectedIdForKind(kind, slices)}
onRowClick={handleRowClick}
isLoading={loading}
/>
) : (
<StatisticsBreakdownDataGrid
rows={rowsForKind(viewData, dim)}
kind={dim}
selectedId={selectedIdForKind(dim, slices)}
onRowClick={(row) => handleRowClick(dim, row)}
isLoading={loading}
/>
)}
</DataPageCard>
</div>
</div>
)
}
export default function StatisticsPage() {
return (
<Suspense fallback={null}>
<StatisticsPageInner />
</Suspense>
)
}
+1 -5
View File
@@ -952,10 +952,6 @@ export default function TrafficPage() {
setFlowAnalytics(null)
return
}
if (range === "5m") {
setFlowAnalytics(null)
return
}
if (range === "30d") {
const month = new Date().toISOString().slice(0, 7)
void getFlowMonthly(backendUrl, {
@@ -1128,7 +1124,7 @@ export default function TrafficPage() {
const ingestLine = flowIngestLine(flowStats)
const collectorAlive = Boolean(flowStats?.listenerBound || flowStats?.packetsReceived)
const flowError = liveError
|| (flowLiveError && !(collectorAlive && /live HTTP 500/.test(flowLiveError)) ? flowLiveError : null)
|| flowLiveError
|| (displayedFlow?.degraded ? "Коллектор перегружен: упрощённая аналитика" : null)
const flowKpiItems = [
+190 -45
View File
@@ -1,30 +1,63 @@
"use client"
import { useMemo, useState } from "react"
import { useCallback, useEffect, useMemo, useState } from "react"
import { PageHeader } from "@/components/page-header"
import { vxlanTunnels, servers } from "@/lib/data"
import type { VxlanTunnel } from "@/lib/data"
import { vxlanTunnels as mockVxlanTunnels, servers as mockServers } from "@/lib/data"
import type { Server, VxlanTunnel } from "@/lib/data"
import { KpiStatGrid } from "@/components/reui-kit/kpi-stat-grid"
import { OpsPanel } from "@/components/ops-panel"
import { DataPageCard } from "@/components/data-page-card"
import { DataPageToolbar } from "@/components/data-page-toolbar"
import { VxlanDataGrid } from "@/components/data-grids/vxlan-data-grid"
import { Button } from "@/components/ui/button"
import { Alert, AlertDescription } from "@/components/ui/alert"
import {
NetworkIcon, PlusIcon, CodeXmlIcon, LayersIcon,
NetworkIcon, PlusIcon, LayersIcon, RefreshCwIcon, AlertCircleIcon,
} from "lucide-react"
import { CodeExportSheet } from "@/components/reui-kit/code-export-sheet"
import { useDataSource } from "@/lib/data-source"
import { requestJson } from "@/shared/api/http-client"
import { cn } from "@/lib/utils"
import { ServerRailLayout, ServerRailMobileButton } from "@/components/server-rail-layout"
import { ALL_SERVERS_ID, type ServerTileItem } from "@/components/server-tile-rail"
// ─── helpers ──────────────────────────────────────────────────────────────────
function serverFor(id: string) {
return servers.find((s) => s.id === id)
interface BackendServer {
id: number
name: string
host: string
type?: Server["type"]
site?: string
country: string
asn?: string
enabled: boolean
status?: Server["status"]
latency?: number | null
}
// ─── RSC generator ───────────────────────────────────────────────────────────
interface VxlanApiResponse {
tunnels: VxlanTunnel[]
}
function generateVxlanRsc(t: VxlanTunnel): string {
const srv = serverFor(t.serverId)
function mapBackendServer(s: BackendServer): Server {
return {
id: String(s.id),
name: s.name || s.host,
host: s.host,
model: "—",
os: "—",
site: s.site ?? "",
country: s.country || "UN",
asn: s.asn ?? "",
type: s.type ?? "exit-node",
enabled: s.enabled,
status: s.status ?? "online",
latency: s.latency ?? null,
sessions: 0,
}
}
function generateVxlanRsc(t: VxlanTunnel, serverById: Record<string, Server>): string {
const srv = serverById[t.serverId]
const lines: string[] = []
lines.push(`# VXLAN — ${t.name} · VNI ${t.vni}`)
if (srv) lines.push(`# Сервер: ${srv.name} (${srv.host})`)
@@ -42,7 +75,6 @@ function generateVxlanRsc(t: VxlanTunnel): string {
if (!t.enabled) lines.push(` disabled=yes \\`)
lines.push(``)
// FDB entries for remote VTEPs
for (const vtep of t.remoteVteps) {
lines.push(`/interface/vxlan/vteps/add \\`)
lines.push(` interface=${t.name} \\`)
@@ -50,7 +82,6 @@ function generateVxlanRsc(t: VxlanTunnel): string {
lines.push(``)
}
// Bridge
lines.push(`# Добавить в bridge:`)
lines.push(`/interface/bridge/port/add \\`)
lines.push(` bridge=bridge-overlay \\`)
@@ -59,12 +90,18 @@ function generateVxlanRsc(t: VxlanTunnel): string {
return lines.join("\n")
}
// ─── Export Sheet ─────────────────────────────────────────────────────────────
function ExportSheet({ open, tunnel, onClose }: {
open: boolean; tunnel: VxlanTunnel | null; onClose: () => void
function ExportSheet({
open, tunnel, onClose, serverById,
}: {
open: boolean
tunnel: VxlanTunnel | null
onClose: () => void
serverById: Record<string, Server>
}) {
const code = useMemo(() => tunnel ? generateVxlanRsc(tunnel) : "", [tunnel])
const code = useMemo(
() => (tunnel ? generateVxlanRsc(tunnel, serverById) : ""),
[tunnel, serverById],
)
return (
<CodeExportSheet
@@ -84,46 +121,156 @@ function ExportSheet({ open, tunnel, onClose }: {
)
}
// ─── Export Sheet ─────────────────────────────────────────────────────────────
export default function VxlanPage() {
const [search, setSearch] = useState("")
const { mode, backendUrl } = useDataSource()
const isLive = mode === "live"
const [search, setSearch] = useState("")
const [exportTunnel, setExportTunnel] = useState<VxlanTunnel | null>(null)
const [selectedServerId, setSelectedServerId] = useState(ALL_SERVERS_ID)
const [liveTunnels, setLiveTunnels] = useState<VxlanTunnel[]>([])
const [liveServers, setLiveServers] = useState<Server[]>([])
const [loading, setLoading] = useState(false)
const [liveError, setLiveError] = useState<string | null>(null)
const loadLive = useCallback(async () => {
if (!isLive) return
setLoading(true)
setLiveError(null)
try {
const [tunnelsRes, serversRes] = await Promise.all([
requestJson<VxlanApiResponse>(backendUrl, "/api/vxlan"),
requestJson<BackendServer[]>(backendUrl, "/api/servers"),
])
setLiveTunnels(tunnelsRes.tunnels ?? [])
setLiveServers(serversRes.filter((s) => s.enabled).map(mapBackendServer))
} catch (e) {
setLiveError(e instanceof Error ? e.message : "Ошибка загрузки")
setLiveTunnels([])
} finally {
setLoading(false)
}
}, [isLive, backendUrl])
useEffect(() => {
if (!isLive) {
queueMicrotask(() => {
setLiveTunnels([])
setLiveServers([])
setLiveError(null)
})
return
}
queueMicrotask(() => {
void loadLive()
})
}, [isLive, loadLive])
const displayTunnels = isLive ? liveTunnels : mockVxlanTunnels
const displayServers = isLive ? liveServers : mockServers.filter((s) => s.enabled)
const effectiveServerId =
selectedServerId === ALL_SERVERS_ID || displayServers.some((s) => s.id === selectedServerId)
? selectedServerId
: ALL_SERVERS_ID
const scopedTunnels = useMemo(() => {
if (effectiveServerId === ALL_SERVERS_ID) return displayTunnels
return displayTunnels.filter((t) => t.serverId === effectiveServerId)
}, [displayTunnels, effectiveServerId])
const serverById = useMemo(
() => Object.fromEntries(displayServers.map((s) => [s.id, s])),
[displayServers],
)
const railItems = useMemo<ServerTileItem[]>(() => (
displayServers.map((s) => ({
id: s.id,
name: s.name,
host: s.host,
site: s.site,
country: s.country,
status: s.status,
type: s.type,
enabled: s.enabled,
meta: String(displayTunnels.filter((t) => t.serverId === s.id).length),
}))
), [displayServers, displayTunnels])
const filtered = useMemo(() => {
if (!search) return vxlanTunnels
if (!search) return scopedTunnels
const q = search.toLowerCase()
return vxlanTunnels.filter((t) =>
return scopedTunnels.filter((t) =>
t.name.includes(q) ||
String(t.vni).includes(q) ||
t.vtepIp.includes(q) ||
(serverFor(t.serverId)?.name.toLowerCase().includes(q) ?? false)
(serverById[t.serverId]?.name.toLowerCase().includes(q) ?? false),
)
}, [search])
}, [search, scopedTunnels, serverById])
const upCount = vxlanTunnels.filter((t) => t.status === "up").length
const vnis = new Set(vxlanTunnels.map((t) => t.vni)).size
const upCount = scopedTunnels.filter((t) => t.status === "up").length
const vnis = new Set(scopedTunnels.map((t) => t.vni)).size
return (
<div className="flex flex-col h-full">
<PageHeader
crumbs={[{ label: "Управление" }, { label: "VXLAN" }]}
actions={
<Button size="sm">
<PlusIcon className="size-4" />Новый VXLAN
</Button>
}
/>
<div className="flex-1 overflow-y-auto p-6">
<>
<ServerRailLayout
items={railItems}
selectedId={effectiveServerId}
onSelect={setSelectedServerId}
showAll
allCount={displayServers.length}
loading={isLive && loading && displayServers.length === 0}
header={
<PageHeader
crumbs={[{ label: "Управление" }, { label: "VXLAN" }]}
actions={
<>
<ServerRailMobileButton />
<Button
variant="outline"
size="sm"
onClick={() => { void loadLive() }}
disabled={!isLive || loading}
>
<RefreshCwIcon className={cn("size-4", loading && "animate-spin")} />
Обновить
</Button>
<Button size="sm">
<PlusIcon className="size-4" />Новый VXLAN
</Button>
</>
}
/>
}
>
<div className="flex flex-col gap-5">
{isLive && liveError && (
<Alert variant="warning" className="py-2">
<AlertCircleIcon />
<AlertDescription className="text-xs">Ошибка загрузки: {liveError}</AlertDescription>
</Alert>
)}
{isLive && !loading && displayTunnels.length === 0 && !liveError && (
<div className="rounded-md border border-border bg-muted/30 px-4 py-6 text-center text-sm text-muted-foreground">
На опрошенных серверах нет VXLAN-интерфейсов
</div>
)}
{mode === "mock" && (
<span className="inline-flex w-fit items-center gap-1.5 rounded-full border border-border bg-muted/40 px-2.5 py-0.5 text-[11px] font-medium text-muted-foreground">
Моковые данные
</span>
)}
<KpiStatGrid
aria-label="Сводка VXLAN"
items={[
{
id: "tunnels",
label: "Туннелей",
value: vxlanTunnels.length,
value: scopedTunnels.length,
icon: <NetworkIcon className="size-4" />,
iconClassName: "text-muted-foreground",
},
@@ -144,14 +291,13 @@ export default function VxlanPage() {
{
id: "servers",
label: "Серверов",
value: new Set(vxlanTunnels.map((t) => t.serverId)).size,
value: new Set(scopedTunnels.map((t) => t.serverId)).size,
icon: <NetworkIcon className="size-4" />,
iconClassName: "text-primary",
},
]}
/>
{/* Info banner */}
<div className="flex items-start gap-3 rounded-lg bg-sky-500/5 border border-sky-500/20 px-4 py-3 text-sm">
<NetworkIcon className="size-5 text-sky-500 shrink-0 mt-0.5" />
<div>
@@ -163,7 +309,6 @@ export default function VxlanPage() {
</div>
</div>
{/* Table */}
<DataPageCard>
<DataPageToolbar
search={search}
@@ -173,12 +318,11 @@ export default function VxlanPage() {
/>
<VxlanDataGrid
tunnels={filtered}
servers={servers}
servers={displayServers}
onExport={setExportTunnel}
/>
</DataPageCard>
{/* Reference */}
<OpsPanel title="RouterOS 7 · /interface/vxlan — быстрые команды" contentClassName="px-5 py-4">
<div className="grid grid-cols-1 sm:grid-cols-3 gap-4 text-xs font-mono">
{[
@@ -232,13 +376,14 @@ export default function VxlanPage() {
</OpsPanel>
</div>
</div>
</ServerRailLayout>
<ExportSheet
open={!!exportTunnel}
tunnel={exportTunnel}
onClose={() => setExportTunnel(null)}
serverById={serverById}
/>
</div>
</>
)
}
+71 -1
View File
@@ -57,10 +57,12 @@ import {
type ServerTileItem,
} from "@/components/server-tile-rail"
import { toast } from "sonner"
import { ConfigHistorySheet } from "@/components/config-history-sheet"
import type { ConfigRevisionDto } from "@/lib/config-revisions"
import {
ShieldCheckIcon, PlusIcon, KeyRoundIcon,
UsersIcon, ActivityIcon, RefreshCwIcon, UploadIcon, InfoIcon,
Trash2Icon, CodeXmlIcon, AlertCircleIcon,
Trash2Icon, CodeXmlIcon, AlertCircleIcon, HistoryIcon,
} from "lucide-react"
type WgWorkspaceTab = "interfaces" | "peers" | "cli"
@@ -188,6 +190,10 @@ export default function WireGuardPage() {
const [exportPeerId, setExportPeerId] = useState<string | null>(null)
const [peerIface, setPeerIface] = useState<WgIfaceWithServer | null>(null)
const [pendingDelete, setPendingDelete] = useState<PendingDelete | null>(null)
const [historyOpen, setHistoryOpen] = useState(false)
const [revisions, setRevisions] = useState<ConfigRevisionDto[]>([])
const [historyLoading, setHistoryLoading] = useState(false)
const [historyRestoring, setHistoryRestoring] = useState(false)
const [liveExport, setLiveExport] = useState<{
rsc?: string
conf?: string
@@ -242,6 +248,44 @@ export default function WireGuardPage() {
? selectedServerId
: ALL_SERVERS_ID
const historyServerId = effectiveServerId === ALL_SERVERS_ID ? null : effectiveServerId
const loadRevisions = useCallback(async () => {
if (!isLive || !historyServerId) return
setHistoryLoading(true)
try {
const res = await requestJson<{ revisions: ConfigRevisionDto[] }>(
backendUrl,
`/api/wireguard/revisions?serverId=${encodeURIComponent(historyServerId)}`,
)
setRevisions(res.revisions)
} catch (err) {
toast.error("Не удалось загрузить историю", { description: String(err) })
setRevisions([])
} finally {
setHistoryLoading(false)
}
}, [isLive, historyServerId, backendUrl])
const restoreRevision = useCallback(async (id: string) => {
if (!isLive || !historyServerId) return
setHistoryRestoring(true)
try {
await requestJson(
backendUrl,
`/api/wireguard/revisions/${encodeURIComponent(id)}/restore`,
{ method: "POST", body: JSON.stringify({ serverId: historyServerId }) },
)
toast.success("Версия применена на роутер")
await loadLive()
await loadRevisions()
} catch (err) {
toast.error("Не удалось откатить", { description: String(err) })
} finally {
setHistoryRestoring(false)
}
}, [isLive, historyServerId, backendUrl, loadLive, loadRevisions])
const scopedIfaces = useMemo(() => {
if (effectiveServerId === ALL_SERVERS_ID) return displayIfaces
return displayIfaces.filter((i) => i.serverId === effectiveServerId)
@@ -544,6 +588,7 @@ export default function WireGuardPage() {
<>
<ServerRailMobileButton />
{isLive && (
<>
<Button
size="sm"
variant="outline"
@@ -553,6 +598,20 @@ export default function WireGuardPage() {
<RefreshCwIcon className={`size-4 ${loading ? "animate-spin" : ""}`} />
Обновить
</Button>
<Button
size="sm"
variant="outline"
disabled={loading || !historyServerId}
title={!historyServerId ? "Выберите сервер, чтобы смотреть историю" : "История версий и откат на CHR"}
onClick={() => {
setHistoryOpen(true)
void loadRevisions()
}}
>
<HistoryIcon className="size-4" />
История
</Button>
</>
)}
<Button size="sm" variant="outline" onClick={() => setImportOpen(true)}>
<UploadIcon className="size-4" />
@@ -822,6 +881,17 @@ export default function WireGuardPage() {
</AlertDialogFooter>
</AlertDialogContent>
</AlertDialog>
<ConfigHistorySheet
open={historyOpen}
onOpenChange={setHistoryOpen}
title="История WireGuard"
itemLabel="интерфейсов"
revisions={revisions}
loading={historyLoading}
restoring={historyRestoring}
onRestore={restoreRevision}
/>
</>
)
}
+1
View File
@@ -5,3 +5,4 @@ dist/
*.db-wal
.env
storage/backups/
storage/geoip/
+620
View File
@@ -0,0 +1,620 @@
-- Compact PostgreSQL 18 schema: wipe all app tables (not schema_migrations),
-- recreate with smaller time-series rows. SQLite is re-imported after this.
DO $$
DECLARE
r record;
BEGIN
FOR r IN
SELECT tablename
FROM pg_tables
WHERE schemaname = 'public'
AND tablename <> 'schema_migrations'
LOOP
EXECUTE format('DROP TABLE IF EXISTS public.%I CASCADE', r.tablename);
END LOOP;
END $$;
CREATE TABLE IF NOT EXISTS schema_migrations (
id TEXT PRIMARY KEY,
applied_at TIMESTAMPTZ NOT NULL DEFAULT now()
);
CREATE TABLE IF NOT EXISTS servers (
id BIGINT GENERATED BY DEFAULT AS IDENTITY PRIMARY KEY,
name TEXT NOT NULL DEFAULT '',
host TEXT NOT NULL,
port INTEGER NOT NULL DEFAULT 443,
username TEXT NOT NULL DEFAULT 'admin',
password TEXT NOT NULL DEFAULT '',
use_ssl BOOLEAN NOT NULL DEFAULT TRUE,
verify_ssl BOOLEAN NOT NULL DEFAULT FALSE,
type TEXT NOT NULL DEFAULT 'home-router'
CHECK (type IN ('jump-host', 'exit-node', 'home-router')),
site TEXT NOT NULL DEFAULT '',
country TEXT NOT NULL DEFAULT '',
asn TEXT NOT NULL DEFAULT '',
comment TEXT NOT NULL DEFAULT '',
enabled BOOLEAN NOT NULL DEFAULT TRUE,
lan_subnet TEXT NOT NULL DEFAULT '',
wan_uplinks JSONB COMPRESSION lz4 NOT NULL DEFAULT '[]'::jsonb,
mgmt_tunnel_ip TEXT NOT NULL DEFAULT '',
created_at TIMESTAMPTZ NOT NULL DEFAULT now(),
updated_at TIMESTAMPTZ NOT NULL DEFAULT now()
);
CREATE TABLE IF NOT EXISTS traffic_settings (
id BIGINT PRIMARY KEY CHECK (id = 1),
enabled BOOLEAN NOT NULL DEFAULT TRUE,
interval_sec INTEGER NOT NULL DEFAULT 30,
retention_days INTEGER NOT NULL DEFAULT 14,
last_collected_at TIMESTAMPTZ,
last_duration_ms INTEGER,
last_error TEXT,
created_at TIMESTAMPTZ NOT NULL DEFAULT now(),
updated_at TIMESTAMPTZ NOT NULL DEFAULT now()
);
CREATE TABLE IF NOT EXISTS servers_api_ping_settings (
id BIGINT PRIMARY KEY CHECK (id = 1),
enabled BOOLEAN NOT NULL DEFAULT FALSE,
interval_sec INTEGER NOT NULL DEFAULT 120,
last_collected_at TIMESTAMPTZ,
last_duration_ms INTEGER,
last_error TEXT,
created_at TIMESTAMPTZ NOT NULL DEFAULT now(),
updated_at TIMESTAMPTZ NOT NULL DEFAULT now()
);
CREATE TABLE IF NOT EXISTS traffic_flow_settings (
id BIGINT PRIMARY KEY CHECK (id = 1),
enabled BOOLEAN NOT NULL DEFAULT FALSE,
collector_ip TEXT NOT NULL DEFAULT '10.255.254.1',
flow_listen_port INTEGER NOT NULL DEFAULT 4739,
wg_listen_port INTEGER NOT NULL DEFAULT 51821,
prefix TEXT NOT NULL DEFAULT '10.255.254.0/24',
public_endpoint TEXT NOT NULL DEFAULT '',
host_public_key TEXT NOT NULL DEFAULT '',
host_private_key TEXT NOT NULL DEFAULT '',
hub_server_id BIGINT,
retention_hours INTEGER NOT NULL DEFAULT 24,
top_n INTEGER NOT NULL DEFAULT 200,
map_service_min_share_pct DOUBLE PRECISION NOT NULL DEFAULT 5,
last_datagram_at TIMESTAMPTZ,
last_exporter_ip TEXT,
last_error TEXT,
packets_received BIGINT NOT NULL DEFAULT 0,
peers_json JSONB COMPRESSION lz4 NOT NULL DEFAULT '[]'::jsonb,
created_at TIMESTAMPTZ NOT NULL DEFAULT now(),
updated_at TIMESTAMPTZ NOT NULL DEFAULT now()
);
CREATE TABLE IF NOT EXISTS uptime_settings (
id BIGINT PRIMARY KEY CHECK (id = 1),
enabled BOOLEAN NOT NULL DEFAULT TRUE,
resources_enabled BOOLEAN NOT NULL DEFAULT TRUE,
ping_enabled BOOLEAN NOT NULL DEFAULT TRUE,
speed_enabled BOOLEAN NOT NULL DEFAULT TRUE,
interval_sec INTEGER NOT NULL DEFAULT 15,
probe_interval_sec INTEGER NOT NULL DEFAULT 15,
speed_interval_sec INTEGER NOT NULL DEFAULT 60,
retention_days INTEGER NOT NULL DEFAULT 14,
last_collected_at TIMESTAMPTZ,
last_duration_ms INTEGER,
last_error TEXT,
created_at TIMESTAMPTZ NOT NULL DEFAULT now(),
updated_at TIMESTAMPTZ NOT NULL DEFAULT now()
);
CREATE TABLE IF NOT EXISTS evobgp_settings (
id BIGINT PRIMARY KEY CHECK (id = 1),
base_url TEXT NOT NULL DEFAULT '',
api_key TEXT NOT NULL DEFAULT '',
enabled BOOLEAN NOT NULL DEFAULT FALSE,
updated_at TIMESTAMPTZ NOT NULL DEFAULT now()
);
CREATE TABLE IF NOT EXISTS alert_telegram_settings (
id BIGINT PRIMARY KEY CHECK (id = 1),
bot_token TEXT NOT NULL DEFAULT '',
chat_id TEXT NOT NULL DEFAULT '',
message_thread_id INTEGER,
updated_at TIMESTAMPTZ NOT NULL DEFAULT now()
);
CREATE TABLE IF NOT EXISTS acme_settings (
id BIGINT PRIMARY KEY CHECK (id = 1),
directory_url TEXT NOT NULL DEFAULT 'https://acme-v02.api.letsencrypt.org/directory',
cloudflare_api_token TEXT NOT NULL DEFAULT '',
default_zone_id TEXT NOT NULL DEFAULT '',
account_private_key TEXT NOT NULL DEFAULT '',
updated_at TIMESTAMPTZ NOT NULL DEFAULT now()
);
CREATE TABLE IF NOT EXISTS certificate_renew_settings (
id BIGINT PRIMARY KEY CHECK (id = 1),
enabled BOOLEAN NOT NULL DEFAULT TRUE,
interval_sec INTEGER NOT NULL DEFAULT 21600,
renew_before_days INTEGER NOT NULL DEFAULT 30,
last_collected_at TIMESTAMPTZ,
last_duration_ms INTEGER,
last_error TEXT,
updated_at TIMESTAMPTZ NOT NULL DEFAULT now()
);
CREATE TABLE IF NOT EXISTS backup_schedule_settings (
id BIGINT PRIMARY KEY CHECK (id = 1),
enabled BOOLEAN NOT NULL DEFAULT TRUE,
frequency TEXT NOT NULL DEFAULT 'daily' CHECK (frequency IN ('daily', 'weekly', 'monthly')),
hour INTEGER NOT NULL DEFAULT 3,
minute INTEGER NOT NULL DEFAULT 0,
week_day INTEGER NOT NULL DEFAULT 0,
month_day INTEGER NOT NULL DEFAULT 1,
keep_count INTEGER NOT NULL DEFAULT 7,
format TEXT NOT NULL DEFAULT 'rsc' CHECK (format IN ('rsc', 'backup')),
server_ids_json JSONB COMPRESSION lz4 NOT NULL DEFAULT '[]'::jsonb,
last_run_at TIMESTAMPTZ,
last_duration_ms INTEGER,
last_error TEXT,
updated_at TIMESTAMPTZ NOT NULL DEFAULT now()
);
CREATE TABLE IF NOT EXISTS internet_path_settings (
id BIGINT PRIMARY KEY CHECK (id = 1),
enabled BOOLEAN NOT NULL DEFAULT TRUE,
interval_sec INTEGER NOT NULL DEFAULT 300,
retention_days INTEGER NOT NULL DEFAULT 14,
last_collected_at TIMESTAMPTZ,
last_duration_ms INTEGER,
last_error TEXT,
created_at TIMESTAMPTZ NOT NULL DEFAULT now(),
updated_at TIMESTAMPTZ NOT NULL DEFAULT now()
);
CREATE TABLE IF NOT EXISTS alert_engine_cursor (
id BIGINT PRIMARY KEY CHECK (id = 1),
last_source_finished_at TIMESTAMPTZ,
updated_at TIMESTAMPTZ NOT NULL DEFAULT now()
);
CREATE TABLE IF NOT EXISTS data_migration (
id INTEGER PRIMARY KEY CHECK (id = 1),
sqlite_imported_at TIMESTAMPTZ,
sqlite_path TEXT,
sqlite_sha256 TEXT,
report_json JSONB COMPRESSION lz4
);
CREATE TABLE IF NOT EXISTS filter_rules (
id BIGINT GENERATED BY DEFAULT AS IDENTITY PRIMARY KEY,
server_id BIGINT NOT NULL REFERENCES servers(id) ON DELETE CASCADE,
sort_order INTEGER NOT NULL DEFAULT 0,
community TEXT NOT NULL,
community_name TEXT,
action TEXT NOT NULL DEFAULT 'route' CHECK (action IN ('route', 'blackhole')),
gateway TEXT NOT NULL DEFAULT '',
gateway_tunnel_id TEXT NOT NULL DEFAULT '',
description TEXT NOT NULL DEFAULT '',
created_at TIMESTAMPTZ NOT NULL DEFAULT now(),
updated_at TIMESTAMPTZ NOT NULL DEFAULT now()
);
CREATE INDEX IF NOT EXISTS idx_filter_rules_server_sort ON filter_rules(server_id, sort_order);
CREATE TABLE IF NOT EXISTS recursive_routes (
id BIGINT GENERATED BY DEFAULT AS IDENTITY PRIMARY KEY,
server_id BIGINT NOT NULL REFERENCES servers(id) ON DELETE CASCADE,
sort_order INTEGER NOT NULL DEFAULT 0,
dst_address TEXT NOT NULL,
gateway TEXT NOT NULL,
distance INTEGER NOT NULL DEFAULT 1,
scope INTEGER,
target_scope INTEGER,
routing_table TEXT NOT NULL DEFAULT 'main',
check_gateway TEXT NOT NULL DEFAULT '',
country TEXT NOT NULL DEFAULT '',
comment TEXT NOT NULL DEFAULT '',
disabled BOOLEAN NOT NULL DEFAULT FALSE,
created_at TIMESTAMPTZ NOT NULL DEFAULT now(),
updated_at TIMESTAMPTZ NOT NULL DEFAULT now()
);
CREATE INDEX IF NOT EXISTS idx_recursive_routes_server_sort ON recursive_routes(server_id, sort_order);
CREATE TABLE IF NOT EXISTS server_snapshots (
id BIGINT GENERATED BY DEFAULT AS IDENTITY,
server_id BIGINT NOT NULL REFERENCES servers(id) ON DELETE CASCADE,
polled_at TIMESTAMPTZ NOT NULL,
status TEXT NOT NULL CHECK (status IN ('online', 'offline')),
latency_ms DOUBLE PRECISION,
ros_version TEXT,
board_name TEXT,
uptime TEXT,
cpu_load INTEGER,
free_memory BIGINT,
total_memory BIGINT,
identity_name TEXT,
raw_interfaces JSONB COMPRESSION lz4,
raw_ip_addresses JSONB COMPRESSION lz4,
PRIMARY KEY (id, polled_at)
) PARTITION BY RANGE (polled_at);
CREATE INDEX IF NOT EXISTS idx_server_snapshots_server_time ON server_snapshots(server_id, polled_at);
CREATE TABLE IF NOT EXISTS traffic_samples (
server_id BIGINT NOT NULL REFERENCES servers(id) ON DELETE CASCADE,
interface_name TEXT NOT NULL,
peer_public_key TEXT NOT NULL DEFAULT '',
sampled_at TIMESTAMPTZ NOT NULL,
rx_bytes BIGINT NOT NULL DEFAULT 0,
tx_bytes BIGINT NOT NULL DEFAULT 0,
rx_bps BIGINT NOT NULL DEFAULT 0,
tx_bps BIGINT NOT NULL DEFAULT 0,
flags SMALLINT NOT NULL DEFAULT 0,
PRIMARY KEY (server_id, sampled_at, interface_name, peer_public_key)
) PARTITION BY RANGE (sampled_at);
CREATE INDEX IF NOT EXISTS idx_traffic_samples_server_iface_time ON traffic_samples(server_id, interface_name, sampled_at);
CREATE TABLE IF NOT EXISTS servers_rest_ping_samples (
server_id BIGINT NOT NULL REFERENCES servers(id) ON DELETE CASCADE,
sampled_at TIMESTAMPTZ NOT NULL,
ok BOOLEAN NOT NULL,
latency_ms INTEGER,
error TEXT,
PRIMARY KEY (server_id, sampled_at)
) PARTITION BY RANGE (sampled_at);
CREATE TABLE IF NOT EXISTS flow_buckets (
server_id BIGINT NOT NULL REFERENCES servers(id) ON DELETE CASCADE,
bucket_at TIMESTAMPTZ NOT NULL,
src INET NOT NULL,
dst INET NOT NULL,
proto SMALLINT NOT NULL DEFAULT 0,
src_port INTEGER NOT NULL DEFAULT 0,
dst_port INTEGER NOT NULL DEFAULT 0,
bytes BIGINT NOT NULL DEFAULT 0,
packets BIGINT NOT NULL DEFAULT 0,
in_iface TEXT NOT NULL DEFAULT '',
out_iface TEXT NOT NULL DEFAULT '',
next_hop INET,
flow_start_ms BIGINT NOT NULL DEFAULT 0,
flow_end_ms BIGINT NOT NULL DEFAULT 0,
PRIMARY KEY (server_id, bucket_at, src, dst, proto, src_port, dst_port, in_iface)
) PARTITION BY RANGE (bucket_at);
CREATE INDEX IF NOT EXISTS idx_flow_buckets_server_time ON flow_buckets(server_id, bucket_at);
CREATE TABLE IF NOT EXISTS flow_minute_stats (
server_id BIGINT NOT NULL REFERENCES servers(id) ON DELETE CASCADE,
bucket_at TIMESTAMPTZ NOT NULL,
bytes BIGINT NOT NULL DEFAULT 0,
packets BIGINT NOT NULL DEFAULT 0,
unique_src INTEGER NOT NULL DEFAULT 0,
unique_dst INTEGER NOT NULL DEFAULT 0,
conversations INTEGER NOT NULL DEFAULT 0,
PRIMARY KEY (server_id, bucket_at)
) PARTITION BY RANGE (bucket_at);
CREATE TABLE IF NOT EXISTS flow_minute_dims (
server_id BIGINT NOT NULL REFERENCES servers(id) ON DELETE CASCADE,
bucket_at TIMESTAMPTZ NOT NULL,
dim TEXT NOT NULL,
key TEXT NOT NULL,
bytes BIGINT NOT NULL DEFAULT 0,
packets BIGINT NOT NULL DEFAULT 0,
PRIMARY KEY (server_id, bucket_at, dim, key)
) PARTITION BY RANGE (bucket_at);
CREATE INDEX IF NOT EXISTS idx_flow_minute_dims_time ON flow_minute_dims(bucket_at, dim);
CREATE TABLE IF NOT EXISTS flow_daily_dims (
server_id BIGINT NOT NULL REFERENCES servers(id) ON DELETE CASCADE,
day DATE NOT NULL,
dim TEXT NOT NULL,
key TEXT NOT NULL,
bytes BIGINT NOT NULL DEFAULT 0,
packets BIGINT NOT NULL DEFAULT 0,
PRIMARY KEY (server_id, day, dim, key)
) PARTITION BY RANGE (day);
CREATE INDEX IF NOT EXISTS idx_flow_daily_dims_day ON flow_daily_dims(day, dim);
CREATE TABLE IF NOT EXISTS flow_ip_meta (
prefix TEXT PRIMARY KEY,
asn INTEGER NOT NULL DEFAULT 0,
country TEXT NOT NULL DEFAULT '',
lat DOUBLE PRECISION,
lng DOUBLE PRECISION,
holder TEXT NOT NULL DEFAULT '',
ok INTEGER NOT NULL DEFAULT 1,
fetched_at TIMESTAMPTZ NOT NULL
);
CREATE TABLE IF NOT EXISTS flow_asn_meta (
asn INTEGER PRIMARY KEY,
holder TEXT NOT NULL DEFAULT '',
fetched_at TIMESTAMPTZ NOT NULL
);
CREATE TABLE IF NOT EXISTS uptime_probes (
id TEXT PRIMARY KEY,
src_server_id BIGINT NOT NULL REFERENCES servers(id) ON DELETE CASCADE,
src_interface TEXT NOT NULL DEFAULT '',
name TEXT NOT NULL,
target TEXT NOT NULL,
probe_filter TEXT NOT NULL DEFAULT '',
enabled BOOLEAN NOT NULL DEFAULT TRUE,
interval_sec INTEGER NOT NULL DEFAULT 0,
show_on_dashboard BOOLEAN NOT NULL DEFAULT FALSE,
sort_order INTEGER NOT NULL DEFAULT 0,
created_at TIMESTAMPTZ NOT NULL DEFAULT now(),
updated_at TIMESTAMPTZ NOT NULL DEFAULT now()
);
CREATE INDEX IF NOT EXISTS idx_uptime_probes_server_sort ON uptime_probes(src_server_id, sort_order);
CREATE TABLE IF NOT EXISTS uptime_probe_samples (
probe_id TEXT NOT NULL REFERENCES uptime_probes(id) ON DELETE CASCADE,
sampled_at TIMESTAMPTZ NOT NULL,
rtt_ms INTEGER,
loss_pct INTEGER NOT NULL DEFAULT 0,
status TEXT NOT NULL DEFAULT 'down' CHECK (status IN ('up', 'warn', 'down')),
PRIMARY KEY (probe_id, sampled_at)
) PARTITION BY RANGE (sampled_at);
CREATE TABLE IF NOT EXISTS uptime_resource_samples (
server_id BIGINT NOT NULL REFERENCES servers(id) ON DELETE CASCADE,
sampled_at TIMESTAMPTZ NOT NULL,
status TEXT NOT NULL DEFAULT 'offline' CHECK (status IN ('online', 'offline')),
cpu_load INTEGER NOT NULL DEFAULT 0,
free_memory BIGINT NOT NULL DEFAULT 0,
total_memory BIGINT NOT NULL DEFAULT 0,
free_hdd_space BIGINT NOT NULL DEFAULT 0,
total_hdd_space BIGINT NOT NULL DEFAULT 0,
uptime_seconds BIGINT NOT NULL DEFAULT 0,
PRIMARY KEY (server_id, sampled_at)
) PARTITION BY RANGE (sampled_at);
CREATE TABLE IF NOT EXISTS uptime_speed_probes (
id TEXT PRIMARY KEY,
src_server_id BIGINT NOT NULL REFERENCES servers(id) ON DELETE CASCADE,
dst_server_id BIGINT NOT NULL REFERENCES servers(id) ON DELETE CASCADE,
src_interface TEXT NOT NULL DEFAULT '',
dst_interface TEXT NOT NULL DEFAULT '',
protocol TEXT NOT NULL DEFAULT 'tcp' CHECK (protocol IN ('tcp', 'udp')),
direction TEXT NOT NULL DEFAULT 'both' CHECK (direction IN ('transmit', 'receive', 'both')),
duration_sec INTEGER NOT NULL DEFAULT 10,
enabled BOOLEAN NOT NULL DEFAULT TRUE,
last_run_at TIMESTAMPTZ,
last_tx_avg_mbps DOUBLE PRECISION,
last_rx_avg_mbps DOUBLE PRECISION,
last_status TEXT CHECK (last_status IN ('done', 'error')),
last_error TEXT,
last_ping_rtt_ms INTEGER,
last_ping_loss_pct INTEGER,
last_ping_at TIMESTAMPTZ,
last_ping_error TEXT,
sort_order INTEGER NOT NULL DEFAULT 0,
created_at TIMESTAMPTZ NOT NULL DEFAULT now(),
updated_at TIMESTAMPTZ NOT NULL DEFAULT now()
);
CREATE INDEX IF NOT EXISTS idx_uptime_speed_probes_src_sort ON uptime_speed_probes(src_server_id, sort_order);
CREATE TABLE IF NOT EXISTS uptime_speed_test_runs (
id TEXT PRIMARY KEY,
probe_id TEXT,
src_server_id BIGINT NOT NULL REFERENCES servers(id) ON DELETE CASCADE,
dst_server_id BIGINT NOT NULL REFERENCES servers(id) ON DELETE CASCADE,
src_interface TEXT NOT NULL DEFAULT '',
dst_interface TEXT NOT NULL DEFAULT '',
src_address TEXT,
dst_address TEXT,
src_interface_address TEXT,
dst_interface_address TEXT,
protocol TEXT NOT NULL DEFAULT 'tcp' CHECK (protocol IN ('tcp', 'udp')),
direction TEXT NOT NULL DEFAULT 'both' CHECK (direction IN ('transmit', 'receive', 'both')),
duration_sec INTEGER NOT NULL DEFAULT 10,
tx_avg_mbps DOUBLE PRECISION,
rx_avg_mbps DOUBLE PRECISION,
ping_rtt_ms INTEGER,
ping_loss_pct INTEGER,
ping_error TEXT,
status TEXT NOT NULL DEFAULT 'done' CHECK (status IN ('done', 'error')),
error TEXT,
created_at TIMESTAMPTZ NOT NULL DEFAULT now()
);
CREATE INDEX IF NOT EXISTS idx_uptime_speed_test_runs_created_at ON uptime_speed_test_runs(created_at);
CREATE TABLE IF NOT EXISTS certificate_issue_jobs (
id TEXT PRIMARY KEY,
status TEXT NOT NULL DEFAULT 'queued' CHECK (status IN ('queued', 'running', 'done', 'failed')),
step TEXT NOT NULL DEFAULT 'queued',
source TEXT NOT NULL DEFAULT 'manual' CHECK (source IN ('manual', 'scheduler')),
server_id BIGINT NOT NULL REFERENCES servers(id) ON DELETE CASCADE,
cert_name TEXT NOT NULL,
domain_names JSONB COMPRESSION lz4 NOT NULL,
key_type TEXT NOT NULL DEFAULT 'rsa2048',
trust_store TEXT NOT NULL DEFAULT 'www,api',
requested_at TIMESTAMPTZ NOT NULL DEFAULT now(),
started_at TIMESTAMPTZ,
finished_at TIMESTAMPTZ,
error TEXT
);
CREATE TABLE IF NOT EXISTS backup_entries (
id TEXT PRIMARY KEY,
server_id BIGINT REFERENCES servers(id) ON DELETE SET NULL,
server_name TEXT NOT NULL,
filename TEXT NOT NULL,
size_bytes BIGINT NOT NULL,
kind TEXT NOT NULL DEFAULT 'manual' CHECK (kind IN ('manual', 'auto')),
notes TEXT,
created_at TIMESTAMPTZ NOT NULL
);
CREATE UNIQUE INDEX IF NOT EXISTS idx_backup_entries_filename ON backup_entries(filename);
CREATE INDEX IF NOT EXISTS idx_backup_entries_server_created ON backup_entries(server_id, created_at);
CREATE TABLE IF NOT EXISTS alert_groups (
id TEXT PRIMARY KEY,
name TEXT NOT NULL,
combine_mode TEXT NOT NULL DEFAULT 'any' CHECK (combine_mode IN ('any', 'all')),
enabled BOOLEAN NOT NULL DEFAULT TRUE,
cooldown_override TEXT,
created_at TIMESTAMPTZ NOT NULL DEFAULT now(),
updated_at TIMESTAMPTZ NOT NULL DEFAULT now()
);
CREATE TABLE IF NOT EXISTS alert_rules (
id TEXT PRIMARY KEY,
name TEXT NOT NULL,
type TEXT NOT NULL,
target TEXT NOT NULL,
condition TEXT NOT NULL,
severity TEXT NOT NULL CHECK (severity IN ('critical', 'warning', 'info')),
enabled BOOLEAN NOT NULL DEFAULT TRUE,
cooldown TEXT NOT NULL DEFAULT '',
rule_chat_id TEXT NOT NULL DEFAULT '',
confirm_stability_sec INTEGER,
recovery_mode TEXT NOT NULL DEFAULT 'always' CHECK (recovery_mode IN ('always', 'never', 'conditional')),
recovery_stability_sec INTEGER,
group_id TEXT REFERENCES alert_groups(id) ON DELETE SET NULL,
created_at TIMESTAMPTZ NOT NULL DEFAULT now(),
updated_at TIMESTAMPTZ NOT NULL DEFAULT now()
);
CREATE TABLE IF NOT EXISTS alert_rule_targets (
id TEXT PRIMARY KEY,
rule_id TEXT NOT NULL REFERENCES alert_rules(id) ON DELETE CASCADE,
target TEXT NOT NULL,
sort_index INTEGER NOT NULL DEFAULT 0
);
CREATE INDEX IF NOT EXISTS idx_alert_rule_targets_rule ON alert_rule_targets(rule_id);
CREATE TABLE IF NOT EXISTS alert_rule_conditions (
id TEXT PRIMARY KEY,
rule_id TEXT NOT NULL REFERENCES alert_rules(id) ON DELETE CASCADE,
condition_line TEXT NOT NULL,
sort_index INTEGER NOT NULL DEFAULT 0
);
CREATE INDEX IF NOT EXISTS idx_alert_rule_conditions_rule ON alert_rule_conditions(rule_id);
CREATE TABLE IF NOT EXISTS alert_engine_state (
scope_key TEXT PRIMARY KEY,
last_fired_at TEXT NOT NULL DEFAULT '',
last_payload_hash TEXT
);
CREATE TABLE IF NOT EXISTS alert_engine_prev_live (
kind TEXT PRIMARY KEY CHECK (kind IN ('gre', 'bgp')),
payload_json JSONB COMPRESSION lz4 NOT NULL DEFAULT '{}'::jsonb,
updated_at TIMESTAMPTZ NOT NULL DEFAULT now()
);
CREATE TABLE IF NOT EXISTS alert_engine_confirm_pending (
rule_id TEXT PRIMARY KEY,
payload_hash TEXT NOT NULL,
since_at TIMESTAMPTZ NOT NULL
);
CREATE TABLE IF NOT EXISTS alert_history (
id TEXT PRIMARY KEY,
rule_id TEXT,
group_id TEXT,
rule_name TEXT NOT NULL,
severity TEXT NOT NULL CHECK (severity IN ('critical', 'warning', 'info')),
message TEXT NOT NULL,
sent_ok BOOLEAN NOT NULL DEFAULT TRUE,
fired_at TIMESTAMPTZ NOT NULL
);
CREATE INDEX IF NOT EXISTS idx_alert_history_fired_at ON alert_history(fired_at);
CREATE INDEX IF NOT EXISTS idx_alert_history_rule_id ON alert_history(rule_id);
CREATE TABLE IF NOT EXISTS alert_outbox (
id TEXT PRIMARY KEY,
dedupe_key TEXT NOT NULL,
channel TEXT NOT NULL DEFAULT 'telegram' CHECK (channel IN ('telegram')),
status TEXT NOT NULL DEFAULT 'pending' CHECK (status IN ('pending', 'sent', 'failed')),
retry_count INTEGER NOT NULL DEFAULT 0,
max_retries INTEGER NOT NULL DEFAULT 3,
next_attempt_at TIMESTAMPTZ NOT NULL,
payload_json JSONB COMPRESSION lz4 NOT NULL,
created_at TIMESTAMPTZ NOT NULL DEFAULT now(),
sent_at TIMESTAMPTZ,
last_error TEXT
);
CREATE UNIQUE INDEX IF NOT EXISTS idx_alert_outbox_dedupe ON alert_outbox(dedupe_key);
CREATE INDEX IF NOT EXISTS idx_alert_outbox_status_next_attempt ON alert_outbox(status, next_attempt_at);
CREATE INDEX IF NOT EXISTS idx_alert_outbox_pending_next ON alert_outbox(next_attempt_at) WHERE status = 'pending';
CREATE TABLE IF NOT EXISTS scheduler_runs (
id TEXT PRIMARY KEY,
job_key TEXT NOT NULL,
started_at TIMESTAMPTZ NOT NULL,
finished_at TIMESTAMPTZ NOT NULL,
status TEXT NOT NULL CHECK (status IN ('ok', 'error')),
error TEXT,
duration_ms INTEGER NOT NULL DEFAULT 0,
result_json JSONB COMPRESSION lz4
);
CREATE INDEX IF NOT EXISTS idx_scheduler_runs_job_time ON scheduler_runs(job_key, started_at);
CREATE TABLE IF NOT EXISTS events (
id TEXT PRIMARY KEY,
created_at TIMESTAMPTZ NOT NULL,
level TEXT NOT NULL CHECK (level IN ('critical', 'warning', 'info')),
event_type TEXT NOT NULL,
source_module TEXT NOT NULL,
title TEXT NOT NULL,
message TEXT NOT NULL,
entity_type TEXT,
entity_id TEXT,
payload_json JSONB COMPRESSION lz4
);
CREATE INDEX IF NOT EXISTS idx_events_created_at ON events(created_at DESC);
CREATE INDEX IF NOT EXISTS idx_events_level_created_at ON events(level, created_at DESC);
CREATE INDEX IF NOT EXISTS idx_events_source_created_at ON events(source_module, created_at DESC);
CREATE INDEX IF NOT EXISTS idx_events_event_type_created_at ON events(event_type, created_at DESC);
CREATE TABLE IF NOT EXISTS app_users (
id TEXT PRIMARY KEY,
name TEXT NOT NULL DEFAULT '',
login TEXT NOT NULL UNIQUE,
email TEXT NOT NULL DEFAULT '',
role TEXT NOT NULL DEFAULT 'viewer' CHECK (role IN ('admin', 'operator', 'viewer')),
active BOOLEAN NOT NULL DEFAULT TRUE,
avatar TEXT NOT NULL DEFAULT '',
last_seen TIMESTAMPTZ,
sections_json JSONB COMPRESSION lz4 NOT NULL DEFAULT '[]'::jsonb,
servers_json JSONB COMPRESSION lz4 NOT NULL DEFAULT '[]'::jsonb,
created_at TIMESTAMPTZ NOT NULL DEFAULT now(),
updated_at TIMESTAMPTZ NOT NULL DEFAULT now()
);
CREATE TABLE IF NOT EXISTS user_interface_bindings (
id TEXT PRIMARY KEY,
user_id TEXT NOT NULL REFERENCES app_users(id) ON DELETE CASCADE,
server_id BIGINT NOT NULL REFERENCES servers(id) ON DELETE CASCADE,
interface_name TEXT NOT NULL,
interface_type TEXT NOT NULL DEFAULT 'other' CHECK (interface_type IN ('ether', 'gre', 'wg', 'other')),
peer_public_key TEXT NOT NULL DEFAULT '',
peer_name TEXT NOT NULL DEFAULT '',
comment TEXT NOT NULL DEFAULT '',
created_at TIMESTAMPTZ NOT NULL DEFAULT now(),
updated_at TIMESTAMPTZ NOT NULL DEFAULT now(),
UNIQUE (server_id, interface_name, peer_public_key)
);
CREATE INDEX IF NOT EXISTS idx_user_iface_bind_user ON user_interface_bindings(user_id);
CREATE TABLE IF NOT EXISTS internet_path_snapshots (
id BIGINT GENERATED BY DEFAULT AS IDENTITY,
sampled_at TIMESTAMPTZ NOT NULL,
payload_json JSONB COMPRESSION lz4 NOT NULL,
PRIMARY KEY (id, sampled_at)
) PARTITION BY RANGE (sampled_at);
CREATE INDEX IF NOT EXISTS idx_internet_path_snapshots_sampled ON internet_path_snapshots(sampled_at);
INSERT INTO traffic_settings (id) VALUES (1) ON CONFLICT (id) DO NOTHING;
INSERT INTO traffic_flow_settings (id) VALUES (1) ON CONFLICT (id) DO NOTHING;
INSERT INTO uptime_settings (id) VALUES (1) ON CONFLICT (id) DO NOTHING;
INSERT INTO evobgp_settings (id) VALUES (1) ON CONFLICT (id) DO NOTHING;
INSERT INTO servers_api_ping_settings (id) VALUES (1) ON CONFLICT (id) DO NOTHING;
INSERT INTO internet_path_settings (id) VALUES (1) ON CONFLICT (id) DO NOTHING;
INSERT INTO alert_telegram_settings (id) VALUES (1) ON CONFLICT (id) DO NOTHING;
INSERT INTO acme_settings (id) VALUES (1) ON CONFLICT (id) DO NOTHING;
INSERT INTO certificate_renew_settings (id) VALUES (1) ON CONFLICT (id) DO NOTHING;
INSERT INTO backup_schedule_settings (id) VALUES (1) ON CONFLICT (id) DO NOTHING;
INSERT INTO alert_engine_cursor (id) VALUES (1) ON CONFLICT (id) DO NOTHING;
INSERT INTO data_migration (id) VALUES (1) ON CONFLICT (id) DO NOTHING;
+44
View File
@@ -0,0 +1,44 @@
-- S3-compatible storage for RouterOS backups
CREATE TABLE IF NOT EXISTS backup_storage_settings (
id BIGINT PRIMARY KEY CHECK (id = 1),
provider TEXT NOT NULL DEFAULT 'local' CHECK (provider IN ('local', 's3')),
s3_endpoint TEXT NOT NULL DEFAULT '',
s3_region TEXT NOT NULL DEFAULT 'us-east-1',
s3_bucket TEXT NOT NULL DEFAULT '',
s3_prefix TEXT NOT NULL DEFAULT 'mikrotik',
s3_access_key_id TEXT NOT NULL DEFAULT '',
s3_secret_access_key TEXT NOT NULL DEFAULT '',
s3_force_path_style BOOLEAN NOT NULL DEFAULT TRUE,
keep_local_copy BOOLEAN NOT NULL DEFAULT TRUE,
last_test_at TIMESTAMPTZ,
last_test_error TEXT,
updated_at TIMESTAMPTZ NOT NULL DEFAULT now()
);
INSERT INTO backup_storage_settings (id)
VALUES (1)
ON CONFLICT (id) DO NOTHING;
ALTER TABLE backup_entries
ADD COLUMN IF NOT EXISTS storage TEXT NOT NULL DEFAULT 'local';
ALTER TABLE backup_entries
ADD COLUMN IF NOT EXISTS s3_key TEXT;
ALTER TABLE backup_entries
ADD COLUMN IF NOT EXISTS s3_etag TEXT;
ALTER TABLE backup_entries
ADD COLUMN IF NOT EXISTS upload_error TEXT;
DO $$
BEGIN
IF NOT EXISTS (
SELECT 1 FROM pg_constraint WHERE conname = 'backup_entries_storage_check'
) THEN
ALTER TABLE backup_entries
ADD CONSTRAINT backup_entries_storage_check
CHECK (storage IN ('local', 's3', 'both'));
END IF;
END $$;
+19
View File
@@ -0,0 +1,19 @@
-- GeoLite2 mmdb (страна/ASN для netflow): настройки автообновления зеркала P3TERX
CREATE TABLE IF NOT EXISTS geoip_settings (
id BIGINT PRIMARY KEY CHECK (id = 1),
enabled BOOLEAN NOT NULL DEFAULT TRUE,
update_interval_sec INTEGER NOT NULL DEFAULT 604800,
last_check_at TIMESTAMPTZ,
last_success_at TIMESTAMPTZ,
last_error TEXT,
country_build_at TIMESTAMPTZ,
asn_build_at TIMESTAMPTZ,
etags_json JSONB NOT NULL DEFAULT '{}'::jsonb,
created_at TIMESTAMPTZ NOT NULL DEFAULT now(),
updated_at TIMESTAMPTZ NOT NULL DEFAULT now()
);
INSERT INTO geoip_settings (id)
VALUES (1)
ON CONFLICT (id) DO NOTHING;
+28
View File
@@ -0,0 +1,28 @@
-- Statistics cube: hour + daily facts (server × iface × country × service × ASN).
-- Compact types. FILLFACTOR/autovacuum нельзя на partitioned parent (PG 42809) —
-- задаются на листовых партициях в ensurePartitionFor.
-- Retention: DROP partitions only (see PARTITION_SPECS).
CREATE TABLE IF NOT EXISTS flow_hour_facts (
server_id BIGINT NOT NULL REFERENCES servers(id) ON DELETE CASCADE,
bucket_at TIMESTAMPTZ NOT NULL,
iface TEXT NOT NULL,
country CHAR(2) NOT NULL,
service TEXT NOT NULL,
asn INTEGER NOT NULL,
bytes BIGINT NOT NULL DEFAULT 0,
packets BIGINT NOT NULL DEFAULT 0,
PRIMARY KEY (server_id, bucket_at, iface, country, service, asn)
) PARTITION BY RANGE (bucket_at);
CREATE TABLE IF NOT EXISTS flow_daily_facts (
server_id BIGINT NOT NULL REFERENCES servers(id) ON DELETE CASCADE,
day DATE NOT NULL,
iface TEXT NOT NULL,
country CHAR(2) NOT NULL,
service TEXT NOT NULL,
asn INTEGER NOT NULL,
bytes BIGINT NOT NULL DEFAULT 0,
packets BIGINT NOT NULL DEFAULT 0,
PRIMARY KEY (server_id, day, iface, country, service, asn)
) PARTITION BY RANGE (day);
+16
View File
@@ -0,0 +1,16 @@
-- История desired/actual снапшотов managed-секций (фильтры, рекурсивные маршруты).
-- Retention — prune в сервисе (последние 50 на пару server+section).
CREATE TABLE IF NOT EXISTS config_revisions (
id TEXT PRIMARY KEY,
server_id BIGINT NOT NULL REFERENCES servers(id) ON DELETE CASCADE,
section TEXT NOT NULL,
source TEXT NOT NULL,
fingerprint TEXT NOT NULL,
payload JSONB NOT NULL DEFAULT '[]'::jsonb,
note TEXT,
created_at TIMESTAMPTZ NOT NULL DEFAULT now()
);
CREATE INDEX IF NOT EXISTS idx_config_revisions_server_section_created
ON config_revisions (server_id, section, created_at DESC);
@@ -0,0 +1,7 @@
-- IPFIX postNAT (IANA 225/226) + postNAPT ports (IANA 227/228) from MikroTik Traffic Flow.
-- Needed to rebuild facts with the same internet dest as the network map.
ALTER TABLE flow_buckets ADD COLUMN IF NOT EXISTS nat_src INET;
ALTER TABLE flow_buckets ADD COLUMN IF NOT EXISTS nat_dst INET;
ALTER TABLE flow_buckets ADD COLUMN IF NOT EXISTS nat_src_port INTEGER NOT NULL DEFAULT 0;
ALTER TABLE flow_buckets ADD COLUMN IF NOT EXISTS nat_dst_port INTEGER NOT NULL DEFAULT 0;
+10 -3
View File
@@ -12,15 +12,21 @@
"db:migrate": "drizzle-kit migrate",
"db:studio": "drizzle-kit studio",
"db:migrate-from-sqlite": "tsx src/scripts/migrate-sqlite-to-pg.ts",
"facts:rebuild": "tsx src/scripts/rebuild-flow-facts.ts",
"test:auth": "tsx src/lib/permissions.test.ts && tsx src/plugins/auth.smoke.test.ts",
"test:wireguard": "npx tsx src/services/wireguard-config.test.ts",
"test:traffic-rate": "tsx src/services/traffic-rate.test.ts",
"test:traffic-flow": "tsx src/services/traffic-flow-parse.test.ts && tsx src/services/traffic-flow-map-exporter.test.ts && tsx src/services/traffic-flow-ifaces.test.ts && tsx src/services/traffic-flow-dedup.test.ts && tsx src/services/traffic-flow-planes.test.ts && tsx src/services/traffic-flow-ip.test.ts && tsx src/services/traffic-flow-classify.test.ts && tsx src/services/traffic-flow-ripe.test.ts && tsx src/services/traffic-flow-brands.test.ts && tsx src/services/traffic-flow-ingest.test.ts && tsx src/services/traffic-flow-analytics.test.ts && tsx src/services/traffic-flow-map-hops.test.ts && tsx src/services/traffic-flow-purge.test.ts",
"test:traffic-flow": "tsx src/services/traffic-flow-parse.test.ts && tsx src/services/traffic-flow-map-exporter.test.ts && tsx src/services/traffic-flow-ifaces.test.ts && tsx src/services/traffic-flow-ifindex.test.ts && tsx src/services/traffic-flow-dedup.test.ts && tsx src/services/traffic-flow-planes.test.ts && tsx src/services/traffic-flow-ip.test.ts && tsx src/services/traffic-flow-dest.test.ts && tsx src/services/traffic-flow-classify.test.ts && tsx src/services/traffic-flow-ripe.test.ts && tsx src/services/traffic-flow-brands.test.ts && tsx src/services/traffic-flow-ingest.test.ts && tsx src/services/traffic-flow-analytics.test.ts && tsx src/services/traffic-flow-map-hops.test.ts && tsx src/services/traffic-flow-purge.test.ts && tsx src/services/traffic-flow-geoip.test.ts && tsx src/services/traffic-flow-facts.test.ts && tsx src/services/traffic-flow-facts-filter.test.ts && tsx src/services/traffic-flow-facts-rebuild.test.ts && tsx src/services/statistics-aggregate.test.ts",
"test:users": "tsx src/modules/users/iface-type.test.ts && tsx src/modules/users/bindings.test.ts",
"test:pg": "tsx src/db/sql-bind.test.ts && tsx src/db/sqlite-json.test.ts && tsx src/db/pg-schema.test.ts",
"test": "npm run test:alert-engine && npm run test:auth && npm run test:wireguard && npm run test:traffic-rate && npm run test:traffic-flow && npm run test:users && npm run test:pg"
"test:pg": "tsx src/db/sql-bind.test.ts && tsx src/db/sqlite-json.test.ts && tsx src/db/traffic-flags.test.ts && tsx src/db/pg-schema.test.ts && tsx src/services/config-revisions.test.ts",
"test:config-sync": "tsx src/services/config-apply-plan.test.ts && tsx src/services/entity-snapshots.test.ts",
"test:backups": "tsx src/services/s3-backup-client.test.ts",
"test:live-maps": "tsx src/services/ospf-route-parse.test.ts && tsx src/services/vxlan-live.test.ts && tsx src/services/containers-live.test.ts",
"test": "npm run test:alert-engine && npm run test:auth && npm run test:wireguard && npm run test:traffic-rate && npm run test:traffic-flow && npm run test:users && npm run test:pg && npm run test:backups && npm run test:live-maps && npm run test:config-sync",
"test:geoip": "tsx src/services/traffic-flow-geoip.test.ts"
},
"dependencies": {
"@aws-sdk/client-s3": "^3.888.0",
"@fastify/cors": "^11.2.0",
"@fastify/jwt": "^10.2.2",
"@fastify/type-provider-zod": "^1.0.0",
@@ -31,6 +37,7 @@
"drizzle-orm": "^0.45.2",
"fastify": "^5.8.5",
"fastify-plugin": "^5.1.0",
"maxmind": "^5.0.7",
"pg": "^8.23.0",
"undici": "^8.1.0",
"zod": "^4.4.1"
+10 -1
View File
@@ -1,7 +1,7 @@
import { pool } from "./index.js"
import { applySqlMigrations } from "./migrate.js"
import { dropExpiredPartitions, ensurePartitionsAround } from "./partitions.js"
import { importSqliteToPostgres, shouldImportSqlite } from "./sqlite-import.js"
import { importSqliteToPostgres, shouldImportSqlite, sqliteFileLooksPresent } from "./sqlite-import.js"
import { env } from "../config.js"
const ETL_LOCK = 8723101
@@ -19,6 +19,15 @@ export async function initDatabase(): Promise<void> {
console.log(
`SQLite → PostgreSQL: готово за ${report.durationMs}ms, таблиц ${Object.keys(report.tables).length}`,
)
} else {
const marker = await pool.query<{ sqlite_imported_at: string | null }>(
`SELECT sqlite_imported_at FROM data_migration WHERE id = 1`,
)
if (!marker.rows[0]?.sqlite_imported_at && !sqliteFileLooksPresent(env.DATABASE_PATH)) {
console.warn(
`SQLite → PostgreSQL: файл ${env.DATABASE_PATH} не найден, база после wipe остаётся пустой (defaults settings)`,
)
}
}
} finally {
try {
+23 -12
View File
@@ -1,9 +1,9 @@
import { readFileSync } from "node:fs"
import { readdirSync, readFileSync } from "node:fs"
import { dirname, join } from "node:path"
import { fileURLToPath } from "node:url"
import type { Pool } from "pg"
const MIGRATION_ID = "0000_postgresql"
const FIRST_MIGRATION = "0000_postgresql.sql"
function migrationsDir(): string {
const here = dirname(fileURLToPath(import.meta.url))
@@ -14,7 +14,7 @@ function migrationsDir(): string {
]
for (const dir of candidates) {
try {
readFileSync(join(dir, `${MIGRATION_ID}.sql`), "utf8")
readFileSync(join(dir, FIRST_MIGRATION), "utf8")
return dir
} catch {
/* try next */
@@ -23,6 +23,10 @@ function migrationsDir(): string {
throw new Error("Не найден backend/drizzle/0000_postgresql.sql")
}
function migrationId(file: string): string {
return file.replace(/\.sql$/i, "")
}
export async function applySqlMigrations(pool: Pool): Promise<void> {
await pool.query(`
CREATE TABLE IF NOT EXISTS schema_migrations (
@@ -30,14 +34,21 @@ export async function applySqlMigrations(pool: Pool): Promise<void> {
applied_at TIMESTAMPTZ NOT NULL DEFAULT now()
)
`)
const { rows } = await pool.query<{ id: string }>(
`SELECT id FROM schema_migrations WHERE id = $1`,
[MIGRATION_ID],
const dir = migrationsDir()
const files = readdirSync(dir)
.filter((f) => /^\d{4}_.+\.sql$/i.test(f))
.sort((a, b) => a.localeCompare(b))
const applied = new Set(
(await pool.query<{ id: string }>(`SELECT id FROM schema_migrations`)).rows.map((r) => r.id),
)
if (rows.length > 0) return
const sql = readFileSync(join(migrationsDir(), `${MIGRATION_ID}.sql`), "utf8")
await pool.query(sql)
await pool.query(`INSERT INTO schema_migrations (id) VALUES ($1) ON CONFLICT (id) DO NOTHING`, [
MIGRATION_ID,
])
for (const file of files) {
const id = migrationId(file)
if (applied.has(id)) continue
const sql = readFileSync(join(dir, file), "utf8")
await pool.query(sql)
await pool.query(
`INSERT INTO schema_migrations (id) VALUES ($1) ON CONFLICT (id) DO NOTHING`,
[id],
)
}
}
+65 -2
View File
@@ -8,11 +8,19 @@ export interface PartitionSpec {
keepDays: number
}
/** Leaf-only: PG forbids storage params on partitioned parents (SQLSTATE 42809). */
const FACT_LEAF_STORAGE =
"fillfactor = 70, autovacuum_vacuum_scale_factor = 0.05, autovacuum_vacuum_cost_limit = 2000"
const FACT_PARENTS = new Set(["flow_hour_facts", "flow_daily_facts"])
export const PARTITION_SPECS: PartitionSpec[] = [
{ parent: "flow_buckets", kind: "day", keepDays: 4 },
{ parent: "flow_minute_stats", kind: "day", keepDays: 4 },
{ parent: "flow_minute_dims", kind: "day", keepDays: 4 },
{ parent: "flow_daily_dims", kind: "month", keepDays: 420 },
{ parent: "flow_hour_facts", kind: "day", keepDays: 3 },
{ parent: "flow_daily_facts", kind: "month", keepDays: 420 },
{ parent: "traffic_samples", kind: "week", keepDays: 21 },
{ parent: "servers_rest_ping_samples", kind: "week", keepDays: 35 },
{ parent: "uptime_probe_samples", kind: "week", keepDays: 21 },
@@ -21,6 +29,37 @@ export const PARTITION_SPECS: PartitionSpec[] = [
{ parent: "internet_path_snapshots", kind: "week", keepDays: 21 },
]
const WEEK_SLACK_DAYS = 7
async function resolveKeepDays(pool: Pool): Promise<Map<string, number>> {
const map = new Map(PARTITION_SPECS.map((s) => [s.parent, s.keepDays]))
try {
const traffic = await pool.query<{ retention_days: number }>(
`SELECT retention_days FROM traffic_settings WHERE id = 1`,
)
const td = Number(traffic.rows[0]?.retention_days)
if (Number.isFinite(td) && td > 0) map.set("traffic_samples", td + WEEK_SLACK_DAYS)
const uptime = await pool.query<{ retention_days: number }>(
`SELECT retention_days FROM uptime_settings WHERE id = 1`,
)
const ud = Number(uptime.rows[0]?.retention_days)
if (Number.isFinite(ud) && ud > 0) {
map.set("uptime_probe_samples", ud + WEEK_SLACK_DAYS)
map.set("uptime_resource_samples", ud + WEEK_SLACK_DAYS)
}
const path = await pool.query<{ retention_days: number }>(
`SELECT retention_days FROM internet_path_settings WHERE id = 1`,
)
const pd = Number(path.rows[0]?.retention_days)
if (Number.isFinite(pd) && pd > 0) map.set("internet_path_snapshots", pd + WEEK_SLACK_DAYS)
} catch {
/* settings may be absent mid-migration */
}
return map
}
function utcDate(d: Date): Date {
return new Date(Date.UTC(d.getUTCFullYear(), d.getUTCMonth(), d.getUTCDate()))
}
@@ -79,13 +118,35 @@ export async function ensurePartitionFor(
await pool.query(
`CREATE TABLE IF NOT EXISTS ${name} PARTITION OF ${parent} FOR VALUES FROM ('${from}') TO ('${to}')`,
)
if (FACT_PARENTS.has(parent)) {
await pool.query(`ALTER TABLE ${name} SET (${FACT_LEAF_STORAGE})`)
}
return name
}
export async function ensurePartitionsBetween(
pool: Pool,
parent: string,
kind: PartitionKind,
from: Date,
to: Date,
): Promise<void> {
const start = from.getTime() <= to.getTime() ? from : to
const end = from.getTime() <= to.getTime() ? to : from
for (let t = new Date(start.getTime()); t <= end; ) {
await ensurePartitionFor(pool, parent, kind, t)
if (kind === "day") t = addUtcDays(t, 1)
else if (kind === "week") t = addUtcDays(t, 7)
else t = new Date(Date.UTC(t.getUTCFullYear(), t.getUTCMonth() + 1, 1))
}
}
export async function ensurePartitionsAround(pool: Pool, around = new Date()): Promise<void> {
const keepDays = await resolveKeepDays(pool)
for (const spec of PARTITION_SPECS) {
const keep = keepDays.get(spec.parent) ?? spec.keepDays
const daysAhead = spec.kind === "month" ? 40 : spec.kind === "week" ? 21 : 8
const start = addUtcDays(around, -spec.keepDays)
const start = addUtcDays(around, -keep)
const end = addUtcDays(around, daysAhead)
for (let t = new Date(start.getTime()); t < end; ) {
await ensurePartitionFor(pool, spec.parent, spec.kind, t)
@@ -97,8 +158,10 @@ export async function ensurePartitionsAround(pool: Pool, around = new Date()): P
}
export async function dropExpiredPartitions(pool: Pool, around = new Date()): Promise<void> {
const keepDays = await resolveKeepDays(pool)
for (const spec of PARTITION_SPECS) {
const cutoff = addUtcDays(around, -spec.keepDays)
const keep = keepDays.get(spec.parent) ?? spec.keepDays
const cutoff = addUtcDays(around, -keep)
const { rows } = await pool.query<{ relname: string }>(
`SELECT c.relname
FROM pg_inherits i
+119
View File
@@ -1,6 +1,7 @@
import assert from "node:assert/strict"
import { withPgOrSkip } from "../test/pg.js"
import { dbQuery, pool } from "./index.js"
import { applySqlMigrations } from "./migrate.js"
import { ensurePartitionFor } from "./partitions.js"
if (!(await withPgOrSkip())) {
@@ -8,6 +9,8 @@ if (!(await withPgOrSkip())) {
process.exit(0)
}
await applySqlMigrations(pool)
{
const { rows } = await dbQuery<{ n: string }>(`SELECT COUNT(*)::text AS n FROM servers`)
assert.ok(rows[0])
@@ -80,4 +83,120 @@ if (!(await withPgOrSkip())) {
assert.equal(arrayAsPgArrayFailed, true, "JS array must not be bound as jsonb without stringify")
}
{
const { rows } = await dbQuery<{ attname: string }>(`
SELECT a.attname
FROM pg_index i
JOIN unnest(i.indkey) WITH ORDINALITY AS k(attnum, ord) ON true
JOIN pg_attribute a ON a.attrelid = i.indrelid AND a.attnum = k.attnum
WHERE i.indrelid = 'traffic_samples'::regclass AND i.indisprimary
ORDER BY k.ord
`)
assert.deepEqual(
rows.map((r) => r.attname),
["server_id", "sampled_at", "interface_name", "peer_public_key"],
"traffic_samples PK без id",
)
}
{
const { rows } = await dbQuery<{ column_name: string }>(`
SELECT column_name FROM information_schema.columns
WHERE table_schema = 'public' AND table_name = 'traffic_samples'
`)
const cols = new Set(rows.map((r) => r.column_name))
assert.equal(cols.has("id"), false)
assert.equal(cols.has("running"), false)
assert.equal(cols.has("disabled"), false)
assert.equal(cols.has("flags"), true)
}
{
const { rows } = await dbQuery<{ column_name: string; udt_name: string }>(`
SELECT column_name, udt_name
FROM information_schema.columns
WHERE table_schema = 'public' AND table_name = 'flow_buckets'
AND column_name IN ('src', 'dst', 'next_hop', 'proto', 'nat_src', 'nat_dst', 'nat_src_port', 'nat_dst_port')
`)
const by = Object.fromEntries(rows.map((r) => [r.column_name, r.udt_name]))
assert.equal(by.src, "inet")
assert.equal(by.dst, "inet")
assert.equal(by.next_hop, "inet")
assert.equal(by.proto, "int2")
assert.equal(by.nat_src, "inet")
assert.equal(by.nat_dst, "inet")
assert.equal(by.nat_src_port, "int4")
assert.equal(by.nat_dst_port, "int4")
}
{
const { rows } = await dbQuery<{ indexdef: string }>(`
SELECT indexdef FROM pg_indexes
WHERE schemaname = 'public' AND tablename = 'traffic_samples'
`)
const defs = rows.map((r) => r.indexdef.toLowerCase())
assert.equal(defs.some((d) => d.includes("using brin")), false, "нет BRIN на traffic_samples")
assert.equal(
defs.filter((d) => d.includes("idx_traffic_samples_server_iface_time")).length,
1,
"один btree (server_id, interface_name, sampled_at)",
)
assert.equal(defs.some((d) => d.includes("idx_traffic_samples_server_time")), false)
}
{
const { rows } = await dbQuery<{ column_name: string }>(`
SELECT column_name FROM information_schema.columns
WHERE table_schema = 'public' AND table_name = 'uptime_resource_samples'
`)
const cols = new Set(rows.map((r) => r.column_name))
assert.equal(cols.has("board_name"), false)
assert.equal(cols.has("ros_version"), false)
}
{
const mig = await dbQuery<{ id: string }>(
`SELECT id FROM schema_migrations WHERE id = '0002_compact_schema'`,
)
assert.equal(mig.rows.length, 1, "0002 применена")
await dbQuery(`INSERT INTO servers (name, host) VALUES ('pg-wipe-idempotent', '127.0.0.1')`)
await applySqlMigrations(pool)
const still = await dbQuery<{ n: string }>(
`SELECT COUNT(*)::text AS n FROM servers WHERE name = 'pg-wipe-idempotent'`,
)
assert.equal(still.rows[0]?.n, "1", "повторный applySqlMigrations не wipe")
await dbQuery(`DELETE FROM servers WHERE name = 'pg-wipe-idempotent'`)
}
{
const mig = await dbQuery<{ id: string }>(
`SELECT id FROM schema_migrations WHERE id = '0006_config_revisions'`,
)
assert.equal(mig.rows.length, 1, "0006 применена")
const { rows } = await dbQuery<{ column_name: string; udt_name: string }>(`
SELECT column_name, udt_name FROM information_schema.columns
WHERE table_schema = 'public' AND table_name = 'config_revisions'
`)
const by = Object.fromEntries(rows.map((r) => [r.column_name, r.udt_name]))
assert.equal(by.payload, "jsonb")
assert.equal(by.fingerprint, "text")
assert.equal(by.section, "text")
}
{
const mig = await dbQuery<{ id: string }>(
`SELECT id FROM schema_migrations WHERE id = '0007_flow_buckets_nat'`,
)
assert.equal(mig.rows.length, 1, "0007 применена")
}
{
const marker = await dbQuery<{ sqlite_imported_at: string | null }>(
`SELECT sqlite_imported_at FROM data_migration WHERE id = 1`,
)
assert.ok(marker.rows[0], "data_migration singleton после wipe")
}
console.log("pg-schema.test.ts: ok")
+94 -21
View File
@@ -5,10 +5,12 @@ import {
date,
doublePrecision,
index,
inet,
integer,
jsonb,
pgTable,
primaryKey,
smallint,
text,
timestamp,
uniqueIndex,
@@ -91,6 +93,19 @@ export const filterRules = pgTable("filter_rules", {
index("idx_filter_rules_server_sort").on(t.serverId, t.sortOrder),
])
export const configRevisions = pgTable("config_revisions", {
id: text("id").primaryKey(),
serverId: intPkRef().references(() => servers.id, { onDelete: "cascade" }),
section: text("section", { enum: ["filters", "recursive-routes", "firewall", "wireguard", "ipsec", "gre"] }).notNull(),
source: text("source", { enum: ["apply", "rollback", "observed", "copy"] }).notNull(),
fingerprint: text("fingerprint").notNull(),
payload: jsonb("payload").$type<unknown>().notNull().default(sql`'[]'::jsonb`),
note: text("note"),
createdAt: ts("created_at").notNull().defaultNow(),
}, (t) => [
index("idx_config_revisions_server_section_created").on(t.serverId, t.section, t.createdAt),
])
export const recursiveRoutes = pgTable("recursive_routes", {
id: idIdentity().primaryKey(),
serverId: intPkRef().references(() => servers.id, { onDelete: "cascade" }),
@@ -135,15 +150,13 @@ export const serversApiPingSettings = pgTable("servers_api_ping_settings", {
})
export const serversRestPingSamples = pgTable("servers_rest_ping_samples", {
id: idIdentity(),
serverId: intPkRef().references(() => servers.id, { onDelete: "cascade" }),
sampledAt: ts("sampled_at").notNull(),
ok: boolean("ok").notNull(),
latencyMs: integer("latency_ms"),
error: text("error"),
}, (t) => [
primaryKey({ columns: [t.id, t.sampledAt] }),
index("idx_servers_rest_ping_samples_server_id").on(t.serverId, t.sampledAt),
primaryKey({ columns: [t.serverId, t.sampledAt] }),
])
export const trafficFlowSettings = pgTable("traffic_flow_settings", {
@@ -208,21 +221,55 @@ export const flowDailyDims = pgTable("flow_daily_dims", {
index("idx_flow_daily_dims_day").on(t.day, t.dim),
])
/** Hour-grain traffic cube for statistics (≤48h). No secondary indexes. */
export const flowHourFacts = pgTable("flow_hour_facts", {
serverId: bigint("server_id", { mode: "number" }).notNull()
.references(() => servers.id, { onDelete: "cascade" }),
bucketAt: ts("bucket_at").notNull(),
iface: text("iface").notNull(),
country: text("country").notNull(),
service: text("service").notNull(),
asn: integer("asn").notNull(),
bytes: bigint("bytes", { mode: "number" }).notNull().default(0),
packets: bigint("packets", { mode: "number" }).notNull().default(0),
}, (t) => [
primaryKey({ columns: [t.serverId, t.bucketAt, t.iface, t.country, t.service, t.asn] }),
])
/** Daily-grain traffic cube for statistics (long window). No secondary indexes. */
export const flowDailyFacts = pgTable("flow_daily_facts", {
serverId: bigint("server_id", { mode: "number" }).notNull()
.references(() => servers.id, { onDelete: "cascade" }),
day: date("day", { mode: "string" }).notNull(),
iface: text("iface").notNull(),
country: text("country").notNull(),
service: text("service").notNull(),
asn: integer("asn").notNull(),
bytes: bigint("bytes", { mode: "number" }).notNull().default(0),
packets: bigint("packets", { mode: "number" }).notNull().default(0),
}, (t) => [
primaryKey({ columns: [t.serverId, t.day, t.iface, t.country, t.service, t.asn] }),
])
export const flowBuckets = pgTable("flow_buckets", {
serverId: intPkRef().references(() => servers.id, { onDelete: "cascade" }),
bucketAt: ts("bucket_at").notNull(),
src: text("src").notNull(),
dst: text("dst").notNull(),
proto: integer("proto").notNull().default(0),
src: inet("src").notNull(),
dst: inet("dst").notNull(),
proto: smallint("proto").notNull().default(0),
srcPort: integer("src_port").notNull().default(0),
dstPort: integer("dst_port").notNull().default(0),
bytes: bigint("bytes", { mode: "number" }).notNull().default(0),
packets: bigint("packets", { mode: "number" }).notNull().default(0),
inIface: text("in_iface").notNull().default(""),
outIface: text("out_iface").notNull().default(""),
nextHop: text("next_hop").notNull().default(""),
nextHop: inet("next_hop"),
flowStartMs: bigint("flow_start_ms", { mode: "number" }).notNull().default(0),
flowEndMs: bigint("flow_end_ms", { mode: "number" }).notNull().default(0),
natSrc: inet("nat_src"),
natDst: inet("nat_dst"),
natSrcPort: integer("nat_src_port").notNull().default(0),
natDstPort: integer("nat_dst_port").notNull().default(0),
}, (t) => [
primaryKey({
name: "flow_buckets_pkey",
@@ -231,6 +278,20 @@ export const flowBuckets = pgTable("flow_buckets", {
index("idx_flow_buckets_server_time").on(t.serverId, t.bucketAt),
])
export const geoipSettings = pgTable("geoip_settings", {
id: idSingleton(),
enabled: boolean("enabled").notNull().default(true),
updateIntervalSec: integer("update_interval_sec").notNull().default(604800),
lastCheckAt: ts("last_check_at"),
lastSuccessAt: ts("last_success_at"),
lastError: text("last_error"),
countryBuildAt: ts("country_build_at"),
asnBuildAt: ts("asn_build_at"),
etagsJson: jsonb("etags_json").notNull().default(sql`'{}'::jsonb`),
createdAt: ts("created_at").notNull().defaultNow(),
updatedAt: ts("updated_at").notNull().defaultNow(),
})
export const flowIpMeta = pgTable("flow_ip_meta", {
prefix: text("prefix").primaryKey(),
asn: integer("asn").notNull().default(0),
@@ -249,7 +310,6 @@ export const flowAsnMeta = pgTable("flow_asn_meta", {
})
export const trafficSamples = pgTable("traffic_samples", {
id: idIdentity(),
serverId: intPkRef().references(() => servers.id, { onDelete: "cascade" }),
interfaceName: text("interface_name").notNull(),
peerPublicKey: text("peer_public_key").notNull().default(""),
@@ -258,11 +318,9 @@ export const trafficSamples = pgTable("traffic_samples", {
txBytes: bigint("tx_bytes", { mode: "number" }).notNull().default(0),
rxBps: bigint("rx_bps", { mode: "number" }).notNull().default(0),
txBps: bigint("tx_bps", { mode: "number" }).notNull().default(0),
running: boolean("running").notNull().default(false),
disabled: boolean("disabled").notNull().default(false),
flags: smallint("flags").notNull().default(0),
}, (t) => [
primaryKey({ columns: [t.id, t.sampledAt] }),
index("idx_traffic_samples_server_time").on(t.serverId, t.sampledAt),
primaryKey({ columns: [t.serverId, t.sampledAt, t.interfaceName, t.peerPublicKey] }),
index("idx_traffic_samples_server_iface_time").on(t.serverId, t.interfaceName, t.sampledAt),
])
@@ -302,19 +360,16 @@ export const uptimeProbes = pgTable("uptime_probes", {
])
export const uptimeProbeSamples = pgTable("uptime_probe_samples", {
id: idIdentity(),
probeId: text("probe_id").notNull().references(() => uptimeProbes.id, { onDelete: "cascade" }),
sampledAt: ts("sampled_at").notNull(),
rttMs: integer("rtt_ms"),
lossPct: integer("loss_pct").notNull().default(0),
status: text("status", { enum: ["up", "warn", "down"] }).notNull().default("down"),
}, (t) => [
primaryKey({ columns: [t.id, t.sampledAt] }),
index("idx_uptime_probe_samples_probe_time").on(t.probeId, t.sampledAt),
primaryKey({ columns: [t.probeId, t.sampledAt] }),
])
export const uptimeResourceSamples = pgTable("uptime_resource_samples", {
id: idIdentity(),
serverId: intPkRef().references(() => servers.id, { onDelete: "cascade" }),
sampledAt: ts("sampled_at").notNull(),
status: text("status", { enum: ["online", "offline"] }).notNull().default("offline"),
@@ -324,11 +379,8 @@ export const uptimeResourceSamples = pgTable("uptime_resource_samples", {
freeHddSpace: bigint("free_hdd_space", { mode: "number" }).notNull().default(0),
totalHddSpace: bigint("total_hdd_space", { mode: "number" }).notNull().default(0),
uptimeSeconds: bigint("uptime_seconds", { mode: "number" }).notNull().default(0),
boardName: text("board_name").notNull().default(""),
rosVersion: text("ros_version").notNull().default(""),
}, (t) => [
primaryKey({ columns: [t.id, t.sampledAt] }),
index("idx_uptime_resource_samples_server_time").on(t.serverId, t.sampledAt),
primaryKey({ columns: [t.serverId, t.sampledAt] }),
])
export const uptimeSpeedProbes = pgTable("uptime_speed_probes", {
@@ -429,6 +481,22 @@ export const backupScheduleSettings = pgTable("backup_schedule_settings", {
updatedAt: ts("updated_at").notNull().defaultNow(),
})
export const backupStorageSettings = pgTable("backup_storage_settings", {
id: idSingleton(),
provider: text("provider", { enum: ["local", "s3"] }).notNull().default("local"),
s3Endpoint: text("s3_endpoint").notNull().default(""),
s3Region: text("s3_region").notNull().default("us-east-1"),
s3Bucket: text("s3_bucket").notNull().default(""),
s3Prefix: text("s3_prefix").notNull().default("mikrotik"),
s3AccessKeyId: text("s3_access_key_id").notNull().default(""),
s3SecretAccessKey: text("s3_secret_access_key").notNull().default(""),
s3ForcePathStyle: boolean("s3_force_path_style").notNull().default(true),
keepLocalCopy: boolean("keep_local_copy").notNull().default(true),
lastTestAt: ts("last_test_at"),
lastTestError: text("last_test_error"),
updatedAt: ts("updated_at").notNull().defaultNow(),
})
export const backupEntries = pgTable("backup_entries", {
id: text("id").primaryKey(),
serverId: bigint("server_id", { mode: "number" })
@@ -438,6 +506,10 @@ export const backupEntries = pgTable("backup_entries", {
sizeBytes: bigint("size_bytes", { mode: "number" }).notNull(),
kind: text("kind", { enum: ["manual", "auto"] }).notNull().default("manual"),
notes: text("notes"),
storage: text("storage", { enum: ["local", "s3", "both"] }).notNull().default("local"),
s3Key: text("s3_key"),
s3Etag: text("s3_etag"),
uploadError: text("upload_error"),
createdAt: ts("created_at").notNull(),
}, (t) => [
uniqueIndex("idx_backup_entries_filename").on(t.filename),
@@ -644,7 +716,7 @@ export const userInterfaceBindings = pgTable("user_interface_bindings", {
userId: text("user_id").notNull().references(() => appUsers.id, { onDelete: "cascade" }),
serverId: intPkRef().references(() => servers.id, { onDelete: "cascade" }),
interfaceName: text("interface_name").notNull(),
interfaceType: text("interface_type", { enum: ["ether", "gre", "wg", "other"] })
interfaceType: text("interface_type", { enum: ["ether", "gre", "wg", "ipsec", "other"] })
.notNull().default("other"),
peerPublicKey: text("peer_public_key").notNull().default(""),
peerName: text("peer_name").notNull().default(""),
@@ -678,6 +750,7 @@ export type ServerInsert = typeof servers.$inferInsert
export type Snapshot = typeof serverSnapshots.$inferSelect
export type SnapshotInsert = typeof serverSnapshots.$inferInsert
export type FilterRuleRow = typeof filterRules.$inferSelect
export type ConfigRevisionRow = typeof configRevisions.$inferSelect
export type RecursiveRouteRow = typeof recursiveRoutes.$inferSelect
export type TrafficSettingsRow = typeof trafficSettings.$inferSelect
export type TrafficFlowSettingsRow = typeof trafficFlowSettings.$inferSelect
+81 -33
View File
@@ -3,7 +3,8 @@ import { existsSync, readFileSync } from "node:fs"
import Database from "better-sqlite3"
import type { Pool } from "pg"
import { env } from "../config.js"
import { ensurePartitionFor, specForParent } from "./partitions.js"
import { ensurePartitionsBetween, specForParent } from "./partitions.js"
import { encodeTrafficFlags } from "./traffic-flags.js"
export interface ImportReport {
sqlitePath: string
@@ -15,7 +16,9 @@ export interface ImportReport {
const SNAPSHOT_RETENTION_DAYS = 14
type ColKind = "ts" | "date" | "bool" | "json" | "json-null" | "bigint-id" | "int" | "text" | "num"
type ColKind = "ts" | "date" | "bool" | "json" | "json-null" | "bigint-id" | "int" | "text" | "num" | "flags" | "inet"
const INSERT_CHUNK = 1000
interface TableCopy {
table: string
@@ -78,6 +81,12 @@ const TABLES: TableCopy[] = [
["server_ids_json", "json"], ["last_run_at", "ts"], ["last_duration_ms", "int"],
["last_error", "text"], ["updated_at", "ts"],
]},
{ table: "backup_storage_settings", upsert: true, columns: [
["id", "int"], ["provider", "text"], ["s3_endpoint", "text"], ["s3_region", "text"],
["s3_bucket", "text"], ["s3_prefix", "text"], ["s3_access_key_id", "text"],
["s3_secret_access_key", "text"], ["s3_force_path_style", "bool"], ["keep_local_copy", "bool"],
["last_test_at", "ts"], ["last_test_error", "text"], ["updated_at", "ts"],
]},
{ table: "internet_path_settings", upsert: true, columns: [
["id", "int"], ["enabled", "bool"], ["interval_sec", "int"], ["retention_days", "int"],
["last_collected_at", "ts"], ["last_duration_ms", "int"], ["last_error", "text"],
@@ -103,19 +112,20 @@ const TABLES: TableCopy[] = [
["free_memory", "int"], ["total_memory", "int"], ["identity_name", "text"],
["raw_interfaces", "json-null"], ["raw_ip_addresses", "json-null"],
]},
{ table: "traffic_samples", identity: true, timeCol: "sampled_at", retentionDays: 14, columns: [
["id", "int"], ["server_id", "int"], ["interface_name", "text"], ["peer_public_key", "text"],
{ table: "traffic_samples", timeCol: "sampled_at", retentionDays: 14, columns: [
["server_id", "int"], ["interface_name", "text"], ["peer_public_key", "text"],
["sampled_at", "ts"], ["rx_bytes", "int"], ["tx_bytes", "int"], ["rx_bps", "int"], ["tx_bps", "int"],
["running", "bool"], ["disabled", "bool"],
["flags", "flags"],
]},
{ table: "servers_rest_ping_samples", identity: true, timeCol: "sampled_at", retentionDays: 30, columns: [
["id", "int"], ["server_id", "int"], ["sampled_at", "ts"], ["ok", "bool"], ["latency_ms", "int"], ["error", "text"],
{ table: "servers_rest_ping_samples", timeCol: "sampled_at", retentionDays: 30, columns: [
["server_id", "int"], ["sampled_at", "ts"], ["ok", "bool"], ["latency_ms", "int"], ["error", "text"],
]},
{ table: "flow_buckets", timeCol: "bucket_at", retentionDays: 2, columns: [
["server_id", "int"], ["bucket_at", "ts"], ["src", "text"], ["dst", "text"], ["proto", "int"],
["server_id", "int"], ["bucket_at", "ts"], ["src", "inet"], ["dst", "inet"], ["proto", "int"],
["src_port", "int"], ["dst_port", "int"], ["bytes", "int"], ["packets", "int"],
["in_iface", "text"], ["out_iface", "text"], ["next_hop", "text"],
["in_iface", "text"], ["out_iface", "text"], ["next_hop", "inet"],
["flow_start_ms", "int"], ["flow_end_ms", "int"],
["nat_src", "inet"], ["nat_dst", "inet"], ["nat_src_port", "int"], ["nat_dst_port", "int"],
]},
{ table: "flow_minute_stats", timeCol: "bucket_at", retentionDays: 3, columns: [
["server_id", "int"], ["bucket_at", "ts"], ["bytes", "int"], ["packets", "int"],
@@ -139,13 +149,13 @@ const TABLES: TableCopy[] = [
["target", "text"], ["probe_filter", "text"], ["enabled", "bool"], ["interval_sec", "int"],
["show_on_dashboard", "bool"], ["sort_order", "int"], ["created_at", "ts"], ["updated_at", "ts"],
]},
{ table: "uptime_probe_samples", identity: true, timeCol: "sampled_at", retentionDays: 14, columns: [
["id", "int"], ["probe_id", "text"], ["sampled_at", "ts"], ["rtt_ms", "int"], ["loss_pct", "int"], ["status", "text"],
{ table: "uptime_probe_samples", timeCol: "sampled_at", retentionDays: 14, columns: [
["probe_id", "text"], ["sampled_at", "ts"], ["rtt_ms", "int"], ["loss_pct", "int"], ["status", "text"],
]},
{ table: "uptime_resource_samples", identity: true, timeCol: "sampled_at", retentionDays: 14, columns: [
["id", "int"], ["server_id", "int"], ["sampled_at", "ts"], ["status", "text"], ["cpu_load", "int"],
{ table: "uptime_resource_samples", timeCol: "sampled_at", retentionDays: 14, columns: [
["server_id", "int"], ["sampled_at", "ts"], ["status", "text"], ["cpu_load", "int"],
["free_memory", "int"], ["total_memory", "int"], ["free_hdd_space", "int"], ["total_hdd_space", "int"],
["uptime_seconds", "int"], ["board_name", "text"], ["ros_version", "text"],
["uptime_seconds", "int"],
]},
{ table: "uptime_speed_probes", columns: [
["id", "text"], ["src_server_id", "int"], ["dst_server_id", "int"], ["src_interface", "text"],
@@ -269,7 +279,20 @@ function parseJson(value: unknown, fallback: unknown): unknown {
}
}
function coerce(kind: ColKind, value: unknown, strict: boolean, rejects: string[], ctx: string): unknown {
function parseInet(value: unknown): string | null {
const s = String(value ?? "").trim()
if (!s) return null
return s
}
function coerce(
kind: ColKind,
value: unknown,
strict: boolean,
rejects: string[],
ctx: string,
row?: Record<string, unknown>,
): unknown {
switch (kind) {
case "ts":
return parseTs(value, strict, rejects, ctx)
@@ -303,6 +326,10 @@ function coerce(kind: ColKind, value: unknown, strict: boolean, rejects: string[
return Number(value)
case "text":
return value == null ? "" : String(value)
case "flags":
return encodeTrafficFlags(parseBool(row?.running), parseBool(row?.disabled))
case "inet":
return parseInet(value)
default:
return value == null ? null : String(value)
}
@@ -326,6 +353,35 @@ async function setval(pool: Pool, table: string): Promise<void> {
)
}
function placeholderFor(kind: ColKind, index: number): string {
if (kind === "json" || kind === "json-null") return `$${index}::jsonb`
if (kind === "inet") return `$${index}::inet`
return `$${index}`
}
async function precreatePartitions(
sqlite: Database.Database,
pool: Pool,
spec: TableCopy,
where: string,
): Promise<void> {
const part = specForParent(spec.table)
if (!part || !spec.timeCol) return
const bounds = sqlite.prepare(
`SELECT MIN(${spec.timeCol}) AS a, MAX(${spec.timeCol}) AS b FROM ${spec.table}${where}`,
).get() as { a?: unknown; b?: unknown }
if (bounds?.a == null || bounds?.b == null) return
const isDate = spec.columns.find((c) => c[0] === spec.timeCol)?.[1] === "date"
const fromIso = isDate
? `${String(bounds.a).slice(0, 10)}T00:00:00Z`
: parseTs(bounds.a, false, [], spec.table)
const toIso = isDate
? `${String(bounds.b).slice(0, 10)}T00:00:00Z`
: parseTs(bounds.b, false, [], spec.table)
if (!fromIso || !toIso) return
await ensurePartitionsBetween(pool, spec.table, part.kind, new Date(fromIso), new Date(toIso))
}
async function copyTable(
sqlite: Database.Database,
pool: Pool,
@@ -341,28 +397,27 @@ async function copyTable(
where = ` WHERE ${spec.timeCol} >= '${cutoff.replace("T", " ").slice(0, 19)}' OR ${spec.timeCol} >= '${cutoff}'`
}
const total = (sqlite.prepare(`SELECT COUNT(*) AS c FROM ${spec.table}${where}`).get() as { c: number }).c
const part = specForParent(spec.table)
await precreatePartitions(sqlite, pool, spec, where)
const cols = spec.columns.map(([c]) => c)
const placeholders = spec.columns
.map(([, kind], i) => (kind === "json" || kind === "json-null" ? `$${i + 1}::jsonb` : `$${i + 1}`))
.join(", ")
const conflictSql = spec.upsert
? `ON CONFLICT (id) DO UPDATE SET ${cols.filter((c) => c !== "id").map((c) => `${c} = EXCLUDED.${c}`).join(", ")}`
: `ON CONFLICT DO NOTHING`
const insertSql = `INSERT INTO ${spec.table} (${cols.join(", ")}) VALUES (${placeholders}) ${conflictSql}`
let copied = 0
let skipped = 0
const stmt = sqlite.prepare(`SELECT * FROM ${spec.table}${where}`)
const batch: unknown[][] = []
const flush = async () => {
if (batch.length === 0) return
const n = spec.columns.length
const valuesSql = batch.map((_, i) =>
`(${spec.columns.map(([, kind], j) => placeholderFor(kind, i * n + j + 1)).join(", ")})`,
).join(", ")
const insertSql = `INSERT INTO ${spec.table} (${cols.join(", ")}) VALUES ${valuesSql} ${conflictSql}`
const client = await pool.connect()
try {
await client.query("BEGIN")
for (const values of batch) {
await client.query(insertSql, values)
copied += 1
}
await client.query(insertSql, batch.flat())
copied += batch.length
await client.query("COMMIT")
} catch (err) {
await client.query("ROLLBACK")
@@ -375,22 +430,15 @@ async function copyTable(
}
for (const row of stmt.iterate() as Iterable<Record<string, unknown>>) {
try {
if (part && spec.timeCol) {
const raw = row[spec.timeCol]
const ts = spec.columns.find((c) => c[0] === spec.timeCol)?.[1] === "date"
? `${String(raw).slice(0, 10)}T00:00:00Z`
: parseTs(raw, false, opts.rejects, spec.table)
if (ts) await ensurePartitionFor(pool, spec.table, part.kind, new Date(ts))
}
const values = spec.columns.map(([col, kind]) =>
coerce(kind, row[col], opts.strict, opts.rejects, `${spec.table}.${col}`),
coerce(kind, row[col], opts.strict, opts.rejects, `${spec.table}.${col}`, row),
)
if (spec.table === "certificate_issue_jobs" && values[4] == null) {
skipped += 1
continue
}
batch.push(values)
if (batch.length >= 200) await flush()
if (batch.length >= INSERT_CHUNK) await flush()
} catch (err) {
skipped += 1
const msg = `${spec.table}: ${err instanceof Error ? err.message : String(err)}`
+19
View File
@@ -0,0 +1,19 @@
import assert from "node:assert/strict"
import {
decodeTrafficSampleFlags,
encodeTrafficFlags,
trafficFlagDisabled,
trafficFlagRunning,
} from "./traffic-flags.js"
assert.equal(encodeTrafficFlags(true, false), 1)
assert.equal(encodeTrafficFlags(false, true), 2)
assert.equal(encodeTrafficFlags(true, true), 3)
assert.equal(encodeTrafficFlags(false, false), 0)
assert.equal(trafficFlagRunning(1), true)
assert.equal(trafficFlagDisabled(1), false)
assert.deepEqual(decodeTrafficSampleFlags(1), { running: true, disabled: false })
assert.deepEqual(decodeTrafficSampleFlags(0), { running: false, disabled: false })
assert.deepEqual(decodeTrafficSampleFlags(undefined), { running: false, disabled: false })
console.log("traffic-flags.test.ts: ok")
+24
View File
@@ -0,0 +1,24 @@
export const TRAFFIC_FLAG_RUNNING = 1
export const TRAFFIC_FLAG_DISABLED = 2
export function encodeTrafficFlags(running: boolean, disabled: boolean): number {
return (running ? TRAFFIC_FLAG_RUNNING : 0) | (disabled ? TRAFFIC_FLAG_DISABLED : 0)
}
export function trafficFlagRunning(flags: number | null | undefined): boolean {
return ((Number(flags) || 0) & TRAFFIC_FLAG_RUNNING) !== 0
}
export function trafficFlagDisabled(flags: number | null | undefined): boolean {
return ((Number(flags) || 0) & TRAFFIC_FLAG_DISABLED) !== 0
}
export function decodeTrafficSampleFlags(flags: number | null | undefined): {
running: boolean
disabled: boolean
} {
return {
running: trafficFlagRunning(flags),
disabled: trafficFlagDisabled(flags),
}
}
+14
View File
@@ -14,6 +14,7 @@ import filtersRoutes from "./routes/filters.js"
import recursiveRoutes from "./routes/recursive-routes.js"
import trafficRoutes from "./routes/traffic.js"
import trafficFlowRoutes from "./routes/traffic-flow.js"
import geoipRoutes from "./routes/geoip.js"
import serversApiPingRoutes from "./routes/servers-api-ping.js"
import uptimeRoutes from "./routes/uptime.js"
import networkRoutes from "./routes/network.js"
@@ -28,10 +29,16 @@ import certificatesRoutes from "./routes/certificates.js"
import systemDatabaseRoutes from "./routes/system-database.js"
import eventsRoutes from "./routes/events.js"
import wireguardRoutes from "./routes/wireguard.js"
import ipsecRoutes from "./routes/ipsec.js"
import vxlanRoutes from "./routes/vxlan.js"
import containersRoutes from "./routes/containers.js"
import firewallRoutes from "./routes/firewall.js"
import greRoutes from "./routes/gre.js"
import usersRoutes from "./routes/users.js"
import statisticsRoutes from "./routes/statistics.js"
import { refreshScheduler, stopScheduler } from "./services/scheduler.js"
import { getFlowWorkerHealth, startTrafficFlowListener, stopTrafficFlowListener } from "./services/traffic-flow-ingest.js"
import { initGeoip } from "./services/traffic-flow-geoip.js"
const eventLoopDelay = monitorEventLoopDelay({ resolution: 20 })
eventLoopDelay.enable()
@@ -116,6 +123,7 @@ export async function buildApp(opts?: {
await app.register(recursiveRoutes, { prefix: "/api" })
await app.register(trafficRoutes, { prefix: "/api" })
await app.register(trafficFlowRoutes, { prefix: "/api" })
await app.register(geoipRoutes, { prefix: "/api" })
await app.register(serversApiPingRoutes, { prefix: "/api" })
await app.register(uptimeRoutes, { prefix: "/api" })
await app.register(networkRoutes, { prefix: "/api" })
@@ -130,11 +138,17 @@ export async function buildApp(opts?: {
await app.register(systemDatabaseRoutes, { prefix: "/api" })
await app.register(eventsRoutes, { prefix: "/api" })
await app.register(wireguardRoutes, { prefix: "/api" })
await app.register(ipsecRoutes, { prefix: "/api" })
await app.register(vxlanRoutes, { prefix: "/api" })
await app.register(containersRoutes, { prefix: "/api" })
await app.register(firewallRoutes, { prefix: "/api" })
await app.register(greRoutes, { prefix: "/api" })
await app.register(usersRoutes, { prefix: "/api" })
await app.register(statisticsRoutes, { prefix: "/api" })
if (opts?.startScheduler !== false) {
await refreshScheduler()
await initGeoip()
await startTrafficFlowListener()
app.addHook("onClose", async () => {
stopScheduler()
+9 -1
View File
@@ -18,7 +18,7 @@ assert.equal(
"mm:settings:admin",
)
assert.equal(
permissionForRequest("GET", "/api/traffic/servers/1/live"),
permissionForRequest("GET", "/api/statistics"),
"mm:traffic:read",
)
assert.equal(
@@ -41,6 +41,14 @@ assert.equal(
permissionForRequest("GET", "/api/firewall/all"),
"mm:network:read",
)
assert.equal(
permissionForRequest("GET", "/api/gre/tunnels"),
"mm:network:read",
)
assert.equal(
permissionForRequest("POST", "/api/gre/tunnels"),
"mm:network:write",
)
assert.equal(
permissionForRequest("GET", "/api/users"),
"mm:users:read",
+7 -3
View File
@@ -107,7 +107,7 @@ const RULES: Rule[] = [
},
{
methods: ["GET"],
match: (p) => p.startsWith("/api/traffic"),
match: (p) => p.startsWith("/api/traffic") || p.startsWith("/api/statistics"),
permission: "mm:traffic:read",
},
{
@@ -155,7 +155,9 @@ const RULES: Rule[] = [
p.startsWith("/api/internet-path") ||
p.startsWith("/api/exec") ||
p.startsWith("/api/wireguard") ||
p.startsWith("/api/firewall"),
p.startsWith("/api/ipsec") ||
p.startsWith("/api/firewall") ||
p.startsWith("/api/gre"),
permission: "mm:network:read",
},
{
@@ -168,7 +170,9 @@ const RULES: Rule[] = [
p.startsWith("/api/internet-path") ||
p.startsWith("/api/exec") ||
p.startsWith("/api/wireguard") ||
p.startsWith("/api/firewall"),
p.startsWith("/api/ipsec") ||
p.startsWith("/api/firewall") ||
p.startsWith("/api/gre"),
permission: "mm:network:write",
},
]
+27 -1
View File
@@ -19,5 +19,31 @@ export function managedComment(label: string): string {
}
export function managedRecursiveComment(comment?: string | null): string {
return comment ? `${PRODUCT_NAME}:recursive ${comment}` : `${PRODUCT_NAME}:recursive`
const stripped = stripManagedRecursiveComment(comment ?? "")
return stripped ? `${PRODUCT_NAME}:recursive ${stripped}` : `${PRODUCT_NAME}:recursive`
}
const LEGACY_RECURSIVE_PREFIX = /^recursive:\s*/i
export function stripManagedRecursiveComment(comment: string): string {
const value = comment.trim()
if (!value) return ""
const managedPrefixes = [
`${PRODUCT_NAME}:recursive`,
`${LEGACY_PRODUCT_NAME}:recursive`,
]
for (const prefix of managedPrefixes) {
if (value.startsWith(prefix)) return value.slice(prefix.length).trim()
}
if (LEGACY_RECURSIVE_PREFIX.test(value)) {
return value.replace(LEGACY_RECURSIVE_PREFIX, "").trim()
}
return value
}
/** Owned recursive route: MM/legacy prefix or old `recursive:` mask. */
export function isOwnedRecursiveComment(comment: string | undefined): boolean {
if (!comment) return false
const value = comment.trim()
return hasManagedRecursiveComment(value) || LEGACY_RECURSIVE_PREFIX.test(value)
}
+2 -1
View File
@@ -1,4 +1,4 @@
export type InterfaceType = "ether" | "gre" | "wg" | "other"
export type InterfaceType = "ether" | "gre" | "wg" | "ipsec" | "other"
export function mapRosInterfaceType(raw: string | undefined | null, name?: string): InterfaceType {
const t = String(raw ?? "").trim().toLowerCase()
@@ -10,6 +10,7 @@ export function mapRosInterfaceType(raw: string | undefined | null, name?: strin
const n = String(name ?? "").trim().toLowerCase()
if (n.startsWith("gre") || n.includes("gre-tunnel")) return "gre"
if (n.startsWith("wg-") || n.startsWith("wireguard")) return "wg"
if (n.startsWith("ipsec")) return "ipsec"
if (n.startsWith("ether") || n.startsWith("sfp")) return "ether"
return "other"
}
+7 -1
View File
@@ -28,7 +28,7 @@ export function peerDisplayName(opts: {
return truncPeerKey(opts.publicKey)
}
/** Ether/GRE — пустой ключ. WG — обязательный public-key. */
/** Ether/GRE — пустой ключ. WG — обязательный public-key. IPsec — CN сертификата клиента. */
export function normalizeBindingPeer(
type: InterfaceType,
peerPublicKey: string | undefined,
@@ -40,5 +40,11 @@ export function normalizeBindingPeer(
}
return key
}
if (type === "ipsec") {
if (!key) {
throw new PeerBindError("Для IPsec укажите клиента (CN сертификата)", 400)
}
return key
}
return ""
}
@@ -13,6 +13,7 @@ import type {
} from "@mmapp/contracts/users"
import { db } from "../../../db/index.js"
import { parseJsonArray } from "../../../db/json.js"
import { decodeTrafficSampleFlags } from "../../../db/traffic-flags.js"
import { servers, trafficSamples } from "../../../db/schema.js"
import {
createBindingRow,
@@ -42,6 +43,7 @@ import {
peerDisplayName,
} from "../peer-bind.js"
import { listWireGuardPeersForCatalog } from "../../../services/wireguard-live.js"
import { listIpsecClientsForCatalog } from "../../../services/ipsec-live.js"
export class UsersServiceError extends Error {
constructor(
@@ -270,8 +272,7 @@ export async function listInterfaceCatalog(serverId: number): Promise<CatalogInt
.select({
interfaceName: trafficSamples.interfaceName,
peerPublicKey: trafficSamples.peerPublicKey,
running: trafficSamples.running,
disabled: trafficSamples.disabled,
flags: trafficSamples.flags,
})
.from(trafficSamples)
.where(eq(trafficSamples.serverId, serverId)))
@@ -281,11 +282,12 @@ export async function listInterfaceCatalog(serverId: number): Promise<CatalogInt
for (const r of rows) {
if (seen.has(r.interfaceName)) continue
seen.add(r.interfaceName)
const decoded = decodeTrafficSampleFlags(r.flags)
ifaces.push({
name: r.interfaceName,
type: mapRosInterfaceType("", r.interfaceName),
running: Boolean(r.running),
disabled: Boolean(r.disabled),
running: decoded.running,
disabled: decoded.disabled,
})
}
}
@@ -304,7 +306,10 @@ export async function listInterfaceCatalog(serverId: number): Promise<CatalogInt
peersByIface.set(peer.interfaceName, list)
}
return ifaces.map((iface) => {
// IKEv2-клиенты — псевдо-интерфейс «ipsec-vpn» с пирами = клиенты (ключ = CN сертификата).
const ipsecLive = await listIpsecClientsForCatalog(serverId)
const entries: CatalogInterface[] = ifaces.map((iface) => {
const ifaceBind = bindings.find((b) => b.interfaceName === iface.name && !(b.peerPublicKey ?? ""))
const owner = ifaceBind ? usersById.get(ifaceBind.userId) : undefined
const base: CatalogInterface = {
@@ -334,7 +339,37 @@ export async function listInterfaceCatalog(serverId: number): Promise<CatalogInt
}
}),
}
}).sort((a, b) => a.name.localeCompare(b.name))
})
if (ipsecLive.clients.length > 0) {
const ifaceBind = bindings.find((b) => b.interfaceName === "ipsec-vpn" && !(b.peerPublicKey ?? ""))
const owner = ifaceBind ? usersById.get(ifaceBind.userId) : undefined
entries.push({
name: "ipsec-vpn",
type: "ipsec",
running: true,
disabled: false,
boundUserId: ifaceBind?.userId ?? null,
boundUserLogin: owner?.login ?? null,
peersError: ipsecLive.error,
peers: ipsecLive.clients.map((c) => {
const cn = c.commonName ?? c.name
const bind = bindings.find((b) => b.interfaceName === "ipsec-vpn" && b.peerPublicKey === cn)
const peerOwner = bind ? usersById.get(bind.userId) : undefined
return {
publicKey: cn,
name: c.name,
comment: c.comment ?? "",
allowedIps: c.staticIp ? [c.staticIp] : [],
latestHandshake: c.activeSince,
boundUserId: bind?.userId ?? null,
boundUserLogin: peerOwner?.login ?? null,
}
}),
})
}
return entries.sort((a, b) => a.name.localeCompare(b.name))
}
export { parseRawInterfaces, mapRosInterfaceType }
+71 -10
View File
@@ -1,20 +1,26 @@
import { randomUUID } from "node:crypto"
import { readFile } from "node:fs/promises"
import path from "node:path"
import { z } from "zod"
import type { FastifyPluginAsyncZod } from "@fastify/type-provider-zod"
import { putBackupScheduleSettingsSchema } from "@mmapp/contracts/backups"
import {
putBackupScheduleSettingsSchema,
putBackupStorageSettingsSchema,
} from "@mmapp/contracts/backups"
import { listServersRead } from "../modules/servers/service/servers-service.js"
import { appendEvent } from "../modules/events/service/events-service.js"
import { refreshScheduler } from "../services/scheduler.js"
import {
deleteBackupRecord,
getBackupById,
getBackupsDir,
getBackupScheduleSettings,
getBackupStorageSettings,
listBackups,
readBackupContent,
restoreBackupToDevice,
runBackupForServer,
syncBackupsFromS3,
testBackupStorageConnection,
updateBackupScheduleSettings,
updateBackupStorageSettings,
type BackupMeta,
} from "../services/backup-service.js"
@@ -97,6 +103,39 @@ const backupsRoutes: FastifyPluginAsyncZod = async (app) => {
return reply.send(result)
})
app.get("/backups/storage", async (_req, reply) => {
return reply.send(await getBackupStorageSettings())
})
app.put("/backups/storage", async (req, reply) => {
const parsed = putBackupStorageSettingsSchema.safeParse(req.body ?? {})
if (!parsed.success) {
return reply.status(400).send({ error: "Некорректное тело запроса", details: parsed.error.flatten() })
}
return reply.send(await updateBackupStorageSettings(parsed.data))
})
app.post("/backups/storage/test", async (_req, reply) => {
try {
return reply.send(await testBackupStorageConnection())
} catch (err) {
return reply.status(400).send({
error: err instanceof Error ? err.message : "Ошибка проверки S3",
settings: await getBackupStorageSettings(),
})
}
})
app.post("/backups/storage/sync", async (_req, reply) => {
try {
return reply.send(await syncBackupsFromS3())
} catch (err) {
return reply.status(400).send({
error: err instanceof Error ? err.message : "Ошибка синхронизации S3",
})
}
})
app.post("/backups/create", { schema: { body: CreateBackupBodySchema } }, async (req, reply) => {
const inputIds = req.body.serverIds.map((x) => String(x))
const notes = req.body.notes?.trim() || undefined
@@ -169,12 +208,34 @@ const backupsRoutes: FastifyPluginAsyncZod = async (app) => {
app.get("/backups/:id/download", { schema: { params: BackupIdParamSchema } }, async (req, reply) => {
const hit = await getBackupById(req.params.id)
if (!hit) return reply.status(404).send({ error: "Бэкап не найден" })
const filePath = path.join(getBackupsDir(), hit.filename)
const content = await readFile(filePath, "utf8").catch(() => null)
if (content == null) return reply.status(404).send({ error: "Файл бэкапа не найден" })
reply.header("Content-Type", "text/plain; charset=utf-8")
reply.header("Content-Disposition", `attachment; filename="${hit.filename}"`)
return reply.send(content)
try {
const content = await readBackupContent(hit)
reply.header("Content-Type", "text/plain; charset=utf-8")
reply.header("Content-Disposition", `attachment; filename="${hit.filename}"`)
return reply.send(content)
} catch {
return reply.status(404).send({ error: "Файл бэкапа не найден" })
}
})
app.post("/backups/:id/restore", { schema: { params: BackupIdParamSchema } }, async (req, reply) => {
try {
const result = await restoreBackupToDevice(req.params.id)
await appendEvent({
level: "warning",
eventType: "backups.restore",
sourceModule: "backups",
title: "Восстановление бэкапа",
message: `${result.filename}${result.serverName}`,
entityType: "backup",
entityId: req.params.id,
})
return reply.send(result)
} catch (err) {
return reply.status(400).send({
error: err instanceof Error ? err.message : "Не удалось восстановить бэкап",
})
}
})
app.delete("/backups/:id", { schema: { params: BackupIdParamSchema } }, async (req, reply) => {
+58
View File
@@ -0,0 +1,58 @@
import { z } from "zod"
import type { FastifyPluginAsyncZod } from "@fastify/type-provider-zod"
import {
getEnabledServerById,
listContainers,
listContainersForServer,
removeContainer,
restartContainer,
startContainer,
stopContainer,
} from "../services/containers-live.js"
import { ServerIdParamSchema, type ServerIdParams } from "../types/server.js"
const RosIdBodySchema = z.object({
rosId: z.string().min(1),
})
type RosIdBody = z.infer<typeof RosIdBodySchema>
type MutateFn = typeof startContainer
const containersRoutes: FastifyPluginAsyncZod = async (app) => {
app.get("/containers", async (_req, reply) => {
const containers = await listContainers()
return reply.send({ containers })
})
app.get("/servers/:id/containers", { schema: { params: ServerIdParamSchema } }, async (req, reply) => {
const params = req.params as ServerIdParams
const server = await getEnabledServerById(params.id)
if (!server) return reply.status(404).send({ error: "Server not found" })
const containers = await listContainersForServer(server)
return reply.send({ containers })
})
function registerMutate(path: string, fn: MutateFn) {
app.post(path, { schema: { params: ServerIdParamSchema, body: RosIdBodySchema } }, async (req, reply) => {
const params = req.params as ServerIdParams
const body = req.body as RosIdBody
const server = await getEnabledServerById(params.id)
if (!server) return reply.status(404).send({ error: "Server not found" })
try {
await fn(server, body.rosId)
return reply.send({ ok: true })
} catch (err) {
return reply.status(502).send({
error: err instanceof Error ? err.message : "Ошибка RouterOS",
})
}
})
}
registerMutate("/servers/:id/containers/start", startContainer)
registerMutate("/servers/:id/containers/stop", stopContainer)
registerMutate("/servers/:id/containers/restart", restartContainer)
registerMutate("/servers/:id/containers/remove", removeContainer)
}
export default containersRoutes
+204 -259
View File
@@ -1,14 +1,20 @@
import { and, asc, eq, inArray } from "drizzle-orm"
import { and, asc, eq } from "drizzle-orm"
import { z } from "zod"
import type { FastifyPluginAsyncZod } from "@fastify/type-provider-zod"
import { db } from "../db/index.js"
import { filterRules, recursiveRoutes, servers } from "../db/schema.js"
import { MikrotikClient } from "../services/mikrotik.js"
import { parseDbServerId } from "../utils/server-id.js"
import { appendEvent } from "../modules/events/service/events-service.js"
import { managedComment } from "../managed-markers.js"
import { planBgpInApply } from "../services/config-apply-plan.js"
import {
hasManagedCommentPrefix,
managedComment,
} from "../managed-markers.js"
appendRevisionIfChanged,
canonicalFilterRules,
getRevisionById,
listRevisions,
type ConfigRevisionSource,
} from "../services/config-revisions.js"
type ServerRow = typeof servers.$inferSelect
@@ -296,47 +302,6 @@ async function resolveRouteTargets(serverId: number, rule: ApiFilterRule): Promi
return { gateway: rule.gateway, outIface: tid }
}
function normalizeCommunity(c: string): string {
return (c ?? "").trim()
}
/** Одинаковый эффект на роутере при одинаковой community (blackhole vs gateway + out-interface) */
async function ruleEffectSignature(serverId: number, r: ApiFilterRule): Promise<string> {
if (r.action === "blackhole") return `bh:${normalizeCommunity(r.community)}`
const { gateway, outIface } = await resolveRouteTargets(serverId, r)
return `rt:${normalizeCommunity(r.community)}:${gateway}:${outIface}`
}
export type FilterRouterCompareStatus = "synced" | "drift" | "missing"
async function compareDbRulesWithRouter(
serverId: number,
dbRules: ApiFilterRule[],
remoteRules: ApiFilterRule[],
): Promise<Record<string, FilterRouterCompareStatus>> {
const remoteSigByComm = new Map<string, string>()
for (const rr of remoteRules) {
const c = normalizeCommunity(rr.community)
if (!remoteSigByComm.has(c)) {
remoteSigByComm.set(c, await ruleEffectSignature(serverId, rr))
}
}
const out: Record<string, FilterRouterCompareStatus> = {}
for (const dr of dbRules) {
const c = normalizeCommunity(dr.community)
const sigD = await ruleEffectSignature(serverId, dr)
const sigR = remoteSigByComm.get(c)
if (sigR === undefined) {
out[c] = "missing"
} else if (sigR !== sigD) {
out[c] = "drift"
} else {
out[c] = "synced"
}
}
return out
}
async function toRouterRuleBody(serverId: number, rules: ApiFilterRule[]): Promise<string> {
if (rules.length === 0) return ""
// Группируем по эффекту (action + gateway + out-interface). Communities с одним и тем же
@@ -421,44 +386,80 @@ async function replaceDbRules(serverId: number, rules: ApiFilterRule[]) {
)
}
const filtersRoutes: FastifyPluginAsyncZod = async (app) => {
/** Сравнение правил в БД с живым bgp-in на MikroTik (один запрос API к роутеру) */
app.get("/filters/router-compare", async (req, reply) => {
const q = req.query as { serverId?: string | number }
const serverId = parseDbServerId(q.serverId)
if (serverId === null) {
return reply.status(400).send({ error: "serverId is required" })
}
async function cacheRulesetsForServer(serverId: number): Promise<ApiFilterRule[]> {
const rows = await db
.select()
.from(filterRules)
.where(eq(filterRules.serverId, serverId))
.orderBy(asc(filterRules.sortOrder))
return rows.map((r) => ({
id: String(r.id),
community: r.community,
communityName: r.communityName ?? undefined,
action: r.action,
gateway: r.gateway,
gatewayTunnelId: r.gatewayTunnelId,
description: r.description,
}))
}
const server = (await db.select().from(servers).where(eq(servers.id, serverId)).limit(1))[0]
if (!server) return reply.status(404).send({ error: "Server not found" })
async function applyFiltersToServer(
server: ServerRow,
rules: ApiFilterRule[],
source: ConfigRevisionSource,
): Promise<{ pushed: number; action: string; conflictsRemoved: number }> {
const client = MikrotikClient.fromServer(server)
const existing = await client.get<RosFilterRule[]>("/routing/filter/rule")
const plan = planBgpInApply(existing, rules.length)
const managedCommentValue = managedComment(server.name || server.host)
try {
const remote = await fetchServerFilters(server)
const rows = await db
.select()
.from(filterRules)
.where(eq(filterRules.serverId, serverId))
.orderBy(asc(filterRules.sortOrder))
if (plan.action === "patch" && plan.managedId) {
const ruleBody = await toRouterRuleBody(server.id, rules)
await client.patch(
`/routing/filter/rule/${encodeURIComponent(plan.managedId)}`,
{
chain: "bgp-in",
comment: managedCommentValue,
rule: ruleBody,
disabled: "no",
},
)
} else if (plan.action === "create") {
const ruleBody = await toRouterRuleBody(server.id, rules)
await client.post("/routing/filter/rule/add", {
chain: "bgp-in",
comment: managedCommentValue,
rule: ruleBody,
})
} else if (plan.action === "delete" && plan.managedId) {
await client.delete(
`/routing/filter/rule/${encodeURIComponent(plan.managedId)}`,
)
}
const dbRules: ApiFilterRule[] = rows.map(r => ({
id: String(r.id),
community: r.community,
communityName: r.communityName ?? undefined,
action: r.action,
gateway: r.gateway,
gatewayTunnelId: r.gatewayTunnelId,
description: r.description,
}))
for (const id of plan.conflictIds) {
await client.delete(`/routing/filter/rule/${encodeURIComponent(id)}`)
}
const byCommunity = await compareDbRulesWithRouter(serverId, dbRules, remote.rules)
return reply.send({ byCommunity })
} catch (err) {
app.log.error({ serverId, err: String(err) }, "filters router-compare failed")
return reply.status(500).send({ error: String(err) })
}
await replaceDbRules(server.id, rules)
await appendRevisionIfChanged({
serverId: server.id,
section: "filters",
source,
payload: canonicalFilterRules(rules),
})
return {
pushed: rules.length,
action: plan.action,
conflictsRemoved: plan.conflictIds.length,
}
}
const RevisionIdParamSchema = z.object({ id: z.string().min(1) })
const filtersRoutes: FastifyPluginAsyncZod = async (app) => {
/** GRE с роутеров: один сервер (?serverId) или все включённые (без query) — для /gre, карты сети */
app.get("/filters/gre-tunnels", async (req, reply) => {
const q = req.query as { serverId?: string | number }
@@ -487,74 +488,103 @@ const filtersRoutes: FastifyPluginAsyncZod = async (app) => {
return reply.send({ tunnels: results.flat() })
})
/** Только правила фильтров из БД (без опроса MikroTik за GRE) */
app.get("/filters/rules", async (_req, reply) => {
/** Без serverId — cache для дашборда. С serverId — live с CHR, cache fallback. */
app.get("/filters/rules", async (req, reply) => {
const q = req.query as { serverId?: string | number }
const serverId = parseDbServerId(q.serverId)
const allServers = await db.select().from(servers).where(eq(servers.enabled, true))
const dbRulesets = await toApiRulesets(allServers)
return reply.send({
rulesets: dbRulesets,
greTunnels: [] as LiveGreTunnel[],
})
})
app.put("/filters/rules", async (req, reply) => {
const body = req.body as { rulesets?: Array<{ serverId: string; rules: ApiFilterRule[] }> }
const payload = body.rulesets ?? []
const serverIds = payload.map(r => Number.parseInt(r.serverId, 10)).filter(Number.isFinite)
if (serverIds.length > 0) {
await db.delete(filterRules).where(inArray(filterRules.serverId, serverIds))
}
for (const rs of payload) {
const sid = Number.parseInt(rs.serverId, 10)
if (!Number.isFinite(sid)) continue
await replaceDbRules(sid, rs.rules ?? [])
}
return reply.send({ ok: true })
})
app.post("/filters/sync/from-router", async (_req, reply) => {
const body = _req.body as { serverId?: string | number } | undefined
const rawServerId = body?.serverId
const serverId = parseDbServerId(rawServerId)
if (serverId === null) {
return reply.status(400).send({ error: "serverId is required" })
const dbRulesets = await toApiRulesets(allServers)
return reply.send({
rulesets: dbRulesets,
greTunnels: [] as LiveGreTunnel[],
live: false,
stale: false,
})
}
const server = (await db.select().from(servers).where(eq(servers.id, serverId)).limit(1))[0]
const server = allServers.find((s) => s.id === serverId)
?? (await db.select().from(servers).where(eq(servers.id, serverId)).limit(1))[0]
if (!server) return reply.status(404).send({ error: "Server not found" })
try {
app.log.info({ serverId, host: server.host }, "Filters sync from router started")
await appendEvent({
level: "info",
eventType: "filters.sync.from_router.started",
sourceModule: "filters",
title: "Синхронизация фильтров запущена",
message: `${server.name || server.host} → БД`,
entityType: "server",
entityId: String(serverId),
})
const remote = await fetchServerFilters(server)
await replaceDbRules(server.id, remote.rules)
app.log.info({ serverId, totalRules: remote.rules.length }, "Filters sync from router completed")
await appendRevisionIfChanged({
serverId: server.id,
section: "filters",
source: "observed",
payload: canonicalFilterRules(remote.rules),
})
const cached = await cacheRulesetsForServer(server.id)
return reply.send({
rulesets: [{ serverId: String(server.id), rules: cached }],
greTunnels: remote.tunnels,
live: true,
stale: false,
})
} catch (err) {
app.log.warn({ serverId, err: String(err) }, "filters live GET failed, serving cache")
const cached = await cacheRulesetsForServer(server.id)
return reply.send({
rulesets: [{ serverId: String(server.id), rules: cached }],
greTunnels: [] as LiveGreTunnel[],
live: false,
stale: true,
error: String(err),
})
}
})
app.put("/filters/rules", async (req, reply) => {
const body = req.body as {
serverId?: string | number
rules?: ApiFilterRule[]
source?: ConfigRevisionSource
}
const serverId = parseDbServerId(body.serverId)
if (serverId === null) return reply.status(400).send({ error: "serverId is required" })
const server = (await db.select().from(servers).where(eq(servers.id, serverId)).limit(1))[0]
if (!server) return reply.status(404).send({ error: "Server not found" })
const rules = body.rules ?? []
const source: ConfigRevisionSource = body.source === "copy" ? "copy" : "apply"
try {
await appendEvent({
level: "info",
eventType: "filters.sync.from_router.done",
eventType: "filters.apply.started",
sourceModule: "filters",
title: "Синхронизация фильтров завершена",
message: `${server.name || server.host}: ${remote.rules.length} правил`,
title: "Применение фильтров на роутер",
message: `${server.name || server.host}: ${rules.length} правил`,
entityType: "server",
entityId: String(serverId),
})
return reply.send({ ok: true, updatedServers: 1, totalRules: remote.rules.length, serverId })
const result = await applyFiltersToServer(server, rules, source)
const cached = await cacheRulesetsForServer(server.id)
await appendEvent({
level: "info",
eventType: "filters.apply.done",
sourceModule: "filters",
title: "Фильтры применены",
message: `${server.name || server.host}: ${result.pushed} правил (${result.action})`,
entityType: "server",
entityId: String(serverId),
})
return reply.send({
ok: true,
serverId,
pushedRules: result.pushed,
action: result.action,
rules: cached,
})
} catch (err) {
app.log.error({ serverId, err: String(err) }, "Filters sync from router failed")
app.log.error({ serverId, err: String(err) }, "filters apply failed")
await appendEvent({
level: "critical",
eventType: "filters.sync.from_router.failed",
eventType: "filters.apply.failed",
sourceModule: "filters",
title: "Ошибка синхронизации фильтров",
title: "Ошибка применения фильтров",
message: `${server.name || server.host}: ${String(err)}`,
entityType: "server",
entityId: String(serverId),
@@ -563,145 +593,60 @@ const filtersRoutes: FastifyPluginAsyncZod = async (app) => {
}
})
app.post("/filters/sync/to-router", async (req, reply) => {
app.get("/filters/revisions", async (req, reply) => {
const q = req.query as { serverId?: string | number }
const serverId = parseDbServerId(q.serverId)
if (serverId === null) return reply.status(400).send({ error: "serverId is required" })
const revisions = await listRevisions(serverId, "filters")
return reply.send({ revisions })
})
app.post("/filters/revisions/:id/restore", {
schema: { params: RevisionIdParamSchema },
}, async (req, reply) => {
const { id } = req.params
const body = req.body as { serverId?: string | number } | undefined
const requestedServerId = parseDbServerId(body?.serverId)
const allServers = await db.select().from(servers).where(eq(servers.enabled, true))
const targetServers = requestedServerId !== null
? allServers.filter(s => s.id === requestedServerId)
: allServers
if (requestedServerId !== null && targetServers.length === 0) {
return reply.status(404).send({ error: "Server not found" })
const rev = await getRevisionById(id)
if (!rev) return reply.status(404).send({ error: "Revision not found" })
if (rev.section !== "filters") return reply.status(400).send({ error: "Revision section mismatch" })
const requested = parseDbServerId(body?.serverId)
if (requested !== null && requested !== rev.serverId) {
return reply.status(400).send({ error: "Revision belongs to another server" })
}
const server = (await db.select().from(servers).where(eq(servers.id, rev.serverId)).limit(1))[0]
if (!server) return reply.status(404).send({ error: "Server not found" })
let updatedServers = 0
let pushedRules = 0
const errors: Array<{ serverId: number; error: string }> = []
await appendEvent({
level: "info",
eventType: "filters.sync.to_router.started",
sourceModule: "filters",
title: "Отправка фильтров на роутеры запущена",
message: `Целевых серверов: ${targetServers.length}`,
payload: { requestedServerId },
})
for (const server of targetServers) {
try {
app.log.info({ serverId: server.id, host: server.host }, "Filters sync to router started")
const client = MikrotikClient.fromServer(server)
const existing = await client.get<RosFilterRule[]>("/routing/filter/rule")
const isInBgpIn = (r: RosFilterRule) =>
(r.chain ?? "").trim().toLowerCase() === "bgp-in"
const managedCommentValue = managedComment(server.name || server.host)
// Уже созданное нами правило — будем PATCH'ить, чтобы сохранить ID/позицию в цепочке.
const managedRule = existing.find(
r => isInBgpIn(r) && hasManagedCommentPrefix(r.comment ?? ""),
)
// Конфликтующие легаси-правила в bgp-in (без нашего comment, но с bgp-communities) —
// удаляем после успешного upsert: иначе старое правило с `else { reject; }`
// отрабатывает первым и перебивает наш upsert.
const conflictIds = existing
.filter(r =>
isInBgpIn(r) &&
!hasManagedCommentPrefix(r.comment ?? "") &&
/bgp-communities/i.test(r.rule ?? ""),
)
.map(r => r[".id"])
.filter((id): id is string => Boolean(id))
const rows = await db.select().from(filterRules)
.where(and(eq(filterRules.serverId, server.id)))
.orderBy(asc(filterRules.sortOrder))
const rules: ApiFilterRule[] = rows.map(r => ({
id: String(r.id),
community: r.community,
communityName: r.communityName ?? undefined,
action: r.action,
gateway: r.gateway,
gatewayTunnelId: r.gatewayTunnelId,
description: r.description,
}))
// Upsert: PATCH существующего managed-правила или POST /add нового.
// Если ошибка — конфликтные правила НЕ удаляем (роутер не остаётся с пустым bgp-in).
// Путь `/routing/filter/rule/add` обязателен: голый POST на коллекцию RouterOS REST
// трактует как «вызов команды» и отдаёт 400 «no such command».
// См. https://help.mikrotik.com/docs/spaces/ROS/pages/47579162/REST+API
if (rules.length > 0) {
const ruleBody = await toRouterRuleBody(server.id, rules)
if (managedRule && managedRule[".id"]) {
await client.patch(
`/routing/filter/rule/${encodeURIComponent(managedRule[".id"])}`,
{
chain: "bgp-in",
comment: managedCommentValue,
rule: ruleBody,
disabled: "no",
},
)
app.log.info({ serverId: server.id, id: managedRule[".id"] }, "bgp-in rule updated")
} else {
await client.post("/routing/filter/rule/add", {
chain: "bgp-in",
comment: managedCommentValue,
rule: ruleBody,
})
app.log.info({ serverId: server.id }, "bgp-in rule created")
}
pushedRules += rules.length
} else if (managedRule && managedRule[".id"]) {
// В БД нет правил → удаляем наш managed-rule на роутере.
await client.delete(
`/routing/filter/rule/${encodeURIComponent(managedRule[".id"])}`,
)
app.log.info({ serverId: server.id }, "bgp-in rule removed (no rules in DB)")
}
for (const id of conflictIds) {
await client.delete(`/routing/filter/rule/${encodeURIComponent(id)}`)
}
updatedServers += 1
app.log.info(
{
serverId: server.id,
mode: managedRule ? "patch" : "create",
conflictsRemoved: conflictIds.length,
pushed: rules.length,
},
"Filters sync to router completed",
)
} catch (err) {
app.log.error({ serverId: server.id, err: String(err) }, "filters sync to-router failed")
errors.push({ serverId: server.id, error: String(err) })
const raw = Array.isArray(rev.payload) ? rev.payload : []
const rules: ApiFilterRule[] = raw.map((item, idx) => {
const r = item as Partial<ApiFilterRule>
return {
id: `rev-${idx}`,
community: r.community ?? "",
communityName: r.communityName,
action: r.action === "blackhole" ? "blackhole" : "route",
gateway: r.gateway ?? "",
gatewayTunnelId: r.gatewayTunnelId ?? "",
description: r.description ?? "",
}
}
})
await appendEvent({
level: errors.length === 0 ? "info" : "warning",
eventType: errors.length === 0 ? "filters.sync.to_router.done" : "filters.sync.to_router.partial",
sourceModule: "filters",
title: errors.length === 0 ? "Отправка фильтров завершена" : "Отправка фильтров завершена с ошибками",
message: `Успешно: ${updatedServers}, ошибок: ${errors.length}, правил: ${pushedRules}`,
payload: {
updatedServers,
pushedRules,
errors,
},
})
return reply.send({
ok: errors.length === 0,
updatedServers,
pushedRules,
errors,
})
try {
const result = await applyFiltersToServer(server, rules, "rollback")
const cached = await cacheRulesetsForServer(server.id)
await appendEvent({
level: "info",
eventType: "filters.rollback.done",
sourceModule: "filters",
title: "Откат фильтров",
message: `${server.name || server.host}: ${result.pushed} правил`,
entityType: "server",
entityId: String(server.id),
})
return reply.send({ ok: true, rules: cached, pushedRules: result.pushed })
} catch (err) {
app.log.error({ serverId: server.id, err: String(err) }, "filters restore failed")
return reply.status(500).send({ error: String(err) })
}
})
}
+73 -1
View File
@@ -2,7 +2,20 @@ import type { FastifyPluginAsyncZod } from "@fastify/type-provider-zod"
import { z } from "zod"
import { MikrotikClient, MikrotikError, encodeRosId, firewallRestPath } from "../services/mikrotik.js"
import { getEnabledServerById } from "../services/wireguard-live.js"
import { listFirewallAll } from "../services/firewall-live.js"
import {
captureFirewallSnapshot,
fetchFirewallState,
listFirewallAll,
} from "../services/firewall-live.js"
import {
captureAndAppendRevision,
listRevisions,
loadRevisionForRestore,
type ConfigRevisionSource,
} from "../services/config-revisions.js"
import { parseFirewallSnapshot, planFirewallRestore } from "../services/entity-snapshots.js"
import { executeRosOps } from "../services/ros-ops.js"
import { parseDbServerId } from "../utils/server-id.js"
import type { FirewallFamily, FirewallTable } from "../types/server.js"
const FamilySchema = z.enum(["ip", "ip6"])
@@ -120,6 +133,20 @@ async function requireServer(serverId: string) {
return await getEnabledServerById(serverId)
}
async function recordFirewall(
server: NonNullable<Awaited<ReturnType<typeof requireServer>>>,
source: ConfigRevisionSource,
) {
await captureAndAppendRevision({
serverId: server.id,
section: "firewall",
source,
capture: () => captureFirewallSnapshot(server),
})
}
const RevisionIdParamSchema = z.object({ id: z.string().min(1) })
const firewallRoutes: FastifyPluginAsyncZod = async (app) => {
app.get("/firewall/all", async (_req, reply) => {
const data = await listFirewallAll()
@@ -138,6 +165,7 @@ const firewallRoutes: FastifyPluginAsyncZod = async (app) => {
const path = firewallRestPath(body.family as FirewallFamily, body.table as FirewallTable)
try {
await client.put(path, ruleToRos(body))
await recordFirewall(server, "apply")
return reply.status(201).send({ ok: true })
} catch (e) {
return reply.status(502).send({ error: `RouterOS: ${rosErr(e)}` })
@@ -156,6 +184,7 @@ const firewallRoutes: FastifyPluginAsyncZod = async (app) => {
const path = `${firewallRestPath(body.family as FirewallFamily, body.table as FirewallTable)}/${encodeRosId(body.rosId)}`
try {
await client.patch(path, ruleToRos(body))
await recordFirewall(server, "apply")
return reply.send({ ok: true })
} catch (e) {
return reply.status(502).send({ error: `RouterOS: ${rosErr(e)}` })
@@ -174,6 +203,7 @@ const firewallRoutes: FastifyPluginAsyncZod = async (app) => {
const path = `${firewallRestPath(body.family, body.table)}/${encodeRosId(body.rosId)}`
try {
await client.patch(path, { disabled: body.disabled ? "yes" : "no" })
await recordFirewall(server, "apply")
return reply.send({ ok: true })
} catch (e) {
return reply.status(502).send({ error: `RouterOS: ${rosErr(e)}` })
@@ -192,6 +222,7 @@ const firewallRoutes: FastifyPluginAsyncZod = async (app) => {
const path = `${firewallRestPath(body.family, body.table)}/${encodeRosId(body.rosId)}`
try {
await client.delete(path)
await recordFirewall(server, "apply")
return reply.send({ ok: true })
} catch (e) {
return reply.status(502).send({ error: `RouterOS: ${rosErr(e)}` })
@@ -213,6 +244,7 @@ const firewallRoutes: FastifyPluginAsyncZod = async (app) => {
numbers: body.rosId,
...(body.destinationRosId ? { destination: body.destinationRosId } : {}),
})
await recordFirewall(server, "apply")
return reply.send({ ok: true })
} catch (e) {
return reply.status(502).send({ error: `RouterOS: ${rosErr(e)}` })
@@ -230,6 +262,7 @@ const firewallRoutes: FastifyPluginAsyncZod = async (app) => {
const client = MikrotikClient.fromServer(server)
try {
await client.put(firewallRestPath(body.family, "address-list"), addressToRos(body))
await recordFirewall(server, "apply")
return reply.status(201).send({ ok: true })
} catch (e) {
return reply.status(502).send({ error: `RouterOS: ${rosErr(e)}` })
@@ -248,6 +281,7 @@ const firewallRoutes: FastifyPluginAsyncZod = async (app) => {
const path = `${firewallRestPath(body.family, "address-list")}/${encodeRosId(body.rosId)}`
try {
await client.patch(path, addressToRos(body))
await recordFirewall(server, "apply")
return reply.send({ ok: true })
} catch (e) {
return reply.status(502).send({ error: `RouterOS: ${rosErr(e)}` })
@@ -266,6 +300,7 @@ const firewallRoutes: FastifyPluginAsyncZod = async (app) => {
const path = `${firewallRestPath(body.family, "address-list")}/${encodeRosId(body.rosId)}`
try {
await client.patch(path, { disabled: body.disabled ? "yes" : "no" })
await recordFirewall(server, "apply")
return reply.send({ ok: true })
} catch (e) {
return reply.status(502).send({ error: `RouterOS: ${rosErr(e)}` })
@@ -284,11 +319,48 @@ const firewallRoutes: FastifyPluginAsyncZod = async (app) => {
const path = `${firewallRestPath(body.family, "address-list")}/${encodeRosId(body.rosId)}`
try {
await client.delete(path)
await recordFirewall(server, "apply")
return reply.send({ ok: true })
} catch (e) {
return reply.status(502).send({ error: `RouterOS: ${rosErr(e)}` })
}
})
app.get("/firewall/revisions", async (req, reply) => {
const q = req.query as { serverId?: string | number }
const serverId = parseDbServerId(q.serverId)
if (serverId === null) return reply.status(400).send({ error: "serverId is required" })
const revisions = await listRevisions(serverId, "firewall")
return reply.send({ revisions })
})
app.post("/firewall/revisions/:id/restore", {
schema: { params: RevisionIdParamSchema },
}, async (req, reply) => {
const { id } = req.params
const body = req.body as { serverId?: string | number } | undefined
const loaded = await loadRevisionForRestore({
id,
section: "firewall",
requestedServerId: parseDbServerId(body?.serverId),
})
if (!loaded.ok) return reply.status(loaded.status).send({ error: loaded.error })
const client = MikrotikClient.fromServer(loaded.server)
try {
const desired = parseFirewallSnapshot(loaded.row.payload)
const state = await fetchFirewallState(loaded.server)
const ops = planFirewallRestore(desired, {
rules: state.liveRules,
addressLists: state.liveLists,
})
await executeRosOps(client, ops)
await recordFirewall(loaded.server, "rollback")
const next = await fetchFirewallState(loaded.server)
return reply.send({ ok: true, rules: next.rules, addressLists: next.addressLists })
} catch (e) {
return reply.status(502).send({ error: `RouterOS: ${rosErr(e)}` })
}
})
}
export default firewallRoutes
+57
View File
@@ -0,0 +1,57 @@
import type { FastifyPluginAsyncZod } from "@fastify/type-provider-zod"
import { geoipSettingsPatchSchema } from "@mmapp/contracts/geoip"
import { refreshScheduler } from "../services/scheduler.js"
import { getGeoipSettings, updateGeoipSettings } from "../services/geoip-settings.js"
import {
GEOIP_ASN_FILE,
GEOIP_COUNTRY_FILE,
geoipReadersStatus,
initGeoip,
} from "../services/traffic-flow-geoip.js"
import { collectGeoipUpdateOnce, getGeoipUpdateState } from "../services/geoip-update-collector.js"
async function buildGeoipStatus() {
await initGeoip()
const readers = geoipReadersStatus()
return {
ready: readers.countryLoaded && readers.asnLoaded,
countryLoaded: readers.countryLoaded,
asnLoaded: readers.asnLoaded,
countryFile: GEOIP_COUNTRY_FILE,
asnFile: GEOIP_ASN_FILE,
dir: readers.dir,
running: getGeoipUpdateState().running,
settings: await getGeoipSettings(),
}
}
const geoipRoutes: FastifyPluginAsyncZod = async (app) => {
app.get("/geoip", async (_req, reply) => {
return reply.send(await buildGeoipStatus())
})
app.put("/geoip", async (req, reply) => {
const parsed = geoipSettingsPatchSchema.safeParse(req.body ?? {})
if (!parsed.success) {
return reply
.status(400)
.send({ error: "Некорректное тело запроса", details: parsed.error.flatten() })
}
await updateGeoipSettings(parsed.data)
await refreshScheduler()
return reply.send({ ok: true, status: await buildGeoipStatus() })
})
app.post("/geoip/update", async (_req, reply) => {
try {
const snapshot = await collectGeoipUpdateOnce({ force: true })
return reply.send({ ok: !snapshot.fatalError && snapshot.errors.length === 0, snapshot })
} catch (e) {
const status = (e as { statusCode?: number }).statusCode ?? 502
const msg = e instanceof Error ? e.message : String(e)
return reply.status(status).send({ error: msg })
}
})
}
export default geoipRoutes
+229
View File
@@ -0,0 +1,229 @@
import type { FastifyPluginAsyncZod } from "@fastify/type-provider-zod"
import { z } from "zod"
import { MikrotikClient, MikrotikError } from "../services/mikrotik.js"
import { getEnabledServerById } from "../services/wireguard-live.js"
import {
captureGreSnapshot,
fetchGreState,
formatKeepalive,
listGreTunnels,
parseKeepalive,
} from "../services/gre-live.js"
import {
canonicalGreSnapshot,
parseGreSnapshot,
planGreCreate,
planGreDelete,
planGreRestore,
} from "../services/entity-snapshots.js"
import { executeRosOps } from "../services/ros-ops.js"
import {
captureAndAppendRevision,
listRevisions,
loadRevisionForRestore,
type ConfigRevisionSource,
} from "../services/config-revisions.js"
import { parseDbServerId } from "../utils/server-id.js"
const TunnelWriteSchema = z.object({
serverId: z.string().min(1),
name: z.string().min(1),
rosId: z.string().optional(),
localAddress: z.string().optional(),
remoteAddress: z.string().min(1),
localInnerIp: z.string().optional(),
remoteInnerIp: z.string().optional(),
comment: z.string().optional(),
enabled: z.boolean().optional(),
mtu: z.number().optional(),
keepaliveInterval: z.number().optional(),
keepaliveRetries: z.number().optional(),
dscp: z.union([z.literal("inherit"), z.number(), z.string()]).optional(),
clampTcpMss: z.boolean().optional(),
allowFastPath: z.boolean().optional(),
ipsecSecret: z.string().optional(),
})
const TunnelKeySchema = z.object({
serverId: z.string().min(1),
rosId: z.string().optional(),
name: z.string().optional(),
disabled: z.boolean().optional(),
})
const RevisionIdParamSchema = z.object({ id: z.string().min(1) })
function rosErr(e: unknown): string {
if (e instanceof MikrotikError) return e.message
if (e instanceof Error) return e.message
return String(e)
}
async function recordGre(
server: NonNullable<Awaited<ReturnType<typeof getEnabledServerById>>>,
source: ConfigRevisionSource,
) {
await captureAndAppendRevision({
serverId: server.id,
section: "gre",
source,
capture: () => captureGreSnapshot(server),
})
}
function tunnelFromBody(body: z.infer<typeof TunnelWriteSchema> & { keepaliveInterval?: number; keepaliveRetries?: number }) {
const dscp = body.dscp == null
? "inherit"
: typeof body.dscp === "number"
? String(body.dscp)
: body.dscp
const keepalive = body.keepaliveInterval === undefined && body.keepaliveRetries === undefined
? undefined
: formatKeepalive(body.keepaliveInterval ?? 0, body.keepaliveRetries ?? 10)
return canonicalGreSnapshot({
tunnels: [{
name: body.name,
localAddress: body.localAddress ?? "",
remoteAddress: body.remoteAddress,
localInnerIp: body.localInnerIp ?? "",
remoteInnerIp: body.remoteInnerIp ?? "",
comment: body.comment ?? "",
disabled: body.enabled === false,
mtu: body.mtu ?? 1476,
keepalive: keepalive ?? "0",
dscp,
clampTcpMss: body.clampTcpMss,
allowFastPath: body.allowFastPath,
ipsecSecret: body.ipsecSecret ?? "",
}],
}).tunnels[0]!
}
const greRoutes: FastifyPluginAsyncZod = async (app) => {
app.get("/gre/tunnels", async (req, reply) => {
const q = req.query as { serverId?: string | number }
const sid = parseDbServerId(q.serverId)
const result = await listGreTunnels({ serverId: sid !== null ? String(sid) : undefined })
return reply.send(result)
})
app.post("/gre/tunnels", async (req, reply) => {
const parsed = TunnelWriteSchema.safeParse(req.body ?? {})
if (!parsed.success) {
return reply.status(400).send({ error: "Некорректное тело запроса", details: parsed.error.flatten() })
}
const body = parsed.data
const server = await getEnabledServerById(body.serverId)
if (!server) return reply.status(404).send({ error: "Сервер не найден" })
const client = MikrotikClient.fromServer(server)
try {
const tunnel = tunnelFromBody(body)
await executeRosOps(client, planGreCreate(tunnel))
await recordGre(server, "apply")
const state = await fetchGreState(server)
const created = state.tunnels.find((t) => t.name === tunnel.name)
return reply.status(201).send(created ?? { ok: true, name: tunnel.name })
} catch (e) {
return reply.status(502).send({ error: `RouterOS: ${rosErr(e)}` })
}
})
app.patch("/gre/tunnels", async (req, reply) => {
const parsed = TunnelWriteSchema.partial().required({ serverId: true }).safeParse(req.body ?? {})
if (!parsed.success) {
return reply.status(400).send({ error: "Некорректное тело запроса", details: parsed.error.flatten() })
}
const body = parsed.data
const server = await getEnabledServerById(body.serverId)
if (!server) return reply.status(404).send({ error: "Сервер не найден" })
try {
const state = await fetchGreState(server)
const live = state.gre.find((g) =>
(body.rosId && g.rosId === body.rosId) || (body.name && g.name === body.name),
)
if (!live) return reply.status(404).send({ error: "Туннель не найден" })
const merged = tunnelFromBody({
serverId: body.serverId,
name: body.name || live.name,
localAddress: body.localAddress ?? live.localAddress,
remoteAddress: body.remoteAddress || live.remoteAddress,
localInnerIp: body.localInnerIp ?? state.addrs.find((a) => a.interfaceName === live.name)?.address ?? "",
remoteInnerIp: body.remoteInnerIp,
comment: body.comment ?? live.comment,
enabled: body.enabled ?? !live.disabled,
mtu: body.mtu ?? live.mtu,
keepaliveInterval: body.keepaliveInterval ?? parseKeepalive(live.keepalive).interval,
keepaliveRetries: body.keepaliveRetries ?? parseKeepalive(live.keepalive).retries,
dscp: body.dscp ?? live.dscp,
clampTcpMss: body.clampTcpMss ?? live.clampTcpMss,
allowFastPath: body.allowFastPath ?? live.allowFastPath,
ipsecSecret: body.ipsecSecret ?? live.ipsecSecret,
})
const ops = planGreRestore(
{ tunnels: state.snapshot.tunnels.map((t) => t.name === live.name ? merged : t) },
{ gre: state.gre, addrs: state.addrs },
)
await executeRosOps(state.client, ops)
await recordGre(server, "apply")
return reply.send({ ok: true })
} catch (e) {
return reply.status(502).send({ error: `RouterOS: ${rosErr(e)}` })
}
})
app.delete("/gre/tunnels", async (req, reply) => {
const parsed = TunnelKeySchema.safeParse(req.body ?? {})
if (!parsed.success) {
return reply.status(400).send({ error: "Некорректное тело запроса", details: parsed.error.flatten() })
}
const body = parsed.data
const server = await getEnabledServerById(body.serverId)
if (!server) return reply.status(404).send({ error: "Сервер не найден" })
try {
const state = await fetchGreState(server)
const live = state.gre.find((g) =>
(body.rosId && g.rosId === body.rosId) || (body.name && g.name === body.name),
)
if (!live) return reply.status(404).send({ error: "Туннель не найден" })
await executeRosOps(state.client, planGreDelete(live.name, { gre: state.gre, addrs: state.addrs }))
await recordGre(server, "apply")
return reply.send({ ok: true })
} catch (e) {
return reply.status(502).send({ error: `RouterOS: ${rosErr(e)}` })
}
})
app.get("/gre/revisions", async (req, reply) => {
const q = req.query as { serverId?: string | number }
const serverId = parseDbServerId(q.serverId)
if (serverId === null) return reply.status(400).send({ error: "serverId is required" })
const revisions = await listRevisions(serverId, "gre")
return reply.send({ revisions })
})
app.post("/gre/revisions/:id/restore", {
schema: { params: RevisionIdParamSchema },
}, async (req, reply) => {
const { id } = req.params
const body = req.body as { serverId?: string | number } | undefined
const loaded = await loadRevisionForRestore({
id,
section: "gre",
requestedServerId: parseDbServerId(body?.serverId),
})
if (!loaded.ok) return reply.status(loaded.status).send({ error: loaded.error })
try {
const desired = parseGreSnapshot(loaded.row.payload)
const state = await fetchGreState(loaded.server)
const ops = planGreRestore(desired, { gre: state.gre, addrs: state.addrs })
await executeRosOps(state.client, ops)
await recordGre(loaded.server, "rollback")
const next = await fetchGreState(loaded.server)
return reply.send({ ok: true, tunnels: next.tunnels })
} catch (e) {
return reply.status(502).send({ error: `RouterOS: ${rosErr(e)}` })
}
})
}
export default greRoutes
+737
View File
@@ -0,0 +1,737 @@
import type { FastifyPluginAsyncZod } from "@fastify/type-provider-zod"
import type { FastifyReply } from "fastify"
import {
ipsecCertDeleteRequestSchema,
ipsecCertExportByNameRequestSchema,
ipsecCertExportRequestSchema,
ipsecInitRequestSchema,
ipsecPeerPatchSchema,
ipsecUserCreateRequestSchema,
ipsecUserPatchSchema,
type IpsecCertBundle,
type IpsecClientDto,
} from "@mmapp/contracts/ipsec"
import { MikrotikClient, MikrotikError } from "../services/mikrotik.js"
import {
IPSEC_CA_CERT,
IPSEC_COMMON_NAME,
IPSEC_SERVER_CERT,
buildClientInstructions,
buildSswanConfig,
clientCertName,
identityDisplayName,
ipsecManagedComment,
ipsecUserComment,
isIke2RemoteAccessIdentity,
isIpsecManagedComment,
parseIpsecUserComment,
poolRangesFromCidr,
resolveIke2CaName,
resolveIke2ServerCert,
selectIke2Peers,
selectSharedIke2Identity,
userModeConfigName,
} from "../services/ipsec-config.js"
import {
ensureIpsecPeer,
ensureIpsecPool,
ensureIpsecProfile,
ensureIpsecProposal,
ensureNatRule,
ensurePolicyTemplate,
ensureSharedModeConfig,
putUserModeConfig,
deleteUserModeConfig,
putIdentity,
patchIdentity,
deleteIdentity,
patchPeer,
deletePeer,
ensureSharedIke2Identity,
listByPath,
} from "../services/ipsec-ros.js"
import {
ensureCaCertificate,
ensureServerCertificate,
exportCertificateP12ByName,
findCertificate,
issueClientCertificate,
} from "../services/ipsec-ca.js"
import {
captureIpsecSnapshot,
fetchIpsecRestoreState,
fetchIpsecState,
getEnabledIpsecServerById,
listIpsec,
mapClients,
mapServerSummary,
} from "../services/ipsec-live.js"
import {
captureAndAppendRevision,
listRevisions,
loadRevisionForRestore,
type ConfigRevisionSource,
} from "../services/config-revisions.js"
import { parseIpsecSnapshot, planIpsecRestore } from "../services/entity-snapshots.js"
import { executeRosOps } from "../services/ros-ops.js"
import { parseDbServerId } from "../utils/server-id.js"
import { randomBytes } from "node:crypto"
import { z } from "zod"
const IPV4_RE = /^\d{1,3}(?:\.\d{1,3}){3}$/
function generatePassphrase(): string {
return randomBytes(9).toString("base64url")
}
function serverIdParam(v: string): string {
return decodeURIComponent(v)
}
function rosIdParam(v: string): string {
return decodeURIComponent(v)
}
function errReply(reply: FastifyReply, e: unknown) {
const msg = e instanceof MikrotikError ? e.message : e instanceof Error ? e.message : String(e)
return reply.status(502).send({ error: `RouterOS: ${msg}` })
}
async function recordIpsec(
server: NonNullable<Awaited<ReturnType<typeof getEnabledIpsecServerById>>>,
source: ConfigRevisionSource,
) {
await captureAndAppendRevision({
serverId: server.id,
section: "ipsec",
source,
capture: () => captureIpsecSnapshot(server),
})
}
/** Существующие статические IP клиентов (персональные mode-config). */
function takenStaticIps(modeConfigs: Array<{ name?: string; address?: string; "address-prefix"?: string }>): string[] {
return modeConfigs
.filter((m) => (m.name ?? "").startsWith("mc-ipsec-"))
.map((m) => String(m.address ?? m["address-prefix"] ?? "").replace(/\/\d+$/, "").trim())
.filter(Boolean)
}
async function buildCertBundle(
client: MikrotikClient,
args: { userName: string; certName?: string; serverEndpoint: string; passphrase: string; dns?: string },
): Promise<IpsecCertBundle> {
const certName = args.certName?.trim() || clientCertName(args.userName)
const { fileName, content } = await exportCertificateP12ByName(client, certName, args.passphrase)
const p12B64 = content.toString("base64")
return {
user: args.userName,
serverEndpoint: args.serverEndpoint,
filename: fileName,
contentB64: p12B64,
mime: "application/x-pkcs12",
passphrase: args.passphrase,
sswanFilename: `${certName}.sswan`,
sswanContent: buildSswanConfig({
name: `IKEv2 ${args.serverEndpoint}`,
serverEndpoint: args.serverEndpoint,
serverId: args.serverEndpoint,
p12B64,
}),
instructions: buildClientInstructions({
userName: args.userName,
serverEndpoint: args.serverEndpoint,
p12Filename: fileName,
passphrase: args.passphrase,
dns: args.dns,
}),
}
}
const RevisionIdParamSchema = z.object({ id: z.string().min(1) })
const ipsecRoutes: FastifyPluginAsyncZod = async (app) => {
app.get("/ipsec", async (req, reply) => {
const q = req.query as { serverId?: string }
const result = await listIpsec({ serverId: q.serverId })
const sid = parseDbServerId(q.serverId)
if (sid !== null) {
const server = await getEnabledIpsecServerById(sid)
if (server) await recordIpsec(server, "observed")
}
return reply.send(result)
})
app.post("/ipsec/server/init", async (req, reply) => {
const parsed = ipsecInitRequestSchema.safeParse(req.body ?? {})
if (!parsed.success) {
return reply.status(400).send({ error: "Некорректное тело запроса", details: parsed.error.flatten() })
}
const body = parsed.data
const ranges = poolRangesFromCidr(body.poolCidr)
if (!ranges) {
return reply.status(400).send({ error: `Некорректная подсеть пула: ${body.poolCidr}` })
}
const server = await getEnabledIpsecServerById(body.serverId)
if (!server) return reply.status(404).send({ error: "Сервер не найден" })
const client = MikrotikClient.fromServer(server)
try {
const caCert = await ensureCaCertificate(client, body.caDaysValid)
const serverCert = await ensureServerCertificate(client, {
serverEndpoint: body.serverEndpoint,
caCertName: caCert,
daysValid: body.serverDaysValid,
})
const comment = ipsecManagedComment("IKEv2 road-warrior")
await ensureIpsecProfile(client, IPSEC_COMMON_NAME, comment)
await ensureIpsecProposal(client, IPSEC_COMMON_NAME, comment)
await ensureIpsecPool(client, IPSEC_COMMON_NAME, ranges, comment)
await ensureSharedModeConfig(client, IPSEC_COMMON_NAME, IPSEC_COMMON_NAME, body.dns, comment)
await ensurePolicyTemplate(client, body.poolCidr, IPSEC_COMMON_NAME, comment)
await ensureIpsecPeer(client, IPSEC_COMMON_NAME, serverCert, IPSEC_COMMON_NAME, comment)
if (body.createNatRule) {
await ensureNatRule(client, body.poolCidr, ipsecManagedComment("интернет клиентам VPN"))
}
await recordIpsec(server, "apply")
const state = await fetchIpsecState(server)
return reply.status(201).send(mapServerSummary(state, mapClients(state)))
} catch (e) {
return errReply(reply, e)
}
})
app.delete("/ipsec/server/:serverId", async (req, reply) => {
const { serverId } = req.params as { serverId: string }
const q = req.query as { removeCertificates?: string }
const removeCertificates = q.removeCertificates !== "false"
const server = await getEnabledIpsecServerById(serverIdParam(serverId))
if (!server) return reply.status(404).send({ error: "Сервер не найден" })
const client = MikrotikClient.fromServer(server)
try {
for (const path of ["/ip/ipsec/identity", "/ip/ipsec/mode-config", "/ip/ipsec/peer", "/ip/ipsec/policy", "/ip/pool"]) {
const rows = await listByPath(client, path)
for (const row of rows) {
if (!row[".id"]) continue
if (isIpsecManagedComment(row.comment) || (path === "/ip/ipsec/policy" && (row.template === "true" || row.template === "yes"))) {
await client.delete(`${path}/${encodeURIComponent(row[".id"])}`).catch(() => undefined)
}
}
}
const nat = await listByPath(client, "/ip/firewall/nat")
for (const row of nat) {
if (row[".id"] && isIpsecManagedComment(row.comment)) {
await client.delete(`/ip/firewall/nat/${encodeURIComponent(row[".id"])}`).catch(() => undefined)
}
}
// profile/proposal не критично оставлять; сертификаты — по флагу
if (removeCertificates) {
const certs = await client.getCertificates()
for (const c of certs) {
const name = String(c.name ?? "")
if (c[".id"] && (name === IPSEC_CA_CERT || name === IPSEC_SERVER_CERT || name.startsWith("ipsec-user-"))) {
await client.delete(`/certificate/${encodeURIComponent(c[".id"])}`).catch(() => undefined)
}
}
}
await recordIpsec(server, "apply")
return reply.send({ ok: true })
} catch (e) {
return errReply(reply, e)
}
})
app.post("/ipsec/users", async (req, reply) => {
const parsed = ipsecUserCreateRequestSchema.safeParse(req.body ?? {})
if (!parsed.success) {
return reply.status(400).send({ error: "Некорректное тело запроса", details: parsed.error.flatten() })
}
const body = parsed.data
const server = await getEnabledIpsecServerById(body.serverId)
if (!server) return reply.status(404).send({ error: "Сервер не найден" })
const client = MikrotikClient.fromServer(server)
try {
const state = await fetchIpsecState(server)
const ike2Peers = selectIke2Peers(state.peers)
const peer = (body.peerName
? state.peers.find((p) => (p.name ?? "").trim() === body.peerName!.trim())
: undefined)
?? ike2Peers.find((p) => isIpsecManagedComment(p.comment))
?? ike2Peers.find((p) => (p.name ?? "").trim() === IPSEC_COMMON_NAME)
?? ike2Peers[0]
?? state.peers[0]
if (!peer) {
return reply.status(400).send({ error: "На роутере нет ни одного IPsec peer — создайте peer на вкладке «Сервер»" })
}
const ike2PeerNames = ike2Peers.map((p) => (p.name ?? "").trim()).filter(Boolean)
const ike2Identities = state.identities.filter(
(i) => isIpsecManagedComment(i.comment) || isIke2RemoteAccessIdentity(i, ike2PeerNames),
)
const idMcName = ike2Identities
.map((i) => (i["mode-config"] ?? "").trim())
.find((n) => n && !n.startsWith("mc-ipsec-"))
const sharedMc = (idMcName ? state.modeConfigs.find((m) => (m.name ?? "").trim() === idMcName) : undefined)
?? state.modeConfigs.find((m) => (m.name ?? "").trim() === IPSEC_COMMON_NAME)
const serverCertRow = resolveIke2ServerCert(state.certs, ike2Peers)
const caName = resolveIke2CaName(state.certs, serverCertRow)
if (body.authMethod === "certificate" && (!caName || !serverCertRow)) {
return reply.status(400).send({
error: "Нет CA/серверного сертификата — для сертификатного клиента запустите мастер или используйте PSK",
})
}
const peerName = (peer.name ?? "").trim()
const serverEndpoint = String(serverCertRow?.["common-name"] ?? "").trim() || server.host
const dns = sharedMc?.["static-dns"]?.trim() || undefined
if (body.authMethod === "pre-shared-key" && !body.psk) {
return reply.status(400).send({ error: "Для PSK-клиента укажите secret (psk)" })
}
const sharedIdentity = selectSharedIke2Identity(state.identities, ike2PeerNames)
const sharedIdentityId = sharedIdentity?.[".id"] ? String(sharedIdentity[".id"]) : undefined
let modeConfig = (sharedMc?.name ?? "").trim()
if (body.staticIp) {
const ip = body.staticIp.trim()
if (!IPV4_RE.test(ip)) return reply.status(400).send({ error: `Некорректный IP: ${ip}` })
const taken = takenStaticIps(state.modeConfigs)
if (taken.includes(ip)) {
return reply.status(409).send({ error: `IP ${ip} уже назначен другому клиенту` })
}
modeConfig = userModeConfigName(body.name)
await putUserModeConfig(client, modeConfig, ip, ipsecManagedComment(`клиент ${body.name.trim()}`))
}
let certName: string | undefined
if (body.authMethod === "certificate") {
const issued = await issueClientCertificate(client, {
userName: body.name,
caCertName: caName!,
daysValid: body.daysValid ?? 1825,
})
if (issued.existed) {
return reply.status(409).send({ error: `Клиент с сертификатом ${issued.certName} уже существует` })
}
certName = issued.certName
}
if (body.authMethod === "certificate" && !body.staticIp) {
// Гибрид: клиент «вешается» на общую identity peer — отдельная identity не нужна,
// RouterOS сам принимает любой клиентский серт, подписанный доверенным CA.
await ensureSharedIke2Identity(client, {
peerName,
serverCertName: String(serverCertRow?.name ?? IPSEC_SERVER_CERT),
modeConfigName: modeConfig,
comment: ipsecManagedComment("IKEv2 listener"),
})
} else {
// PSK или персональный статический IP: отдельная identity (match-by=certificate для cert).
// place-before гарантирует матчинг раньше общей listener-identity.
await putIdentity(client, {
peerName,
modeConfig,
comment: ipsecUserComment(body.name),
authMethod: body.authMethod,
certificate: body.authMethod === "certificate" ? String(serverCertRow?.name ?? IPSEC_SERVER_CERT) : undefined,
remoteCertificate: certName,
secret: body.psk,
remoteId: body.authMethod === "pre-shared-key" ? (body.remoteId ?? body.name) : undefined,
placeBefore: sharedIdentityId,
})
}
let bundle: IpsecCertBundle | undefined
if (body.authMethod === "certificate" && certName) {
const passphrase = body.passphrase?.trim() || generatePassphrase()
bundle = await buildCertBundle(client, {
userName: body.name,
serverEndpoint,
passphrase,
dns,
})
}
await recordIpsec(server, "apply")
const fresh = await fetchIpsecState(server)
const clients = mapClients(fresh)
const created: IpsecClientDto | undefined = clients.find(
(c) => c.name === body.name.trim(),
) ?? clients.find((c) => c.certificateName === certName)
return reply.status(201).send({ client: created ?? null, bundle })
} catch (e) {
return errReply(reply, e)
}
})
app.patch("/ipsec/users/:serverId/:rosId", async (req, reply) => {
const { serverId, rosId } = req.params as { serverId: string; rosId: string }
const parsed = ipsecUserPatchSchema.safeParse(req.body ?? {})
if (!parsed.success) {
return reply.status(400).send({ error: "Некорректное тело запроса", details: parsed.error.flatten() })
}
const d = parsed.data
const server = await getEnabledIpsecServerById(serverIdParam(serverId))
if (!server) return reply.status(404).send({ error: "Сервер не найден" })
const client = MikrotikClient.fromServer(server)
try {
const state = await fetchIpsecState(server)
const targetRosId = rosIdParam(rosId)
const identity = state.identities.find((i) => String(i[".id"] ?? "") === targetRosId)
const certRow = identity
? undefined
: state.certs.find((c) => String(c[".id"] ?? "") === targetRosId)
?? state.certs.find((c) => String(c.name ?? "").trim() === targetRosId)
if (!identity && !certRow) return reply.status(404).send({ error: "Клиент не найден" })
const ike2Peers = selectIke2Peers(state.peers)
const ike2PeerNames = ike2Peers.map((p) => (p.name ?? "").trim()).filter(Boolean)
const serverCertRow = resolveIke2ServerCert(state.certs, ike2Peers)
const managedIdentity = identity
? isIpsecManagedComment(identity.comment)
: String(certRow?.name ?? "").startsWith("ipsec-user-")
const oldName = identity
? (parseIpsecUserComment(identity.comment) ?? identity.comment ?? "")
: String(certRow?.["common-name"] ?? certRow?.name ?? "").trim()
const sharedMc = state.modeConfigs.find((m) => (m.name ?? "").trim() === IPSEC_COMMON_NAME)
const sharedMcName = (sharedMc?.name ?? "").trim()
const newName = d.name?.trim() || oldName
if (d.name && d.name !== oldName) {
if (identity) {
// managed: user=<name>; существующий RouterOS identity: обычный comment
await patchIdentity(client, identity[".id"]!, {
comment: managedIdentity ? ipsecUserComment(d.name) : d.name.trim(),
})
}
// managed-серт переименовываем на конвенционное имя (client1/anakondra на устройстве не трогаем)
if (certRow?.[".id"] && managedIdentity) {
await client.patch(`/certificate/${encodeURIComponent(String(certRow[".id"]))}`, { name: clientCertName(d.name) })
}
}
if (identity && d.staticIp !== undefined) {
const currentMc = (identity["mode-config"] ?? "").trim()
if (d.staticIp == null) {
// вернуть выдачу из пула
if (sharedMcName) await patchIdentity(client, identity[".id"]!, { "mode-config": sharedMcName })
const personal = currentMc && currentMc !== sharedMcName ? currentMc : ""
if (personal) await deleteUserModeConfig(client, personal)
} else {
const ip = d.staticIp.trim()
if (!IPV4_RE.test(ip)) return reply.status(400).send({ error: `Некорректный IP: ${ip}` })
const others = takenStaticIps(
state.modeConfigs.filter((m) => (m.name ?? "").trim() !== currentMc),
)
if (others.includes(ip)) return reply.status(409).send({ error: `IP ${ip} уже назначен другому клиенту` })
const name = userModeConfigName(newName)
await putUserModeConfig(client, name, ip, ipsecManagedComment(`клиент ${newName.trim()}`))
await patchIdentity(client, identity[".id"]!, { "mode-config": name })
}
} else if (!identity && certRow && d.staticIp) {
// cert-клиент без identity: персональная identity (match-by=certificate) + статический mode-config
const ip = d.staticIp.trim()
if (!IPV4_RE.test(ip)) return reply.status(400).send({ error: `Некорректный IP: ${ip}` })
const taken = takenStaticIps(state.modeConfigs)
if (taken.includes(ip)) return reply.status(409).send({ error: `IP ${ip} уже назначен другому клиенту` })
const peer = ike2Peers.find((p) => isIpsecManagedComment(p.comment)) ?? ike2Peers[0]
if (!peer) return reply.status(400).send({ error: "На роутере нет IKEv2 peer для привязки клиента" })
const shared = selectSharedIke2Identity(state.identities, ike2PeerNames)
const mcName = userModeConfigName(newName)
await putUserModeConfig(client, mcName, ip, ipsecManagedComment(`клиент ${newName.trim()}`))
await putIdentity(client, {
peerName: (peer.name ?? "").trim(),
modeConfig: mcName,
comment: ipsecUserComment(newName),
authMethod: "certificate",
certificate: String(serverCertRow?.name ?? IPSEC_SERVER_CERT),
remoteCertificate: String(certRow.name ?? "").trim(),
placeBefore: shared?.[".id"] ? String(shared[".id"]) : undefined,
})
}
if (identity) {
const patch: Record<string, string> = {}
if (d.psk) patch.secret = d.psk
if (d.remoteId !== undefined) patch["remote-id"] = d.remoteId
if (d.disabled === true) patch.disabled = "yes"
if (d.disabled === false) patch.disabled = "no"
if (Object.keys(patch).length) await patchIdentity(client, identity[".id"]!, patch)
}
await recordIpsec(server, "apply")
return reply.send({ ok: true })
} catch (e) {
return errReply(reply, e)
}
})
app.delete("/ipsec/users/:serverId/:rosId", async (req, reply) => {
const { serverId, rosId } = req.params as { serverId: string; rosId: string }
const q = req.query as { removeCertificate?: string }
const removeCertificate = q.removeCertificate !== "false"
const server = await getEnabledIpsecServerById(serverIdParam(serverId))
if (!server) return reply.status(404).send({ error: "Сервер не найден" })
const client = MikrotikClient.fromServer(server)
try {
const state = await fetchIpsecState(server)
const targetRosId = rosIdParam(rosId)
const identity = state.identities.find((i) => String(i[".id"] ?? "") === targetRosId)
const certRow = identity
? undefined
: state.certs.find((c) => String(c[".id"] ?? "") === targetRosId)
?? state.certs.find((c) => String(c.name ?? "").trim() === targetRosId)
if (!identity && !certRow) return reply.status(404).send({ error: "Клиент не найден" })
const managed = identity
? isIpsecManagedComment(identity.comment)
: String(certRow?.name ?? "").startsWith("ipsec-user-")
const userName = identity ? (parseIpsecUserComment(identity.comment) ?? "") : ""
const sharedMcName = (state.modeConfigs.find((m) => (m.name ?? "").trim() === IPSEC_COMMON_NAME)?.name ?? "").trim()
const personal = identity ? (identity["mode-config"] ?? "").trim() : ""
const remoteCert = identity
? (identity["remote-certificate"] ?? "").trim()
: String(certRow?.name ?? "").trim()
if (identity) {
await deleteIdentity(client, identity[".id"]!)
if (personal && personal !== sharedMcName) await deleteUserModeConfig(client, personal)
}
if (removeCertificate) {
// удаляем только клиентский серт (managed / выбранный явно); CA и серверный не трогаем
const ike2Peers = selectIke2Peers(state.peers)
const caName = resolveIke2CaName(state.certs, resolveIke2ServerCert(state.certs, ike2Peers))
const serverCertName = String(resolveIke2ServerCert(state.certs, ike2Peers)?.name ?? "").trim()
const certName = remoteCert || (managed && userName ? clientCertName(userName) : "")
if (certName && certName !== caName && certName !== serverCertName) {
await client.removeCertificate(certName).catch(() => undefined)
}
}
await recordIpsec(server, "apply")
return reply.send({ ok: true })
} catch (e) {
return errReply(reply, e)
}
})
app.patch("/ipsec/peers/:serverId/:rosId", async (req, reply) => {
const { serverId, rosId } = req.params as { serverId: string; rosId: string }
const parsed = ipsecPeerPatchSchema.safeParse(req.body ?? {})
if (!parsed.success) {
return reply.status(400).send({ error: "Некорректное тело запроса", details: parsed.error.flatten() })
}
const d = parsed.data
const server = await getEnabledIpsecServerById(serverIdParam(serverId))
if (!server) return reply.status(404).send({ error: "Сервер не найден" })
const client = MikrotikClient.fromServer(server)
try {
const state = await fetchIpsecState(server)
const peer = state.peers.find((p) => String(p[".id"] ?? "") === rosIdParam(rosId))
if (!peer) return reply.status(404).send({ error: "Peer не найден" })
const body: Record<string, string> = {}
if (d.name !== undefined) body.name = d.name
if (d.address !== undefined) body.address = d.address
if (d.exchangeMode !== undefined) body["exchange-mode"] = d.exchangeMode
if (d.passive !== undefined) body.passive = d.passive ? "yes" : "no"
if (d.certificate !== undefined) body.certificate = d.certificate
if (d.profile !== undefined) body.profile = d.profile
if (d.disabled !== undefined) body.disabled = d.disabled ? "yes" : "no"
if (Object.keys(body).length) await patchPeer(client, rosIdParam(rosId), body)
await recordIpsec(server, "apply")
return reply.send({ ok: true })
} catch (e) {
return errReply(reply, e)
}
})
app.delete("/ipsec/peers/:serverId/:rosId", async (req, reply) => {
const { serverId, rosId } = req.params as { serverId: string; rosId: string }
const q = req.query as { force?: string }
const server = await getEnabledIpsecServerById(serverIdParam(serverId))
if (!server) return reply.status(404).send({ error: "Сервер не найден" })
const client = MikrotikClient.fromServer(server)
try {
const state = await fetchIpsecState(server)
const peer = state.peers.find((p) => String(p[".id"] ?? "") === rosIdParam(rosId))
if (!peer) return reply.status(404).send({ error: "Peer не найден" })
const peerName = (peer.name ?? "").trim()
const linked = state.identities
.filter((i) => (i.peer ?? "").trim() === peerName)
.map((i) => identityDisplayName(i, state.certs))
if (linked.length > 0 && q.force !== "true") {
return reply.status(409).send({
error: `На peer «${peerName}» ссылаются identity: ${linked.join(", ")}. Удаление разорвёт их.`,
identities: linked,
})
}
await deletePeer(client, rosIdParam(rosId))
await recordIpsec(server, "apply")
return reply.send({ ok: true })
} catch (e) {
return errReply(reply, e)
}
})
app.delete("/ipsec/certs/:serverId", async (req, reply) => {
const { serverId } = req.params as { serverId: string }
const parsed = ipsecCertDeleteRequestSchema.safeParse(req.body ?? {})
if (!parsed.success) {
return reply.status(400).send({ error: "Некорректное тело запроса", details: parsed.error.flatten() })
}
const q = req.query as { force?: string }
const server = await getEnabledIpsecServerById(serverIdParam(serverId))
if (!server) return reply.status(404).send({ error: "Сервер не найден" })
const client = MikrotikClient.fromServer(server)
try {
const state = await fetchIpsecState(server)
const name = parsed.data.name
const cert = state.certs.find((c) => String(c.name ?? "").trim() === name)
if (!cert?.[".id"]) return reply.status(404).send({ error: `Сертификат ${name} не найден` })
const usedBy = [
...state.peers.filter((p) => (p.certificate ?? "").trim() === name).map((p) => `peer ${(p.name ?? "").trim()}`),
...state.identities
.filter((i) => (i.certificate ?? "").trim() === name || (i["remote-certificate"] ?? "").trim() === name)
.map((i) => `identity ${identityDisplayName(i, state.certs)}`),
]
if (usedBy.length > 0 && q.force !== "true") {
return reply.status(409).send({
error: `Сертификат «${name}» используется: ${usedBy.join(", ")}.`,
usedBy,
})
}
await client.delete(`/certificate/${encodeURIComponent(cert[".id"])}`)
await recordIpsec(server, "apply")
return reply.send({ ok: true })
} catch (e) {
return errReply(reply, e)
}
})
app.post("/ipsec/users/:serverId/:rosId/cert", async (req, reply) => {
const { serverId, rosId } = req.params as { serverId: string; rosId: string }
const parsed = ipsecCertExportRequestSchema.safeParse({ ...(req.body as object), serverId, clientId: rosId })
if (!parsed.success) {
return reply.status(400).send({ error: "Некорректное тело запроса", details: parsed.error.flatten() })
}
const body = parsed.data
const server = await getEnabledIpsecServerById(serverIdParam(serverId))
if (!server) return reply.status(404).send({ error: "Сервер не найден" })
const client = MikrotikClient.fromServer(server)
try {
const state = await fetchIpsecState(server)
const identity = state.identities.find((i) => String(i[".id"] ?? "") === rosIdParam(rosId))
if (!identity) return reply.status(404).send({ error: "Клиент не найден" })
const userName = parseIpsecUserComment(identity.comment) ?? ""
if (!userName) return reply.status(400).send({ error: "Не managed-клиент" })
const certName = identity["remote-certificate"]?.trim() || clientCertName(userName)
if (!(await findCertificate(client, certName))) {
return reply.status(404).send({ error: `Сертификат ${certName} не найден на роутере` })
}
const serverCert = state.certs.find((c) => String(c.name ?? "") === IPSEC_SERVER_CERT)
const serverEndpoint = String(serverCert?.["common-name"] ?? "").trim() || server.host
const sharedMc = state.modeConfigs.find((m) => (m.name ?? "").trim() === IPSEC_COMMON_NAME)
const dns = sharedMc?.["static-dns"]?.trim() || undefined
// export работает по имени сертификата (certName), не по userName
const { fileName, content } = await client.exportCertificatePkcs12({
name: certName,
passphrase: body.passphrase,
})
const p12B64 = content.toString("base64")
const bundle: IpsecCertBundle = {
user: userName,
serverEndpoint,
filename: fileName,
contentB64: p12B64,
mime: "application/x-pkcs12",
passphrase: body.passphrase,
sswanFilename: `${certName}.sswan`,
sswanContent: buildSswanConfig({
name: `IKEv2 ${serverEndpoint}`,
serverEndpoint,
serverId: serverEndpoint,
p12B64,
}),
instructions: buildClientInstructions({
userName,
serverEndpoint,
p12Filename: fileName,
passphrase: body.passphrase,
dns,
}),
}
return reply.send(bundle)
} catch (e) {
return errReply(reply, e)
}
})
/** Экспорт .p12 существующего клиентского сертификата по имени (client1/anakondra и т.п.). */
app.post("/ipsec/certs/:serverId/export", async (req, reply) => {
const { serverId } = req.params as { serverId: string }
const parsed = ipsecCertExportByNameRequestSchema.safeParse(req.body ?? {})
if (!parsed.success) {
return reply.status(400).send({ error: "Некорректное тело запроса", details: parsed.error.flatten() })
}
const { name, passphrase } = parsed.data
const server = await getEnabledIpsecServerById(serverIdParam(serverId))
if (!server) return reply.status(404).send({ error: "Сервер не найден" })
const client = MikrotikClient.fromServer(server)
try {
const state = await fetchIpsecState(server)
const cert = state.certs.find((c) => String(c.name ?? "").trim() === name)
if (!cert?.[".id"]) return reply.status(404).send({ error: `Сертификат ${name} не найден` })
const serverCertRow = resolveIke2ServerCert(state.certs, selectIke2Peers(state.peers))
const serverEndpoint = String(serverCertRow?.["common-name"] ?? "").trim() || server.host
const userName = String(cert["common-name"] ?? "").trim() || name
const sharedMc = state.modeConfigs.find((m) => (m.name ?? "").trim() === IPSEC_COMMON_NAME)
const dns = sharedMc?.["static-dns"]?.trim() || undefined
const bundle = await buildCertBundle(client, {
userName,
certName: name,
serverEndpoint,
passphrase,
dns,
})
return reply.send(bundle)
} catch (e) {
return errReply(reply, e)
}
})
app.get("/ipsec/revisions", async (req, reply) => {
const q = req.query as { serverId?: string | number }
const serverId = parseDbServerId(q.serverId)
if (serverId === null) return reply.status(400).send({ error: "serverId is required" })
const revisions = await listRevisions(serverId, "ipsec")
return reply.send({ revisions })
})
app.post("/ipsec/revisions/:id/restore", {
schema: { params: RevisionIdParamSchema },
}, async (req, reply) => {
const { id } = req.params
const body = req.body as { serverId?: string | number } | undefined
const loaded = await loadRevisionForRestore({
id,
section: "ipsec",
requestedServerId: parseDbServerId(body?.serverId),
})
if (!loaded.ok) return reply.status(loaded.status).send({ error: loaded.error })
try {
const desired = parseIpsecSnapshot(loaded.row.payload)
const state = await fetchIpsecRestoreState(loaded.server)
const ops = planIpsecRestore(desired, {
peers: state.peers,
identities: state.identities,
modeConfigs: state.modeConfigs,
pools: state.pools,
nat: state.nat,
})
await executeRosOps(state.client, ops)
await recordIpsec(loaded.server, "rollback")
const result = await listIpsec({ serverId: String(loaded.server.id) })
return reply.send({ ok: true, ...result })
} catch (e) {
return errReply(reply, e)
}
})
}
export default ipsecRoutes
+36 -8
View File
@@ -6,9 +6,10 @@ import { MikrotikClient } from "../services/mikrotik.js"
import { ServerIdParamSchema, type ServerIdParams } from "../types/server.js"
import type {
RosOspfNeighbor, RosOspfArea, RosOspfInterfaceTemplate, RosOspfInstance,
RosBfdSession,
OspfNeighborRead, OspfInterfaceRead, OspfInstanceRead, BfdSessionRead,
RosBfdSession, RosIpRoute,
OspfNeighborRead, OspfInterfaceRead, OspfInstanceRead, OspfRouteRead, BfdSessionRead,
} from "../types/server.js"
import { parseOspfGateway, parseOspfRouteType } from "../services/ospf-route-parse.js"
import { z } from "zod"
type ServerRow = typeof servers.$inferSelect
@@ -74,14 +75,15 @@ function parseAddrIface(addr: string): { ip: string; iface: string } {
/** Fetch all OSPF + BFD data for one server */
async function fetchServerOspf(server: ServerRow) {
const client = MikrotikClient.fromServer(server)
const [neighbors, areas, ifaceTemplates, instances, bfdSessions] = await Promise.all([
const [neighbors, areas, ifaceTemplates, instances, bfdSessions, ipRoutes] = await Promise.all([
client.getOspfNeighbors(),
client.getOspfAreas(),
client.getOspfInterfaceTemplates(),
client.getOspfInstances(),
client.getBfdSessions().catch(() => [] as RosBfdSession[]), // BFD is optional
client.getIpRoutes().catch(() => [] as RosIpRoute[]),
])
return { neighbors, areas, ifaceTemplates, instances, bfdSessions }
return { neighbors, areas, ifaceTemplates, instances, bfdSessions, ipRoutes }
}
// ── BFD parser ────────────────────────────────────────────────────────────────
@@ -200,6 +202,29 @@ function parseInstances(
}))
}
function parseOspfRoutes(server: ServerRow, routes: RosIpRoute[]): OspfRouteRead[] {
const out: OspfRouteRead[] = []
for (const [idx, r] of routes.entries()) {
const type = parseOspfRouteType(r)
if (!type) continue
const { nextHop, via } = parseOspfGateway(r)
const metric = parseInt(r["ospf-metric"] ?? r.distance ?? "0") || 0
out.push({
id: r[".id"] ?? String(idx),
serverId: server.id,
serverName: server.name || server.host,
serverSite: server.site,
destination: r["dst-address"] ?? "",
type,
cost: metric,
nextHop,
via,
area: r["ospf-area"] ?? "",
})
}
return out
}
function calcRouteScore(pingMs: number, dlMbps: number, ulMbps: number, pingWeight: number) {
const pingScore = Math.max(0, 100 - pingMs * 0.6)
const speedScore = Math.min(100, (dlMbps + ulMbps) / 18)
@@ -217,7 +242,7 @@ async function getLatestSnapshotLatencyMs(serverId: number): Promise<number> {
return latest.length > 0 ? Math.max(1, Math.round(latest[0].latencyMs ?? 100)) : 100
}
async function latestTrafficByInterface(serverId: number): Promise<Map<string, { id: number; serverId: number; disabled: boolean; sampledAt: string; interfaceName: string; peerPublicKey: string; rxBytes: number; txBytes: number; rxBps: number; txBps: number; running: boolean; }>> {
async function latestTrafficByInterface(serverId: number): Promise<Map<string, TrafficSampleRow>> {
const rows = await db
.select()
.from(trafficSamples)
@@ -584,16 +609,17 @@ const ospfRoutes: FastifyPluginAsyncZod = async (app) => {
const perServer = await Promise.all(
allServers.map(async (server) => {
try {
const { neighbors, areas, ifaceTemplates, instances, bfdSessions } = await fetchServerOspf(server)
const { neighbors, areas, ifaceTemplates, instances, bfdSessions, ipRoutes } = await fetchServerOspf(server)
const areaMap = buildAreaMap(areas)
return {
neighbors: parseNeighbors(server, neighbors, areaMap),
interfaces: parseInterfaces(server, ifaceTemplates, areas, instances, areaMap),
instances: parseInstances(server, instances),
bfdSessions: parseBfdSessions(server, bfdSessions),
routes: parseOspfRoutes(server, ipRoutes),
}
} catch {
return { neighbors: [], interfaces: [], instances: [], bfdSessions: [] }
return { neighbors: [], interfaces: [], instances: [], bfdSessions: [], routes: [] }
}
}),
)
@@ -603,6 +629,7 @@ const ospfRoutes: FastifyPluginAsyncZod = async (app) => {
interfaces: perServer.flatMap(r => r.interfaces),
instances: perServer.flatMap(r => r.instances),
bfdSessions: perServer.flatMap(r => r.bfdSessions),
routes: perServer.flatMap(r => r.routes),
})
})
@@ -634,13 +661,14 @@ const ospfRoutes: FastifyPluginAsyncZod = async (app) => {
if (!server) return reply.status(404).send({ error: "Server not found" })
try {
const { neighbors, areas, ifaceTemplates, instances, bfdSessions } = await fetchServerOspf(server)
const { neighbors, areas, ifaceTemplates, instances, bfdSessions, ipRoutes } = await fetchServerOspf(server)
const areaMap = buildAreaMap(areas)
return reply.send({
neighbors: parseNeighbors(server, neighbors, areaMap),
interfaces: parseInterfaces(server, ifaceTemplates, areas, instances, areaMap),
instances: parseInstances(server, instances),
bfdSessions: parseBfdSessions(server, bfdSessions),
routes: parseOspfRoutes(server, ipRoutes),
areas: areas.map(a => ({ name: a.name, areaId: a["area-id"] ?? "0.0.0.0", type: a.type, disabled: a.disabled === "true", inactive: a.inactive === "true", instance: a.instance })),
})
} catch (err) {
+141 -80
View File
@@ -1,13 +1,23 @@
import { asc, eq } from "drizzle-orm"
import { z } from "zod"
import type { FastifyPluginAsyncZod } from "@fastify/type-provider-zod"
import { db } from "../db/index.js"
import { recursiveRoutes, servers } from "../db/schema.js"
import { MikrotikClient } from "../services/mikrotik.js"
import { parseDbServerId } from "../utils/server-id.js"
import { managedRecursiveComment } from "../managed-markers.js"
import {
hasManagedRecursiveComment,
managedRecursiveComment,
} from "../managed-markers.js"
mapRosManagedRoutes,
planRecursiveApply,
userRecursiveComment,
} from "../services/config-apply-plan.js"
import {
appendRevisionIfChanged,
canonicalRecursiveRoutes,
getRevisionById,
listRevisions,
type ConfigRevisionSource,
} from "../services/config-revisions.js"
type ServerRow = typeof servers.$inferSelect
@@ -51,27 +61,6 @@ interface RecursiveRouteDto {
disabled: boolean
}
function isIpGateway(gw: string): boolean {
return /^\d{1,3}(\.\d{1,3}){3}(?:%\S+)?$/.test(gw.trim())
}
function isRecursiveRoute(r: RosRoute): boolean {
if ((r.static ?? "false") !== "true") return false
if ((r.dynamic ?? "false") === "true") return false
if ((r.blackhole ?? "false") === "true") return false
if ((r.unreachable ?? "false") === "true") return false
if ((r.prohibit ?? "false") === "true") return false
const dst = r["dst-address"] ?? ""
const gw = r.gateway ?? ""
if (!dst || !gw) return false
return isIpGateway(gw)
}
function hasRecursiveCommentMask(comment: string | undefined): boolean {
if (!comment) return false
return /^recursive:\s*/i.test(comment.trim())
}
function splitGateway(raw: string): { ip: string; name: string } | null {
const v = raw.trim()
if (!v) return null
@@ -102,6 +91,21 @@ async function mapDbRoutes(serverId: number): Promise<RecursiveRouteDto[]> {
}))
}
function mergeCachedCountry(
live: RecursiveRouteDto[],
cached: RecursiveRouteDto[],
): RecursiveRouteDto[] {
return live.map((row) => {
if (row.country) return row
const match = cached.find((c) =>
c.dstAddress === row.dstAddress &&
c.gateway === row.gateway &&
c.distance === row.distance,
)
return match?.country ? { ...row, country: match.country } : row
})
}
function toRouterPayload(route: RecursiveRouteDto): Record<string, string> {
return {
"dst-address": route.dstAddress,
@@ -132,7 +136,7 @@ async function replaceDbRoutes(serverId: number, routes: RecursiveRouteDto[]) {
routingTable: r.routingTable || "main",
checkGateway: r.checkGateway ?? "",
country: r.country ?? "",
comment: r.comment ?? "",
comment: userRecursiveComment(r.comment),
disabled: r.disabled,
createdAt: now,
updatedAt: now,
@@ -140,6 +144,34 @@ async function replaceDbRoutes(serverId: number, routes: RecursiveRouteDto[]) {
)
}
async function applyRecursiveToServer(
server: ServerRow,
routes: RecursiveRouteDto[],
source: ConfigRevisionSource,
): Promise<{ pushed: number; deleted: number }> {
const client = MikrotikClient.fromServer(server)
const existing = await client.get<RosRoute[]>("/ip/route")
const { deleteIds } = planRecursiveApply(existing)
for (const route of routes) {
await client.post("/ip/route", toRouterPayload(route))
}
for (const id of deleteIds) {
await client.delete(`/ip/route/${encodeURIComponent(id)}`)
}
await replaceDbRoutes(server.id, routes)
await appendRevisionIfChanged({
serverId: server.id,
section: "recursive-routes",
source,
payload: canonicalRecursiveRoutes(routes),
})
return { pushed: routes.length, deleted: deleteIds.length }
}
const RevisionIdParamSchema = z.object({ id: z.string().min(1) })
const recursiveRoutesPlugin: FastifyPluginAsyncZod = async (app) => {
app.get("/recursive-routes/gateways", async (req, reply) => {
const q = req.query as { serverId?: string | number }
@@ -175,79 +207,108 @@ const recursiveRoutesPlugin: FastifyPluginAsyncZod = async (app) => {
const q = req.query as { serverId?: string | number }
const serverId = parseDbServerId(q.serverId)
if (serverId === null) return reply.status(400).send({ error: "serverId is required" })
const server = (await db.select().from(servers).where(eq(servers.id, serverId)).limit(1))[0]
if (!server) return reply.status(404).send({ error: "Server not found" })
return reply.send({ routes: await mapDbRoutes(serverId) })
})
app.put("/recursive-routes", async (req, reply) => {
const body = req.body as { serverId?: string | number; routes?: RecursiveRouteDto[] }
const serverId = parseDbServerId(body.serverId)
if (serverId === null) return reply.status(400).send({ error: "serverId is required" })
const server = (await db.select().from(servers).where(eq(servers.id, serverId)).limit(1))[0]
if (!server) return reply.status(404).send({ error: "Server not found" })
await replaceDbRoutes(serverId, body.routes ?? [])
return reply.send({ ok: true })
})
app.post("/recursive-routes/sync/from-router", async (req, reply) => {
const body = req.body as { serverId?: string | number } | undefined
const serverId = parseDbServerId(body?.serverId)
if (serverId === null) return reply.status(400).send({ error: "serverId is required" })
const server: ServerRow | undefined = (await db
.select().from(servers)
.where(eq(servers.id, serverId))
.limit(1))[0]
if (!server) return reply.status(404).send({ error: "Server not found" })
const cached = await mapDbRoutes(serverId)
try {
const client = MikrotikClient.fromServer(server)
const rosRoutes = await client.get<RosRoute[]>("/ip/route")
const rec = rosRoutes.filter(r =>
isRecursiveRoute(r) && hasRecursiveCommentMask(r.comment),
)
const mapped: RecursiveRouteDto[] = rec.map((r, i) => ({
id: r[".id"] ?? `ros-${i}`,
dstAddress: r["dst-address"] ?? "",
gateway: r.gateway ?? "",
distance: Number.parseInt(r.distance ?? "1", 10) || 1,
scope: r.scope ? (Number.parseInt(r.scope, 10) || null) : null,
targetScope: r["target-scope"] ? (Number.parseInt(r["target-scope"], 10) || null) : null,
routingTable: r["routing-table"] ?? "main",
checkGateway: r["check-gateway"] ?? "",
country: "",
comment: r.comment ?? "",
disabled: r.disabled === "true",
}))
await replaceDbRoutes(serverId, mapped)
return reply.send({ ok: true, serverId, totalRoutes: mapped.length })
const live = mergeCachedCountry(mapRosManagedRoutes(rosRoutes), cached)
await replaceDbRoutes(serverId, live)
await appendRevisionIfChanged({
serverId,
section: "recursive-routes",
source: "observed",
payload: canonicalRecursiveRoutes(live),
})
const stored = await mapDbRoutes(serverId)
return reply.send({ routes: stored, live: true, stale: false })
} catch (err) {
app.log.warn({ serverId, err: String(err) }, "recursive live GET failed, serving cache")
return reply.send({
routes: cached,
live: false,
stale: true,
error: String(err),
})
}
})
app.put("/recursive-routes", async (req, reply) => {
const body = req.body as {
serverId?: string | number
routes?: RecursiveRouteDto[]
source?: ConfigRevisionSource
}
const serverId = parseDbServerId(body.serverId)
if (serverId === null) return reply.status(400).send({ error: "serverId is required" })
const server = (await db.select().from(servers).where(eq(servers.id, serverId)).limit(1))[0]
if (!server) return reply.status(404).send({ error: "Server not found" })
const routes = (body.routes ?? []).map((r) => ({
...r,
comment: userRecursiveComment(r.comment),
}))
const source: ConfigRevisionSource = body.source === "copy" ? "copy" : "apply"
try {
const result = await applyRecursiveToServer(server, routes, source)
const stored = await mapDbRoutes(serverId)
return reply.send({ ok: true, routes: stored, pushedRoutes: result.pushed })
} catch (err) {
return reply.status(500).send({ error: String(err) })
}
})
app.post("/recursive-routes/sync/to-router", async (req, reply) => {
const body = req.body as { serverId?: string | number } | undefined
const serverId = parseDbServerId(body?.serverId)
app.get("/recursive-routes/revisions", async (req, reply) => {
const q = req.query as { serverId?: string | number }
const serverId = parseDbServerId(q.serverId)
if (serverId === null) return reply.status(400).send({ error: "serverId is required" })
const server = (await db.select().from(servers).where(eq(servers.id, serverId)).limit(1))[0]
const revisions = await listRevisions(serverId, "recursive-routes")
return reply.send({ revisions })
})
app.post("/recursive-routes/revisions/:id/restore", {
schema: { params: RevisionIdParamSchema },
}, async (req, reply) => {
const { id } = req.params
const body = req.body as { serverId?: string | number } | undefined
const rev = await getRevisionById(id)
if (!rev) return reply.status(404).send({ error: "Revision not found" })
if (rev.section !== "recursive-routes") {
return reply.status(400).send({ error: "Revision section mismatch" })
}
const requested = parseDbServerId(body?.serverId)
if (requested !== null && requested !== rev.serverId) {
return reply.status(400).send({ error: "Revision belongs to another server" })
}
const server = (await db.select().from(servers).where(eq(servers.id, rev.serverId)).limit(1))[0]
if (!server) return reply.status(404).send({ error: "Server not found" })
const raw = Array.isArray(rev.payload) ? rev.payload : []
const routes: RecursiveRouteDto[] = raw.map((item, idx) => {
const r = item as Partial<RecursiveRouteDto>
return {
id: `rev-${idx}`,
dstAddress: r.dstAddress ?? "",
gateway: r.gateway ?? "",
distance: r.distance ?? 1,
scope: r.scope ?? null,
targetScope: r.targetScope ?? null,
routingTable: r.routingTable || "main",
checkGateway: r.checkGateway ?? "",
country: r.country ?? "",
comment: userRecursiveComment(r.comment),
disabled: Boolean(r.disabled),
}
})
try {
const client = MikrotikClient.fromServer(server)
const existing = await client.get<RosRoute[]>("/ip/route")
const managed = existing.filter(r => hasManagedRecursiveComment(r.comment ?? ""))
for (const r of managed) {
if (!r[".id"]) continue
await client.delete(`/ip/route/${encodeURIComponent(r[".id"])}`)
}
const dbRows = await mapDbRoutes(serverId)
for (const route of dbRows) {
await client.post("/ip/route", toRouterPayload(route))
}
return reply.send({ ok: true, serverId, pushedRoutes: dbRows.length })
const result = await applyRecursiveToServer(server, routes, "rollback")
const stored = await mapDbRoutes(server.id)
return reply.send({ ok: true, routes: stored, pushedRoutes: result.pushed })
} catch (err) {
return reply.status(500).send({ error: String(err) })
}
+13 -1
View File
@@ -2,6 +2,10 @@ import type { FastifyPluginAsyncZod } from "@fastify/type-provider-zod"
import { count } from "drizzle-orm"
import { listCertificatesFromServers } from "../services/certificates-service.js"
import { countWireGuardInterfaces } from "../services/wireguard-live.js"
import { countIpsecClients } from "../services/ipsec-live.js"
import { countVxlanTunnels } from "../services/vxlan-live.js"
import { countContainers } from "../services/containers-live.js"
import { countBgpSessions } from "../services/bgp-peers-live.js"
import { db } from "../db/index.js"
import {
filterRules,
@@ -24,9 +28,13 @@ const sidebarCountsRoutes: FastifyPluginAsyncZod = async (app) => {
const uptimeProbesTotal = await tableCount(uptimeProbes)
const uptimeSpeedProbesTotal = await tableCount(uptimeSpeedProbes)
const recursiveRoutesTotal = await tableCount(recursiveRoutes)
const [certRes, wireguardTotal] = await Promise.all([
const [certRes, wireguardTotal, ipsecTotal, bgpTotal, vxlanTotal, containersTotal] = await Promise.all([
listCertificatesFromServers(),
countWireGuardInterfaces().catch(() => 0),
countIpsecClients().catch(() => 0),
countBgpSessions().catch(() => 0),
countVxlanTunnels().catch(() => 0),
countContainers().catch(() => 0),
])
const certificatesTotal = certRes.certificates.length
const usersTotal = (await listUsers()).length
@@ -40,7 +48,11 @@ const sidebarCountsRoutes: FastifyPluginAsyncZod = async (app) => {
recursiveRoutes: recursiveRoutesTotal,
certificates: certificatesTotal,
wireguard: wireguardTotal,
ipsec: ipsecTotal,
users: usersTotal,
bgpSessions: bgpTotal,
vxlan: vxlanTotal,
containers: containersTotal,
})
})
}
+26
View File
@@ -0,0 +1,26 @@
import type { FastifyPluginAsyncZod } from "@fastify/type-provider-zod"
import { statisticsPivotQuerySchema, statisticsQuerySchema } from "@mmapp/contracts/statistics"
import { getStatistics, getStatisticsPivot, pivotDimsConflict } from "../services/statistics-aggregate.js"
const statisticsRoutes: FastifyPluginAsyncZod = async (app) => {
app.get("/statistics", async (req, reply) => {
const parsed = statisticsQuerySchema.safeParse(req.query ?? {})
if (!parsed.success) {
return reply.status(400).send({ error: "Некорректный период или фильтры", details: parsed.error.flatten() })
}
return reply.send(await getStatistics(parsed.data))
})
app.get("/statistics/pivot", async (req, reply) => {
const parsed = statisticsPivotQuerySchema.safeParse(req.query ?? {})
if (!parsed.success) {
return reply.status(400).send({ error: "Некорректный период или измерения", details: parsed.error.flatten() })
}
if (pivotDimsConflict(parsed.data.row, parsed.data.col)) {
return reply.status(400).send({ error: "Строки и колонки должны отличаться" })
}
return reply.send(await getStatisticsPivot(parsed.data))
})
}
export default statisticsRoutes
+24 -1
View File
@@ -27,6 +27,7 @@ import {
import { buildFlowMapHops } from "../services/traffic-flow-map-hops.js"
import { applyFlowOverlay } from "../services/traffic-flow-overlay.js"
import { listTrafficFlowHostFiles } from "../services/traffic-flow-host-files.js"
import { rebuildFlowFactsFromBuckets } from "../services/traffic-flow-facts-rebuild.js"
import { appendEvent } from "../modules/events/service/events-service.js"
const LIVE_TICK_MS = 2000
@@ -112,6 +113,7 @@ async function applyOverlayHandler(req: FastifyRequest, reply: FastifyReply) {
const result = await applyFlowOverlay(parsed.data.serverId, {
publicEndpoint: parsed.data.publicEndpoint,
requestHost: requestPublicHost(req),
disableGreFastPath: parsed.data.disableGreFastPath,
})
return reply.send(result)
} catch (e) {
@@ -203,6 +205,27 @@ const trafficFlowRoutes: FastifyPluginAsyncZod = async (app) => {
}
})
app.post("/traffic/flow/rebuild-facts", async (_req, reply) => {
try {
const result = await rebuildFlowFactsFromBuckets()
await appendEvent({
level: "info",
eventType: "traffic.flow.rebuild_facts",
sourceModule: "traffic",
title: "Пересчитан куб NetFlow",
message: `Факты ${result.facts} из ${result.buckets} сессий, дней ${result.days.length}`,
entityType: "traffic_flow",
entityId: "rebuild-facts",
payload: { buckets: result.buckets, facts: result.facts, days: result.days },
})
return reply.send(result)
} catch (err) {
const message = err instanceof Error ? err.message : String(err)
const status = message.includes("уже выполняется") ? 409 : 500
return reply.status(status).send({ error: message })
}
})
app.post("/traffic/flow/overlay", applyOverlayHandler)
app.post("/traffic/flow-overlay", applyOverlayHandler)
@@ -275,7 +298,7 @@ const trafficFlowRoutes: FastifyPluginAsyncZod = async (app) => {
try {
while (!abort.signal.aborted) {
const payload = safeBuildLiveFlowSample(liveQuery)
const payload = await safeBuildLiveFlowSample(liveQuery)
writeSse(reply.raw, payload.event, payload.data)
await sleep(LIVE_TICK_MS, abort.signal)
}
+10 -3
View File
@@ -1,7 +1,7 @@
import type { FastifyPluginAsyncZod } from "@fastify/type-provider-zod"
import { asc, desc, eq } from "drizzle-orm"
import { db } from "../db/index.js"
import { servers, uptimeSpeedProbes, uptimeSpeedTestRuns } from "../db/schema.js"
import { servers, serverSnapshots, uptimeSpeedProbes, uptimeSpeedTestRuns } from "../db/schema.js"
import { MikrotikClient } from "../services/mikrotik.js"
import { scheduleAlertEngineAfterDataCollectors } from "../services/alert-collector-hooks.js"
import { refreshScheduler, getSchedulerStatus } from "../services/scheduler.js"
@@ -502,7 +502,14 @@ const uptimeRoutes: FastifyPluginAsyncZod = async (app) => {
})
const resources = await Promise.all(allServers.map(async (s) => {
const rows = await readResourceSamplesSince(sinceIso, s.id)
const [rows, snap] = await Promise.all([
readResourceSamplesSince(sinceIso, s.id),
db.select({ boardName: serverSnapshots.boardName })
.from(serverSnapshots)
.where(eq(serverSnapshots.serverId, s.id))
.orderBy(desc(serverSnapshots.polledAt))
.limit(1),
])
const { row: pick, hasData } = pickResourceDisplayRow(rows, resourceFallbackMaxGapMs)
const cpuHistory = toSeries(rows.map((r) => r.cpuLoad), 40)
return {
@@ -515,7 +522,7 @@ const uptimeRoutes: FastifyPluginAsyncZod = async (app) => {
hddUsed: Math.max(0, ((pick?.totalHddSpace ?? 0) - (pick?.freeHddSpace ?? 0)) / (1024 * 1024)),
hddTotal: Math.max(0, (pick?.totalHddSpace ?? 0) / (1024 * 1024)),
uptimeSeconds: pick?.uptimeSeconds ?? 0,
boardName: pick?.boardName || "RouterBOARD",
boardName: snap[0]?.boardName || "RouterBOARD",
temp: undefined as number | undefined,
}
}))
+23
View File
@@ -0,0 +1,23 @@
import { eq } from "drizzle-orm"
import type { FastifyPluginAsyncZod } from "@fastify/type-provider-zod"
import { db } from "../db/index.js"
import { servers } from "../db/schema.js"
import { listVxlanTunnels, listVxlanTunnelsForServer } from "../services/vxlan-live.js"
import { ServerIdParamSchema, type ServerIdParams } from "../types/server.js"
const vxlanRoutes: FastifyPluginAsyncZod = async (app) => {
app.get("/vxlan", async (_req, reply) => {
const tunnels = await listVxlanTunnels()
return reply.send({ tunnels })
})
app.get("/servers/:id/vxlan", { schema: { params: ServerIdParamSchema } }, async (req, reply) => {
const params = req.params as ServerIdParams
const server = (await db.select().from(servers).where(eq(servers.id, params.id)).limit(1))[0]
if (!server) return reply.status(404).send({ error: "Server not found" })
const tunnels = await listVxlanTunnelsForServer(server)
return reply.send({ tunnels })
})
}
export default vxlanRoutes
+78
View File
@@ -18,6 +18,8 @@ import {
type WgParsedConfig,
} from "../services/wireguard-config.js"
import {
captureWireguardSnapshot,
fetchWireguardRestoreState,
getEnabledServerById,
listWireGuardInterfaces,
} from "../services/wireguard-live.js"
@@ -27,6 +29,16 @@ import {
putWireguardPeer,
toRosBody,
} from "../services/wireguard-ros.js"
import {
captureAndAppendRevision,
listRevisions,
loadRevisionForRestore,
type ConfigRevisionSource,
} from "../services/config-revisions.js"
import { parseWireguardSnapshot, planWireguardRestore } from "../services/entity-snapshots.js"
import { executeRosOps } from "../services/ros-ops.js"
import { parseDbServerId } from "../utils/server-id.js"
import { z } from "zod"
function serverIdParam(v: string): string {
return decodeURIComponent(v)
@@ -137,6 +149,20 @@ function findIface(
return list.find((i) => i.serverId === serverId && i.name === interfaceName)
}
async function recordWireguard(
server: NonNullable<Awaited<ReturnType<typeof getEnabledServerById>>>,
source: ConfigRevisionSource,
) {
await captureAndAppendRevision({
serverId: server.id,
section: "wireguard",
source,
capture: () => captureWireguardSnapshot(server),
})
}
const RevisionIdParamSchema = z.object({ id: z.string().min(1) })
const wireguardRoutes: FastifyPluginAsyncZod = async (app) => {
app.get("/wireguard", async (req, reply) => {
const q = req.query as { serverId?: string; includePrivateKey?: string }
@@ -145,6 +171,11 @@ const wireguardRoutes: FastifyPluginAsyncZod = async (app) => {
serverId: q.serverId,
includePrivateKey,
})
const sid = parseDbServerId(q.serverId)
if (sid !== null) {
const server = await getEnabledServerById(sid)
if (server) await recordWireguard(server, "observed")
}
return reply.send(result)
})
@@ -181,6 +212,7 @@ const wireguardRoutes: FastifyPluginAsyncZod = async (app) => {
includePrivateKey: true,
})
const created = list.interfaces.find((i) => i.name === body.name)
await recordWireguard(server, "apply")
return reply.status(201).send(created ?? { ok: true, name: body.name })
} catch (e) {
const msg = e instanceof MikrotikError ? e.message : e instanceof Error ? e.message : String(e)
@@ -210,6 +242,7 @@ const wireguardRoutes: FastifyPluginAsyncZod = async (app) => {
disabled: d.disabled === true ? "yes" : d.disabled === false ? "no" : undefined,
}),
)
await recordWireguard(server, "apply")
return reply.send({ ok: true })
} catch (e) {
const msg = e instanceof Error ? e.message : String(e)
@@ -224,6 +257,7 @@ const wireguardRoutes: FastifyPluginAsyncZod = async (app) => {
const client = MikrotikClient.fromServer(server)
try {
await client.delete(`/interface/wireguard/${encodeURIComponent(rosIdParam(rosId))}`)
await recordWireguard(server, "apply")
return reply.send({ ok: true })
} catch (e) {
const msg = e instanceof Error ? e.message : String(e)
@@ -242,6 +276,7 @@ const wireguardRoutes: FastifyPluginAsyncZod = async (app) => {
const client = MikrotikClient.fromServer(server)
try {
await putWireguardPeer(client, peerToRosBody(body))
await recordWireguard(server, "apply")
return reply.status(201).send({ ok: true })
} catch (e) {
const msg = e instanceof Error ? e.message : String(e)
@@ -277,6 +312,7 @@ const wireguardRoutes: FastifyPluginAsyncZod = async (app) => {
disabled: d.disabled === true ? "yes" : d.disabled === false ? "no" : undefined,
}),
)
await recordWireguard(server, "apply")
return reply.send({ ok: true })
} catch (e) {
const msg = e instanceof Error ? e.message : String(e)
@@ -291,6 +327,7 @@ const wireguardRoutes: FastifyPluginAsyncZod = async (app) => {
const client = MikrotikClient.fromServer(server)
try {
await client.delete(`/interface/wireguard/peers/${encodeURIComponent(rosIdParam(rosId))}`)
await recordWireguard(server, "apply")
return reply.send({ ok: true })
} catch (e) {
const msg = e instanceof Error ? e.message : String(e)
@@ -320,6 +357,7 @@ const wireguardRoutes: FastifyPluginAsyncZod = async (app) => {
const client = MikrotikClient.fromServer(server)
try {
const applied = await applyParsedConfig(client, config)
await recordWireguard(server, "copy")
return reply.send({ dryRun: false, preview, applied })
} catch (e) {
const msg = e instanceof Error ? e.message : String(e)
@@ -437,6 +475,46 @@ const wireguardRoutes: FastifyPluginAsyncZod = async (app) => {
content,
})
})
app.get("/wireguard/revisions", async (req, reply) => {
const q = req.query as { serverId?: string | number }
const serverId = parseDbServerId(q.serverId)
if (serverId === null) return reply.status(400).send({ error: "serverId is required" })
const revisions = await listRevisions(serverId, "wireguard")
return reply.send({ revisions })
})
app.post("/wireguard/revisions/:id/restore", {
schema: { params: RevisionIdParamSchema },
}, async (req, reply) => {
const { id } = req.params
const body = req.body as { serverId?: string | number } | undefined
const loaded = await loadRevisionForRestore({
id,
section: "wireguard",
requestedServerId: parseDbServerId(body?.serverId),
})
if (!loaded.ok) return reply.status(loaded.status).send({ error: loaded.error })
try {
const desired = parseWireguardSnapshot(loaded.row.payload)
const state = await fetchWireguardRestoreState(loaded.server)
const ops = planWireguardRestore(desired, {
ifaces: state.ifaces,
peers: state.peers,
addrs: state.addrs,
})
await executeRosOps(state.client, ops)
await recordWireguard(loaded.server, "rollback")
const list = await listWireGuardInterfaces({
serverId: String(loaded.server.id),
includePrivateKey: true,
})
return reply.send({ ok: true, interfaces: list.interfaces })
} catch (e) {
const msg = e instanceof Error ? e.message : String(e)
return reply.status(502).send({ error: `RouterOS: ${msg}` })
}
})
}
export default wireguardRoutes
@@ -0,0 +1,8 @@
import { initDatabase } from "../db/bootstrap.js"
import { closePool } from "../db/index.js"
import { rebuildFlowFactsFromBuckets } from "../services/traffic-flow-facts-rebuild.js"
await initDatabase()
const result = await rebuildFlowFactsFromBuckets()
console.log(JSON.stringify(result, null, 2))
await closePool()
+301 -12
View File
@@ -1,14 +1,30 @@
import { randomUUID } from "node:crypto"
import { mkdir, rm, stat, writeFile } from "node:fs/promises"
import { mkdir, rm, stat, writeFile, readFile } from "node:fs/promises"
import path from "node:path"
import { desc, eq } from "drizzle-orm"
import type { BackupScheduleSettingsDto } from "@mmapp/contracts/backups"
import type {
BackupScheduleSettingsDto,
BackupStorageSettingsDto,
PutBackupStorageSettings,
} from "@mmapp/contracts/backups"
import { db } from "../db/index.js"
import { parseJsonArray } from "../db/json.js"
import { backupEntries, backupScheduleSettings } from "../db/schema.js"
import { backupEntries, backupScheduleSettings, backupStorageSettings } from "../db/schema.js"
import { getServerRowById } from "../modules/servers/repository/servers-repository.js"
import { listServersRead } from "../modules/servers/service/servers-service.js"
import { MikrotikClient } from "./mikrotik.js"
import {
buildS3ObjectKey,
createS3ClientFromConfig,
parseS3ObjectKey,
sanitizeServerName,
s3DeleteObject,
s3GetObject,
s3ListObjects,
s3PutObject,
s3TestConnection,
type S3BackupConfig,
} from "./s3-backup-client.js"
const SETTINGS_ID = 1
const BACKUPS_DIR = path.resolve(process.cwd(), "storage", "backups")
@@ -22,9 +38,14 @@ export type BackupMeta = {
createdAt: string
kind: "manual" | "auto"
notes?: string
storage: "local" | "s3" | "both"
s3Key?: string | null
uploadError?: string | null
}
function rowToMeta(row: typeof backupEntries.$inferSelect): BackupMeta {
type BackupRow = typeof backupEntries.$inferSelect
function rowToMeta(row: BackupRow): BackupMeta {
return {
id: row.id,
serverId: row.serverId,
@@ -34,6 +55,9 @@ function rowToMeta(row: typeof backupEntries.$inferSelect): BackupMeta {
createdAt: row.createdAt,
kind: row.kind,
notes: row.notes ?? undefined,
storage: row.storage ?? "local",
s3Key: row.s3Key,
uploadError: row.uploadError,
}
}
@@ -60,16 +84,36 @@ export async function insertBackup(meta: BackupMeta): Promise<void> {
sizeBytes: meta.sizeBytes,
kind: meta.kind,
notes: meta.notes ?? null,
storage: meta.storage,
s3Key: meta.s3Key ?? null,
s3Etag: null,
uploadError: meta.uploadError ?? null,
createdAt: meta.createdAt,
})
}
async function getBackupRow(id: string): Promise<BackupRow | undefined> {
return (await db.select().from(backupEntries).where(eq(backupEntries.id, id)).limit(1))[0]
}
export async function deleteBackupRecord(id: string): Promise<BackupMeta | null> {
const hit = await getBackupById(id)
if (!hit) return null
const row = await getBackupRow(id)
if (!row) return null
const meta = rowToMeta(row)
if (row.s3Key) {
try {
const cfg = await getS3ConfigIfEnabled()
if (cfg) {
const client = createS3ClientFromConfig(cfg)
await s3DeleteObject(client, cfg.bucket, row.s3Key)
}
} catch {
/* объект мог уже отсутствовать */
}
}
await db.delete(backupEntries).where(eq(backupEntries.id, id))
await rm(path.join(BACKUPS_DIR, hit.filename), { force: true })
return hit
await rm(path.join(BACKUPS_DIR, row.filename), { force: true })
return meta
}
function fmtTs(d = new Date()): string {
@@ -167,6 +211,127 @@ export async function touchBackupScheduleRunMeta(patch: {
}).where(eq(backupScheduleSettings.id, SETTINGS_ID))
}
function defaultStorageRow() {
return {
id: SETTINGS_ID,
provider: "local" as const,
s3Endpoint: "",
s3Region: "us-east-1",
s3Bucket: "",
s3Prefix: "mikrotik",
s3AccessKeyId: "",
s3SecretAccessKey: "",
s3ForcePathStyle: true,
keepLocalCopy: true,
lastTestAt: null as string | null,
lastTestError: null as string | null,
updatedAt: new Date().toISOString(),
}
}
async function getBackupStorageSettingsRow() {
return (await db.select().from(backupStorageSettings).where(eq(backupStorageSettings.id, SETTINGS_ID)).limit(1))[0]
?? defaultStorageRow()
}
function toStorageDto(row: Awaited<ReturnType<typeof getBackupStorageSettingsRow>>): BackupStorageSettingsDto {
return {
provider: row.provider,
s3Endpoint: row.s3Endpoint,
s3Region: row.s3Region,
s3Bucket: row.s3Bucket,
s3Prefix: row.s3Prefix,
s3AccessKeyId: row.s3AccessKeyId,
secretConfigured: Boolean(row.s3SecretAccessKey),
s3ForcePathStyle: row.s3ForcePathStyle,
keepLocalCopy: row.keepLocalCopy,
lastTestAt: row.lastTestAt ?? null,
lastTestError: row.lastTestError ?? null,
updatedAt: row.updatedAt,
}
}
export async function getBackupStorageSettings(): Promise<BackupStorageSettingsDto> {
return toStorageDto(await getBackupStorageSettingsRow())
}
export async function updateBackupStorageSettings(
patch: PutBackupStorageSettings,
): Promise<BackupStorageSettingsDto> {
const prev = await getBackupStorageSettingsRow()
const now = new Date().toISOString()
const secret = patch.s3SecretAccessKey
const next = {
provider: patch.provider ?? prev.provider,
s3Endpoint: patch.s3Endpoint ?? prev.s3Endpoint,
s3Region: patch.s3Region ?? prev.s3Region,
s3Bucket: patch.s3Bucket ?? prev.s3Bucket,
s3Prefix: patch.s3Prefix ?? prev.s3Prefix,
s3AccessKeyId: patch.s3AccessKeyId ?? prev.s3AccessKeyId,
s3SecretAccessKey: secret && secret.length > 0 ? secret : prev.s3SecretAccessKey,
s3ForcePathStyle: patch.s3ForcePathStyle ?? prev.s3ForcePathStyle,
keepLocalCopy: patch.keepLocalCopy ?? prev.keepLocalCopy,
updatedAt: now,
}
if ((await db.select().from(backupStorageSettings).where(eq(backupStorageSettings.id, SETTINGS_ID)).limit(1))[0]) {
await db.update(backupStorageSettings).set(next).where(eq(backupStorageSettings.id, SETTINGS_ID))
} else {
await db.insert(backupStorageSettings).values({ id: SETTINGS_ID, ...next })
}
return await getBackupStorageSettings()
}
function rowToS3Config(row: Awaited<ReturnType<typeof getBackupStorageSettingsRow>>): S3BackupConfig | null {
if (row.provider !== "s3") return null
if (!row.s3Bucket.trim() || !row.s3AccessKeyId.trim() || !row.s3SecretAccessKey) return null
return {
endpoint: row.s3Endpoint,
region: row.s3Region,
bucket: row.s3Bucket.trim(),
prefix: row.s3Prefix,
accessKeyId: row.s3AccessKeyId,
secretAccessKey: row.s3SecretAccessKey,
forcePathStyle: row.s3ForcePathStyle,
}
}
async function getS3ConfigIfEnabled(): Promise<S3BackupConfig | null> {
return rowToS3Config(await getBackupStorageSettingsRow())
}
export async function testBackupStorageConnection(): Promise<BackupStorageSettingsDto> {
const row = await getBackupStorageSettingsRow()
const cfg = rowToS3Config(row)
const now = new Date().toISOString()
if (!cfg) {
const error = row.provider === "s3"
? "Заполните bucket, ключ доступа и секрет"
: "S3 не выбран"
await persistStorageTest(now, error)
throw new Error(error)
}
try {
const client = createS3ClientFromConfig(cfg)
await s3TestConnection(client, cfg.bucket)
await persistStorageTest(now, null)
} catch (err) {
const message = err instanceof Error ? err.message : String(err)
await persistStorageTest(now, message)
throw new Error(message)
}
return await getBackupStorageSettings()
}
async function persistStorageTest(at: string, error: string | null) {
const exists = (await db.select().from(backupStorageSettings).where(eq(backupStorageSettings.id, SETTINGS_ID)).limit(1))[0]
const patch = { lastTestAt: at, lastTestError: error, updatedAt: at }
if (exists) {
await db.update(backupStorageSettings).set(patch).where(eq(backupStorageSettings.id, SETTINGS_ID))
} else {
await db.insert(backupStorageSettings).values({ id: SETTINGS_ID, ...patch })
}
}
export async function resolveBackupServerIds(settings: BackupScheduleSettingsDto): Promise<string[]> {
const enabled = new Set((await listServersRead()).map((s) => String(s.id)))
const requested = settings.serverIds.length > 0 ? settings.serverIds : [...enabled]
@@ -214,6 +379,23 @@ export function isBackupDue(now: Date, settings: BackupScheduleSettingsDto, last
return true
}
async function uploadBackupToS3(params: {
serverName: string
filename: string
body: string
}): Promise<{ key: string; etag?: string } | { error: string }> {
const cfg = await getS3ConfigIfEnabled()
if (!cfg) return { error: "S3 не настроен" }
try {
const client = createS3ClientFromConfig(cfg)
const key = buildS3ObjectKey(cfg.prefix, sanitizeServerName(params.serverName), params.filename)
const put = await s3PutObject(client, cfg.bucket, key, params.body)
return { key, etag: put.etag }
} catch (err) {
return { error: err instanceof Error ? err.message : String(err) }
}
}
export async function runBackupForServer(
id: string,
kind: BackupMeta["kind"],
@@ -230,12 +412,33 @@ export async function runBackupForServer(
const client = MikrotikClient.fromServer(row)
const script = await client.exportConfigScript()
const ts = fmtTs()
const safeServer = row.name.replace(/[^a-zA-Z0-9._-]+/g, "_")
const safeServer = sanitizeServerName(row.name)
const filename = `${safeServer}_${ts}.rsc`
const filePath = path.join(BACKUPS_DIR, filename)
await ensureBackupStorage()
await writeFile(filePath, script, "utf8")
const st = await stat(filePath)
const storageRow = await getBackupStorageSettingsRow()
let storage: BackupMeta["storage"] = "local"
let s3Key: string | null = null
let s3Etag: string | null = null
let uploadError: string | null = null
if (storageRow.provider === "s3") {
const uploaded = await uploadBackupToS3({ serverName: row.name, filename, body: script })
if ("key" in uploaded) {
s3Key = uploaded.key
s3Etag = uploaded.etag ?? null
storage = storageRow.keepLocalCopy ? "both" : "s3"
if (!storageRow.keepLocalCopy) {
await rm(filePath, { force: true })
}
} else {
uploadError = uploaded.error
storage = "local"
}
}
const meta: BackupMeta = {
id: randomUUID(),
serverId: row.id,
@@ -245,8 +448,24 @@ export async function runBackupForServer(
createdAt: new Date().toISOString(),
kind,
notes,
storage,
s3Key,
uploadError,
}
await insertBackup(meta)
await db.insert(backupEntries).values({
id: meta.id,
serverId: meta.serverId,
serverName: meta.serverName,
filename: meta.filename,
sizeBytes: meta.sizeBytes,
kind: meta.kind,
notes: meta.notes ?? null,
storage: meta.storage,
s3Key: meta.s3Key ?? null,
s3Etag,
uploadError: meta.uploadError ?? null,
createdAt: meta.createdAt,
})
return meta
}
@@ -257,12 +476,82 @@ export async function pruneBackupsForServer(serverId: string, keepCount: number)
if (rows.length <= keepCount) return 0
const toDelete = rows.slice(keepCount)
for (const hit of toDelete) {
await db.delete(backupEntries).where(eq(backupEntries.id, hit.id))
await rm(path.join(BACKUPS_DIR, hit.filename), { force: true })
await deleteBackupRecord(hit.id)
}
return toDelete.length
}
export async function readBackupContent(meta: BackupMeta): Promise<Buffer> {
if ((meta.storage === "s3" || meta.storage === "both") && meta.s3Key) {
try {
const cfg = await getS3ConfigIfEnabled()
if (cfg) {
const client = createS3ClientFromConfig(cfg)
return await s3GetObject(client, cfg.bucket, meta.s3Key)
}
} catch {
/* fallback: локальная копия, если есть */
}
}
return await readFile(path.join(BACKUPS_DIR, meta.filename))
}
export async function restoreBackupToDevice(id: string): Promise<{ filename: string; serverName: string }> {
const meta = await getBackupById(id)
if (!meta) throw new Error("Бэкап не найден")
if (!meta.serverId) throw new Error("Сервер бэкапа удалён — восстановить нельзя")
const row = await getServerRowById(meta.serverId)
if (!row) throw new Error("Сервер не найден")
const content = await readBackupContent(meta)
const client = MikrotikClient.fromServer(row)
const uploaded = await client.uploadTextFile(meta.filename, content.toString("utf8"), 60_000)
await client.importUploadedFile(uploaded)
return { filename: meta.filename, serverName: row.name }
}
export async function syncBackupsFromS3(): Promise<{ imported: number; skipped: number }> {
const cfg = await getS3ConfigIfEnabled()
if (!cfg) throw new Error("S3 не настроен")
const client = createS3ClientFromConfig(cfg)
const objects = await s3ListObjects(client, cfg.bucket, cfg.prefix)
const existing = new Set(
(await db.select({ filename: backupEntries.filename, s3Key: backupEntries.s3Key }).from(backupEntries))
.flatMap((row) => [row.filename, row.s3Key].filter((v): v is string => Boolean(v))),
)
let imported = 0
let skipped = 0
for (const obj of objects) {
if (!obj.key.toLowerCase().endsWith(".rsc")) {
skipped += 1
continue
}
const parsed = parseS3ObjectKey(obj.key)
if (existing.has(obj.key) || existing.has(parsed.filename)) {
skipped += 1
continue
}
const createdAt = obj.lastModified ?? new Date().toISOString()
await db.insert(backupEntries).values({
id: randomUUID(),
serverId: null,
serverName: parsed.serverName,
filename: parsed.filename,
sizeBytes: obj.size,
kind: "auto",
notes: "Импорт из S3",
storage: "s3",
s3Key: obj.key,
s3Etag: null,
uploadError: null,
createdAt,
})
existing.add(obj.key)
existing.add(parsed.filename)
imported += 1
}
return { imported, skipped }
}
export function getBackupsDir(): string {
return BACKUPS_DIR
}
+13
View File
@@ -28,3 +28,16 @@ export async function fetchBgpSessionsForAlerts(): Promise<BgpSessionRead[]> {
)
return results.flat()
}
export async function countBgpSessions(): Promise<number> {
try {
const result = await Promise.race([
fetchBgpSessionsForAlerts(),
new Promise<null>((resolve) => setTimeout(() => resolve(null), 8_000)),
])
if (!result) return 0
return result.length
} catch {
return 0
}
}
@@ -0,0 +1,65 @@
import assert from "node:assert/strict"
import {
certificateRole,
isCaCertificate,
mapRosCertificateRow,
type RosCertificateRow,
} from "./certificate-parse.js"
// Реальные серты пользователя: MyCA (root) → vpn-server, client1, anakondra.
const myCa: RosCertificateRow = {
name: "MyCA",
"common-name": "MyCA",
"key-usage": "key-cert-sign,crl-sign",
ca: "",
flags: "KAT",
authority: "true",
}
const vpnServer: RosCertificateRow = {
name: "vpn-server",
"common-name": "vpn.example.com",
"key-usage": "digital-signature,key-encipherment,key-cert-sign,crl-sign,tls-server,tls-client",
ca: "MyCA",
flags: "KLAT",
}
const client1: RosCertificateRow = {
name: "client1",
"common-name": "client1",
"key-usage": "digital-signature,key-encipherment,key-cert-sign,crl-sign,tls-server,tls-client",
ca: "MyCA",
flags: "KLAT",
}
const anakondra: RosCertificateRow = { ...client1, name: "anakondra", "common-name": "anakondra" }
{
assert.ok(isCaCertificate(myCa))
// default key-usage содержит key-cert-sign, но ca заполнен → не CA
assert.ok(!isCaCertificate(vpnServer))
assert.ok(!isCaCertificate(client1))
}
{
const ctx = {
peerCertNames: ["vpn-server"],
identityCertNames: ["client1", "anakondra"],
}
assert.equal(certificateRole(myCa, ctx), "ca")
assert.equal(certificateRole(vpnServer, ctx), "server")
// reference-based важнее key-usage: у client1 в key-usage есть tls-server, но он client
assert.equal(certificateRole(client1, ctx), "client")
assert.equal(certificateRole(anakondra, ctx), "client")
// без reference-контекста: подпись локальным CA делает серт клиентским, несмотря на tls-server в usage
assert.equal(certificateRole(client1), "client")
assert.equal(certificateRole(anakondra), "client")
// без контекста — по key-usage (default содержит оба, поэтому server)
assert.equal(certificateRole({ name: "x", "key-usage": "tls-client" }), "client")
assert.equal(certificateRole({ name: "y", "key-usage": "digital-signature" }), "other")
}
{
const dto = mapRosCertificateRow(1, "mt", client1)
assert.equal(dto?.signedByCertName, "MyCA")
assert.equal(mapRosCertificateRow(1, "mt", myCa)?.signedByCertName, undefined)
}
console.log("certificate-parse.test.ts: ok")
+60
View File
@@ -2,6 +2,64 @@ import type { CertificateDto } from "@mmapp/contracts/certificates"
export type RosCertificateRow = Record<string, string | undefined>
/** Роль сертификата в контексте IPsec/IKEv2. */
export type CertRole = "ca" | "server" | "client" | "other"
/** Контекст для reference-based определения роли (peer/identity ссылки точнее, чем key-usage). */
export interface CertificateRoleContext {
/** Имена сертификатов из `peer.certificate`. */
peerCertNames?: Iterable<string>
/** Имена сертификатов из `identity.remote-certificate`. */
identityCertNames?: Iterable<string>
}
function rosTrue(v: string | undefined): boolean {
return v === "true" || v === "yes"
}
/** Флаги RouterOS (`flags` = строка вида "KLAT"). */
function certFlags(row: RosCertificateRow): string {
return (row.flags ?? "").toUpperCase()
}
/**
* CA: `key-cert-sign`, серт self-signed (read-only `ca` пуст).
* Только флаг authority/`A` не годится: RouterOS ставит `A` и серверным сертам,
* т.к. дефолтный `key-usage` содержит `key-cert-sign`. Реальный признак отсутствие подписавшего.
*/
export function isCaCertificate(row: RosCertificateRow): boolean {
const usage = row["key-usage"] ?? ""
const signer = (row.ca ?? "").trim()
if (signer !== "") return false
return certFlags(row).includes("A") || rosTrue(row.authority) || usage.includes("key-cert-sign")
}
/**
* Роль сертификата. Приоритет:
* 1. CA (authority / key-cert-sign без подписавшего);
* 2. серт из `peer.certificate` server;
* 3. серт из `identity.remote-certificate` client;
* 4. `key-usage` tls-server server, tls-client client (reference-based важнее: RouterOS
* дефолт содержит и tls-server, и tls-client).
*/
export function certificateRole(
row: RosCertificateRow,
ctx: CertificateRoleContext = {},
): CertRole {
const name = (row.name ?? "").trim()
if (isCaCertificate(row)) return "ca"
if (name && new Set(ctx.peerCertNames ?? []).has(name)) return "server"
if (name && new Set(ctx.identityCertNames ?? []).has(name)) return "client"
// Серт, подписанный локальным CA (`ca` заполнен), но не используемый как peer-серт — клиентский.
// `client1`/`anakondra` имеют дефолтный key-usage (tls-server+tls-client), поэтому по usage
// они выглядели бы «server».
if ((row.ca ?? "").trim() !== "") return "client"
const usage = row["key-usage"] ?? ""
if (usage.includes("tls-server")) return "server"
if (usage.includes("tls-client")) return "client"
return "other"
}
function parseRosDate(raw: string | undefined): Date | null {
if (!raw?.trim()) return null
const d = new Date(raw)
@@ -88,6 +146,8 @@ export function mapRosCertificateRow(
trusted: row.trusted === "true",
status,
acmeStatus: row["acme-status"]?.trim() || undefined,
/** Имя CA, которым серт подписан на устройстве (read-only поле `ca`). */
signedByCertName: (row.ca ?? "").trim() || undefined,
}
}
@@ -113,6 +113,15 @@ export async function collectCertificatesRenewOnce(): Promise<CertificatesRenewR
continue
}
const stillOn = await getCertificateRenewSettings()
if (!stillOn.enabled) {
item.action = "skipped"
item.message = "Автообновление выключено"
snapshot.skippedTargets += 1
snapshot.targets?.push(item)
continue
}
const jobId = randomUUID()
await createIssueJobRecord({
id: jobId,
@@ -0,0 +1,96 @@
import assert from "node:assert/strict"
import { hasManagedCommentPrefix, isOwnedRecursiveComment, managedRecursiveComment, stripManagedRecursiveComment } from "../managed-markers.js"
import {
planBgpInApply,
planRecursiveApply,
unmanagedRouteIds,
} from "./config-apply-plan.js"
import {
canonicalFilterRules,
fingerprintPayload,
} from "./config-revisions.js"
import { mapRosManagedRoutes } from "./config-apply-plan.js"
{
const fp1 = fingerprintPayload(canonicalFilterRules([
{ community: "65001:100", action: "route", gateway: "10.0.0.1", gatewayTunnelId: "gre1", description: "a" },
]))
const fp2 = fingerprintPayload(canonicalFilterRules([
{ community: "65001:100", action: "route", gateway: "10.0.0.1", gatewayTunnelId: "gre1", description: "a" },
]))
const fp3 = fingerprintPayload(canonicalFilterRules([
{ community: "65001:100", action: "route", gateway: "10.0.0.2", gatewayTunnelId: "gre1", description: "a" },
]))
assert.equal(fp1, fp2)
assert.notEqual(fp1, fp3)
}
{
const existing = [
{ ".id": "*1", chain: "bgp-in", comment: "MikrotikManager: msk", rule: "if (true) { accept; }" },
{ ".id": "*2", chain: "bgp-in", comment: "legacy", rule: "if (bgp-communities includes 1:1) { reject; }" },
{ ".id": "*3", chain: "bgp-out", comment: "MikrotikManager: other", rule: "if (bgp-communities includes 1:1) { accept; }" },
]
const patch = planBgpInApply(existing, 3)
assert.equal(patch.action, "patch")
assert.equal(patch.managedId, "*1")
assert.deepEqual(patch.conflictIds, ["*2"])
const create = planBgpInApply(existing.filter((r) => r[".id"] !== "*1"), 1)
assert.equal(create.action, "create")
assert.equal(create.managedId, undefined)
const del = planBgpInApply(existing, 0)
assert.equal(del.action, "delete")
assert.equal(del.managedId, "*1")
const noop = planBgpInApply([], 0)
assert.equal(noop.action, "noop")
}
{
const routes = [
{ ".id": "*10", comment: "MikrotikManager:recursive via de", static: "true", "dst-address": "8.8.8.8/32", gateway: "1.1.1.1" },
{ ".id": "*11", comment: "user static", static: "true", "dst-address": "1.1.1.1/32", gateway: "9.9.9.9" },
{ ".id": "*12", comment: "recursive: old", static: "true", "dst-address": "9.9.9.9/32", gateway: "1.1.1.1" },
]
const plan = planRecursiveApply(routes)
assert.deepEqual(plan.deleteIds, ["*10", "*12"])
assert.deepEqual(unmanagedRouteIds(routes), ["*11"])
}
{
assert.equal(stripManagedRecursiveComment("MikrotikManager:recursive via de"), "via de")
assert.equal(stripManagedRecursiveComment("recursive: old"), "old")
assert.equal(managedRecursiveComment("MikrotikManager:recursive via de"), "MikrotikManager:recursive via de")
assert.equal(isOwnedRecursiveComment("MikrotikManager:recursive via de"), true)
assert.equal(isOwnedRecursiveComment("recursive: x"), true)
assert.equal(isOwnedRecursiveComment("user static"), false)
assert.equal(hasManagedCommentPrefix("MikrotikManager: msk"), true)
}
{
const mapped = mapRosManagedRoutes([
{
".id": "*1",
static: "true",
"dst-address": "10.9.9.2/32",
gateway: "1.2.3.4",
comment: "MikrotikManager:recursive hop-de",
distance: "1",
},
{
".id": "*2",
static: "true",
"dst-address": "10.9.9.3/32",
gateway: "1.2.3.4",
comment: "not ours",
distance: "1",
},
])
assert.equal(mapped.length, 1)
assert.equal(mapped[0]?.dstAddress, "10.9.9.2/32")
assert.equal(mapped[0]?.comment, "hop-de")
}
console.log("config-apply-plan.test.ts: ok")
+140
View File
@@ -0,0 +1,140 @@
import {
hasManagedCommentPrefix,
isOwnedRecursiveComment,
stripManagedRecursiveComment,
} from "../managed-markers.js"
export type RosFilterRuleLike = {
".id"?: string
chain?: string
rule?: string
comment?: string
}
export type BgpInApplyAction = "patch" | "create" | "delete" | "noop"
export interface BgpInApplyPlan {
action: BgpInApplyAction
managedId?: string
conflictIds: string[]
}
function isInBgpIn(rule: RosFilterRuleLike): boolean {
return (rule.chain ?? "").trim().toLowerCase() === "bgp-in"
}
export function planBgpInApply(
existing: RosFilterRuleLike[],
rulesCount: number,
): BgpInApplyPlan {
const managed = existing.find(
(r) => isInBgpIn(r) && hasManagedCommentPrefix(r.comment ?? ""),
)
const conflictIds = existing
.filter((r) =>
isInBgpIn(r) &&
!hasManagedCommentPrefix(r.comment ?? "") &&
/bgp-communities/i.test(r.rule ?? ""),
)
.map((r) => r[".id"])
.filter((id): id is string => Boolean(id))
if (rulesCount > 0) {
return {
action: managed?.[".id"] ? "patch" : "create",
managedId: managed?.[".id"],
conflictIds,
}
}
if (managed?.[".id"]) {
return { action: "delete", managedId: managed[".id"], conflictIds }
}
return { action: "noop", conflictIds }
}
export type RosRouteLike = {
".id"?: string
comment?: string
static?: string
dynamic?: string
blackhole?: string
unreachable?: string
prohibit?: string
"dst-address"?: string
gateway?: string
}
export function planRecursiveApply(existing: RosRouteLike[]): { deleteIds: string[] } {
return {
deleteIds: existing
.filter((r) => isOwnedRecursiveComment(r.comment))
.map((r) => r[".id"])
.filter((id): id is string => Boolean(id)),
}
}
export function unmanagedRouteIds(existing: RosRouteLike[]): string[] {
return existing
.filter((r) => Boolean(r[".id"]) && !isOwnedRecursiveComment(r.comment))
.map((r) => r[".id"] as string)
}
export function userRecursiveComment(comment: string | undefined): string {
return stripManagedRecursiveComment(comment ?? "")
}
function isIpGateway(gw: string): boolean {
return /^\d{1,3}(\.\d{1,3}){3}(?:%\S+)?$/.test(gw.trim())
}
export function isManagedRecursiveRoute(r: RosRouteLike): boolean {
if ((r.static ?? "false") !== "true") return false
if ((r.dynamic ?? "false") === "true") return false
if ((r.blackhole ?? "false") === "true") return false
if ((r.unreachable ?? "false") === "true") return false
if ((r.prohibit ?? "false") === "true") return false
const dst = r["dst-address"] ?? ""
const gw = r.gateway ?? ""
if (!dst || !gw) return false
if (!isIpGateway(gw)) return false
return isOwnedRecursiveComment(r.comment)
}
export interface MappedRecursiveRoute {
id: string
dstAddress: string
gateway: string
distance: number
scope: number | null
targetScope: number | null
routingTable: string
checkGateway: string
country: string
comment: string
disabled: boolean
}
export function mapRosManagedRoutes(
rosRoutes: Array<RosRouteLike & {
distance?: string
scope?: string
"target-scope"?: string
"routing-table"?: string
"check-gateway"?: string
disabled?: string
}>,
): MappedRecursiveRoute[] {
return rosRoutes.filter(isManagedRecursiveRoute).map((r, i) => ({
id: r[".id"] ?? `ros-${i}`,
dstAddress: r["dst-address"] ?? "",
gateway: r.gateway ?? "",
distance: Number.parseInt(r.distance ?? "1", 10) || 1,
scope: r.scope ? (Number.parseInt(r.scope, 10) || null) : null,
targetScope: r["target-scope"] ? (Number.parseInt(r["target-scope"], 10) || null) : null,
routingTable: r["routing-table"] ?? "main",
checkGateway: r["check-gateway"] ?? "",
country: "",
comment: userRecursiveComment(r.comment),
disabled: r.disabled === "true",
}))
}
@@ -0,0 +1,101 @@
import assert from "node:assert/strict"
import { withPgOrSkip } from "../test/pg.js"
import { dbQuery } from "../db/index.js"
import {
appendRevisionIfChanged,
fingerprintPayload,
getRevisionById,
listRevisions,
pruneRevisions,
} from "./config-revisions.js"
if (!(await withPgOrSkip())) {
console.log("config-revisions.test.ts: skip")
process.exit(0)
}
const tag = `rev-test-${Date.now()}`
await dbQuery(`INSERT INTO servers (name, host) VALUES ($1, '127.0.0.1')`, [tag])
const { rows } = await dbQuery<{ id: number }>(`SELECT id FROM servers WHERE name = $1 LIMIT 1`, [tag])
const serverId = rows[0]?.id
assert.ok(serverId)
try {
const payloadA = [{ community: "1:1", action: "route" }]
const first = await appendRevisionIfChanged({
serverId,
section: "filters",
source: "apply",
payload: payloadA,
})
assert.equal(first.created, true)
assert.equal(first.revision.source, "apply")
const dup = await appendRevisionIfChanged({
serverId,
section: "filters",
source: "observed",
payload: payloadA,
})
assert.equal(dup.created, false)
assert.equal(dup.revision.id, first.revision.id)
const payloadB = [{ community: "1:2", action: "blackhole" }]
const second = await appendRevisionIfChanged({
serverId,
section: "filters",
source: "rollback",
payload: payloadB,
})
assert.equal(second.created, true)
assert.equal(second.revision.source, "rollback")
assert.notEqual(second.revision.fingerprint, first.revision.fingerprint)
const listed = await listRevisions(serverId, "filters")
assert.equal(listed.length, 2)
assert.equal(listed[0]?.source, "rollback")
const stored = await getRevisionById(second.revision.id)
assert.ok(stored)
assert.equal(fingerprintPayload(stored.payload), second.revision.fingerprint)
const objPayload = { rules: [{ chain: "input" }], addressLists: [{ list: "vip" }] }
const objRev = await appendRevisionIfChanged({
serverId,
section: "firewall",
source: "apply",
payload: objPayload,
})
assert.equal(objRev.created, true)
assert.equal(objRev.revision.itemCount, 2)
const objStored = await getRevisionById(objRev.revision.id)
assert.ok(objStored)
assert.ok(!Array.isArray(objStored.payload))
assert.equal(fingerprintPayload(objStored.payload), objRev.revision.fingerprint)
const objDup = await appendRevisionIfChanged({
serverId,
section: "firewall",
source: "rollback",
payload: objPayload,
})
assert.equal(objDup.created, false)
assert.equal(objDup.revision.source, "apply")
for (let i = 0; i < 4; i++) {
await appendRevisionIfChanged({
serverId,
section: "filters",
source: "apply",
payload: [{ community: `9:${i}`, action: "route" }],
})
}
const pruned = await pruneRevisions(serverId, "filters", 3)
assert.ok(pruned >= 1)
const after = await listRevisions(serverId, "filters")
assert.equal(after.length, 3)
} finally {
await dbQuery(`DELETE FROM servers WHERE id = $1`, [serverId])
}
console.log("config-revisions.test.ts: ok")
+237
View File
@@ -0,0 +1,237 @@
import { createHash, randomUUID } from "node:crypto"
import { and, desc, eq } from "drizzle-orm"
import { db } from "../db/index.js"
import { configRevisions, servers, type ConfigRevisionRow } from "../db/schema.js"
export const CONFIG_REVISION_KEEP = 50
export const CONFIG_SECTIONS = [
"filters",
"recursive-routes",
"firewall",
"wireguard",
"ipsec",
"gre",
] as const
export type ConfigSection = (typeof CONFIG_SECTIONS)[number]
export type ConfigRevisionSource = "apply" | "rollback" | "observed" | "copy"
export interface ConfigRevisionDto {
id: string
serverId: string
section: ConfigSection
source: ConfigRevisionSource
fingerprint: string
createdAt: string
note: string | null
itemCount: number
}
export function stableStringify(value: unknown): string {
if (value === null || typeof value !== "object") return JSON.stringify(value)
if (Array.isArray(value)) return `[${value.map(stableStringify).join(",")}]`
const obj = value as Record<string, unknown>
const keys = Object.keys(obj).sort()
return `{${keys.map((k) => `${JSON.stringify(k)}:${stableStringify(obj[k])}`).join(",")}}`
}
export function fingerprintPayload(payload: unknown): string {
return createHash("sha256").update(stableStringify(payload)).digest("hex")
}
export function revisionItemCount(payload: unknown): number {
if (Array.isArray(payload)) return payload.length
if (payload && typeof payload === "object") {
let n = 0
for (const value of Object.values(payload as Record<string, unknown>)) {
if (Array.isArray(value)) n += value.length
}
return n
}
return 0
}
export function persistablePayload(payload: unknown): unknown {
if (payload === undefined) return []
return payload
}
export function canonicalFilterRules(
rules: Array<{
community?: string
action?: string
gateway?: string
gatewayTunnelId?: string
description?: string
}>,
): unknown[] {
return rules.map((r) => ({
community: (r.community ?? "").trim(),
action: r.action === "blackhole" ? "blackhole" : "route",
gateway: r.gateway ?? "",
gatewayTunnelId: r.gatewayTunnelId ?? "",
description: r.description ?? "",
}))
}
export function canonicalRecursiveRoutes(
routes: Array<{
dstAddress?: string
gateway?: string
distance?: number
scope?: number | null
targetScope?: number | null
routingTable?: string
checkGateway?: string
comment?: string
disabled?: boolean
country?: string
}>,
): unknown[] {
return routes.map((r) => ({
dstAddress: (r.dstAddress ?? "").trim(),
gateway: r.gateway ?? "",
distance: r.distance ?? 1,
scope: r.scope ?? null,
targetScope: r.targetScope ?? null,
routingTable: r.routingTable || "main",
checkGateway: r.checkGateway ?? "",
comment: r.comment ?? "",
disabled: Boolean(r.disabled),
country: r.country ?? "",
}))
}
export function toRevisionDto(row: ConfigRevisionRow): ConfigRevisionDto {
return {
id: row.id,
serverId: String(row.serverId),
section: row.section as ConfigSection,
source: row.source,
fingerprint: row.fingerprint,
createdAt: row.createdAt,
note: row.note ?? null,
itemCount: revisionItemCount(row.payload),
}
}
export async function listRevisions(
serverId: number,
section: ConfigSection,
limit = CONFIG_REVISION_KEEP,
): Promise<ConfigRevisionDto[]> {
const rows = await db
.select()
.from(configRevisions)
.where(and(eq(configRevisions.serverId, serverId), eq(configRevisions.section, section)))
.orderBy(desc(configRevisions.createdAt))
.limit(limit)
return rows.map(toRevisionDto)
}
export async function getRevisionById(id: string): Promise<ConfigRevisionRow | undefined> {
return (await db.select().from(configRevisions).where(eq(configRevisions.id, id)).limit(1))[0]
}
export async function pruneRevisions(
serverId: number,
section: ConfigSection,
keep = CONFIG_REVISION_KEEP,
): Promise<number> {
const rows = await db
.select({ id: configRevisions.id })
.from(configRevisions)
.where(and(eq(configRevisions.serverId, serverId), eq(configRevisions.section, section)))
.orderBy(desc(configRevisions.createdAt))
const extra = rows.slice(keep)
if (extra.length === 0) return 0
for (const row of extra) {
await db.delete(configRevisions).where(eq(configRevisions.id, row.id))
}
return extra.length
}
export async function appendRevisionIfChanged(input: {
serverId: number
section: ConfigSection
source: ConfigRevisionSource
payload: unknown
note?: string | null
}): Promise<{ created: boolean; revision: ConfigRevisionDto }> {
const payload = persistablePayload(input.payload)
const fingerprint = fingerprintPayload(payload)
const latest = (await db
.select()
.from(configRevisions)
.where(and(
eq(configRevisions.serverId, input.serverId),
eq(configRevisions.section, input.section),
))
.orderBy(desc(configRevisions.createdAt))
.limit(1))[0]
if (latest?.fingerprint === fingerprint) {
return { created: false, revision: toRevisionDto(latest) }
}
const now = new Date().toISOString()
const id = randomUUID()
await db.insert(configRevisions).values({
id,
serverId: input.serverId,
section: input.section,
source: input.source,
fingerprint,
payload,
note: input.note ?? null,
createdAt: now,
})
await pruneRevisions(input.serverId, input.section)
const row = await getRevisionById(id)
if (!row) throw new Error("config-revisions: insert vanished")
return { created: true, revision: toRevisionDto(row) }
}
/** После успешного mutate: capture live → append, ошибки snapshot не валят мутацию. */
export async function captureAndAppendRevision(input: {
serverId: number
section: ConfigSection
source: ConfigRevisionSource
capture: () => Promise<unknown>
note?: string | null
}): Promise<{ created: boolean; revision: ConfigRevisionDto } | null> {
try {
const payload = await input.capture()
return await appendRevisionIfChanged({
serverId: input.serverId,
section: input.section,
source: input.source,
payload,
note: input.note,
})
} catch {
return null
}
}
export async function loadRevisionForRestore(opts: {
id: string
section: ConfigSection
requestedServerId: number | null
}): Promise<
| { ok: true; row: ConfigRevisionRow; server: typeof servers.$inferSelect }
| { ok: false; status: number; error: string }
> {
const row = await getRevisionById(opts.id)
if (!row) return { ok: false, status: 404, error: "Revision not found" }
if (row.section !== opts.section) {
return { ok: false, status: 400, error: "Revision section mismatch" }
}
if (opts.requestedServerId !== null && opts.requestedServerId !== row.serverId) {
return { ok: false, status: 400, error: "Revision belongs to another server" }
}
const server = (await db.select().from(servers).where(eq(servers.id, row.serverId)).limit(1))[0]
if (!server) return { ok: false, status: 404, error: "Server not found" }
return { ok: true, row, server }
}
@@ -0,0 +1,46 @@
import assert from "node:assert/strict"
import { mapContainerRow } from "./containers-live.js"
const server = {
id: 3,
name: "mt-spb",
host: "10.0.1.1",
} as Parameters<typeof mapContainerRow>[0]
const row = mapContainerRow(
server,
{
".id": "*A",
name: "adguard",
"remote-image": "adguard/adguardhome:latest",
interface: "veth-adguard",
envlist: "adguard-envs",
mounts: "agh-conf,agh-work",
status: "running",
"start-on-boot": "true",
comment: "DNS",
},
[
{ name: "adguard-envs", key: "FOO", value: "bar" },
{ name: "other", key: "SKIP", value: "x" },
],
[
{ name: "agh-conf", dst: "/opt/conf", src: "/disk1/conf" },
{ name: "agh-work", dst: "/opt/work" },
],
0,
)
assert.equal(row.rosId, "*A")
assert.equal(row.image, "adguard/adguardhome")
assert.equal(row.tag, "latest")
assert.equal(row.status, "running")
assert.deepEqual(row.interfaces, ["veth-adguard"])
assert.deepEqual(row.envs, [{ key: "FOO", value: "bar" }])
assert.deepEqual(row.mounts, [
{ dst: "/opt/conf", src: "/disk1/conf" },
{ dst: "/opt/work", src: undefined },
])
assert.equal(row.startOnBoot, true)
console.log("containers-live.test.ts: ok")
+215
View File
@@ -0,0 +1,215 @@
import { eq } from "drizzle-orm"
import { db } from "../db/index.js"
import { servers } from "../db/schema.js"
import { MikrotikClient, MikrotikError } from "./mikrotik.js"
type ServerRow = typeof servers.$inferSelect
interface RosContainer {
".id"?: string
name?: string
tag?: string
"remote-image"?: string
interface?: string
envlist?: string
mounts?: string
cmd?: string
"start-on-boot"?: string
comment?: string
status?: string
"memory-high"?: string
cpu?: string
}
interface RosContainerEnv {
name?: string
key?: string
value?: string
}
interface RosContainerMount {
name?: string
src?: string
dst?: string
}
export type ContainerLiveStatus = "running" | "stopped" | "error"
export type ContainerLive = {
id: string
rosId: string
name: string
serverId: string
image: string
tag: string
status: ContainerLiveStatus
envs: { key: string; value: string }[]
mounts: { dst: string; src?: string }[]
interfaces: string[]
cmd?: string
startOnBoot: boolean
comment: string
uptime?: string
cpu?: number
memMb?: number
}
function rosYes(v: string | undefined): boolean {
return v === "true" || v === "yes"
}
function mapStatus(raw: string | undefined): ContainerLiveStatus {
const s = (raw ?? "").toLowerCase()
if (s === "running") return "running"
if (s === "error" || s === "failed") return "error"
return "stopped"
}
function splitCsv(v: string | undefined): string[] {
return (v ?? "")
.split(",")
.map((x) => x.trim())
.filter(Boolean)
}
function parseImageTag(c: RosContainer): { image: string; tag: string } {
const remote = (c["remote-image"] ?? "").trim()
if (remote) {
const idx = remote.lastIndexOf(":")
if (idx > 0 && !remote.slice(idx + 1).includes("/")) {
return { image: remote.slice(0, idx), tag: remote.slice(idx + 1) }
}
return { image: remote, tag: (c.tag ?? "latest").trim() || "latest" }
}
return { image: (c.name ?? "").trim(), tag: (c.tag ?? "latest").trim() || "latest" }
}
function isMissingPackage(err: unknown): boolean {
if (err instanceof MikrotikError) {
if (err.statusCode === 404) return true
const body = err.body.toLowerCase()
return body.includes("no such command") || body.includes("not found") || body.includes("unknown")
}
const msg = err instanceof Error ? err.message.toLowerCase() : String(err).toLowerCase()
return msg.includes("no such command") || msg.includes("404")
}
export function mapContainerRow(
server: ServerRow,
c: RosContainer,
envs: RosContainerEnv[],
mounts: RosContainerMount[],
idx: number,
): ContainerLive {
const rosId = String(c[".id"] ?? `c-${idx}`)
const name = (c.name ?? "").trim() || `container-${idx + 1}`
const { image, tag } = parseImageTag(c)
const envlist = (c.envlist ?? "").trim()
const mountNames = new Set(splitCsv(c.mounts))
const envRows = envlist
? envs.filter((e) => (e.name ?? "").trim() === envlist && (e.key ?? "").trim())
: []
const mountRows = mounts.filter((m) => mountNames.has((m.name ?? "").trim()) && (m.dst ?? "").trim())
const cpuRaw = Number.parseInt(c.cpu ?? "", 10)
const memRaw = Number.parseInt(c["memory-high"] ?? "", 10)
return {
id: `${server.id}-${rosId}`,
rosId,
name,
serverId: String(server.id),
image,
tag,
status: mapStatus(c.status),
envs: envRows.map((e) => ({ key: e.key ?? "", value: e.value ?? "" })),
mounts: mountRows.map((m) => ({ dst: m.dst ?? "", src: m.src || undefined })),
interfaces: splitCsv(c.interface),
cmd: (c.cmd ?? "").trim() || undefined,
startOnBoot: rosYes(c["start-on-boot"]),
comment: c.comment ?? "",
cpu: Number.isFinite(cpuRaw) ? cpuRaw : undefined,
memMb: Number.isFinite(memRaw) ? Math.round(memRaw / (1024 * 1024)) || undefined : undefined,
}
}
async function fetchContainersForServer(server: ServerRow): Promise<ContainerLive[]> {
const client = MikrotikClient.fromServer(server)
try {
const [raw, envsRaw, mountsRaw] = await Promise.all([
client.get<RosContainer[]>("/container"),
client.get<RosContainerEnv[]>("/container/envs").catch(() => [] as RosContainerEnv[]),
client.get<RosContainerMount[]>("/container/mounts").catch(() => [] as RosContainerMount[]),
])
const list = Array.isArray(raw) ? raw : []
const envs = Array.isArray(envsRaw) ? envsRaw : []
const mounts = Array.isArray(mountsRaw) ? mountsRaw : []
return list.map((c, idx) => mapContainerRow(server, c, envs, mounts, idx))
} catch (err) {
if (isMissingPackage(err)) return []
throw err
}
}
export async function listContainers(): Promise<ContainerLive[]> {
const enabledServers = await db.select().from(servers).where(eq(servers.enabled, true))
const results = await Promise.all(
enabledServers.map(async (server) => {
try {
return await fetchContainersForServer(server)
} catch {
return [] as ContainerLive[]
}
}),
)
return results.flat()
}
export async function listContainersForServer(server: ServerRow): Promise<ContainerLive[]> {
try {
return await fetchContainersForServer(server)
} catch {
return []
}
}
export async function countContainers(): Promise<number> {
try {
const result = await Promise.race([
listContainers(),
new Promise<null>((resolve) => setTimeout(() => resolve(null), 8_000)),
])
if (!result) return 0
return result.length
} catch {
return 0
}
}
function encodeRosId(rosId: string): string {
return encodeURIComponent(rosId)
}
export async function startContainer(server: ServerRow, rosId: string): Promise<void> {
const client = MikrotikClient.fromServer(server)
await client.post("/container/start", { ".id": rosId })
}
export async function stopContainer(server: ServerRow, rosId: string): Promise<void> {
const client = MikrotikClient.fromServer(server)
await client.post("/container/stop", { ".id": rosId })
}
export async function restartContainer(server: ServerRow, rosId: string): Promise<void> {
await stopContainer(server, rosId)
await startContainer(server, rosId)
}
export async function removeContainer(server: ServerRow, rosId: string): Promise<void> {
const client = MikrotikClient.fromServer(server)
await client.delete(`/container/${encodeRosId(rosId)}`)
}
export async function getEnabledServerById(serverId: string | number) {
const id = typeof serverId === "number" ? serverId : Number.parseInt(String(serverId), 10)
if (!Number.isFinite(id)) return null
return (await db.select().from(servers).where(eq(servers.id, id)).limit(1))[0] ?? null
}
@@ -0,0 +1,216 @@
import assert from "node:assert/strict"
import {
canonicalFirewallSnapshot,
canonicalGreSnapshot,
canonicalIpsecSnapshot,
canonicalWireguardSnapshot,
opsPaths,
opsTouchOnly,
parseIpsecSnapshot,
planFirewallRestore,
planGreCreate,
planGreDelete,
planGreRestore,
planIpsecRestore,
planWireguardRestore,
} from "./entity-snapshots.js"
import { fingerprintPayload, revisionItemCount } from "./config-revisions.js"
{
const a = canonicalFirewallSnapshot({
rules: [{ family: "ip", table: "filter", chain: "input", action: "accept", comment: "ssh" }],
addressLists: [{ family: "ip", list: "vip", address: "1.1.1.1" }],
})
const b = canonicalFirewallSnapshot({
rules: [{ family: "ip", table: "filter", chain: "input", action: "accept", comment: "ssh" }],
addressLists: [{ family: "ip", list: "vip", address: "1.1.1.1" }],
})
assert.equal(fingerprintPayload(a), fingerprintPayload(b))
assert.equal(revisionItemCount(a), 2)
}
{
const desired = canonicalFirewallSnapshot({
rules: [{ family: "ip", table: "filter", chain: "input", action: "accept", comment: "keep" }],
addressLists: [],
})
const ops = planFirewallRestore(desired, {
rules: [
{
...desired.rules[0]!,
rosId: "*1",
dynamic: false,
},
{
family: "ip",
table: "filter",
chain: "forward",
action: "drop",
protocol: "",
srcAddress: "",
dstAddress: "",
srcAddressList: "",
dstAddressList: "",
srcPort: "",
dstPort: "",
inInterface: "",
outInterface: "",
connectionState: "",
comment: "extra",
disabled: false,
log: false,
logPrefix: "",
tlsHost: "",
layer7Proto: "",
rosId: "*2",
dynamic: false,
},
],
addressLists: [],
})
assert.ok(ops.some((op) => op.op === "delete" && op.path.includes("/ip/firewall/filter/")))
assert.equal(opsTouchOnly(ops, ["/ip/firewall", "/ipv6/firewall"]), true)
assert.equal(opsPaths(ops).some((p) => p.startsWith("/ip/route") || p.startsWith("/interface/wireguard")), false)
}
{
const snap = canonicalWireguardSnapshot({
interfaces: [{
name: "wg0",
privateKey: "abc",
address: "10.8.0.1/24",
peers: [{ publicKey: "pk", allowedAddresses: ["10.8.0.2/32"] }],
}],
})
const ops = planWireguardRestore(snap, {
ifaces: [{ name: "wg0", rosId: "*w", listenPort: 13231, mtu: 1420, privateKey: "abc", comment: "", disabled: false }],
peers: [{
rosId: "*p",
interfaceName: "wg0",
publicKey: "old",
allowedAddresses: ["0.0.0.0/0"],
endpointAddress: "",
endpointPort: "",
persistentKeepalive: null,
comment: "",
name: "",
disabled: false,
privateKey: "",
clientAddress: "",
clientDns: "",
clientEndpoint: "",
}],
addrs: [{ rosId: "*a", interfaceName: "wg0", address: "10.8.0.1/24" }],
})
assert.ok(ops.some((op) => op.op === "delete" && op.path.includes("/interface/wireguard/peers/")))
assert.ok(ops.some((op) => op.op === "put" && op.path === "/interface/wireguard/peers"))
assert.equal(opsTouchOnly(ops, ["/interface/wireguard", "/ip/address"]), true)
assert.equal(opsPaths(ops).some((p) => p.startsWith("/interface/gre") || p.startsWith("/ip/route")), false)
}
{
const tunnel = canonicalGreSnapshot({
tunnels: [{
name: "gre-a",
remoteAddress: "203.0.113.1",
localInnerIp: "10.200.0.1/30",
ipsecSecret: "psk-secret",
}],
}).tunnels[0]!
const create = planGreCreate(tunnel)
assert.deepEqual(create.map((op) => op.op), ["put", "put"])
assert.equal(create[0]?.path, "/interface/gre")
assert.equal(create[1]?.path, "/ip/address")
assert.equal(create[1] && create[1].op === "put" ? create[1].body.interface : "", "gre-a")
assert.equal(opsPaths(create).some((p) => p.includes("gre-b")), false)
const del = planGreDelete("gre-a", {
gre: [
{ name: "gre-a", rosId: "*1", localAddress: "", remoteAddress: "203.0.113.1", comment: "", disabled: false, mtu: 1476, keepalive: "0", dscp: "inherit", clampTcpMss: true, allowFastPath: true, ipsecSecret: "" },
{ name: "gre-b", rosId: "*2", localAddress: "", remoteAddress: "203.0.113.2", comment: "", disabled: false, mtu: 1476, keepalive: "0", dscp: "inherit", clampTcpMss: true, allowFastPath: true, ipsecSecret: "" },
],
addrs: [
{ rosId: "*a1", interfaceName: "gre-a", address: "10.200.0.1/30" },
{ rosId: "*a2", interfaceName: "gre-b", address: "10.200.0.5/30" },
],
})
assert.ok(del.some((op) => op.path === "/ip/address/*a1"))
assert.ok(del.some((op) => op.path === "/interface/gre/*1"))
assert.equal(opsPaths(del).some((p) => p.includes("*2") || p.includes("*a2")), false)
const restore = planGreRestore(
canonicalGreSnapshot({ tunnels: [tunnel] }),
{
gre: [
{ name: "gre-a", rosId: "*1", localAddress: "", remoteAddress: "203.0.113.1", comment: "", disabled: false, mtu: 1476, keepalive: "0", dscp: "inherit", clampTcpMss: true, allowFastPath: true, ipsecSecret: "psk-secret" },
{ name: "gre-b", rosId: "*2", localAddress: "", remoteAddress: "203.0.113.2", comment: "", disabled: false, mtu: 1476, keepalive: "0", dscp: "inherit", clampTcpMss: true, allowFastPath: true, ipsecSecret: "" },
],
addrs: [
{ rosId: "*a1", interfaceName: "gre-a", address: "10.200.0.1/30" },
{ rosId: "*a2", interfaceName: "gre-b", address: "10.200.0.5/30" },
],
},
)
assert.ok(restore.some((op) => op.path === "/interface/gre/*2"))
assert.ok(restore.some((op) => op.path === "/ip/address/*a2"))
assert.equal(opsTouchOnly(restore, ["/interface/gre", "/ip/address"]), true)
}
{
const p1 = fingerprintPayload({ tunnels: [{ name: "gre-a", mtu: 1476 }] })
const p2 = fingerprintPayload({ tunnels: [{ name: "gre-a", mtu: 1476 }] })
const p3 = fingerprintPayload({ tunnels: [{ name: "gre-a", mtu: 1400 }] })
assert.equal(p1, p2)
assert.notEqual(p1, p3)
}
{
const desired = canonicalIpsecSnapshot({
peers: [{ name: "ipsec-vpn", address: "0.0.0.0/0", exchangeMode: "ike2", passive: true, certificate: "ipsec-server", profile: "ipsec-vpn", comment: "MikrotikManager:ipsec", disabled: false }],
identities: [{ peerName: "ipsec-vpn", authMethod: "rsa-key", certificate: "ipsec-server", remoteCertificate: "ipsec-user-alice", matchBy: "certificate", secret: "", remoteId: "", modeConfig: "ipsec-vpn", generatePolicy: "port-strict", comment: "MikrotikManager:ipsec user=alice", disabled: false }],
modeConfigs: [{ name: "ipsec-vpn", addressPool: "ipsec-vpn", address: "", staticDns: "10.77.0.1", comment: "MikrotikManager:ipsec" }],
pools: [{ name: "ipsec-vpn", ranges: "10.77.0.2-10.77.0.254", comment: "MikrotikManager:ipsec" }],
nat: [{ chain: "srcnat", action: "masquerade", srcAddress: "10.77.0.0/24", comment: "MikrotikManager:ipsec интернет клиентам VPN" }],
})
// round-trip через payload ревизии
const restored = parseIpsecSnapshot(JSON.parse(JSON.stringify(desired)))
assert.equal(fingerprintPayload(restored), fingerprintPayload(desired))
assert.equal(restored.identities.length, 1)
assert.equal(restored.identities[0]?.comment, "MikrotikManager:ipsec user=alice")
const ops = planIpsecRestore(restored, {
peers: [
{ ...restored.peers[0]!, rosId: "*P1" },
{ name: "site-to-site", address: "203.0.113.7", exchangeMode: "ike2", passive: false, certificate: "", profile: "default", comment: "", disabled: false, rosId: "*P2" },
],
identities: [
{ ...restored.identities[0]!, secret: "(hidden)", rosId: "*I1" },
{
peerName: "site-to-site", authMethod: "pre-shared-key", certificate: "", remoteCertificate: "", matchBy: "",
secret: "(hidden)", remoteId: "peer-b", modeConfig: "", generatePolicy: "", comment: "не managed",
disabled: false, rosId: "*I2",
},
],
modeConfigs: [
{ ...restored.modeConfigs[0]!, rosId: "*M1" },
{ name: "mc-ipsec-ghost", addressPool: "", address: "10.77.0.9", staticDns: "", comment: "MikrotikManager:ipsec клиент ghost", rosId: "*M2" },
],
pools: [{ ...restored.pools[0]!, rosId: "*PL1" }],
nat: [{ ...restored.nat[0]!, rosId: "*N1" }],
})
// лишние managed-объекты удаляются, чужие (site-to-site / «не managed») не трогаем
assert.ok(!ops.some((op) => op.path.includes("*P2")), "чужой peer не тронут")
assert.ok(!ops.some((op) => op.path.includes("*I2")), "чужая identity не тронута")
assert.ok(ops.some((op) => op.op === "delete" && op.path === "/ip/ipsec/mode-config/*M2"), "персональный mc лишнего клиента удалён")
const patchOp = ops.find((op) => op.op === "patch" && op.path === "/ip/ipsec/identity/*I1")
assert.ok(patchOp, "identity желаемого клиента патчится")
assert.ok(patchOp?.body.secret === undefined, "секрет (hidden) не перезаписываем")
assert.equal(
opsTouchOnly(ops, ["/ip/ipsec", "/ip/pool", "/ip/firewall/nat"]),
true,
"restore не выходит за пределы ipsec-объектов",
)
}
console.log("entity-snapshots.test.ts: ok")
+960
View File
@@ -0,0 +1,960 @@
/** Канонические снапшоты и планы restore для firewall / WireGuard / GRE / IPsec. */
import { isIpsecManagedComment } from "./ipsec-config.js"
export type FirewallFamily = "ip" | "ip6"
export type FirewallTable = "filter" | "nat" | "mangle" | "raw"
export type RosWriteOp =
| { op: "put"; path: string; body: Record<string, string> }
| { op: "post"; path: string; body: Record<string, string> }
| { op: "patch"; path: string; body: Record<string, string> }
| { op: "delete"; path: string }
| { op: "move"; path: string; body: Record<string, string> }
export interface FirewallSnapshotRule {
family: FirewallFamily
table: FirewallTable
chain: string
action: string
protocol: string
srcAddress: string
dstAddress: string
srcAddressList: string
dstAddressList: string
srcPort: string
dstPort: string
inInterface: string
outInterface: string
connectionState: string
comment: string
disabled: boolean
log: boolean
logPrefix: string
tlsHost: string
layer7Proto: string
}
export interface FirewallSnapshotList {
family: FirewallFamily
list: string
address: string
comment: string
disabled: boolean
timeout: string
}
export interface FirewallSnapshot {
rules: FirewallSnapshotRule[]
addressLists: FirewallSnapshotList[]
}
export interface FirewallLiveRule extends FirewallSnapshotRule {
rosId: string
dynamic: boolean
}
export interface FirewallLiveList extends FirewallSnapshotList {
rosId: string
dynamic: boolean
}
export interface WgSnapshotPeer {
publicKey: string
allowedAddresses: string[]
endpointAddress: string
endpointPort: string
persistentKeepalive: number | null
comment: string
name: string
disabled: boolean
privateKey: string
clientAddress: string
clientDns: string
clientEndpoint: string
}
export interface WgSnapshotIface {
name: string
listenPort: number
mtu: number
privateKey: string
address: string
comment: string
disabled: boolean
peers: WgSnapshotPeer[]
}
export interface WgSnapshot {
interfaces: WgSnapshotIface[]
}
export interface WgLiveIface {
name: string
rosId: string
listenPort: number
mtu: number
privateKey: string
comment: string
disabled: boolean
}
export interface WgLivePeer {
rosId: string
interfaceName: string
publicKey: string
allowedAddresses: string[]
endpointAddress: string
endpointPort: string
persistentKeepalive: number | null
comment: string
name: string
disabled: boolean
privateKey: string
clientAddress: string
clientDns: string
clientEndpoint: string
}
export interface WgLiveAddr {
rosId: string
interfaceName: string
address: string
}
export interface GreSnapshotTunnel {
name: string
localAddress: string
remoteAddress: string
localInnerIp: string
remoteInnerIp: string
comment: string
disabled: boolean
mtu: number
keepalive: string
dscp: string
clampTcpMss: boolean
allowFastPath: boolean
ipsecSecret: string
}
export interface GreSnapshot {
tunnels: GreSnapshotTunnel[]
}
export interface GreLiveIface {
name: string
rosId: string
localAddress: string
remoteAddress: string
comment: string
disabled: boolean
mtu: number
keepalive: string
dscp: string
clampTcpMss: boolean
allowFastPath: boolean
ipsecSecret: string
}
export interface GreLiveAddr {
rosId: string
interfaceName: string
address: string
}
// ── IPsec / IKEv2 (managed-объекты, маркер MikrotikManager:ipsec) ────────────
export interface IpsecSnapshotPeer {
name: string
address: string
exchangeMode: string
passive: boolean
certificate: string
profile: string
comment: string
disabled: boolean
}
export interface IpsecSnapshotIdentity {
peerName: string
authMethod: string
certificate: string
remoteCertificate: string
matchBy: string
secret: string
remoteId: string
modeConfig: string
generatePolicy: string
comment: string
disabled: boolean
}
export interface IpsecSnapshotModeConfig {
name: string
addressPool: string
address: string
staticDns: string
comment: string
}
export interface IpsecSnapshotPool {
name: string
ranges: string
comment: string
}
export interface IpsecSnapshotPolicy {
srcAddress: string
dstAddress: string
proposal: string
comment: string
}
export interface IpsecSnapshotNat {
chain: string
action: string
srcAddress: string
comment: string
}
export interface IpsecSnapshot {
peers: IpsecSnapshotPeer[]
identities: IpsecSnapshotIdentity[]
modeConfigs: IpsecSnapshotModeConfig[]
pools: IpsecSnapshotPool[]
policies: IpsecSnapshotPolicy[]
nat: IpsecSnapshotNat[]
}
export interface IpsecLivePeer extends IpsecSnapshotPeer {
rosId: string
}
export interface IpsecLiveIdentity extends IpsecSnapshotIdentity {
rosId: string
}
export interface IpsecLiveModeConfig extends IpsecSnapshotModeConfig {
rosId: string
}
export interface IpsecLivePool extends IpsecSnapshotPool {
rosId: string
}
export interface IpsecLivePolicy extends IpsecSnapshotPolicy {
rosId: string
}
export interface IpsecLiveNat extends IpsecSnapshotNat {
rosId: string
}
function str(v: unknown): string {
return String(v ?? "").trim()
}
function bool(v: unknown): boolean {
if (typeof v === "boolean") return v
const s = str(v).toLowerCase()
return s === "true" || s === "yes" || s === "1"
}
function num(v: unknown, fallback: number): number {
const n = typeof v === "number" ? v : Number.parseInt(str(v), 10)
return Number.isFinite(n) ? n : fallback
}
export function isHiddenSecret(value: string | undefined): boolean {
const s = str(value)
if (!s) return true
if (s === "(hidden)") return true
return /^\*+$/.test(s)
}
export function firewallRestPath(
family: FirewallFamily,
table: FirewallTable | "address-list",
): string {
const root = family === "ip6" ? "/ipv6/firewall" : "/ip/firewall"
return `${root}/${table}`
}
function rosYesNo(v: boolean | undefined): string | undefined {
if (v === true) return "yes"
if (v === false) return "no"
return undefined
}
function compactBody(obj: Record<string, string | undefined>): Record<string, string> {
const out: Record<string, string> = {}
for (const [k, v] of Object.entries(obj)) {
if (v !== undefined && v !== "") out[k] = v
}
return out
}
export function canonicalFirewallSnapshot(input: {
rules?: Array<Partial<FirewallSnapshotRule>>
addressLists?: Array<Partial<FirewallSnapshotList>>
}): FirewallSnapshot {
const rules = (input.rules ?? []).map((r) => ({
family: r.family === "ip6" ? "ip6" as const : "ip" as const,
table: (["filter", "nat", "mangle", "raw"] as const).includes(r.table as FirewallTable)
? (r.table as FirewallTable)
: "filter",
chain: str(r.chain),
action: str(r.action),
protocol: str(r.protocol),
srcAddress: str(r.srcAddress),
dstAddress: str(r.dstAddress),
srcAddressList: str(r.srcAddressList),
dstAddressList: str(r.dstAddressList),
srcPort: str(r.srcPort),
dstPort: str(r.dstPort),
inInterface: str(r.inInterface),
outInterface: str(r.outInterface),
connectionState: str(r.connectionState),
comment: str(r.comment),
disabled: Boolean(r.disabled),
log: Boolean(r.log),
logPrefix: str(r.logPrefix),
tlsHost: str(r.tlsHost),
layer7Proto: str(r.layer7Proto),
}))
const addressLists = (input.addressLists ?? []).map((e) => ({
family: e.family === "ip6" ? "ip6" as const : "ip" as const,
list: str(e.list),
address: str(e.address),
comment: str(e.comment),
disabled: Boolean(e.disabled),
timeout: str(e.timeout),
}))
return { rules, addressLists }
}
export function parseFirewallSnapshot(payload: unknown): FirewallSnapshot {
if (!payload || typeof payload !== "object" || Array.isArray(payload)) {
return { rules: [], addressLists: [] }
}
const o = payload as Record<string, unknown>
return canonicalFirewallSnapshot({
rules: Array.isArray(o.rules) ? o.rules as Partial<FirewallSnapshotRule>[] : [],
addressLists: Array.isArray(o.addressLists) ? o.addressLists as Partial<FirewallSnapshotList>[] : [],
})
}
function firewallRuleKey(r: FirewallSnapshotRule): string {
return [
r.family, r.table, r.chain, r.action, r.protocol,
r.srcAddress, r.dstAddress, r.srcAddressList, r.dstAddressList,
r.srcPort, r.dstPort, r.inInterface, r.outInterface, r.connectionState,
r.comment, r.disabled ? "1" : "0", r.log ? "1" : "0", r.logPrefix, r.tlsHost, r.layer7Proto,
].join("\0")
}
function firewallListKey(e: FirewallSnapshotList): string {
return [e.family, e.list, e.address, e.comment, e.disabled ? "1" : "0", e.timeout].join("\0")
}
function firewallRuleBody(r: FirewallSnapshotRule): Record<string, string> {
return compactBody({
chain: r.chain,
action: r.action,
protocol: r.protocol && r.protocol !== "all" ? r.protocol : undefined,
"src-address": r.srcAddress,
"dst-address": r.dstAddress,
"src-address-list": r.srcAddressList,
"dst-address-list": r.dstAddressList,
"src-port": r.srcPort,
"dst-port": r.dstPort,
"in-interface": r.inInterface,
"out-interface": r.outInterface,
"connection-state": r.connectionState,
comment: r.comment,
disabled: rosYesNo(r.disabled),
log: rosYesNo(r.log),
"log-prefix": r.logPrefix,
"tls-host": r.tlsHost,
"layer7-protocol": r.layer7Proto,
})
}
export function planFirewallRestore(
desiredInput: FirewallSnapshot,
current: { rules: FirewallLiveRule[]; addressLists: FirewallLiveList[] },
): RosWriteOp[] {
const desired = canonicalFirewallSnapshot(desiredInput)
const ops: RosWriteOp[] = []
const usedRules = new Set<string>()
const usedLists = new Set<string>()
for (const live of current.rules) {
if (live.dynamic) continue
const key = firewallRuleKey(live)
const stillWanted = desired.rules.some((d) => firewallRuleKey(d) === key)
if (!stillWanted) {
ops.push({
op: "delete",
path: `${firewallRestPath(live.family, live.table)}/${live.rosId}`,
})
} else {
usedRules.add(key)
}
}
for (const live of current.addressLists) {
if (live.dynamic) continue
const key = firewallListKey(live)
const stillWanted = desired.addressLists.some((d) => firewallListKey(d) === key)
if (!stillWanted) {
ops.push({
op: "delete",
path: `${firewallRestPath(live.family, "address-list")}/${live.rosId}`,
})
} else {
usedLists.add(key)
}
}
for (const rule of desired.rules) {
if (usedRules.has(firewallRuleKey(rule))) continue
ops.push({
op: "put",
path: firewallRestPath(rule.family, rule.table),
body: firewallRuleBody(rule),
})
}
for (const entry of desired.addressLists) {
if (usedLists.has(firewallListKey(entry))) continue
ops.push({
op: "put",
path: firewallRestPath(entry.family, "address-list"),
body: compactBody({
list: entry.list,
address: entry.address,
comment: entry.comment,
timeout: entry.timeout,
disabled: rosYesNo(entry.disabled),
}),
})
}
return ops
}
function canonicalPeer(p: Partial<WgSnapshotPeer>): WgSnapshotPeer {
const allowed = Array.isArray(p.allowedAddresses)
? p.allowedAddresses.map((a) => str(a)).filter(Boolean)
: str((p as { allowedIps?: unknown }).allowedIps)
.split(",")
.map((s) => s.trim())
.filter(Boolean)
return {
publicKey: str(p.publicKey),
allowedAddresses: allowed,
endpointAddress: str(p.endpointAddress),
endpointPort: str(p.endpointPort),
persistentKeepalive: p.persistentKeepalive == null ? null : num(p.persistentKeepalive, 0) || null,
comment: str(p.comment),
name: str(p.name),
disabled: Boolean(p.disabled),
privateKey: str(p.privateKey),
clientAddress: str(p.clientAddress),
clientDns: str(p.clientDns),
clientEndpoint: str(p.clientEndpoint),
}
}
export function canonicalWireguardSnapshot(input: {
interfaces?: Array<Partial<WgSnapshotIface> & { peers?: Array<Partial<WgSnapshotPeer>> }>
}): WgSnapshot {
const interfaces = (input.interfaces ?? [])
.map((iface) => ({
name: str(iface.name),
listenPort: num(iface.listenPort, 13231),
mtu: num(iface.mtu, 1420),
privateKey: str(iface.privateKey),
address: str(iface.address),
comment: str(iface.comment),
disabled: Boolean(iface.disabled),
peers: (iface.peers ?? []).map(canonicalPeer).sort((a, b) => a.publicKey.localeCompare(b.publicKey)),
}))
.filter((i) => i.name)
.sort((a, b) => a.name.localeCompare(b.name))
return { interfaces }
}
export function parseWireguardSnapshot(payload: unknown): WgSnapshot {
if (!payload || typeof payload !== "object" || Array.isArray(payload)) {
return { interfaces: [] }
}
const o = payload as Record<string, unknown>
return canonicalWireguardSnapshot({
interfaces: Array.isArray(o.interfaces)
? o.interfaces as Array<Partial<WgSnapshotIface> & { peers?: Array<Partial<WgSnapshotPeer>> }>
: [],
})
}
function peerBody(interfaceName: string, p: WgSnapshotPeer): Record<string, string> {
return compactBody({
interface: interfaceName,
"public-key": p.publicKey,
"allowed-address": p.allowedAddresses.join(","),
"endpoint-address": p.endpointAddress,
"endpoint-port": p.endpointPort,
"persistent-keepalive": p.persistentKeepalive != null ? String(p.persistentKeepalive) : undefined,
comment: p.comment,
name: p.name,
"private-key": isHiddenSecret(p.privateKey) ? undefined : p.privateKey,
"client-address": p.clientAddress,
"client-dns": p.clientDns,
"client-endpoint": p.clientEndpoint,
disabled: rosYesNo(p.disabled),
})
}
export function planWireguardRestore(
desiredInput: WgSnapshot,
current: { ifaces: WgLiveIface[]; peers: WgLivePeer[]; addrs: WgLiveAddr[] },
): RosWriteOp[] {
const desired = canonicalWireguardSnapshot(desiredInput)
const wantedNames = new Set(desired.interfaces.map((i) => i.name))
const ops: RosWriteOp[] = []
for (const peer of current.peers) {
const iface = desired.interfaces.find((i) => i.name === peer.interfaceName)
const keep = iface?.peers.some((p) => p.publicKey === peer.publicKey)
if (!keep) {
ops.push({ op: "delete", path: `/interface/wireguard/peers/${peer.rosId}` })
}
}
for (const addr of current.addrs) {
if (!wantedNames.has(addr.interfaceName)) {
ops.push({ op: "delete", path: `/ip/address/${addr.rosId}` })
}
}
for (const iface of current.ifaces) {
if (!wantedNames.has(iface.name)) {
ops.push({ op: "delete", path: `/interface/wireguard/${iface.rosId}` })
}
}
for (const want of desired.interfaces) {
const live = current.ifaces.find((i) => i.name === want.name)
const ifaceBody = compactBody({
name: want.name,
"listen-port": String(want.listenPort),
mtu: String(want.mtu),
"private-key": isHiddenSecret(want.privateKey) ? undefined : want.privateKey,
comment: want.comment,
disabled: rosYesNo(want.disabled),
})
if (!live) {
ops.push({ op: "put", path: "/interface/wireguard", body: ifaceBody })
} else {
ops.push({
op: "patch",
path: `/interface/wireguard/${live.rosId}`,
body: ifaceBody,
})
}
const liveAddr = current.addrs.find((a) => a.interfaceName === want.name)
if (want.address) {
if (!liveAddr) {
ops.push({ op: "put", path: "/ip/address", body: { address: want.address, interface: want.name } })
} else if (liveAddr.address !== want.address) {
ops.push({ op: "delete", path: `/ip/address/${liveAddr.rosId}` })
ops.push({ op: "put", path: "/ip/address", body: { address: want.address, interface: want.name } })
}
} else if (liveAddr) {
ops.push({ op: "delete", path: `/ip/address/${liveAddr.rosId}` })
}
for (const peer of want.peers) {
if (!peer.publicKey) continue
const livePeer = current.peers.find(
(p) => p.interfaceName === want.name && p.publicKey === peer.publicKey,
)
const body = peerBody(want.name, peer)
if (!livePeer) {
ops.push({ op: "put", path: "/interface/wireguard/peers", body })
} else {
ops.push({
op: "patch",
path: `/interface/wireguard/peers/${livePeer.rosId}`,
body,
})
}
}
}
return ops
}
// ── IPsec: canonical / parse / plan ─────────────────────────────────────────
function canonicalIpsecPeerRaw(p: Partial<IpsecSnapshotPeer>): IpsecSnapshotPeer {
return {
name: str(p.name),
address: str(p.address),
exchangeMode: str(p.exchangeMode),
passive: Boolean(p.passive),
certificate: str(p.certificate),
profile: str(p.profile),
comment: str(p.comment),
disabled: Boolean(p.disabled),
}
}
function canonicalIpsecIdentityRaw(i: Partial<IpsecSnapshotIdentity>): IpsecSnapshotIdentity {
return {
peerName: str(i.peerName),
authMethod: str(i.authMethod),
certificate: str(i.certificate),
remoteCertificate: str(i.remoteCertificate),
matchBy: str(i.matchBy),
secret: str(i.secret),
remoteId: str(i.remoteId),
modeConfig: str(i.modeConfig),
generatePolicy: str(i.generatePolicy),
comment: str(i.comment),
disabled: Boolean(i.disabled),
}
}
function canonicalIpsecModeConfigRaw(m: Partial<IpsecSnapshotModeConfig>): IpsecSnapshotModeConfig {
return {
name: str(m.name),
addressPool: str(m.addressPool),
address: str(m.address),
staticDns: str(m.staticDns),
comment: str(m.comment),
}
}
export function canonicalIpsecSnapshot(input: {
peers?: Array<Partial<IpsecSnapshotPeer>>
identities?: Array<Partial<IpsecSnapshotIdentity>>
modeConfigs?: Array<Partial<IpsecSnapshotModeConfig>>
pools?: Array<Partial<IpsecSnapshotPool>>
policies?: Array<Partial<IpsecSnapshotPolicy>>
nat?: Array<Partial<IpsecSnapshotNat>>
}): IpsecSnapshot {
return {
peers: (input.peers ?? []).map(canonicalIpsecPeerRaw).filter((p) => p.name).sort((a, b) => a.name.localeCompare(b.name)),
identities: (input.identities ?? [])
.map(canonicalIpsecIdentityRaw)
.filter((i) => i.comment)
.sort((a, b) => a.comment.localeCompare(b.comment)),
modeConfigs: (input.modeConfigs ?? [])
.map(canonicalIpsecModeConfigRaw)
.filter((m) => m.name)
.sort((a, b) => a.name.localeCompare(b.name)),
pools: (input.pools ?? [])
.map((p) => ({ name: str(p.name), ranges: str(p.ranges), comment: str(p.comment) }))
.filter((p) => p.name)
.sort((a, b) => a.name.localeCompare(b.name)),
policies: (input.policies ?? [])
.map((p) => ({ srcAddress: str(p.srcAddress), dstAddress: str(p.dstAddress), proposal: str(p.proposal), comment: str(p.comment) }))
.filter((p) => p.dstAddress)
.sort((a, b) => a.dstAddress.localeCompare(b.dstAddress)),
nat: (input.nat ?? [])
.map((n) => ({ chain: str(n.chain), action: str(n.action), srcAddress: str(n.srcAddress), comment: str(n.comment) }))
.filter((n) => n.comment)
.sort((a, b) => a.comment.localeCompare(b.comment)),
}
}
export function parseIpsecSnapshot(payload: unknown): IpsecSnapshot {
if (!payload || typeof payload !== "object" || Array.isArray(payload)) {
return canonicalIpsecSnapshot({})
}
const o = payload as Record<string, unknown>
return canonicalIpsecSnapshot({
peers: Array.isArray(o.peers) ? o.peers as Array<Partial<IpsecSnapshotPeer>> : [],
identities: Array.isArray(o.identities) ? o.identities as Array<Partial<IpsecSnapshotIdentity>> : [],
modeConfigs: Array.isArray(o.modeConfigs) ? o.modeConfigs as Array<Partial<IpsecSnapshotModeConfig>> : [],
pools: Array.isArray(o.pools) ? o.pools as Array<Partial<IpsecSnapshotPool>> : [],
policies: Array.isArray(o.policies) ? o.policies as Array<Partial<IpsecSnapshotPolicy>> : [],
nat: Array.isArray(o.nat) ? o.nat as Array<Partial<IpsecSnapshotNat>> : [],
})
}
function ipsecPeerBody(p: IpsecSnapshotPeer): Record<string, string> {
return compactBody({
name: p.name,
address: p.address,
"exchange-mode": p.exchangeMode,
passive: rosYesNo(p.passive),
certificate: p.certificate,
"send-cert": "always",
profile: p.profile,
comment: p.comment,
disabled: rosYesNo(p.disabled),
})
}
function ipsecIdentityBody(i: IpsecSnapshotIdentity): Record<string, string> {
return compactBody({
peer: i.peerName,
"auth-method": i.authMethod,
certificate: i.certificate,
"remote-certificate": i.remoteCertificate,
"match-by": i.matchBy,
secret: isHiddenSecret(i.secret) ? undefined : i.secret,
"remote-id": i.remoteId,
"mode-config": i.modeConfig,
"generate-policy": i.generatePolicy,
comment: i.comment,
disabled: rosYesNo(i.disabled),
})
}
function ipsecModeConfigBody(m: IpsecSnapshotModeConfig): Record<string, string> {
return compactBody({
name: m.name,
"address-pool": m.addressPool,
address: m.address,
"static-dns": m.staticDns,
comment: m.comment,
})
}
function ipsecPoolBody(p: IpsecSnapshotPool): Record<string, string> {
return compactBody({ name: p.name, ranges: p.ranges, comment: p.comment })
}
function ipsecNatBody(n: IpsecSnapshotNat): Record<string, string> {
return compactBody({
chain: n.chain,
action: n.action,
"src-address": n.srcAddress,
comment: n.comment,
})
}
/** Restore только managed-объектов IKEv2 (peer/identity/mode-config/pool/nat; секреты (hidden) не перезаписываем;
* чужие (не ipsec-managed) live-объекты не трогаем даже если их передали). */
export function planIpsecRestore(
desiredInput: IpsecSnapshot,
current: {
peers: IpsecLivePeer[]
identities: IpsecLiveIdentity[]
modeConfigs: IpsecLiveModeConfig[]
pools: IpsecLivePool[]
nat: IpsecLiveNat[]
},
): RosWriteOp[] {
const desired = canonicalIpsecSnapshot(desiredInput)
const ops: RosWriteOp[] = []
const managedPeers = current.peers.filter((p) => isIpsecManagedComment(p.comment))
const managedIdentities = current.identities.filter((i) => isIpsecManagedComment(i.comment))
const managedModeConfigs = current.modeConfigs.filter((m) => isIpsecManagedComment(m.comment))
const managedPools = current.pools.filter((p) => isIpsecManagedComment(p.comment))
const managedNat = current.nat.filter((n) => isIpsecManagedComment(n.comment))
for (const identity of managedIdentities) {
if (!desired.identities.some((i) => i.comment === identity.comment)) {
ops.push({ op: "delete", path: `/ip/ipsec/identity/${identity.rosId}` })
}
}
for (const mc of managedModeConfigs) {
if (!desired.modeConfigs.some((m) => m.name === mc.name)) {
ops.push({ op: "delete", path: `/ip/ipsec/mode-config/${mc.rosId}` })
}
}
for (const peer of managedPeers) {
if (!desired.peers.some((p) => p.name === peer.name)) {
ops.push({ op: "delete", path: `/ip/ipsec/peer/${peer.rosId}` })
}
}
for (const pool of managedPools) {
if (!desired.pools.some((p) => p.name === pool.name)) {
ops.push({ op: "delete", path: `/ip/pool/${pool.rosId}` })
}
}
for (const rule of managedNat) {
if (!desired.nat.some((n) => n.comment === rule.comment)) {
ops.push({ op: "delete", path: `/ip/firewall/nat/${rule.rosId}` })
}
}
for (const want of desired.peers) {
const live = managedPeers.find((p) => p.name === want.name)
const body = ipsecPeerBody(want)
if (!live) ops.push({ op: "put", path: "/ip/ipsec/peer", body })
else ops.push({ op: "patch", path: `/ip/ipsec/peer/${live.rosId}`, body })
}
for (const want of desired.modeConfigs) {
const live = managedModeConfigs.find((m) => m.name === want.name)
const body = ipsecModeConfigBody(want)
if (!live) ops.push({ op: "put", path: "/ip/ipsec/mode-config", body })
else ops.push({ op: "patch", path: `/ip/ipsec/mode-config/${live.rosId}`, body })
}
for (const want of desired.pools) {
const live = managedPools.find((p) => p.name === want.name)
const body = ipsecPoolBody(want)
if (!live) ops.push({ op: "put", path: "/ip/pool", body })
else ops.push({ op: "patch", path: `/ip/pool/${live.rosId}`, body })
}
for (const want of desired.identities) {
const live = managedIdentities.find((i) => i.comment === want.comment)
const body = ipsecIdentityBody(want)
if (!live) ops.push({ op: "put", path: "/ip/ipsec/identity", body })
else ops.push({ op: "patch", path: `/ip/ipsec/identity/${live.rosId}`, body })
}
for (const want of desired.nat) {
const live = managedNat.find((n) => n.comment === want.comment)
const body = ipsecNatBody(want)
if (!live) ops.push({ op: "put", path: "/ip/firewall/nat", body })
else ops.push({ op: "patch", path: `/ip/firewall/nat/${live.rosId}`, body })
}
return ops
}
export function canonicalGreSnapshot(input: {
tunnels?: Array<Partial<GreSnapshotTunnel>>
}): GreSnapshot {
const tunnels = (input.tunnels ?? [])
.map((t) => ({
name: str(t.name),
localAddress: str(t.localAddress),
remoteAddress: str(t.remoteAddress),
localInnerIp: str(t.localInnerIp),
remoteInnerIp: str(t.remoteInnerIp),
comment: str(t.comment),
disabled: Boolean(t.disabled),
mtu: num(t.mtu, 1476),
keepalive: str(t.keepalive) || "0",
dscp: str(t.dscp) || "inherit",
clampTcpMss: t.clampTcpMss !== false,
allowFastPath: t.allowFastPath !== false,
ipsecSecret: str(t.ipsecSecret),
}))
.filter((t) => t.name)
.sort((a, b) => a.name.localeCompare(b.name))
return { tunnels }
}
export function parseGreSnapshot(payload: unknown): GreSnapshot {
if (!payload || typeof payload !== "object" || Array.isArray(payload)) {
return { tunnels: [] }
}
const o = payload as Record<string, unknown>
return canonicalGreSnapshot({
tunnels: Array.isArray(o.tunnels) ? o.tunnels as Array<Partial<GreSnapshotTunnel>> : [],
})
}
export function greInterfaceBody(t: GreSnapshotTunnel): Record<string, string> {
return compactBody({
name: t.name,
"local-address": t.localAddress && t.localAddress !== "0.0.0.0" ? t.localAddress : undefined,
"remote-address": t.remoteAddress,
mtu: String(t.mtu),
keepalive: t.keepalive,
dscp: t.dscp,
"clamp-tcp-mss": t.clampTcpMss ? "yes" : "no",
"allow-fast-path": t.allowFastPath ? "yes" : "no",
comment: t.comment,
disabled: rosYesNo(t.disabled),
"ipsec-secret": isHiddenSecret(t.ipsecSecret) ? undefined : t.ipsecSecret,
})
}
export function planGreCreate(tunnel: GreSnapshotTunnel): RosWriteOp[] {
const t = canonicalGreSnapshot({ tunnels: [tunnel] }).tunnels[0]
if (!t) return []
const ops: RosWriteOp[] = [
{ op: "put", path: "/interface/gre", body: greInterfaceBody(t) },
]
if (t.localInnerIp) {
ops.push({
op: "put",
path: "/ip/address",
body: { address: t.localInnerIp, interface: t.name },
})
}
return ops
}
export function planGreDelete(
name: string,
current: { gre: GreLiveIface[]; addrs: GreLiveAddr[] },
): RosWriteOp[] {
const want = str(name)
const ops: RosWriteOp[] = []
for (const addr of current.addrs) {
if (addr.interfaceName === want) {
ops.push({ op: "delete", path: `/ip/address/${addr.rosId}` })
}
}
for (const gre of current.gre) {
if (gre.name === want) {
ops.push({ op: "delete", path: `/interface/gre/${gre.rosId}` })
}
}
return ops
}
export function planGreRestore(
desiredInput: GreSnapshot,
current: { gre: GreLiveIface[]; addrs: GreLiveAddr[] },
): RosWriteOp[] {
const desired = canonicalGreSnapshot(desiredInput)
const wanted = new Set(desired.tunnels.map((t) => t.name))
const ops: RosWriteOp[] = []
for (const gre of current.gre) {
if (!wanted.has(gre.name)) {
ops.push(...planGreDelete(gre.name, current))
}
}
for (const want of desired.tunnels) {
const live = current.gre.find((g) => g.name === want.name)
const body = greInterfaceBody(want)
if (!live) {
ops.push({ op: "put", path: "/interface/gre", body })
} else {
ops.push({ op: "patch", path: `/interface/gre/${live.rosId}`, body })
}
const liveAddr = current.addrs.find((a) => a.interfaceName === want.name)
if (want.localInnerIp) {
if (!liveAddr) {
ops.push({
op: "put",
path: "/ip/address",
body: { address: want.localInnerIp, interface: want.name },
})
} else if (liveAddr.address !== want.localInnerIp) {
ops.push({ op: "delete", path: `/ip/address/${liveAddr.rosId}` })
ops.push({
op: "put",
path: "/ip/address",
body: { address: want.localInnerIp, interface: want.name },
})
}
} else if (liveAddr) {
ops.push({ op: "delete", path: `/ip/address/${liveAddr.rosId}` })
}
}
return ops
}
export function opsPaths(ops: RosWriteOp[]): string[] {
return ops.map((op) => op.path)
}
export function opsTouchOnly(ops: RosWriteOp[], prefixes: string[]): boolean {
return ops.every((op) => prefixes.some((p) => op.path === p || op.path.startsWith(`${p}/`)))
}
+115 -7
View File
@@ -5,12 +5,21 @@ import {
MikrotikClient,
firewallRestPath,
} from "./mikrotik.js"
import {
captureAndAppendRevision,
} from "./config-revisions.js"
import type {
FirewallFamily,
FirewallTable,
RosFirewallAddressList,
RosFirewallFilter,
} from "../types/server.js"
import {
canonicalFirewallSnapshot,
type FirewallLiveList,
type FirewallLiveRule,
type FirewallSnapshot,
} from "./entity-snapshots.js"
type ServerRow = typeof servers.$inferSelect
@@ -150,29 +159,121 @@ async function safeGet<T>(fn: () => Promise<T[]>, fallback: T[] = []): Promise<T
}
}
export async function fetchServerFirewall(server: ServerRow): Promise<{
function rosYes(v: string | undefined): boolean {
return v === "true" || v === "yes"
}
export function mapFirewallSnapshotRule(
family: FirewallFamily,
table: FirewallTable,
raw: RosFirewallFilter,
): FirewallLiveRule {
return {
rosId: raw[".id"] || "",
dynamic: rosYes(raw.dynamic),
family,
table,
chain: raw.chain || "",
action: raw.action || "",
protocol: raw.protocol || "",
srcAddress: raw["src-address"] ?? "",
dstAddress: raw["dst-address"] ?? "",
srcAddressList: raw["src-address-list"] ?? "",
dstAddressList: raw["dst-address-list"] ?? "",
srcPort: raw["src-port"] ?? "",
dstPort: raw["dst-port"] ?? "",
inInterface: raw["in-interface"] ?? "",
outInterface: raw["out-interface"] ?? "",
connectionState: raw["connection-state"] ?? "",
comment: raw.comment ?? "",
disabled: rosDisabled(raw.disabled),
log: rosYes(raw.log),
logPrefix: raw["log-prefix"] ?? "",
tlsHost: raw["tls-host"] ?? "",
layer7Proto: raw["layer7-protocol"] ?? "",
}
}
export function mapFirewallSnapshotList(
family: FirewallFamily,
raw: RosFirewallAddressList,
): FirewallLiveList {
return {
rosId: raw[".id"] || "",
dynamic: rosYes(raw.dynamic),
family,
list: raw.list || "",
address: raw.address || "",
comment: raw.comment ?? "",
disabled: rosDisabled(raw.disabled),
timeout: raw.timeout ?? "",
}
}
export async function fetchFirewallState(server: ServerRow): Promise<{
rules: FirewallRuleDto[]
addressLists: FirewallAddressListDto[]
liveRules: FirewallLiveRule[]
liveLists: FirewallLiveList[]
snapshot: FirewallSnapshot
}> {
const client = MikrotikClient.fromServer(server)
const ruleJobs = FAMILIES.flatMap((family) =>
TABLES.map(async (table) => {
const raw = await safeGet(() => client.getFirewallRules(family, table))
return raw.map((row, idx) => mapFirewallRule(server, family, table, row, idx))
return { family, table, raw }
}),
)
const listJobs = FAMILIES.map(async (family) => {
const raw = await safeGet(() => client.getFirewallAddressList(family))
return raw.map((row, idx) => mapAddressList(server, family, row, idx))
return { family, raw }
})
const [ruleChunks, listChunks] = await Promise.all([
Promise.all(ruleJobs),
Promise.all(listJobs),
])
return {
rules: ruleChunks.flat(),
addressLists: listChunks.flat(),
const rules: FirewallRuleDto[] = []
const liveRules: FirewallLiveRule[] = []
for (const chunk of ruleChunks) {
chunk.raw.forEach((row, idx) => {
rules.push(mapFirewallRule(server, chunk.family, chunk.table, row, idx))
liveRules.push(mapFirewallSnapshotRule(chunk.family, chunk.table, row))
})
}
const addressLists: FirewallAddressListDto[] = []
const liveLists: FirewallLiveList[] = []
for (const chunk of listChunks) {
chunk.raw.forEach((row, idx) => {
addressLists.push(mapAddressList(server, chunk.family, row, idx))
liveLists.push(mapFirewallSnapshotList(chunk.family, row))
})
}
return {
rules,
addressLists,
liveRules,
liveLists,
snapshot: canonicalFirewallSnapshot({
rules: liveRules.filter((r) => !r.dynamic),
addressLists: liveLists.filter((e) => !e.dynamic),
}),
}
}
export async function fetchServerFirewall(server: ServerRow): Promise<{
rules: FirewallRuleDto[]
addressLists: FirewallAddressListDto[]
}> {
const state = await fetchFirewallState(server)
return { rules: state.rules, addressLists: state.addressLists }
}
export async function captureFirewallSnapshot(server: ServerRow): Promise<FirewallSnapshot> {
const state = await fetchFirewallState(server)
return state.snapshot
}
export async function listFirewallAll(): Promise<{
@@ -183,7 +284,14 @@ export async function listFirewallAll(): Promise<{
const perServer = await Promise.all(
allServers.map(async (server) => {
try {
return await fetchServerFirewall(server)
const state = await fetchFirewallState(server)
await captureAndAppendRevision({
serverId: server.id,
section: "firewall",
source: "observed",
capture: async () => state.snapshot,
})
return { rules: state.rules, addressLists: state.addressLists }
} catch {
return { rules: [] as FirewallRuleDto[], addressLists: [] as FirewallAddressListDto[] }
}
+100
View File
@@ -0,0 +1,100 @@
import { eq } from "drizzle-orm"
import { db } from "../db/index.js"
import { geoipSettings } from "../db/schema.js"
import type { GeoipSettingsDto, GeoipSettingsPatch } from "@mmapp/contracts/geoip"
const SETTINGS_ID = 1
const DEFAULT_INTERVAL_SEC = 604800
let dbEnabled = true
/** Тесты без PostgreSQL: геттеры отдают дефолты, touch/update — no-op. */
export function disableGeoipDbForTests(): void {
dbEnabled = false
}
export function resetGeoipSettingsForTests(): void {
dbEnabled = true
}
type GeoipSettingsRow = typeof geoipSettings.$inferSelect
async function getGeoipSettingsRow(): Promise<GeoipSettingsRow | undefined> {
if (!dbEnabled) return undefined
return (
(await db.select().from(geoipSettings).where(eq(geoipSettings.id, SETTINGS_ID)).limit(1))[0]
)
}
function toDto(row: GeoipSettingsRow | undefined): GeoipSettingsDto {
return {
enabled: row?.enabled ?? true,
updateIntervalSec: row?.updateIntervalSec ?? DEFAULT_INTERVAL_SEC,
lastCheckAt: row?.lastCheckAt ?? null,
lastSuccessAt: row?.lastSuccessAt ?? null,
lastError: row?.lastError ?? null,
countryBuildAt: row?.countryBuildAt ?? null,
asnBuildAt: row?.asnBuildAt ?? null,
updatedAt: row?.updatedAt ?? new Date().toISOString(),
}
}
export async function getGeoipSettings(): Promise<GeoipSettingsDto> {
return toDto(await getGeoipSettingsRow())
}
/** ETag'и зеркала для conditional GET (ключ — имя файла базы). */
export async function getGeoipEtags(): Promise<Record<string, string>> {
const row = await getGeoipSettingsRow()
if (!row) return {}
const raw = row?.etagsJson
if (!raw || typeof raw !== "object") return {}
return Object.fromEntries(
Object.entries(raw as Record<string, unknown>).filter(
(entry): entry is [string, string] => typeof entry[1] === "string",
),
)
}
export async function updateGeoipSettings(patch: GeoipSettingsPatch): Promise<GeoipSettingsDto> {
const prev = await getGeoipSettingsRow()
const next = {
enabled: patch.enabled ?? prev?.enabled ?? true,
updateIntervalSec: patch.updateIntervalSec ?? prev?.updateIntervalSec ?? DEFAULT_INTERVAL_SEC,
updatedAt: new Date().toISOString(),
}
if (prev) {
await db.update(geoipSettings).set(next).where(eq(geoipSettings.id, SETTINGS_ID))
} else {
await db.insert(geoipSettings).values({ id: SETTINGS_ID, ...next })
}
return getGeoipSettings()
}
export async function touchGeoipRunMeta(patch: {
lastCheckAt?: string
lastSuccessAt?: string | null
lastError?: string | null
countryBuildAt?: string | null
asnBuildAt?: string | null
etags?: Record<string, string>
}): Promise<void> {
const prev = await getGeoipSettingsRow()
const set: Partial<typeof geoipSettings.$inferInsert> = {
updatedAt: new Date().toISOString(),
}
if (patch.lastCheckAt !== undefined) set.lastCheckAt = patch.lastCheckAt
if (patch.lastSuccessAt !== undefined) set.lastSuccessAt = patch.lastSuccessAt
if (patch.lastError !== undefined) set.lastError = patch.lastError
if (patch.countryBuildAt !== undefined) set.countryBuildAt = patch.countryBuildAt
if (patch.asnBuildAt !== undefined) set.asnBuildAt = patch.asnBuildAt
if (patch.etags !== undefined) {
const prevEtags = (prev?.etagsJson as Record<string, string> | null) ?? {}
set.etagsJson = { ...prevEtags, ...patch.etags }
}
if (prev) {
await db.update(geoipSettings).set(set).where(eq(geoipSettings.id, SETTINGS_ID))
} else {
await db.insert(geoipSettings).values({ id: SETTINGS_ID, ...set })
}
}
@@ -0,0 +1,198 @@
import { rename, rm, mkdir, writeFile } from "node:fs/promises"
import path from "node:path"
import { open, type AsnResponse, type CountryResponse } from "maxmind"
import type { GeoipUpdateRunSnapshot } from "../types/scheduler-run-snapshot.js"
import { SCHEDULER_RUN_SNAPSHOT_VERSION } from "../types/scheduler-run-snapshot.js"
import { getGeoipEtags, getGeoipSettings, touchGeoipRunMeta } from "./geoip-settings.js"
import {
GEOIP_ASN_FILE,
GEOIP_COUNTRY_FILE,
geoipDir,
reloadGeoipReaders,
} from "./traffic-flow-geoip.js"
/** Зеркало GeoLite2 без регистрации и ключей (см. README: GeoIP). */
const MIRROR_BASE = "https://github.com/P3TERX/GeoLite.mmdb/raw/download"
const DOWNLOAD_TIMEOUT_MS = 120_000
/** Пробный IP для валидации скачанной базы: Google DNS. */
const PROBE_IP = "8.8.8.8"
type GeoipDbKind = "country" | "asn"
let updating = false
let fetchImpl: typeof fetch = globalThis.fetch.bind(globalThis)
export function getGeoipUpdateState(): { running: boolean } {
return { running: updating }
}
async function validateCountryFile(filePath: string): Promise<string> {
const reader = await open<CountryResponse>(filePath)
const rec = reader.get(PROBE_IP)
const iso = rec?.country?.iso_code ?? rec?.registered_country?.iso_code ?? ""
if (iso !== "US") {
throw new Error(`база Country не распознала ${PROBE_IP} как US (${iso || "нет записи"})`)
}
return reader.metadata.buildEpoch.toISOString()
}
async function validateAsnFile(filePath: string): Promise<string> {
const reader = await open<AsnResponse>(filePath)
const rec = reader.get(PROBE_IP)
const asn = rec?.autonomous_system_number ?? 0
if (asn !== 15169) {
throw new Error(`база ASN не распознала ${PROBE_IP} как AS15169 (${asn ? `AS${asn}` : "нет записи"})`)
}
return reader.metadata.buildEpoch.toISOString()
}
let validateCountry = validateCountryFile
let validateAsn = validateAsnFile
export function setGeoipFetchForTests(fn: typeof fetch): void {
fetchImpl = fn
}
export function setGeoipValidateForTests(opts: {
country?: (filePath: string) => Promise<string>
asn?: (filePath: string) => Promise<string>
}): void {
validateCountry = opts.country ?? validateCountryFile
validateAsn = opts.asn ?? validateAsnFile
}
export function resetGeoipUpdateForTests(): void {
updating = false
fetchImpl = globalThis.fetch.bind(globalThis)
validateCountry = validateCountryFile
validateAsn = validateAsnFile
}
/**
* Разовая проверка/доставка баз с зеркала P3TERX. Conditional GET по ETag
* (304 = не меняем файл), валидация пробоем 8.8.8.8, атомарная подмена через rename.
*/
export async function collectGeoipUpdateOnce(
opts: { force?: boolean } = {},
): Promise<GeoipUpdateRunSnapshot> {
const sampledAt = new Date().toISOString()
if (updating) {
if (opts.force) {
throw Object.assign(new Error("Обновление GeoIP уже выполняется"), { statusCode: 409 })
}
return emptySnapshot(sampledAt, true)
}
const settings = await getGeoipSettings()
if (!settings.enabled && !opts.force) {
return emptySnapshot(sampledAt, true)
}
updating = true
const snapshot: GeoipUpdateRunSnapshot = {
v: SCHEDULER_RUN_SNAPSHOT_VERSION,
job: "geoip_update",
sampledAt,
checked: 0,
downloaded: 0,
skippedUnchanged: 0,
bytes: 0,
errors: [],
}
try {
const dir = geoipDir()
await mkdir(dir, { recursive: true })
const storedEtags = await getGeoipEtags()
const etags: Record<string, string> = {}
const buildAt: Partial<Record<GeoipDbKind, string>> = {}
for (const kind of ["country", "asn"] as const) {
snapshot.checked += 1
const file = kind === "country" ? GEOIP_COUNTRY_FILE : GEOIP_ASN_FILE
const target = path.join(dir, file)
const tmp = `${target}.tmp`
const prevEtag = storedEtags[file]
try {
const ac = new AbortController()
const timer = setTimeout(() => ac.abort(), DOWNLOAD_TIMEOUT_MS)
let res: Response
try {
res = await fetchImpl(`${MIRROR_BASE}/${file}`, {
headers: prevEtag ? { "If-None-Match": prevEtag } : {},
signal: ac.signal,
})
} finally {
clearTimeout(timer)
}
if (res.status === 304) {
snapshot.skippedUnchanged += 1
if (prevEtag) etags[file] = prevEtag
continue
}
if (!res.ok) throw new Error(`HTTP ${res.status}`)
const etag = res.headers.get("etag") ?? ""
const body = Buffer.from(await res.arrayBuffer())
snapshot.bytes += body.byteLength
await writeFile(tmp, body)
buildAt[kind] =
kind === "country" ? await validateCountry(tmp) : await validateAsn(tmp)
const prevFile = `${target}.prev`
await rm(prevFile, { force: true })
await rename(target, prevFile).catch(() => {
/* текущего файла могло ещё не быть */
})
await rename(tmp, target)
snapshot.downloaded += 1
if (etag) etags[file] = etag
} catch (e) {
await rm(tmp, { force: true }).catch(() => {
/* best-effort */
})
const message = e instanceof Error ? e.message : String(e)
snapshot.errors.push(`${file}: ${message}`)
}
}
if (snapshot.downloaded > 0) {
await reloadGeoipReaders()
}
await touchGeoipRunMeta({
lastCheckAt: sampledAt,
lastSuccessAt: snapshot.errors.length ? null : sampledAt,
lastError: snapshot.errors.length ? snapshot.errors.join("; ") : null,
countryBuildAt: buildAt.country,
asnBuildAt: buildAt.asn,
etags,
})
return snapshot
} catch (e) {
const message = e instanceof Error ? e.message : String(e)
snapshot.fatalError = message
await touchGeoipRunMeta({
lastCheckAt: sampledAt,
lastError: message,
}).catch(() => {
/* best-effort */
})
return snapshot
} finally {
updating = false
}
}
function emptySnapshot(sampledAt: string, skipped: boolean): GeoipUpdateRunSnapshot {
return {
v: SCHEDULER_RUN_SNAPSHOT_VERSION,
job: "geoip_update",
sampledAt,
skipped,
checked: 0,
downloaded: 0,
skippedUnchanged: 0,
bytes: 0,
errors: [],
}
}
+253
View File
@@ -0,0 +1,253 @@
import { eq } from "drizzle-orm"
import { db } from "../db/index.js"
import { servers } from "../db/schema.js"
import { MikrotikClient } from "./mikrotik.js"
import {
canonicalGreSnapshot,
type GreLiveAddr,
type GreLiveIface,
type GreSnapshot,
} from "./entity-snapshots.js"
import { captureAndAppendRevision } from "./config-revisions.js"
type ServerRow = typeof servers.$inferSelect
export interface RosGre {
".id"?: string
name?: string
"local-address"?: string
"remote-address"?: string
"allow-fast-path"?: string
"clamp-tcp-mss"?: string
mtu?: string
keepalive?: string
dscp?: string
running?: string
disabled?: string
comment?: string
"ipsec-secret"?: string
}
interface RosIpAddress {
".id"?: string
address?: string
interface?: string
disabled?: string
network?: string
}
export interface LiveGreTunnel {
id: string
rosId: string
name: string
serverId: string
localAddress: string
remoteAddress: string
localInnerIp: string
remoteInnerIp: string
poolId: string
ipsec: { secret: string } | null
mtu: number
keepaliveInterval: number
keepaliveRetries: number
dscp: "inherit" | number
clampTcpMss: boolean
allowFastPath: boolean
comment: string
enabled: boolean
status: "up" | "down" | "degraded"
}
export function parseKeepalive(value: string | undefined): { interval: number; retries: number } {
if (!value || value.toLowerCase() === "none") return { interval: 0, retries: 0 }
const [intervalRaw, retriesRaw] = value.split(",")
const interval = Number.parseInt((intervalRaw ?? "").trim(), 10)
const retries = Number.parseInt((retriesRaw ?? "").trim(), 10)
return {
interval: Number.isFinite(interval) ? interval : 0,
retries: Number.isFinite(retries) ? retries : 0,
}
}
export function formatKeepalive(interval: number, retries: number): string {
if (!interval || interval <= 0) return "0"
return `${interval}s,${retries > 0 ? retries : 10}`
}
function parseDscp(value: string | undefined): "inherit" | number {
if (!value || value === "inherit") return "inherit"
const n = Number.parseInt(value, 10)
return Number.isFinite(n) ? n : "inherit"
}
function parseInnerFromComment(comment: string | undefined): { localInnerIp: string; remoteInnerIp: string } {
if (!comment) return { localInnerIp: "", remoteInnerIp: "" }
const local = comment.match(/address\s*=\s*([0-9.]+\/\d+)/)?.[1] ?? ""
const remote = comment.match(/(?:network|gateway)\s*=\s*([0-9.]+\/\d+)/)?.[1] ?? ""
return { localInnerIp: local, remoteInnerIp: remote }
}
function rosDisabled(v: string | undefined): boolean {
return v === "true" || v === "yes"
}
export function mapGreLive(
server: ServerRow,
greRaw: RosGre[],
addrsRaw: RosIpAddress[],
): {
tunnels: LiveGreTunnel[]
snapshot: GreSnapshot
gre: GreLiveIface[]
addrs: GreLiveAddr[]
} {
const addrsByIface = new Map<string, { address: string; rosId: string }[]>()
for (const a of addrsRaw) {
if (rosDisabled(a.disabled)) continue
const iface = (a.interface ?? "").trim()
const address = (a.address ?? "").trim()
const rosId = String(a[".id"] ?? "")
if (!iface || !address || !rosId) continue
const list = addrsByIface.get(iface) ?? []
list.push({ address, rosId })
addrsByIface.set(iface, list)
}
const gre: GreLiveIface[] = []
const addrs: GreLiveAddr[] = []
const tunnels: LiveGreTunnel[] = []
greRaw.forEach((g, idx) => {
const rosId = String(g[".id"] ?? g.name ?? `gre-${idx}`)
const name = (g.name ?? "").trim() || `gre-${idx + 1}`
const keepalive = parseKeepalive(g.keepalive)
const fromComment = parseInnerFromComment(g.comment)
const ifaceAddrs = addrsByIface.get(name) ?? []
const localInnerIp = ifaceAddrs[0]?.address || fromComment.localInnerIp
const secret = (g["ipsec-secret"] ?? "").trim()
const disabled = rosDisabled(g.disabled)
const running = g.running === "true" || g.running === "yes"
gre.push({
name,
rosId,
localAddress: g["local-address"] ?? "",
remoteAddress: g["remote-address"] ?? "",
comment: g.comment ?? "",
disabled,
mtu: Number.parseInt(g.mtu ?? "1476", 10) || 1476,
keepalive: g.keepalive ?? "0",
dscp: g.dscp ?? "inherit",
clampTcpMss: g["clamp-tcp-mss"] !== "false" && g["clamp-tcp-mss"] !== "no",
allowFastPath: g["allow-fast-path"] !== "false" && g["allow-fast-path"] !== "no",
ipsecSecret: secret,
})
for (const a of ifaceAddrs) {
addrs.push({ rosId: a.rosId, interfaceName: name, address: a.address })
}
tunnels.push({
id: rosId || `${server.id}:${name}`,
rosId,
name,
serverId: String(server.id),
localAddress: g["local-address"] ?? "",
remoteAddress: g["remote-address"] ?? "",
localInnerIp,
remoteInnerIp: fromComment.remoteInnerIp,
poolId: "live",
ipsec: secret ? { secret } : null,
mtu: Number.parseInt(g.mtu ?? "1476", 10) || 1476,
keepaliveInterval: keepalive.interval,
keepaliveRetries: keepalive.retries,
dscp: parseDscp(g.dscp),
clampTcpMss: g["clamp-tcp-mss"] !== "false" && g["clamp-tcp-mss"] !== "no",
allowFastPath: g["allow-fast-path"] !== "false" && g["allow-fast-path"] !== "no",
comment: g.comment ?? "",
enabled: !disabled,
status: disabled ? "down" : running ? "up" : "degraded",
})
})
return {
tunnels,
snapshot: canonicalGreSnapshot({
tunnels: gre.map((g) => ({
name: g.name,
localAddress: g.localAddress,
remoteAddress: g.remoteAddress,
localInnerIp: addrs.find((a) => a.interfaceName === g.name)?.address ?? "",
remoteInnerIp: "",
comment: g.comment,
disabled: g.disabled,
mtu: g.mtu,
keepalive: g.keepalive,
dscp: g.dscp,
clampTcpMss: g.clampTcpMss,
allowFastPath: g.allowFastPath,
ipsecSecret: g.ipsecSecret,
})),
}),
gre,
addrs,
}
}
export async function fetchGreState(server: ServerRow) {
const client = MikrotikClient.fromServer(server)
const [greRaw, addrsRaw] = await Promise.all([
client.get<RosGre[]>("/interface/gre"),
client.get<RosIpAddress[]>("/ip/address").catch(() => [] as RosIpAddress[]),
])
return {
client,
...mapGreLive(server, Array.isArray(greRaw) ? greRaw : [], Array.isArray(addrsRaw) ? addrsRaw : []),
}
}
export async function captureGreSnapshot(server: ServerRow): Promise<GreSnapshot> {
const state = await fetchGreState(server)
return state.snapshot
}
export async function listGreTunnels(opts?: { serverId?: string }): Promise<{
tunnels: LiveGreTunnel[]
failures: Array<{ serverId: string; serverName?: string; error: string }>
}> {
let serverRows: ServerRow[]
if (opts?.serverId) {
const id = Number.parseInt(String(opts.serverId), 10)
if (!Number.isFinite(id)) {
return { tunnels: [], failures: [{ serverId: String(opts.serverId), error: "Некорректный serverId" }] }
}
const row = (await db.select().from(servers).where(eq(servers.id, id)).limit(1))[0]
serverRows = row ? [row] : []
} else {
serverRows = await db.select().from(servers).where(eq(servers.enabled, true))
}
const failures: Array<{ serverId: string; serverName?: string; error: string }> = []
const chunks = await Promise.all(
serverRows.map(async (server) => {
try {
const state = await fetchGreState(server)
await captureAndAppendRevision({
serverId: server.id,
section: "gre",
source: "observed",
capture: async () => state.snapshot,
})
return state.tunnels
} catch (e) {
failures.push({
serverId: String(server.id),
serverName: server.name ?? undefined,
error: e instanceof Error ? e.message : String(e),
})
return [] as LiveGreTunnel[]
}
}),
)
return { tunnels: chunks.flat(), failures }
}
@@ -1,4 +1,4 @@
import { and, asc, desc, eq, lt } from "drizzle-orm"
import { and, asc, desc, eq } from "drizzle-orm"
import { db } from "../db/index.js"
import {
filterRules,
@@ -45,11 +45,6 @@ async function getSettingsRow() {
return (await db.select().from(internetPathSettings).where(eq(internetPathSettings.id, 1)).limit(1))[0]
}
async function cleanupSnapshots(retentionDays: number) {
const cutoff = new Date(Date.now() - retentionDays * 24 * 60 * 60 * 1000).toISOString()
await db.delete(internetPathSnapshots).where(lt(internetPathSnapshots.sampledAt, cutoff))
}
async function buildRulesets() {
const enabled = await db.select().from(servers).where(eq(servers.enabled, true))
const rules = await db.select().from(filterRules).orderBy(asc(filterRules.serverId), asc(filterRules.sortOrder))
@@ -273,7 +268,6 @@ export async function collectInternetPathSnapshotOnce(): Promise<InternetPathRun
sampledAt,
payloadJson: payload,
})
await cleanupSnapshots(Math.max(1, settings.retentionDays))
await db.update(internetPathSettings).set({
lastCollectedAt: sampledAt,
lastDurationMs: Date.now() - started,
+130
View File
@@ -0,0 +1,130 @@
import type { MikrotikClient } from "./mikrotik.js"
import { IPSEC_CA_CERT, IPSEC_SERVER_CERT, clientCertName, ipsecManagedComment } from "./ipsec-config.js"
type RosCertRow = Record<string, string | undefined>
const SIGN_POLL_TIMEOUT_MS = 90_000
const SIGN_POLL_INTERVAL_MS = 1_000
function isIp(value: string): boolean {
return /^\d{1,3}(?:\.\d{1,3}){3}$/.test(value.trim())
}
export async function findCertificate(client: MikrotikClient, name: string): Promise<RosCertRow | undefined> {
const certs = await client.getCertificates()
return certs.find((c) => String(c.name ?? "").trim() === name)
}
/** Сертификат подписан: у него заполнен invalid-after. */
export async function isCertificateSigned(client: MikrotikClient, name: string): Promise<boolean> {
const cert = await findCertificate(client, name)
return Boolean(cert && String(cert["invalid-after"] ?? "").trim() !== "")
}
/** sign в RouterOS не мгновенный: ждём появления invalid-after. */
export async function waitCertificateSigned(client: MikrotikClient, name: string, timeoutMs = SIGN_POLL_TIMEOUT_MS): Promise<void> {
const startedAt = Date.now()
while (Date.now() - startedAt < timeoutMs) {
if (await isCertificateSigned(client, name)) return
await new Promise((resolve) => setTimeout(resolve, SIGN_POLL_INTERVAL_MS))
}
throw new Error(`Сертификат ${name} не подписан за ${Math.round(timeoutMs / 1000)} с`)
}
/** Локальный CA для IKEv2 (self-signed); идемпотентно. Возвращает имя сертификата. */
export async function ensureCaCertificate(client: MikrotikClient, daysValid: number): Promise<string> {
const existing = await findCertificate(client, IPSEC_CA_CERT)
if (!existing) {
await client.addCertificate({
name: IPSEC_CA_CERT,
"common-name": "MikrotikManager IPsec CA",
"key-size": "4096",
"key-usage": "key-cert-sign,crl-sign",
"days-valid": String(daysValid),
comment: ipsecManagedComment("CA"),
})
}
if (!(await isCertificateSigned(client, IPSEC_CA_CERT))) {
await client.signCertificate({ name: IPSEC_CA_CERT, daysValid })
await waitCertificateSigned(client, IPSEC_CA_CERT)
}
return IPSEC_CA_CERT
}
/** Серверный сертификат (CN/SAN = адрес, по которому стучатся клиенты); подписывается CA. */
export async function ensureServerCertificate(
client: MikrotikClient,
args: { serverEndpoint: string; caCertName: string; daysValid: number },
): Promise<string> {
const endpoint = args.serverEndpoint.trim()
const san = isIp(endpoint) ? `IP:${endpoint}` : `DNS:${endpoint}`
const existing = await findCertificate(client, IPSEC_SERVER_CERT)
if (!existing) {
await client.addCertificate({
name: IPSEC_SERVER_CERT,
"common-name": endpoint,
"subject-alt-name": san,
"key-size": "2048",
"key-usage": "digital-signature,key-encipherment,tls-server",
"days-valid": String(args.daysValid),
comment: ipsecManagedComment("server"),
})
}
if (!(await isCertificateSigned(client, IPSEC_SERVER_CERT))) {
await client.signCertificate({ name: IPSEC_SERVER_CERT, ca: args.caCertName, daysValid: args.daysValid })
await waitCertificateSigned(client, IPSEC_SERVER_CERT)
}
return IPSEC_SERVER_CERT
}
export interface IssuedClientCert {
certName: string
commonName: string
/** Сертификат с этим CN уже существовал (перевыпуск не выполнялся). */
existed: boolean
}
/** Клиентский сертификат: add + sign CA. CN = имя пользователя. */
export async function issueClientCertificate(
client: MikrotikClient,
args: { userName: string; caCertName: string; daysValid: number },
): Promise<IssuedClientCert> {
const certName = clientCertName(args.userName)
const existing = await findCertificate(client, certName)
if (existing) {
return { certName, commonName: String(existing["common-name"] ?? args.userName), existed: true }
}
await client.addCertificate({
name: certName,
"common-name": args.userName.trim(),
"key-size": "2048",
"key-usage": "digital-signature,key-encipherment,tls-client",
"days-valid": String(args.daysValid),
comment: ipsecManagedComment(`client ${args.userName.trim()}`),
})
await client.signCertificate({ name: certName, ca: args.caCertName, daysValid: args.daysValid })
await waitCertificateSigned(client, certName)
return { certName, commonName: args.userName.trim(), existed: false }
}
/** Экспорт произвольного сертификата по имени (существующие client1/anakondra и т.п.). */
export async function exportCertificateP12ByName(
client: MikrotikClient,
certName: string,
passphrase: string,
): Promise<{ fileName: string; content: Buffer; certName: string }> {
const name = certName.trim()
const cert = await findCertificate(client, name)
if (!cert) throw new Error(`Сертификат ${name} не найден на роутере`)
const { fileName, content } = await client.exportCertificatePkcs12({ name, passphrase })
return { fileName, content, certName: name }
}
/** Экспорт клиентского .p12 (сертификат + ключ; CA в цепочке) с роутера. */
export async function exportClientP12(
client: MikrotikClient,
userName: string,
passphrase: string,
): Promise<{ fileName: string; content: Buffer; certName: string }> {
return exportCertificateP12ByName(client, clientCertName(userName), passphrase)
}
+191
View File
@@ -0,0 +1,191 @@
import assert from "node:assert/strict"
import {
IPSEC_MARKER,
buildClientInstructions,
buildSswanConfig,
clientCertName,
findFreePoolIp,
identityDisplayName,
ipsecManagedComment,
ipsecSlug,
ipsecUserComment,
isIke2Peer,
isIke2RemoteAccessIdentity,
isIpsecManagedComment,
parseIpsecUserComment,
poolRangesFromCidr,
resolveIke2CaName,
resolveIke2ServerCert,
selectIke2Peers,
selectSharedIke2Identity,
userModeConfigName,
} from "./ipsec-config.js"
import { identityRosBody } from "./ipsec-ros.js"
{
assert.equal(ipsecSlug("Alice Cooper"), "alice-cooper")
assert.equal(ipsecSlug("Иван"), "")
assert.equal(clientCertName("Alice Cooper"), "ipsec-user-alice-cooper")
assert.equal(userModeConfigName("Bob"), "mc-ipsec-bob")
}
{
assert.ok(isIpsecManagedComment(ipsecManagedComment("IKEv2 road-warrior")))
assert.ok(isIpsecManagedComment(`${IPSEC_MARKER} user=alice`))
assert.ok(!isIpsecManagedComment("какой-то чужой комментарий"))
assert.ok(!isIpsecManagedComment(undefined))
assert.equal(ipsecUserComment("alice"), `${IPSEC_MARKER} user=alice`)
assert.equal(parseIpsecUserComment(ipsecUserComment("alice")), "alice")
assert.equal(parseIpsecUserComment("не managed"), null)
assert.equal(parseIpsecUserComment(null), null)
}
{
assert.equal(poolRangesFromCidr("10.77.0.0/24"), "10.77.0.2-10.77.0.254")
assert.equal(poolRangesFromCidr("10.77.0.1/24"), "10.77.0.2-10.77.0.254")
assert.equal(poolRangesFromCidr("192.168.7.0/28"), "192.168.7.2-192.168.7.14")
assert.equal(poolRangesFromCidr("10.77.0.0/31"), null, "/31 — нет адресов клиентам")
assert.equal(poolRangesFromCidr("10.77.0.0"), null)
assert.equal(poolRangesFromCidr("не-сидр"), null)
}
{
const range = "10.77.0.2-10.77.0.10"
assert.equal(findFreePoolIp(range, []), "10.77.0.2")
assert.equal(findFreePoolIp(range, ["10.77.0.2"]), "10.77.0.3")
assert.equal(findFreePoolIp(range, ["10.77.0.2", "10.77.0.3/32", "10.77.0.4"]), "10.77.0.5")
assert.equal(findFreePoolIp(range, ["10.77.0.2", "10.77.0.3", "10.77.0.4", "10.77.0.5", "10.77.0.6", "10.77.0.7", "10.77.0.8", "10.77.0.9", "10.77.0.10"]), null)
// одиночный адрес без диапазона
assert.equal(findFreePoolIp("10.77.0.7", []), "10.77.0.7")
assert.equal(findFreePoolIp("мусор", []), null)
}
{
// managed: user= из comment
assert.equal(identityDisplayName({ comment: ipsecUserComment("alice"), ".id": "*1" }, []), "alice")
// существующий RouterOS identity: сырой comment
assert.equal(identityDisplayName({ comment: "home office", ".id": "*2" }, []), "home office")
// remote-id
assert.equal(identityDisplayName({ "remote-id": "[email protected]", ".id": "*3" }, []), "[email protected]")
// CN сертификата по remote-certificate
assert.equal(
identityDisplayName(
{ "remote-certificate": "ipsec-user-bob", ".id": "*4" },
[{ name: "ipsec-user-bob", "common-name": "bob" }],
),
"bob",
)
// fallback: peer#shortId
assert.equal(
identityDisplayName({ peer: "ikev2-srv", ".id": "*AB12CD34" }, []),
"ikev2-srv#AB12CD",
)
// fallback: rosId
assert.equal(identityDisplayName({ ".id": "*FF" }, []), "FF")
assert.equal(identityDisplayName({}, []), "identity")
}
{
const sswan = buildSswanConfig({
name: "IKEv2 vpn.example.com",
serverEndpoint: "vpn.example.com",
serverId: "vpn.example.com",
p12B64: "cDEy",
uuid: "fix-me",
})
const parsed = JSON.parse(sswan) as Record<string, unknown>
assert.equal(parsed.version, 1)
assert.equal(parsed.type, "ikev2-cert")
const remote = parsed.remote as Record<string, string>
const local = parsed.local as Record<string, string>
assert.equal(remote.addr, "vpn.example.com")
assert.equal(remote.id, "vpn.example.com")
assert.equal(local.p12, "cDEy")
}
{
const text = buildClientInstructions({
userName: "alice",
serverEndpoint: "vpn.example.com",
p12Filename: "cert_export_ipsec-user-alice.p12",
passphrase: "s3cret",
dns: "10.77.0.1",
})
assert.ok(text.includes("alice"))
assert.ok(text.includes("vpn.example.com"))
assert.ok(text.includes("s3cret"))
assert.ok(text.includes("strongSwan"))
}
// ── IKEv2-only отбор ────────────────────────────────────────────────────────
{
const grePeer = { name: "gre-tunnel", "exchange-mode": "ike2", address: "1.2.3.4", passive: "no" }
const s2sPeer = { name: "s2s", "exchange-mode": "main", address: "5.6.7.8", certificate: "vpn-server" }
const rwPeer = { name: "vpn-server", "exchange-mode": "ike2", address: "0.0.0.0/0", passive: "yes", certificate: "vpn-server" }
assert.ok(isIke2Peer(rwPeer))
assert.ok(!isIke2Peer(s2sPeer))
const selected = selectIke2Peers([grePeer, s2sPeer, rwPeer])
assert.deepEqual(selected.map((p) => p.name), ["vpn-server"])
// fallback: нет строгого набора → ike2 + сертификат
const fallback = selectIke2Peers([grePeer, { name: "ike2-cert", "exchange-mode": "ike2", certificate: "c" }])
assert.deepEqual(fallback.map((p) => p.name), ["ike2-cert"])
const names = ["vpn-server"]
assert.ok(isIke2RemoteAccessIdentity(
{ peer: "vpn-server", "mode-config": "ipsec-vpn", "auth-method": "rsa-key", certificate: "vpn-server", "remote-certificate": "client1" },
names,
))
assert.ok(isIke2RemoteAccessIdentity({ peer: "vpn-server", "auth-method": "rsa-key", certificate: "vpn-server" }, names))
// GRE/site-to-site PSK-туннели отсеиваются
assert.ok(!isIke2RemoteAccessIdentity({ peer: "s2s", "auth-method": "pre-shared-key" }, names))
assert.ok(!isIke2RemoteAccessIdentity({ peer: "vpn-server", "auth-method": "pre-shared-key" }, names))
// без ограничения по peer (набор пуст) — mode-config/серт всё равно нужен
assert.ok(isIke2RemoteAccessIdentity({ peer: "x", "mode-config": "mc" }, []))
assert.ok(!isIke2RemoteAccessIdentity({ peer: "x", "auth-method": "pre-shared-key" }, []))
}
{
const myCa = { name: "MyCA", "key-usage": "key-cert-sign,crl-sign", ca: "", flags: "KAT", authority: "true" }
const vpnServer = { name: "vpn-server", "key-usage": "digital-signature,key-encipherment,tls-server,tls-client", ca: "MyCA" }
const client1 = { name: "client1", "key-usage": "digital-signature,tls-client,tls-server", ca: "MyCA" }
const certs = [myCa, vpnServer, client1]
assert.equal(resolveIke2ServerCert(certs, [{ name: "vpn-server", certificate: "vpn-server" }])?.name, "vpn-server")
assert.equal(resolveIke2ServerCert([myCa], [])?.name, undefined)
assert.equal(resolveIke2CaName(certs, vpnServer), "MyCA")
assert.equal(resolveIke2CaName([vpnServer, client1], vpnServer), undefined)
}
{
// Общая listener-identity: без remote-certificate, сертификатный auth.
const shared = { ".id": "*S", peer: "vpn-server", "auth-method": "rsa-key", certificate: "vpn-server", "mode-config": "ikev2-modeconf" }
const perClient = { ".id": "*C", peer: "vpn-server", "auth-method": "rsa-key", certificate: "vpn-server", "remote-certificate": "client1", "match-by": "certificate" }
const psk = { ".id": "*P", peer: "vpn-server", "auth-method": "pre-shared-key" }
const foreignPeer = { ".id": "*F", peer: "gre-tunnel", "auth-method": "rsa-key", certificate: "vpn-server" }
assert.equal(selectSharedIke2Identity([perClient, psk, foreignPeer, shared], ["vpn-server"])?.[".id"], "*S")
// только per-client identity (с remote-certificate) общей не считается
assert.equal(selectSharedIke2Identity([perClient, psk], ["vpn-server"]), undefined)
// managed-приоритет
const managed = { ".id": "*M", peer: "vpn-server", "auth-method": "rsa-key", certificate: "vpn-server", comment: ipsecManagedComment("listener") }
assert.equal(selectSharedIke2Identity([shared, managed], ["vpn-server"])?.[".id"], "*M")
}
{
const body = identityRosBody({
peerName: "vpn-server",
modeConfig: "mc-ipsec-alice",
comment: ipsecUserComment("alice"),
authMethod: "certificate",
certificate: "vpn-server",
remoteCertificate: "ipsec-user-alice",
placeBefore: "*S",
})
assert.equal(body["place-before"], "*S")
assert.equal(body["remote-certificate"], "ipsec-user-alice")
assert.equal(body["match-by"], "certificate")
}
console.log("ipsec-config.test.ts: ok")
+303
View File
@@ -0,0 +1,303 @@
import { randomUUID } from "node:crypto"
import { PRODUCT_NAME } from "../managed-markers.js"
import { isCaCertificate } from "./certificate-parse.js"
// ── managed-маркеры (как у filters/recursive, см. managed-markers.ts) ─────────
export const IPSEC_MARKER = `${PRODUCT_NAME}:ipsec`
export function ipsecManagedComment(label = ""): string {
return label ? `${IPSEC_MARKER} ${label}` : IPSEC_MARKER
}
export function isIpsecManagedComment(comment: string | undefined | null): boolean {
return Boolean(comment) && String(comment).trim().startsWith(IPSEC_MARKER)
}
/** Комментарий identity клиента: `MikrotikManager:ipsec user=<имя>`. */
export function ipsecUserComment(name: string): string {
return `${IPSEC_MARKER} user=${name.trim()}`
}
export function parseIpsecUserComment(comment: string | undefined | null): string | null {
if (!comment) return null
const m = /user=([^\s]+)/.exec(comment.trim())
return m?.[1] ?? null
}
export type IdentityLike = {
".id"?: string
comment?: string
"remote-id"?: string
"remote-certificate"?: string
peer?: string
}
/** Identity + поля, нужные для отбора IKEv2 remote-access. */
export interface Ike2IdentityLike extends IdentityLike {
certificate?: string
"auth-method"?: string
"mode-config"?: string
}
/** Peer (/ip/ipsec/peer). */
export interface PeerLike {
".id"?: string
name?: string
address?: string
"exchange-mode"?: string
passive?: string
certificate?: string
comment?: string
}
export interface CertLike {
name?: string
"common-name"?: string
"key-usage"?: string
/** read-only: имя CA, которым подписан серт. */
ca?: string
/** флаги RouterOS (строка вида "KLAT"). */
flags?: string
authority?: string
}
/** Строка сертификата RouterOS (raw). */
type RosRow = Record<string, string | undefined>
/** Отображаемое имя identity: managed user= → сырой comment → remote-id → CN сертификата → peer#id. */
export function identityDisplayName(i: IdentityLike, certs: CertLike[] = []): string {
const comment = (i.comment ?? "").trim()
const managedName = parseIpsecUserComment(comment)
if (managedName) return managedName
if (comment) return comment
const remoteId = (i["remote-id"] ?? "").trim()
if (remoteId) return remoteId
const remoteCert = (i["remote-certificate"] ?? "").trim()
if (remoteCert) {
const cn = String(certs.find((c) => String(c.name ?? "").trim() === remoteCert)?.["common-name"] ?? "").trim()
return cn || remoteCert
}
const rosId = String(i[".id"] ?? "").replace(/^\*/, "")
const peer = (i.peer ?? "").trim()
if (peer) return `${peer}#${rosId.slice(0, 6) || "identity"}`
return rosId || "identity"
}
// ── naming-конвенции управляемых объектов RouterOS ──────────────────────────
export const IPSEC_CA_CERT = "ipsec-ca"
export const IPSEC_SERVER_CERT = "ipsec-server"
/** Общее имя peer / profile / proposal / пула / общего mode-config. */
export const IPSEC_COMMON_NAME = "ipsec-vpn"
export function ipsecSlug(name: string): string {
return name
.trim()
.toLowerCase()
.replace(/[^a-z0-9]+/g, "-")
.replace(/^-+|-+$/g, "")
}
export function clientCertName(userName: string): string {
return `ipsec-user-${ipsecSlug(userName) || "client"}`
}
export function userModeConfigName(userName: string): string {
return `mc-ipsec-${ipsecSlug(userName) || "client"}`
}
// ── отбор IKEv2 remote-access (без GRE/site-to-site) ────────────────────────
function rosBool(v: string | undefined): boolean {
return v === "true" || v === "yes"
}
function isAnyAddress(a: string | undefined): boolean {
const v = (a ?? "").trim()
return v === "" || v === "0.0.0.0/0" || v === "::/0"
}
/** Peer с IKEv2 (exchange-mode ike2). */
export function isIke2Peer(p: PeerLike): boolean {
return (p["exchange-mode"] ?? "").trim().toLowerCase() === "ike2"
}
/**
* Remote-access IKEv2 peers. Строгий набор: ike2 + passive + any-address + сертификат.
* Fallback ike2 + сертификат любые ike2 (чтобы не потерять существующий сервер).
*/
export function selectIke2Peers<T extends PeerLike>(peers: T[]): T[] {
const ike2 = peers.filter(isIke2Peer)
const strict = ike2.filter(
(p) => rosBool(p.passive) && isAnyAddress(p.address) && Boolean((p.certificate ?? "").trim()),
)
if (strict.length > 0) return strict
const withCert = ike2.filter((p) => Boolean((p.certificate ?? "").trim()))
return withCert.length > 0 ? withCert : ike2
}
/**
* Identity участвует в IKEv2 remote-access: привязан к IKEv2 peer И имеет mode-config
* либо сертификат клиента. PSK-туннели без mode-config/сертификата отсеиваются.
* Пустой набор peer-имён не ограничиваем по peer.
*/
export function isIke2RemoteAccessIdentity(
i: Ike2IdentityLike,
ike2PeerNames?: Iterable<string>,
): boolean {
const names = new Set(Array.from(ike2PeerNames ?? [], (n) => n.trim()).filter(Boolean))
if (names.size > 0 && !names.has((i.peer ?? "").trim())) return false
const hasModeConfig = Boolean((i["mode-config"] ?? "").trim())
const hasRemoteCert = Boolean((i["remote-certificate"] ?? "").trim())
const rsaKey = (i["auth-method"] ?? "").trim().toLowerCase() === "rsa-key"
return hasModeConfig || hasRemoteCert || (rsaKey && Boolean((i.certificate ?? "").trim()))
}
/**
* Общая listener-identity на IKEv2 peer: `remote-certificate` пуст, auth-method сертификатный.
* Именно она обслуживает всех клиентов, чей серт подписан доверенным CA (RouterOS валидирует по CA,
* см. docs «If a remote-certificate is not specified then the received certificate is checked against CA»).
* Приоритет managed, иначе первая подходящая (на устройстве это `denozord`/`ikev2-peer`).
*/
export function selectSharedIke2Identity<T extends Ike2IdentityLike>(
identities: T[],
ike2PeerNames?: Iterable<string>,
): T | undefined {
const names = new Set(Array.from(ike2PeerNames ?? [], (n) => n.trim()).filter(Boolean))
const candidates = identities.filter((i) => {
if (names.size > 0 && !names.has((i.peer ?? "").trim())) return false
if ((i["remote-certificate"] ?? "").trim() !== "") return false
const auth = (i["auth-method"] ?? "").trim().toLowerCase()
if (auth !== "rsa-key" && auth !== "digital-signature") return false
return Boolean((i.certificate ?? "").trim())
})
return candidates.find((i) => isIpsecManagedComment(i.comment)) ?? candidates[0]
}
/** Серверный серт IKEv2 — из `peer.certificate` (напр. vpn-server), fallback по имени/usage. */
export function resolveIke2ServerCert<T extends RosRow>(certs: T[], ike2Peers: PeerLike[]): T | undefined {
const names = new Set(ike2Peers.map((p) => (p.certificate ?? "").trim()).filter(Boolean))
return certs.find((c) => names.has(String(c.name ?? "").trim()))
?? certs.find((c) => String(c.name ?? "").trim() === IPSEC_SERVER_CERT)
?? certs.find((c) => String(c["key-usage"] ?? "").includes("tls-server"))
}
/** CA для подписи новых клиентов: `ca` серверного серта (MyCA) → managed ipsec-ca → первый CA. */
export function resolveIke2CaName<T extends RosRow>(certs: T[], serverCert?: T | null): string | undefined {
const signed = String(serverCert?.ca ?? "").trim()
if (signed && certs.some((c) => String(c.name ?? "").trim() === signed)) return signed
if (certs.some((c) => String(c.name ?? "").trim() === IPSEC_CA_CERT)) return IPSEC_CA_CERT
const ca = certs.find((c) => isCaCertificate(c))
return ca ? String(ca.name ?? "").trim() || undefined : undefined
}
// ── пул адресов клиентов ────────────────────────────────────────────────────
function ipToInt(b: Array<number | undefined>): number {
return (((b[0]! << 24) | (b[1]! << 16) | (b[2]! << 8) | b[3]!) >>> 0)
}
function intToIp(n: number): string {
return [(n >>> 24) & 255, (n >>> 16) & 255, (n >>> 8) & 255, n & 255].join(".")
}
function parseIpv4(s: string): number | null {
const parts = s.trim().split(".")
if (parts.length !== 4) return null
const octets = parts.map((p) => Number(p))
if (octets.some((o) => !Number.isInteger(o) || o < 0 || o > 255)) return null
return ipToInt(octets)
}
/** «10.77.0.0/24» → pool ranges «10.77.0.2-10.77.0.254» (шлюзы .1 и broadcast не раздаём). */
export function poolRangesFromCidr(cidr: string): string | null {
const m = /^(\d{1,3}(?:\.\d{1,3}){3})\/(\d{1,2})$/.exec(cidr.trim())
if (!m) return null
const base = parseIpv4(m[1]!)
const prefix = Number(m[2])
if (base == null || !Number.isInteger(prefix) || prefix < 16 || prefix > 30) return null
const mask = prefix === 0 ? 0 : (0xffffffff << (32 - prefix)) >>> 0
const network = (base & mask) >>> 0
const size = 2 ** (32 - prefix)
const from = network + 2
const to = network + size - 2
if (to <= from) return null
return `${intToIp(from)}-${intToIp(to)}`
}
export interface PoolIp {
ip: string
n: number
}
/** Первый свободный IP диапазона пула («a.b.c.d-a.b.c.e», берём первый диапазон из списка), исключая занятые. */
export function findFreePoolIp(range: string, taken: Iterable<string>): string | null {
const takenSet = new Set(
Array.from(taken, (t) => t.replace(/\/\d+$/, "").trim()),
)
const first = range.split(",").map((s) => s.trim()).filter(Boolean)[0] ?? ""
const [fromRaw, toRaw] = first.split("-")
const from = parseIpv4(fromRaw ?? "")
const to = parseIpv4(toRaw ?? fromRaw ?? "")
if (from == null) return null
const last = to ?? from
if (last < from) return null
const cap = Math.min(last, from + 65_534)
for (let n = from; n <= cap; n++) {
const ip = intToIp(n)
if (!takenSet.has(ip)) return ip
}
return null
}
// ── клиентские конфиги ──────────────────────────────────────────────────────
/** strongSwan (Android/iOS) .sswan-профиль с встроенным .p12. */
export function buildSswanConfig(args: {
name: string
serverEndpoint: string
serverId: string
p12B64: string
uuid?: string
}): string {
return JSON.stringify(
{
version: 1,
uuid: args.uuid ?? randomUUID(),
name: args.name,
type: "ikev2-cert",
remote: { addr: args.serverEndpoint, id: args.serverId },
local: { p12: args.p12B64 },
"ike-proposal": "AES256-SHA256-MODP2048",
"esp-proposal": "AES256-SHA256-MODP2048",
},
null,
2,
)
}
/** Короткая текстовая инструкция по подключению (Windows/macOS/iOS/Android). */
export function buildClientInstructions(args: {
userName: string
serverEndpoint: string
p12Filename: string
passphrase?: string
dns?: string
}): string {
const pass = args.passphrase ? `Пароль архива (при импорте): ${args.passphrase}\n` : ""
return [
`IKEv2/IPsec VPN — клиент «${args.userName}»`,
`Сервер: ${args.serverEndpoint}${args.dns ? ` (DNS: ${args.dns})` : ""}`,
"",
`1. Скачайте и импортируйте сертификат ${args.p12Filename}.`,
pass ? ` ${pass.trim()}` : " Пароль архива задаётся при экспорте.",
"2. Windows: Параметры → Сеть → VPN → Добавить: тип IKEv2, «Вход с сертификатом»",
" (сертификат из .p12 должен лежать в хранилище «Личный» текущего пользователя).",
"3. macOS/iOS: импортируйте .p12 в Связку ключей, затем добавьте VPN (IKEv2),",
" аутентификация — сертификат; удалённый ID = CN серверного сертификата.",
"4. Android: strongSwan app → импорт .sswan-профиля (сертификат уже внутри).",
"",
"Адрес выдаётся автоматически при подключении (mode-config).",
].join("\n")
}
+618
View File
@@ -0,0 +1,618 @@
import { eq } from "drizzle-orm"
import { db } from "../db/index.js"
import { servers } from "../db/schema.js"
import { MikrotikClient } from "./mikrotik.js"
import type {
IpsecCertInfoDto,
IpsecClientDto,
IpsecListResponse,
IpsecModeConfigDto,
IpsecPeerDto,
IpsecPoolDto,
IpsecServerSummaryDto,
} from "@mmapp/contracts/ipsec"
import {
IPSEC_CA_CERT,
IPSEC_COMMON_NAME,
IPSEC_SERVER_CERT,
clientCertName,
identityDisplayName,
isIke2RemoteAccessIdentity,
isIpsecManagedComment,
resolveIke2CaName,
resolveIke2ServerCert,
selectIke2Peers,
selectSharedIke2Identity,
} from "./ipsec-config.js"
import { certificateRole, type CertificateRoleContext } from "./certificate-parse.js"
import {
canonicalIpsecSnapshot,
type IpsecLiveIdentity,
type IpsecLiveModeConfig,
type IpsecLiveNat,
type IpsecLivePeer,
type IpsecLivePool,
type IpsecSnapshot,
} from "./entity-snapshots.js"
type ServerRow = typeof servers.$inferSelect
export interface RosIpsecPeer {
".id"?: string
name?: string
address?: string
"exchange-mode"?: string
passive?: string
certificate?: string
profile?: string
disabled?: string
comment?: string
}
export interface RosIpsecIdentity {
".id"?: string
peer?: string
"auth-method"?: string
certificate?: string
"remote-certificate"?: string
"match-by"?: string
secret?: string
"remote-id"?: string
"mode-config"?: string
"generate-policy"?: string
disabled?: string
comment?: string
}
export interface RosIpsecModeConfig {
".id"?: string
name?: string
"address-pool"?: string
"address-prefix"?: string
address?: string
"split-dns"?: string
"static-dns"?: string
comment?: string
}
export interface RosIpsecPool {
".id"?: string
name?: string
ranges?: string
comment?: string
}
export interface RosIpsecPolicy {
".id"?: string
"src-address"?: string
"dst-address"?: string
proposal?: string
template?: string
comment?: string
}
export interface RosFirewallNat {
".id"?: string
chain?: string
action?: string
"src-address"?: string
comment?: string
}
export interface RosIpsecActivePeer {
".id"?: string
address?: string
"remote-id"?: string
identity?: string
established?: string
}
type RosCertRow = Record<string, string | undefined>
function asBool(v: string | undefined): boolean {
return v === "true" || v === "yes"
}
export function mapCertificate(c: RosCertRow, ctx: CertificateRoleContext = {}): IpsecCertInfoDto {
const name = String(c.name ?? "")
const isUser = name.startsWith("ipsec-user-")
const role = certificateRole(c, {
peerCertNames: ctx.peerCertNames,
identityCertNames: ctx.identityCertNames,
})
return {
name,
commonName: c["common-name"] || undefined,
keySize: c["key-size"] || undefined,
fingerprint: c.fingerprint || undefined,
expiresAt: c["invalid-after"] || undefined,
trusted: asBool(c.trusted),
hasPrivateKey: asBool(c["private-key"]),
role,
signedBy: (c.ca ?? "").trim() || undefined,
managed: (name === IPSEC_CA_CERT && role === "ca")
|| name === IPSEC_SERVER_CERT
|| isUser
|| isIpsecManagedComment(c.comment),
}
}
function mapPeer(server: ServerRow, p: RosIpsecPeer): IpsecPeerDto {
return {
id: `${server.id}:${String(p[".id"] ?? p.name ?? "peer")}`,
rosId: String(p[".id"] ?? p.name ?? "peer"),
serverId: String(server.id),
serverName: String(server.name ?? "").trim() || String(server.host ?? server.id),
name: (p.name ?? "").trim(),
address: p.address || undefined,
exchangeMode: p["exchange-mode"] || undefined,
passive: asBool(p.passive),
certificate: p.certificate || undefined,
profile: p.profile || undefined,
disabled: asBool(p.disabled),
comment: p.comment || undefined,
managed: isIpsecManagedComment(p.comment),
}
}
function mapModeConfig(server: ServerRow, m: RosIpsecModeConfig): IpsecModeConfigDto {
return {
id: `${server.id}:${String(m[".id"] ?? m.name ?? "mc")}`,
rosId: String(m[".id"] ?? m.name ?? "mc"),
serverId: String(server.id),
name: (m.name ?? "").trim(),
addressPool: m["address-pool"] || m["address-prefix"] || undefined,
address: m.address || undefined,
splitDns: m["split-dns"] || undefined,
staticDns: m["static-dns"] || undefined,
comment: m.comment || undefined,
managed: isIpsecManagedComment(m.comment),
}
}
function mapPool(server: ServerRow, p: RosIpsecPool): IpsecPoolDto {
return {
id: `${server.id}:${String(p[".id"] ?? p.name ?? "pool")}`,
rosId: String(p[".id"] ?? p.name ?? "pool"),
serverId: String(server.id),
name: (p.name ?? "").trim(),
ranges: (p.ranges ?? "").trim(),
comment: p.comment || undefined,
managed: isIpsecManagedComment(p.comment),
}
}
export interface IpsecServerState {
server: ServerRow
client: MikrotikClient
peers: RosIpsecPeer[]
identities: RosIpsecIdentity[]
modeConfigs: RosIpsecModeConfig[]
pools: RosIpsecPool[]
policies: RosIpsecPolicy[]
nat: RosFirewallNat[]
active: RosIpsecActivePeer[]
certs: RosCertRow[]
}
export async function fetchIpsecState(server: ServerRow): Promise<IpsecServerState> {
const client = MikrotikClient.fromServer(server)
const empty = <T>(v: unknown): T[] => (Array.isArray(v) ? (v as T[]) : [])
const [peers, identities, modeConfigs, pools, policies, nat, active, certs] = await Promise.all([
client.get<unknown>("/ip/ipsec/peer").then((v) => empty<RosIpsecPeer>(v)).catch(() => [] as RosIpsecPeer[]),
client.get<unknown>("/ip/ipsec/identity").then((v) => empty<RosIpsecIdentity>(v)).catch(() => [] as RosIpsecIdentity[]),
client.get<unknown>("/ip/ipsec/mode-config").then((v) => empty<RosIpsecModeConfig>(v)).catch(() => [] as RosIpsecModeConfig[]),
client.get<unknown>("/ip/pool").then((v) => empty<RosIpsecPool>(v)).catch(() => [] as RosIpsecPool[]),
client.get<unknown>("/ip/ipsec/policy").then((v) => empty<RosIpsecPolicy>(v)).catch(() => [] as RosIpsecPolicy[]),
client.get<unknown>("/ip/firewall/nat").then((v) => empty<RosFirewallNat>(v)).catch(() => [] as RosFirewallNat[]),
client.get<unknown>("/ip/ipsec/active-peers").then((v) => empty<RosIpsecActivePeer>(v)).catch(() => [] as RosIpsecActivePeer[]),
client.getCertificates().catch(() => [] as RosCertRow[]),
])
return { server, client, peers, identities, modeConfigs, pools, policies, nat, active, certs }
}
/**
* Клиенты IKEv2:
* 1) клиентские сертификаты (роль client) включая существующие client1/anakondra на устройстве;
* 2) PSK-identity.
* Общая listener-identity (без remote-certificate) клиентом не считается она обслуживает всех.
*/
export function mapClients(state: IpsecServerState): IpsecClientDto[] {
const server = state.server
const ike2Peers = selectIke2Peers(state.peers)
const ike2PeerNames = ike2Peers.map((p) => (p.name ?? "").trim()).filter(Boolean)
const mcByName = new Map(state.modeConfigs.map((m) => [(m.name ?? "").trim(), m]))
const serverName = String(server.name ?? "").trim() || String(server.host ?? server.id)
const roleCtx: CertificateRoleContext = {
peerCertNames: ike2Peers.map((p) => (p.certificate ?? "").trim()).filter(Boolean),
identityCertNames: state.identities.map((i) => (i["remote-certificate"] ?? "").trim()).filter(Boolean),
}
const caName = resolveIke2CaName(state.certs, resolveIke2ServerCert(state.certs, ike2Peers))
const shared = selectSharedIke2Identity(state.identities, ike2PeerNames)
const sharedRosId = shared?.[".id"] ? String(shared[".id"]) : undefined
const activeByRemote = new Map<string, RosIpsecActivePeer>()
for (const a of state.active) {
const rid = String(a["remote-id"] ?? "").trim()
if (rid) activeByRemote.set(rid, a)
}
const identityByRemoteCert = new Map<string, RosIpsecIdentity>()
for (const i of state.identities) {
const rc = (i["remote-certificate"] ?? "").trim()
if (rc) identityByRemoteCert.set(rc, i)
}
const personalStaticIp = (mcNameRaw: string): string | undefined => {
const mc = mcByName.get(mcNameRaw)
if (!mc) return undefined
return (mc.address ?? mc["address-prefix"] ?? "").replace(/\/\d+$/, "").trim() || undefined
}
const out: IpsecClientDto[] = []
// 1. Клиентские сертификаты (роль client), подписанные нашим CA / выданные менеджером / referenced identity.
for (const cert of state.certs) {
const certName = String(cert.name ?? "").trim()
if (!certName) continue
if (certificateRole(cert, roleCtx) !== "client") continue
const signedBy = (cert.ca ?? "").trim()
const isUser = certName.startsWith("ipsec-user-")
const referenced = identityByRemoteCert.has(certName)
if (!isUser && !referenced && !(caName && signedBy === caName)) continue
const identity = identityByRemoteCert.get(certName)
const comment = identity?.comment ?? ""
const mcName = (identity?.["mode-config"] ?? "").trim()
const cn = String(cert["common-name"] ?? "").trim()
const active = (cn ? activeByRemote.get(cn) : undefined)
?? (identity?.["remote-id"] ? activeByRemote.get(String(identity["remote-id"]).trim()) : undefined)
out.push({
id: `${server.id}:${String(identity?.[".id"] ?? cert[".id"] ?? certName)}`,
rosId: String(identity?.[".id"] ?? cert[".id"] ?? certName),
serverId: String(server.id),
serverName,
name: cn || certName,
authMethod: "certificate",
kind: "cert",
certificateName: certName,
certName,
signedBy: signedBy || undefined,
commonName: cn || undefined,
remoteId: (identity?.["remote-id"] ?? "").trim() || undefined,
staticIp: personalStaticIp(mcName),
modeConfigName: mcName || undefined,
peerName: (identity?.peer ?? "").trim() || undefined,
online: Boolean(active),
activeAddress: active?.address || undefined,
activeSince: active?.established || undefined,
disabled: asBool(identity?.disabled),
comment: comment || undefined,
managed: isUser || isIpsecManagedComment(comment),
})
}
// 2. PSK-identity (managed или IKEv2 remote-access).
for (const i of state.identities) {
if ((i["auth-method"] ?? "").trim().toLowerCase() !== "pre-shared-key") continue
const rosId = String(i[".id"] ?? "")
if (sharedRosId && rosId === sharedRosId) continue
const managed = isIpsecManagedComment(i.comment)
if (!managed && !isIke2RemoteAccessIdentity(i, ike2PeerNames)) continue
const mcName = (i["mode-config"] ?? "").trim()
const remoteId = (i["remote-id"] ?? "").trim()
const active = remoteId ? activeByRemote.get(remoteId) : undefined
out.push({
id: `${server.id}:${rosId || "identity"}`,
rosId: rosId || "identity",
serverId: String(server.id),
serverName,
name: identityDisplayName(i, state.certs),
authMethod: "pre-shared-key",
kind: "psk",
remoteId: remoteId || undefined,
staticIp: personalStaticIp(mcName),
modeConfigName: mcName || undefined,
peerName: (i.peer ?? "").trim() || undefined,
online: Boolean(active),
activeAddress: active?.address || undefined,
activeSince: active?.established || undefined,
disabled: asBool(i.disabled),
comment: i.comment || undefined,
managed,
})
}
return out.sort((a, b) => a.name.localeCompare(b.name))
}
export function mapServerSummary(state: IpsecServerState, clients: IpsecClientDto[]): IpsecServerSummaryDto {
const server = state.server
const ike2Peers = selectIke2Peers(state.peers)
const ike2PeerNames = ike2Peers.map((p) => (p.name ?? "").trim()).filter(Boolean)
const ike2Identities = state.identities.filter(
(i) => isIpsecManagedComment(i.comment) || isIke2RemoteAccessIdentity(i, ike2PeerNames),
)
const managedPeer = ike2Peers.find((p) => isIpsecManagedComment(p.comment))
?? ike2Peers.find((p) => (p.name ?? "").trim() === IPSEC_COMMON_NAME)
/** Для отображения: managed IKEv2 peer, иначе первый существующий IKEv2 peer. */
const primaryPeer = managedPeer ?? ike2Peers[0]
// mode-config, на который ссылаются IKEv2 identity (не персональный mc-ipsec-*), fallback managed shared
const idMcName = ike2Identities
.map((i) => (i["mode-config"] ?? "").trim())
.find((n) => n && !n.startsWith("mc-ipsec-"))
const sharedMc = (idMcName ? state.modeConfigs.find((m) => (m.name ?? "").trim() === idMcName) : undefined)
?? state.modeConfigs.find((m) => (m.name ?? "").trim() === IPSEC_COMMON_NAME)
?? state.modeConfigs.find((m) => isIpsecManagedComment(m.comment) && !(m.name ?? "").startsWith("mc-ipsec-"))
const pool = sharedMc
? state.pools.find((p) => (p.name ?? "").trim() === (sharedMc["address-pool"] ?? "").trim())
?? state.pools.find((p) => isIpsecManagedComment(p.comment))
: undefined
const serverCertRow = resolveIke2ServerCert(state.certs, ike2Peers)
const caName = resolveIke2CaName(state.certs, serverCertRow)
const caCertRow = caName ? state.certs.find((c) => String(c.name ?? "").trim() === caName) : undefined
const roleCtx: CertificateRoleContext = {
peerCertNames: ike2Peers.map((p) => (p.certificate ?? "").trim()).filter(Boolean),
identityCertNames: ike2Identities.map((i) => (i["remote-certificate"] ?? "").trim()).filter(Boolean),
}
const referencedCertNames = new Set<string>([
...(roleCtx.peerCertNames ?? []),
...(roleCtx.identityCertNames ?? []),
...ike2Identities.map((i) => (i.certificate ?? "").trim()).filter(Boolean),
])
// только IKEv2-релевантные серты: CA сервера, referenced peer/identity и подписанные этим CA
const relevantCerts = state.certs.filter((c) => {
const n = String(c.name ?? "").trim()
if (caName && n === caName) return true
if (referencedCertNames.has(n)) return true
return Boolean(caName) && (c.ca ?? "").trim() === caName
})
const caCert = caCertRow ? mapCertificate(caCertRow, roleCtx) : undefined
const serverCert = serverCertRow ? mapCertificate(serverCertRow, roleCtx) : undefined
const natRuleManaged = state.nat.some((r) => isIpsecManagedComment(r.comment) && r.chain === "srcnat")
return {
serverId: String(server.id),
serverName: String(server.name ?? "").trim() || String(server.host ?? server.id),
serverCountry: server.country ?? undefined,
initialized: Boolean(managedPeer && caCert && serverCert),
ike2Ready: Boolean(ike2Peers.length > 0 && caCert && serverCert),
serverEndpoint: serverCertRow ? String(serverCertRow["common-name"] ?? "") || undefined : undefined,
peer: primaryPeer ? mapPeer(server, primaryPeer) : undefined,
peers: ike2Peers.map((p) => mapPeer(server, p)),
pool: pool ? mapPool(server, pool) : undefined,
sharedModeConfig: sharedMc ? mapModeConfig(server, sharedMc) : undefined,
caCert,
serverCert,
natRuleManaged,
clientsTotal: clients.length,
clientsOnline: clients.filter((c) => c.online).length,
certs: relevantCerts.map((c) => mapCertificate(c, roleCtx)),
}
}
/**
* Snapshot для истории/отката намеренно остаётся managed-only (`MikrotikManager:ipsec`):
* restore не должен трогать/пересоздавать уже существующие на роутере (не наших) объекты.
*/
export async function captureIpsecSnapshot(server: ServerRow): Promise<IpsecSnapshot> {
const state = await fetchIpsecState(server)
return canonicalIpsecSnapshot({
peers: state.peers
.filter((p) => isIpsecManagedComment(p.comment))
.map((p) => ({
name: (p.name ?? "").trim(),
address: (p.address ?? "").trim(),
exchangeMode: (p["exchange-mode"] ?? "").trim(),
passive: asBool(p.passive),
certificate: (p.certificate ?? "").trim(),
profile: (p.profile ?? "").trim(),
comment: (p.comment ?? "").trim(),
disabled: asBool(p.disabled),
})),
identities: state.identities
.filter((i) => isIpsecManagedComment(i.comment))
.map((i) => ({
peerName: (i.peer ?? "").trim(),
authMethod: (i["auth-method"] ?? "").trim(),
certificate: (i.certificate ?? "").trim(),
remoteCertificate: (i["remote-certificate"] ?? "").trim(),
matchBy: (i["match-by"] ?? "").trim(),
secret: (i.secret ?? "").trim(),
remoteId: (i["remote-id"] ?? "").trim(),
modeConfig: (i["mode-config"] ?? "").trim(),
generatePolicy: (i["generate-policy"] ?? "").trim(),
comment: (i.comment ?? "").trim(),
disabled: asBool(i.disabled),
})),
modeConfigs: state.modeConfigs
.filter((m) => isIpsecManagedComment(m.comment))
.map((m) => ({
name: (m.name ?? "").trim(),
addressPool: (m["address-pool"] ?? "").trim(),
address: (m.address ?? m["address-prefix"] ?? "").trim(),
staticDns: (m["static-dns"] ?? "").trim(),
comment: (m.comment ?? "").trim(),
})),
pools: state.pools
.filter((p) => isIpsecManagedComment(p.comment))
.map((p) => ({
name: (p.name ?? "").trim(),
ranges: (p.ranges ?? "").trim(),
comment: (p.comment ?? "").trim(),
})),
policies: state.policies
.filter((p) => isIpsecManagedComment(p.comment))
.map((p) => ({
srcAddress: (p["src-address"] ?? "").trim(),
dstAddress: (p["dst-address"] ?? "").trim(),
proposal: (p.proposal ?? "").trim(),
comment: (p.comment ?? "").trim(),
})),
nat: state.nat
.filter((n) => isIpsecManagedComment(n.comment))
.map((n) => ({
chain: (n.chain ?? "").trim(),
action: (n.action ?? "").trim(),
srcAddress: (n["src-address"] ?? "").trim(),
comment: (n.comment ?? "").trim(),
})),
})
}
export async function fetchIpsecRestoreState(server: ServerRow): Promise<{
client: MikrotikClient
peers: IpsecLivePeer[]
identities: IpsecLiveIdentity[]
modeConfigs: IpsecLiveModeConfig[]
pools: IpsecLivePool[]
nat: IpsecLiveNat[]
snapshot: IpsecSnapshot
}> {
const state = await fetchIpsecState(server)
return {
client: state.client,
peers: state.peers
.filter((p) => isIpsecManagedComment(p.comment))
.map((p) => ({
rosId: String(p[".id"] ?? ""),
name: (p.name ?? "").trim(),
address: (p.address ?? "").trim(),
exchangeMode: (p["exchange-mode"] ?? "").trim(),
passive: asBool(p.passive),
certificate: (p.certificate ?? "").trim(),
profile: (p.profile ?? "").trim(),
comment: (p.comment ?? "").trim(),
disabled: asBool(p.disabled),
})),
identities: state.identities
.filter((i) => isIpsecManagedComment(i.comment))
.map((i) => ({
rosId: String(i[".id"] ?? ""),
peerName: (i.peer ?? "").trim(),
authMethod: (i["auth-method"] ?? "") === "pre-shared-key" ? "pre-shared-key" as const : "rsa-key" as const,
certificate: (i.certificate ?? "").trim(),
remoteCertificate: (i["remote-certificate"] ?? "").trim(),
matchBy: (i["match-by"] ?? "").trim(),
secret: (i.secret ?? "").trim(),
remoteId: (i["remote-id"] ?? "").trim(),
modeConfig: (i["mode-config"] ?? "").trim(),
generatePolicy: (i["generate-policy"] ?? "").trim(),
comment: (i.comment ?? "").trim(),
disabled: asBool(i.disabled),
})),
modeConfigs: state.modeConfigs
.filter((m) => isIpsecManagedComment(m.comment))
.map((m) => ({
rosId: String(m[".id"] ?? ""),
name: (m.name ?? "").trim(),
addressPool: (m["address-pool"] ?? "").trim(),
address: (m.address ?? m["address-prefix"] ?? "").trim(),
staticDns: (m["static-dns"] ?? "").trim(),
comment: (m.comment ?? "").trim(),
})),
pools: state.pools
.filter((p) => isIpsecManagedComment(p.comment))
.map((p) => ({
rosId: String(p[".id"] ?? ""),
name: (p.name ?? "").trim(),
ranges: (p.ranges ?? "").trim(),
comment: (p.comment ?? "").trim(),
})),
nat: state.nat
.filter((n) => isIpsecManagedComment(n.comment))
.map((n) => ({
rosId: String(n[".id"] ?? ""),
chain: (n.chain ?? "").trim(),
action: (n.action ?? "").trim(),
srcAddress: (n["src-address"] ?? "").trim(),
comment: (n.comment ?? "").trim(),
})),
snapshot: await captureIpsecSnapshot(server),
}
}
export type IpsecListResult = IpsecListResponse
export async function listIpsec(opts?: { serverId?: string }): Promise<IpsecListResult> {
let serverRows: ServerRow[]
if (opts?.serverId) {
const id = Number.parseInt(String(opts.serverId), 10)
if (!Number.isFinite(id)) {
return { servers: [], clients: [], failures: [{ serverId: String(opts.serverId), error: "Некорректный serverId" }] }
}
const row = (await db.select().from(servers).where(eq(servers.id, id)).limit(1))[0]
serverRows = row ? [row] : []
} else {
serverRows = await db.select().from(servers).where(eq(servers.enabled, true))
}
const failures: IpsecListResult["failures"] = []
const summaries: IpsecServerSummaryDto[] = []
const clients: IpsecClientDto[] = []
await Promise.all(
serverRows.map(async (server) => {
try {
const state = await fetchIpsecState(server)
const serverClients = mapClients(state)
summaries.push(mapServerSummary(state, serverClients))
clients.push(...serverClients)
} catch (e) {
failures.push({
serverId: String(server.id),
serverName: server.name ?? undefined,
error: e instanceof Error ? e.message : String(e),
})
}
}),
)
summaries.sort((a, b) => a.serverName.localeCompare(b.serverName))
clients.sort((a, b) => a.name.localeCompare(b.name))
return { servers: summaries, clients, failures }
}
export async function countIpsecClients(): Promise<number> {
try {
const result = await Promise.race([
listIpsec(),
new Promise<null>((resolve) => setTimeout(() => resolve(null), 8_000)),
])
if (!result) return 0
return result.clients.length
} catch {
return 0
}
}
export async function getEnabledIpsecServerById(serverId: string | number) {
const id = typeof serverId === "number" ? serverId : Number.parseInt(String(serverId), 10)
if (!Number.isFinite(id)) return null
return (await db.select().from(servers).where(eq(servers.id, id)).limit(1))[0] ?? null
}
/** Каталог клиентов для модуля «Пользователи» (привязка app-пользователей по CN сертификата). */
export async function listIpsecClientsForCatalog(serverId: number): Promise<{
clients: IpsecClientDto[]
error?: string
}> {
const row = await getEnabledIpsecServerById(serverId)
if (!row) return { clients: [], error: "Сервер не найден" }
try {
const state = await Promise.race([
fetchIpsecState(row),
new Promise<never>((_, reject) => {
setTimeout(() => reject(new Error("Таймаут RouterOS")), 5_000)
}),
])
return { clients: mapClients(state) }
} catch (e) {
return { clients: [], error: e instanceof Error ? e.message : String(e) }
}
}
export { clientCertName }
+258
View File
@@ -0,0 +1,258 @@
import type { MikrotikClient } from "./mikrotik.js"
import {
ipsecManagedComment,
selectIke2Peers,
selectSharedIke2Identity,
type Ike2IdentityLike,
type PeerLike,
} from "./ipsec-config.js"
export function toRosBody(obj: Record<string, string | number | boolean | undefined | null>): Record<string, string> {
const out: Record<string, string> = {}
for (const [k, v] of Object.entries(obj)) {
if (v == null) continue
const s = String(v)
if (s === "") continue
out[k] = s
}
return out
}
type RosRow = Record<string, string | undefined>
async function listByPath(client: MikrotikClient, path: string): Promise<RosRow[]> {
const raw = await client.get<unknown>(path).catch(() => [])
return Array.isArray(raw) ? (raw as RosRow[]) : []
}
/** PUT (создать) или PATCH по имени — мастер инициализации идемпотентен. */
export async function putOrPatchByName(
client: MikrotikClient,
path: string,
name: string,
body: Record<string, string>,
): Promise<"created" | "patched"> {
const rows = await listByPath(client, path)
const existing = rows.find((r) => String(r.name ?? "").trim() === name)
if (existing?.[".id"]) {
await client.patch(`${path}/${encodeURIComponent(existing[".id"])}`, body)
return "patched"
}
await client.put(path, body)
return "created"
}
async function findRosId(client: MikrotikClient, path: string, name: string): Promise<string | null> {
const rows = await listByPath(client, path)
return rows.find((r) => String(r.name ?? "").trim() === name)?.[".id"] ?? null
}
// ── инициализация IKEv2-сервера ─────────────────────────────────────────────
export async function ensureIpsecProfile(client: MikrotikClient, name: string, comment: string): Promise<void> {
await putOrPatchByName(client, "/ip/ipsec/profile", name, toRosBody({
name,
"hash-algorithm": "sha256",
"enc-algorithm": "aes-256,aes-192,aes-128",
"dh-group": "modp2048,modp1536,modp1024",
"nat-traversal": "yes",
comment,
}))
}
export async function ensureIpsecProposal(client: MikrotikClient, name: string, comment: string): Promise<void> {
await putOrPatchByName(client, "/ip/ipsec/proposal", name, toRosBody({
name,
"auth-algorithms": "sha256,sha1",
"enc-algorithms": "aes-256-cbc,aes-192-cbc,aes-128-cbc",
"pfs-group": "modp2048",
comment,
}))
}
export async function ensureIpsecPool(client: MikrotikClient, name: string, ranges: string, comment: string): Promise<void> {
await putOrPatchByName(client, "/ip/pool", name, toRosBody({ name, ranges, comment }))
}
/** Общий mode-config: адрес клиентам из пула. */
export async function ensureSharedModeConfig(
client: MikrotikClient,
name: string,
poolName: string,
dns: string | undefined,
comment: string,
): Promise<void> {
await putOrPatchByName(client, "/ip/ipsec/mode-config", name, toRosBody({
name,
"address-pool": poolName,
"static-dns": dns,
comment,
}))
}
export async function ensurePolicyTemplate(
client: MikrotikClient,
dstCidr: string,
proposalName: string,
comment: string,
): Promise<void> {
const rows = await listByPath(client, "/ip/ipsec/policy")
const managed = rows.find((r) => (r.comment ?? "").trim() === comment && (r.template === "true" || r.template === "yes"))
if (managed?.[".id"]) {
await client.patch(`/ip/ipsec/policy/${encodeURIComponent(managed[".id"])}`, toRosBody({
"src-address": "0.0.0.0/0",
"dst-address": dstCidr,
proposal: proposalName,
comment,
}))
return
}
await client.put("/ip/ipsec/policy", toRosBody({
"src-address": "0.0.0.0/0",
"dst-address": dstCidr,
proposal: proposalName,
template: "yes",
comment,
}))
}
export async function ensureIpsecPeer(
client: MikrotikClient,
name: string,
serverCertName: string,
profileName: string,
comment: string,
): Promise<void> {
await putOrPatchByName(client, "/ip/ipsec/peer", name, toRosBody({
name,
address: "0.0.0.0/0",
"exchange-mode": "ike2",
passive: "yes",
certificate: serverCertName,
"send-cert": "always",
profile: profileName,
comment,
}))
}
/** Managed srcnat masquerade: интернет клиентам VPN. */
export async function ensureNatRule(client: MikrotikClient, poolCidr: string, comment: string): Promise<void> {
const rows = await listByPath(client, "/ip/firewall/nat")
const existing = rows.find(
(r) => (r.comment ?? "").trim() === comment && r.chain === "srcnat",
)
if (existing?.[".id"]) return
await client.put("/ip/firewall/nat", toRosBody({
chain: "srcnat",
action: "masquerade",
"src-address": poolCidr,
"out-interface-list": "WAN",
comment,
}))
}
// ── клиенты ─────────────────────────────────────────────────────────────────
export async function putUserModeConfig(
client: MikrotikClient,
name: string,
staticIp: string,
comment: string,
): Promise<void> {
await putOrPatchByName(client, "/ip/ipsec/mode-config", name, toRosBody({
name,
address: staticIp,
comment,
}))
}
export async function deleteUserModeConfig(client: MikrotikClient, name: string): Promise<void> {
const id = await findRosId(client, "/ip/ipsec/mode-config", name)
if (id) await client.delete(`/ip/ipsec/mode-config/${encodeURIComponent(id)}`)
}
export interface IdentityFields {
peerName: string
modeConfig: string
comment: string
authMethod: "certificate" | "pre-shared-key"
/** cert: серверный сертификат (router представляет его клиенту). */
certificate?: string
/** cert: строгий мэтч конкретного клиента по его сертификату. */
remoteCertificate?: string
/** psk. */
secret?: string
remoteId?: string
/** Вставить identity перед указанной (per-client match-by=certificate должен идти раньше общей). */
placeBefore?: string
}
export function identityRosBody(f: IdentityFields): Record<string, string> {
return toRosBody({
peer: f.peerName,
"auth-method": f.authMethod === "certificate" ? "rsa-key" : "pre-shared-key",
certificate: f.certificate,
"remote-certificate": f.remoteCertificate,
"match-by": f.authMethod === "certificate" ? "certificate" : undefined,
secret: f.secret,
"remote-id": f.remoteId,
"mode-config": f.modeConfig,
"generate-policy": "port-strict",
"place-before": f.placeBefore,
comment: f.comment,
})
}
export async function putIdentity(client: MikrotikClient, fields: IdentityFields): Promise<void> {
await client.put("/ip/ipsec/identity", identityRosBody(fields))
}
/**
* Общая listener-identity на IKEv2 peer для всех клиентов (reuse существующей, иначе создать).
* Без `remote-certificate`/`match-by` RouterOS принимает любой клиентский серт, подписанный доверенным CA.
*/
export async function ensureSharedIke2Identity(
client: MikrotikClient,
args: { peerName: string; serverCertName: string; modeConfigName?: string; comment: string },
): Promise<string | undefined> {
const identities = await listByPath(client, "/ip/ipsec/identity")
const peers = await listByPath(client, "/ip/ipsec/peer")
const peerNames = selectIke2Peers(peers as PeerLike[]).map((p) => (p.name ?? "").trim()).filter(Boolean)
const existing = selectSharedIke2Identity(
identities as Ike2IdentityLike[],
peerNames.length > 0 ? peerNames : [args.peerName],
)
if (existing?.[".id"]) return String(existing[".id"])
await putIdentity(client, {
peerName: args.peerName,
modeConfig: args.modeConfigName ?? "",
comment: args.comment,
authMethod: "certificate",
certificate: args.serverCertName,
})
return undefined
}
export async function patchIdentity(client: MikrotikClient, rosId: string, body: Record<string, string>): Promise<void> {
await client.patch(`/ip/ipsec/identity/${encodeURIComponent(rosId)}`, body)
}
export async function deleteIdentity(client: MikrotikClient, rosId: string): Promise<void> {
await client.delete(`/ip/ipsec/identity/${encodeURIComponent(rosId)}`)
}
export async function patchPeer(client: MikrotikClient, rosId: string, body: Record<string, string>): Promise<void> {
await client.patch(`/ip/ipsec/peer/${encodeURIComponent(rosId)}`, body)
}
export async function deletePeer(client: MikrotikClient, rosId: string): Promise<void> {
await client.delete(`/ip/ipsec/peer/${encodeURIComponent(rosId)}`)
}
/** Снять с identity персональный mode-config (вернуть выдачу из пула) безопасно: пустой patch не шлём. */
export async function clearIdentityModeConfig(client: MikrotikClient, rosId: string, sharedModeConfig: string): Promise<void> {
await patchIdentity(client, rosId, toRosBody({ "mode-config": sharedModeConfig }))
}
export { listByPath, findRosId, ipsecManagedComment }
+175
View File
@@ -267,6 +267,57 @@ function rosDelete(
})
}
/** GET бинарного содержимого (файлы RouterOS): без utf8-декодирования, JSON-ответ = ошибка. */
function rosDownload(
params: MikrotikConnectParams,
path: string,
timeoutMs: number,
): Promise<Buffer> {
return new Promise((resolve, reject) => {
const basePath = params.apiPath ?? "/rest"
const authHeader = "Basic " + Buffer.from(`${params.username}:${params.password}`).toString("base64")
const options: https.RequestOptions = {
hostname: params.host,
port: params.port,
path: basePath + path,
method: "GET",
headers: { Authorization: authHeader },
rejectUnauthorized: params.useSsl ? params.verifySsl : undefined,
}
const lib = params.useSsl ? https : http
const req = lib.request(options, (res) => {
const chunks: Buffer[] = []
res.on("data", (chunk: Buffer) => { chunks.push(chunk) })
res.on("end", () => {
const buf = Buffer.concat(chunks)
if (!res.statusCode || res.statusCode < 200 || res.statusCode >= 300) {
reject(new MikrotikError(res.statusCode ?? 0, path, buf.toString("utf8").slice(0, 200)))
return
}
const contentType = String(res.headers["content-type"] ?? "")
if (contentType.includes("application/json")) {
reject(new Error(`RouterOS вернул метаданные вместо содержимого файла ${path}`))
return
}
resolve(buf)
})
})
const timer = setTimeout(() => {
req.destroy(new Error(`Connection to ${params.host}:${params.port} timed out after ${timeoutMs / 1000}s`))
}, timeoutMs)
req.on("close", () => clearTimeout(timer))
req.on("error", (err) => {
clearTimeout(timer)
reject(err)
})
req.end()
})
}
function rosPatch(
params: MikrotikConnectParams,
path: string,
@@ -364,6 +415,13 @@ export function encodeRosId(rosId: string): string {
// ── MikrotikClient ─────────────────────────────────────────────────────────────
/** Имя параметра из ошибки RouterOS `unknown parameter <name>` (400) — для деградации запроса. */
function unknownParameterName(error: MikrotikError): string | undefined {
if (error.statusCode !== 400) return undefined
const match = error.body.match(/unknown parameter\s+"?([A-Za-z0-9_-]+)"?/i)
return match?.[1]
}
export class MikrotikClient {
constructor(private readonly params: MikrotikConnectParams) {}
@@ -586,6 +644,14 @@ export class MikrotikClient {
: new Error(`Не удалось загрузить файл ${normalized} на RouterOS`)
}
async importUploadedFile(fileName: string): Promise<unknown> {
try {
return await this.post("/import", { "file-name": fileName }, 120_000)
} catch {
return await this.post("/execute", { script: `/import file-name="${fileName}"` }, 120_000)
}
}
async importCertificate(params: {
fileName: string
name: string
@@ -613,6 +679,115 @@ export class MikrotikClient {
}
}
/** Скачивание содержимого файла RouterOS (GET /rest/file/<name>, бинарно). */
async downloadFile(fileName: string, timeoutMs = 30_000): Promise<Buffer> {
const normalized = routerFileBasename(fileName)
const candidates = [normalized, `flash/${normalized}`]
let lastError: unknown
for (const name of candidates) {
try {
return await rosDownload(this.params, `/file/${encodeURIComponent(name)}`, timeoutMs)
} catch (error) {
lastError = error
}
}
throw lastError instanceof Error
? lastError
: new Error(`Не удалось скачать файл ${normalized} с RouterOS`)
}
/**
* POST с деградацией по несовместимым параметрам: набор полей `/certificate/*` зависит от версии
* RouterOS (напр. `comment`, `days-valid`). При 400 «unknown parameter X» убираем X и повторяем.
*/
private async postTolerant(
path: string,
body: Record<string, string>,
timeoutMs: number,
maxAttempts = 4,
): Promise<unknown> {
const payload: Record<string, string> = { ...body }
for (let attempt = 0; attempt < maxAttempts; attempt += 1) {
try {
return await this.post(path, payload, timeoutMs)
} catch (error) {
const param = error instanceof MikrotikError ? unknownParameterName(error) : undefined
if (!param || !(param in payload)) throw error
delete payload[param]
}
}
throw new Error(`RouterOS: не удалось выполнить ${path} (несовместимые параметры): ${Object.keys(body).join(", ")}`)
}
/** Создание ключевой пары + заявки: /certificate add (поля common-name, key-size, key-usage…). */
async addCertificate(body: Record<string, string>, timeoutMs = 30_000): Promise<unknown> {
return this.postTolerant("/certificate/add", body, timeoutMs)
}
/** Подпись сертификата локальным CA; sign небыстрый — увеличенный таймаут. */
async signCertificate(params: {
name: string
ca?: string
daysValid?: number
}, timeoutMs = 120_000): Promise<unknown> {
const body: Record<string, string> = { name: params.name }
if (params.ca) body.ca = params.ca
if (params.daysValid != null) body["days-valid"] = String(params.daysValid)
try {
return await this.postTolerant("/certificate/sign", body, timeoutMs)
} catch (e) {
// Некоторые версии REST принимают цель подписи только как .id.
const certs = await this.getCertificates()
const row = certs.find((c) => String(c.name ?? "") === params.name)
const id = row?.[".id"]
if (!id) throw e
return await this.postTolerant("/certificate/sign", { ".id": id, ...body }, timeoutMs)
}
}
/** Экспорт сертификата в файл на роутере (pkcs12/pem); возвращает имя созданного файла. */
async exportCertificate(params: {
name: string
type: "pkcs12" | "pem"
passphrase?: string
}, timeoutMs = 60_000): Promise<string> {
const body: Record<string, string> = { name: params.name, type: params.type }
if (params.passphrase?.trim()) body["export-passphrase"] = params.passphrase.trim()
let raw: unknown
try {
raw = await this.postTolerant("/certificate/export-certificate", body, timeoutMs)
} catch (e) {
const certs = await this.getCertificates()
const id = certs.find((c) => String(c.name ?? "") === params.name)?.[".id"]
if (!id) throw e
raw = await this.postTolerant("/certificate/export-certificate", { ".id": id, ...body }, timeoutMs)
}
void raw
// RouterOS создаёт cert_export_<name>.p12 либо <name>.p12 — ищем по списку файлов.
const ext = params.type === "pkcs12" ? "p12" : "crt"
const wanted = [`${params.name}.${ext}`, `cert_export_${params.name}.${ext}`]
const files = await this.listFiles()
const hit = files.find((f) => wanted.includes(f.name))
?? files.find((f) => f.name.endsWith(`.${ext}`) && f.name.includes(params.name))
if (!hit) throw new Error(`Файл экспорта ${params.name}.${ext} не найден на RouterOS`)
return hit.name
}
/** Скачивание .p12 (сертификат + ключ + цепочка) как бинарный Buffer. */
async exportCertificatePkcs12(params: { name: string; passphrase?: string }): Promise<{ fileName: string; content: Buffer }> {
const fileName = await this.exportCertificate({ name: params.name, type: "pkcs12", passphrase: params.passphrase })
const content = await this.downloadFile(fileName)
return { fileName, content }
}
async removeCertificate(nameOrId: string, timeoutMs = 30_000): Promise<void> {
const certs = await this.getCertificates()
const row = certs.find((c) => String(c.name ?? "") === nameOrId || c[".id"] === nameOrId)
const id = row?.[".id"]
if (!id) return
await this.delete(`/certificate/${encodeURIComponent(id)}`, timeoutMs)
}
private async patchIpService(serviceName: string, body: Record<string, string>): Promise<void> {
const pathByName = `/ip/service/${encodeURIComponent(serviceName)}`
try {
@@ -0,0 +1,22 @@
import assert from "node:assert/strict"
import type { RosIpRoute } from "../types/server.js"
import { parseOspfGateway, parseOspfRouteType } from "./ospf-route-parse.js"
function route(partial: Partial<RosIpRoute>): RosIpRoute {
return { ".id": "*1", "dst-address": "10.0.0.0/8", ...partial }
}
assert.equal(parseOspfRouteType(route({ static: "true" })), null)
assert.equal(parseOspfRouteType(route({ bgp: "true" })), null)
assert.equal(parseOspfRouteType(route({ ospf: "true" })), "O")
assert.equal(parseOspfRouteType(route({ "ospf-type": "intra-area" })), "O")
assert.equal(parseOspfRouteType(route({ ospf: "true", "ospf-type": "inter-area" })), "O IA")
assert.equal(parseOspfRouteType(route({ ospf: "true", "ospf-type": "ext-type-1" })), "O E1")
assert.equal(parseOspfRouteType(route({ ospf: "true", "ospf-type": "type-2" })), "O E2")
assert.deepEqual(parseOspfGateway(route({ gateway: "10.200.0.1%gre-msk-spb" })), {
nextHop: "10.200.0.1",
via: "gre-msk-spb",
})
console.log("ospf-route-parse.test.ts: ok")
+28
View File
@@ -0,0 +1,28 @@
import type { RosIpRoute } from "../types/server.js"
export type OspfRouteKind = "O" | "O IA" | "O E1" | "O E2"
/** RouterOS /ip/route → тип OSPF-маршрута UI, либо null если маршрут не OSPF. */
export function parseOspfRouteType(r: RosIpRoute): OspfRouteKind | null {
const ospfFlag = r.ospf === "true" || r.ospf === "yes"
const raw = `${r["ospf-type"] ?? ""} ${r.type ?? ""}`.toLowerCase()
const looksOspf = ospfFlag || raw.includes("ospf") || Boolean(r["ospf-type"])
if (!looksOspf) return null
if (raw.includes("inter")) return "O IA"
if (raw.includes("e1") || raw.includes("type-1") || raw.includes("ext-1") || raw.includes("nssa-ext-type-1")) {
return "O E1"
}
if (raw.includes("e2") || raw.includes("type-2") || raw.includes("ext-2") || raw.includes("nssa-ext-type-2")) {
return "O E2"
}
return "O"
}
export function parseOspfGateway(r: RosIpRoute): { nextHop: string; via: string } {
const gw = (r.gateway ?? r["immediate-gw"] ?? "").trim()
const [ip, iface = ""] = gw.split("%")
return {
nextHop: ip || gw || "—",
via: iface || (r.interface ?? "—"),
}
}
+12 -5
View File
@@ -1,5 +1,5 @@
import { eq, lt } from "drizzle-orm"
import { db, pool } from "../db/index.js"
import { eq } from "drizzle-orm"
import { db, dbQuery, pool } from "../db/index.js"
import { dropExpiredPartitions, ensurePartitionsAround } from "../db/partitions.js"
import { servers, serverSnapshots } from "../db/schema.js"
import type { SnapshotInsert } from "../db/schema.js"
@@ -82,11 +82,18 @@ export async function pollServer(serverId: number): Promise<SnapshotRead> {
.values(partialSnap as SnapshotInsert)
.returning()
const cutoffIso = new Date(Date.now() - 14 * 24 * 3600_000).toISOString()
await db.delete(serverSnapshots).where(lt(serverSnapshots.polledAt, cutoffIso))
if (inserted) {
await dbQuery(
`UPDATE server_snapshots
SET raw_interfaces = NULL, raw_ip_addresses = NULL
WHERE server_id = $1 AND polled_at < $2
AND (raw_interfaces IS NOT NULL OR raw_ip_addresses IS NOT NULL)`,
[serverId, inserted.polledAt],
)
}
void dropExpiredPartitions(pool).then(() => ensurePartitionsAround(pool))
return toSnapshotRead(inserted)
return toSnapshotRead(inserted!)
}
// ── helper ─────────────────────────────────────────────────────────────────────
+32
View File
@@ -0,0 +1,32 @@
import type { MikrotikClient } from "./mikrotik.js"
import type { RosWriteOp } from "./entity-snapshots.js"
function encodeIdSegment(path: string): string {
const i = path.lastIndexOf("/")
if (i < 0) return path
const last = path.slice(i + 1)
if (!last.startsWith("*")) return path
return `${path.slice(0, i + 1)}${encodeURIComponent(last)}`
}
export async function executeRosOps(client: MikrotikClient, ops: RosWriteOp[]): Promise<void> {
for (const op of ops) {
if (op.op === "put") {
await client.put(op.path, op.body)
continue
}
if (op.op === "post") {
await client.post(encodeIdSegment(op.path), op.body)
continue
}
if (op.op === "patch") {
await client.patch(encodeIdSegment(op.path), op.body)
continue
}
if (op.op === "delete") {
await client.delete(encodeIdSegment(op.path))
continue
}
await client.post(encodeIdSegment(op.path), op.body)
}
}
@@ -0,0 +1,89 @@
import assert from "node:assert/strict"
import {
DeleteObjectCommand,
GetObjectCommand,
HeadBucketCommand,
ListObjectsV2Command,
PutObjectCommand,
type S3Client,
} from "@aws-sdk/client-s3"
import {
buildS3ObjectKey,
normalizeS3Prefix,
parseS3ObjectKey,
sanitizeServerName,
s3DeleteObject,
s3GetObject,
s3ListObjects,
s3PutObject,
s3TestConnection,
} from "./s3-backup-client.js"
assert.equal(normalizeS3Prefix("/mikrotik/backups/"), "mikrotik/backups")
assert.equal(sanitizeServerName("MSK CHR 01"), "MSK_CHR_01")
assert.equal(
buildS3ObjectKey("mikrotik", "msk-chr01", "chr_2026-09-08_03-00-00.rsc"),
"mikrotik/msk-chr01/chr_2026-09-08_03-00-00.rsc",
)
assert.deepEqual(
parseS3ObjectKey("mikrotik/msk-chr01/chr_2026-09-08_03-00-00.rsc"),
{ filename: "chr_2026-09-08_03-00-00.rsc", serverName: "msk-chr01" },
)
const store = new Map<string, Buffer>()
let lastCommand = ""
const fake = {
send: async (command: { input?: Record<string, unknown> }) => {
const name = command.constructor.name
lastCommand = name
const input = command.input ?? {}
if (command instanceof HeadBucketCommand || name === "HeadBucketCommand") {
if (input.Bucket !== "backups") throw new Error("no bucket")
return {}
}
if (command instanceof PutObjectCommand || name === "PutObjectCommand") {
const key = String(input.Key)
const body = input.Body
store.set(key, Buffer.isBuffer(body) ? body : Buffer.from(String(body)))
return { ETag: '"etag-1"' }
}
if (command instanceof GetObjectCommand || name === "GetObjectCommand") {
const key = String(input.Key)
const body = store.get(key)
if (!body) throw new Error("not found")
return { Body: { transformToByteArray: async () => new Uint8Array(body) } }
}
if (command instanceof DeleteObjectCommand || name === "DeleteObjectCommand") {
store.delete(String(input.Key))
return {}
}
if (command instanceof ListObjectsV2Command || name === "ListObjectsV2Command") {
const prefix = String(input.Prefix ?? "")
const contents = [...store.entries()]
.filter(([key]) => !prefix || key.startsWith(prefix))
.map(([key, buf]) => ({ Key: key, Size: buf.length, LastModified: new Date("2026-09-08T00:00:00Z") }))
return { Contents: contents, IsTruncated: false }
}
throw new Error(`unexpected command ${name}`)
},
} as unknown as S3Client
await s3TestConnection(fake, "backups")
assert.equal(lastCommand === "HeadBucketCommand" || lastCommand.includes("Head"), true)
const put = await s3PutObject(fake, "backups", "mikrotik/a/file.rsc", "hello")
assert.equal(put.etag, '"etag-1"')
const got = await s3GetObject(fake, "backups", "mikrotik/a/file.rsc")
assert.equal(got.toString("utf8"), "hello")
const listed = await s3ListObjects(fake, "backups", "mikrotik")
assert.equal(listed.length, 1)
assert.equal(listed[0]?.key, "mikrotik/a/file.rsc")
await s3DeleteObject(fake, "backups", "mikrotik/a/file.rsc")
const after = await s3ListObjects(fake, "backups", "mikrotik")
assert.equal(after.length, 0)
console.log("s3-backup-client.test.ts: ok")
+128
View File
@@ -0,0 +1,128 @@
import {
DeleteObjectCommand,
GetObjectCommand,
HeadBucketCommand,
ListObjectsV2Command,
PutObjectCommand,
S3Client,
type S3ClientConfig,
} from "@aws-sdk/client-s3"
export type S3BackupConfig = {
endpoint: string
region: string
bucket: string
prefix: string
accessKeyId: string
secretAccessKey: string
forcePathStyle: boolean
}
export type S3ListedObject = {
key: string
size: number
lastModified?: string
}
export function normalizeS3Prefix(prefix: string): string {
return prefix.trim().replace(/^\/+|\/+$/g, "")
}
export function sanitizeServerName(name: string): string {
const safe = name.replace(/[^a-zA-Z0-9._-]+/g, "_").replace(/^_+|_+$/g, "")
return safe || "server"
}
export function buildS3ObjectKey(prefix: string, serverSafe: string, filename: string): string {
const parts = [normalizeS3Prefix(prefix), sanitizeServerName(serverSafe), filename]
.filter((part) => part.length > 0)
return parts.join("/")
}
export function parseS3ObjectKey(key: string): { filename: string; serverName: string } {
const parts = key.split("/").filter(Boolean)
const filename = parts.pop() ?? key
const folder = parts.pop() ?? ""
const base = filename.replace(/\.(rsc|backup)$/i, "")
const fromFilename = base.replace(/_\d{4}-\d{2}-\d{2}_\d{2}-\d{2}-\d{2}$/, "")
return { filename, serverName: folder || fromFilename || filename }
}
export function createS3ClientFromConfig(cfg: S3BackupConfig): S3Client {
const options: S3ClientConfig = {
region: cfg.region.trim() || "us-east-1",
credentials: {
accessKeyId: cfg.accessKeyId,
secretAccessKey: cfg.secretAccessKey,
},
forcePathStyle: cfg.forcePathStyle,
}
const endpoint = cfg.endpoint.trim()
if (endpoint) options.endpoint = endpoint
return new S3Client(options)
}
export async function s3TestConnection(client: S3Client, bucket: string): Promise<void> {
try {
await client.send(new HeadBucketCommand({ Bucket: bucket }))
} catch {
await client.send(new ListObjectsV2Command({ Bucket: bucket, MaxKeys: 1 }))
}
}
export async function s3PutObject(
client: S3Client,
bucket: string,
key: string,
body: Buffer | string,
): Promise<{ etag?: string }> {
const out = await client.send(new PutObjectCommand({
Bucket: bucket,
Key: key,
Body: body,
ContentType: "text/plain; charset=utf-8",
}))
return { etag: out.ETag }
}
export async function s3GetObject(
client: S3Client,
bucket: string,
key: string,
): Promise<Buffer> {
const out = await client.send(new GetObjectCommand({ Bucket: bucket, Key: key }))
const bytes = await out.Body?.transformToByteArray()
if (!bytes) throw new Error("Пустой объект S3")
return Buffer.from(bytes)
}
export async function s3DeleteObject(client: S3Client, bucket: string, key: string): Promise<void> {
await client.send(new DeleteObjectCommand({ Bucket: bucket, Key: key }))
}
export async function s3ListObjects(
client: S3Client,
bucket: string,
prefix: string,
): Promise<S3ListedObject[]> {
const items: S3ListedObject[] = []
let token: string | undefined
const normalized = normalizeS3Prefix(prefix)
do {
const out = await client.send(new ListObjectsV2Command({
Bucket: bucket,
Prefix: normalized ? `${normalized}/` : undefined,
ContinuationToken: token,
}))
for (const obj of out.Contents ?? []) {
if (!obj.Key) continue
items.push({
key: obj.Key,
size: obj.Size ?? 0,
lastModified: obj.LastModified?.toISOString(),
})
}
token = out.IsTruncated ? out.NextContinuationToken : undefined
} while (token)
return items
}
+20
View File
@@ -46,6 +46,8 @@ import { collectCertificatesRenewOnce } from "./certificate-renew-collector.js"
import { getCertificateRenewSettings } from "./certificates-service.js"
import { collectScheduledBackupsOnce } from "./backup-scheduler-collector.js"
import { getBackupScheduleSettings } from "./backup-service.js"
import { collectGeoipUpdateOnce } from "./geoip-update-collector.js"
import { getGeoipSettings } from "./geoip-settings.js"
import {
endSchedulerJob,
isSchedulerJobRunning,
@@ -63,6 +65,7 @@ export const JOB_KEYS = [
"gre_bgp",
"certificates_renew",
"backups",
"geoip_update",
"alert_engine",
] as const
export type SchedulerJobKey = (typeof JOB_KEYS)[number]
@@ -148,6 +151,9 @@ async function runSchedulerJobBody(jobKey: SchedulerJobKey): Promise<void> {
case "backups":
snapshot = await collectScheduledBackupsOnce()
break
case "geoip_update":
snapshot = await collectGeoipUpdateOnce()
break
case "alert_engine": {
const r = await runAlertEngineOnce()
snapshot = {
@@ -377,6 +383,18 @@ export async function refreshScheduler(): Promise<void> {
)
}
const geoip = await getGeoipSettings()
if (geoip.enabled) {
const geoipMs = Math.max(6 * 3600_000, geoip.updateIntervalSec * 1000)
void executeSchedulerJob("geoip_update").catch(() => {})
timers.set(
"geoip_update",
setInterval(() => {
void executeSchedulerJob("geoip_update").catch(() => {})
}, geoipMs),
)
}
const alertMs = 20_000
void executeSchedulerJob("alert_engine").catch(() => {})
timers.set(
@@ -409,6 +427,7 @@ export async function getSchedulerStatus() {
const internetPath = await getInternetPathSettings()
const certRenew = await getCertificateRenewSettings()
const backupSchedule = await getBackupScheduleSettings()
const geoip = await getGeoipSettings()
const resOn = uptime.resourcesEnabled ?? uptime.enabled
const pingOn = uptime.pingEnabled ?? uptime.enabled
@@ -424,6 +443,7 @@ export async function getSchedulerStatus() {
gre_bgp: { enabled: true, intervalSec: 30 },
certificates_renew: { enabled: certRenew.enabled, intervalSec: certRenew.intervalSec },
backups: { enabled: backupSchedule.enabled, intervalSec: 60 },
geoip_update: { enabled: geoip.enabled, intervalSec: geoip.updateIntervalSec },
alert_engine: { enabled: true, intervalSec: 20 },
}
@@ -162,8 +162,6 @@ export async function collectServersRestPingOnce(): Promise<ServersRestPingRunSn
}
})
await dbQuery(`DELETE FROM servers_rest_ping_samples WHERE sampled_at < now() - interval '30 days'`)
await db.update(serversApiPingSettings)
.set({
lastCollectedAt: sampledAt,
@@ -0,0 +1,322 @@
import assert from "node:assert/strict"
import { getStatistics, getStatisticsPivot, normalizeFactService, parseStatisticsPeriod, pivotDimsConflict } from "./statistics-aggregate.js"
import { rememberServerIfaces, resetIfaceCacheForTests } from "./traffic-flow-ifindex.js"
import { setRefreshIfacesForTests } from "./traffic-flow-ifaces.js"
import { invalidateFlowCatalogCache } from "./traffic-flow-topology.js"
import { withPgOrSkip } from "../test/pg.js"
import { dbQuery } from "../db/index.js"
import { ensurePartitionFor } from "../db/partitions.js"
import { pool } from "../db/index.js"
import { STATISTICS_UNBOUND_USER_ID } from "@mmapp/contracts/statistics"
{
const sameDay = parseStatisticsPeriod("2026-09-10", "2026-09-10")
assert.ok(sameDay)
assert.equal(sameDay.fromDay, "2026-09-10")
assert.equal(sameDay.toDayExclusive, "2026-09-11")
assert.equal(sameDay.grain, "hour")
const month = parseStatisticsPeriod("2026-08-01", "2026-08-31")
assert.ok(month)
assert.equal(month.grain, "day")
assert.equal(month.toDayExclusive, "2026-09-01")
assert.equal(parseStatisticsPeriod("2026-09-10", "2026-09-09"), null)
assert.equal(pivotDimsConflict("country", "country"), true)
assert.equal(pivotDimsConflict("country", "service"), false)
}
{
const nowMs = Date.parse("2026-09-12T12:00:00Z")
// «Сегодня»: окно до сейчас, не до конца суток — иначе avgBps размывается будущими часами.
const today = parseStatisticsPeriod("2026-09-12", "2026-09-12", nowMs)
assert.ok(today)
assert.equal(today.windowSec, 12 * 3600)
assert.equal(today.grain, "hour")
assert.equal(today.toDayExclusive, "2026-09-13", "дневные факты текущего дня не теряем")
// Прошлые периоды не клампятся.
const past = parseStatisticsPeriod("2026-09-10", "2026-09-10", nowMs)
assert.ok(past)
assert.equal(past.windowSec, 86_400)
// ISO-диапазон ровно 24 часа.
const iso24 = parseStatisticsPeriod("2026-09-11T12:00:00Z", "2026-09-12T12:00:00Z", nowMs)
assert.ok(iso24)
assert.equal(iso24.windowSec, 86_400)
assert.equal(iso24.grain, "hour")
// «to» далеко в будущем клампится к сейчас.
const futureTo = parseStatisticsPeriod("2026-09-11", "2026-09-20", nowMs)
assert.ok(futureTo)
assert.equal(futureTo.windowSec, 86_400 + 12 * 3600)
// Полностью будущий диапазон невалиден.
assert.equal(parseStatisticsPeriod("2026-09-13", "2026-09-14", nowMs), null)
}
{
// Таксономия сервисов как на карте: skip-список сворачивается в «Прочее».
assert.equal(normalizeFactService("Google"), "Google")
assert.equal(normalizeFactService("DNS"), "Прочее")
assert.equal(normalizeFactService("SSH"), "Прочее")
assert.equal(normalizeFactService("BGP"), "Прочее")
assert.equal(normalizeFactService("WireGuard"), "Прочее")
assert.equal(normalizeFactService("GRE"), "Прочее")
assert.equal(normalizeFactService("Прочее"), "Прочее")
assert.equal(normalizeFactService(""), "Прочее")
}
if (!(await withPgOrSkip())) {
console.log("statistics-aggregate.test.ts: skip")
process.exit(0)
}
const inserted = await dbQuery<{ id: number }>(`
INSERT INTO servers (name, host, type, wan_uplinks)
VALUES ('stats-cube', '127.0.0.1', 'jump-host', '[{"iface":"wan1"}]'::jsonb)
RETURNING id
`)
const serverId = inserted.rows[0]?.id
if (serverId == null) throw new Error("no server")
const enInserted = await dbQuery<{ id: number }>(`
INSERT INTO servers (name, host, type)
VALUES ('stats-en', '198.51.100.1', 'exit-node')
RETURNING id
`)
const enId = enInserted.rows[0]?.id
if (enId == null) throw new Error("no en server")
await ensurePartitionFor(pool, "flow_daily_facts", "month", new Date("2026-09-01T00:00:00Z"))
await ensurePartitionFor(pool, "flow_hour_facts", "day", new Date("2026-09-10T00:00:00Z"))
await dbQuery(`DELETE FROM flow_daily_facts WHERE server_id IN ($1, $2)`, [serverId, enId])
await dbQuery(`DELETE FROM flow_hour_facts WHERE server_id IN ($1, $2)`, [serverId, enId])
await dbQuery(`DELETE FROM user_interface_bindings WHERE server_id = $1`, [serverId])
await dbQuery(`DELETE FROM server_snapshots WHERE server_id IN ($1, $2)`, [serverId, enId])
await dbQuery(`DELETE FROM app_users WHERE id = 'u-stats-1'`)
await dbQuery(`
INSERT INTO app_users (id, name, login, role, active)
VALUES ('u-stats-1', 'Клиент', 'stats-user', 'viewer', TRUE)
ON CONFLICT (id) DO NOTHING
`)
await dbQuery(`
INSERT INTO user_interface_bindings (id, user_id, server_id, interface_name, interface_type)
VALUES ('bind-stats-1', 'u-stats-1', $1, 'gre-client', 'gre')
`, [serverId])
await dbQuery(`
INSERT INTO server_snapshots (server_id, polled_at, status, raw_interfaces)
VALUES
($1, '2026-09-10T12:00:00Z', 'online', $3::jsonb),
($2, '2026-09-10T12:00:00Z', 'online', $4::jsonb)
`, [
serverId,
enId,
JSON.stringify([
{ name: "gre-client", type: "gre-tunnel" },
{ name: "wan1", type: "ether" },
{ name: "gre-en", type: "gre-tunnel" },
{ name: "NSK-SERVHOST-RTK", type: "gre-tunnel" },
{ name: "wg-mesh", type: "wg" },
{ name: "wg-server", type: "wg" },
{ name: "wg-flow", type: "wg" },
]),
JSON.stringify([
{ name: "ether1", type: "ether" },
{ name: "gre-jh", type: "gre-tunnel" },
]),
])
resetIfaceCacheForTests()
rememberServerIfaces(serverId, [
{ name: "gre-client", ifindex: "2" },
{ name: "wan1", ifindex: "8" },
{ name: "gre-en", ifindex: "9" },
{ name: "NSK-SERVHOST-RTK" },
{ name: "wg-mesh" },
{ name: "wg-server" },
{ name: "wg-flow" },
])
rememberServerIfaces(enId, [
{ name: "ether1", ifindex: "2" },
{ name: "gre-jh", ifindex: "5" },
])
setRefreshIfacesForTests(async () => {})
invalidateFlowCatalogCache()
await dbQuery(`
INSERT INTO flow_daily_facts (server_id, day, iface, country, service, asn, bytes, packets)
VALUES
($1, '2026-09-10', '2', 'US', 'https', 15169, 800, 10),
($1, '2026-09-10', '2', 'DE', 'dns', 15133, 200, 4),
($1, '2026-09-10', 'wan1', 'NL', 'other', 0, 70, 1),
($1, '2026-09-10', '0', 'US', 'https', 0, 999, 3),
($1, '2026-09-10', 'gre-en', 'US', 'https', 15169, 400, 2),
($1, '2026-09-10', 'NSK-SERVHOST-RTK', 'US', 'https', 15169, 300, 2),
($1, '2026-09-10', 'wg-mesh', 'US', 'https', 0, 250, 2),
($1, '2026-09-10', 'wg-flow', 'US', 'https', 0, 80, 1),
($2, '2026-09-10', 'gre-jh', 'US', 'https', 15169, 500, 5),
($2, '2026-09-10', 'ether1', 'US', 'https', 15169, 200, 2)
`, [serverId, enId])
try {
const unique = await getStatistics({ from: "2026-09-01", to: "2026-09-30", planes: "unique" })
assert.equal(unique.grain, "day")
assert.equal(unique.kpis.bytes, 1000)
const uniqueAsnSum = unique.asns.reduce((s, r) => s + r.bytes, 0)
assert.equal(uniqueAsnSum, unique.kpis.bytes, "unique KPI = SUM dest ASN")
assert.equal(unique.kpis.users, 1)
assert.ok(unique.countries.some((r) => r.id === "US"))
assert.ok(unique.users.some((r) => r.id === "u-stats-1"))
assert.equal(unique.users.find((r) => r.id === STATISTICS_UNBOUND_USER_ID), undefined)
assert.ok(unique.servers.some((r) => r.id === String(serverId)))
assert.ok(!unique.servers.some((r) => r.id === String(enId)), "EN-транзит не в сетевом KPI")
const greIface = unique.interfaces.find((r) => r.label.includes("gre-client"))
assert.ok(greIface)
assert.equal(greIface.bytes, 1000)
assert.equal(greIface.id, `${serverId}:gre-client`)
assert.ok(!unique.interfaces.some((r) => /· (?:#)?\d+$/.test(r.label)))
assert.ok(!unique.interfaces.some((r) => r.label.includes(" · —") || r.label.endsWith("· —")))
assert.ok(!unique.interfaces.some((r) => r.label.includes("gre-en")))
assert.ok(!unique.interfaces.some((r) => r.label.includes("NSK-SERVHOST-RTK")))
assert.ok(!unique.interfaces.some((r) => r.label.includes("wg-mesh")))
assert.ok(!unique.interfaces.some((r) => r.label.includes("wg-flow")))
assert.equal(unique.interfaces.find((r) => r.id === `${serverId}:wan1`), undefined, "unique без WAN")
const allPlanes = await getStatistics({ from: "2026-09-01", to: "2026-09-30", planes: "all" })
assert.equal(allPlanes.kpis.bytes, 1000, "KPI unique и all одинаковый")
const wanRow = allPlanes.interfaces.find((r) => r.id === `${serverId}:wan1`)
assert.ok(wanRow)
assert.ok(wanRow.label.includes("WAN · интернет"))
assert.equal(wanRow.bytes, 70)
assert.equal(wanRow.percent, 0)
const overlayGre = allPlanes.interfaces.find((r) => r.id === `${serverId}:gre-en`)
assert.ok(overlayGre)
assert.ok(overlayGre.label.includes("дубль"))
assert.equal(overlayGre.percent, 0)
const overlayCustom = allPlanes.interfaces.find((r) => r.label.includes("NSK-SERVHOST-RTK"))
assert.ok(overlayCustom)
assert.ok(overlayCustom.label.includes("дубль"))
const overlayWg = allPlanes.interfaces.find((r) => r.label.includes("wg-mesh"))
assert.ok(overlayWg)
assert.ok(overlayWg.label.includes("дубль"))
assert.ok(!allPlanes.interfaces.some((r) => r.label.includes("wg-flow")))
const wanSlice = await getStatistics({
from: "2026-09-01",
to: "2026-09-30",
serverId,
iface: "wan1",
})
assert.equal(wanSlice.kpis.bytes, 70)
const nodeSlice = await getStatistics({
from: "2026-09-01",
to: "2026-09-30",
serverId,
planes: "unique",
})
assert.equal(nodeSlice.kpis.bytes, 1000)
assert.equal(nodeSlice.interfaces.find((r) => r.id === `${serverId}:wan1`), undefined)
assert.ok(!nodeSlice.users.some((r) => r.id === STATISTICS_UNBOUND_USER_ID))
const nodeAll = await getStatistics({
from: "2026-09-01",
to: "2026-09-30",
serverId,
planes: "all",
})
assert.equal(nodeAll.kpis.bytes, 1000)
const nodeWan = nodeAll.interfaces.find((r) => r.id === `${serverId}:wan1`)
assert.ok(nodeWan)
assert.equal(nodeWan.percent, 0)
assert.ok(nodeWan.label.includes("WAN · интернет"))
const enSlice = await getStatistics({
from: "2026-09-01",
to: "2026-09-30",
serverId: enId,
planes: "unique",
})
assert.equal(enSlice.kpis.bytes, 0)
assert.ok(!enSlice.interfaces.some((r) => r.label.includes("gre-jh")))
assert.ok(!enSlice.interfaces.some((r) => r.label.includes("WAN · интернет")))
const enAll = await getStatistics({
from: "2026-09-01",
to: "2026-09-30",
serverId: enId,
planes: "all",
})
assert.equal(enAll.kpis.bytes, 0)
assert.ok(enAll.interfaces.some((r) => r.label.includes("WAN · интернет") && r.label.includes("ether1") && r.percent === 0))
assert.ok(enAll.interfaces.some((r) => r.label.includes("gre-jh") && r.label.includes("дубль")))
const sliced = await getStatistics({
from: "2026-09-01",
to: "2026-09-30",
country: "US",
service: "https",
asn: 15169,
})
assert.equal(sliced.kpis.bytes, 800)
assert.equal(sliced.countries.length, 1)
assert.equal(sliced.countries[0]?.id, "US")
assert.ok(sliced.users.some((r) => r.id === "u-stats-1"))
const byUser = await getStatistics({
from: "2026-09-01",
to: "2026-09-30",
userId: "u-stats-1",
})
assert.equal(byUser.kpis.bytes, 1000)
const pivot = await getStatisticsPivot({
from: "2026-09-01",
to: "2026-09-30",
row: "country",
col: "service",
metric: "bytes",
})
const us = pivot.rows.find((r) => r.id === "US")
const de = pivot.rows.find((r) => r.id === "DE")
assert.ok(us)
assert.ok(de)
assert.equal(us.cells.https, 800)
assert.equal(de.cells.dns, 200)
await dbQuery(`
INSERT INTO user_interface_bindings (id, user_id, server_id, interface_name, interface_type)
VALUES ('bind-stats-wg', 'u-stats-1', $1, 'wg-server', 'wg')
`, [serverId])
await dbQuery(`
INSERT INTO flow_daily_facts (server_id, day, iface, country, service, asn, bytes, packets)
VALUES ($1, '2026-09-10', 'wg-server', 'US', 'https', 15169, 150, 2)
`, [serverId])
invalidateFlowCatalogCache()
const withWg = await getStatistics({ from: "2026-09-01", to: "2026-09-30", planes: "unique" })
assert.equal(withWg.kpis.bytes, 1150)
assert.ok(withWg.interfaces.some((r) => r.label.includes("wg-server") && r.bytes === 150))
assert.ok(!withWg.interfaces.some((r) => r.label.includes("wg-flow")))
assert.ok(!withWg.interfaces.some((r) => r.label.includes("wg-mesh")))
await dbQuery(`
INSERT INTO flow_hour_facts (server_id, bucket_at, iface, country, service, asn, bytes, packets)
VALUES ($1, '2026-09-10T10:00:00Z', '2', 'US', 'https', 15169, 40, 2)
`, [serverId])
const hourly = await getStatistics({
from: "2026-09-10T00:00:00.000Z",
to: "2026-09-10T23:00:00.000Z",
})
assert.equal(hourly.grain, "hour")
assert.equal(hourly.kpis.bytes, 40)
assert.ok(hourly.users.some((r) => r.id === "u-stats-1"))
} finally {
setRefreshIfacesForTests(null)
resetIfaceCacheForTests()
invalidateFlowCatalogCache()
await dbQuery(`DELETE FROM flow_daily_facts WHERE server_id IN ($1, $2)`, [serverId, enId])
await dbQuery(`DELETE FROM flow_hour_facts WHERE server_id IN ($1, $2)`, [serverId, enId])
await dbQuery(`DELETE FROM user_interface_bindings WHERE server_id IN ($1, $2)`, [serverId, enId])
await dbQuery(`DELETE FROM server_snapshots WHERE server_id IN ($1, $2)`, [serverId, enId])
await dbQuery(`DELETE FROM servers WHERE id IN ($1, $2)`, [serverId, enId])
}
console.log("statistics-aggregate.test.ts: ok")
@@ -0,0 +1,904 @@
import { eq } from "drizzle-orm"
import { db, dbAll } from "../db/index.js"
import { appUsers, flowAsnMeta, servers, userInterfaceBindings } from "../db/schema.js"
import {
STATISTICS_UNBOUND_USER_ID,
type StatisticsBreakdownRow,
type StatisticsDto,
type StatisticsPivotDim,
type StatisticsPivotDto,
type StatisticsPivotQuery,
type StatisticsQuery,
} from "@mmapp/contracts/statistics"
import {
collapseServerIfaceRows,
displayFactIface,
expandBindingIfaces,
factIfaceAliases,
listCachedIfaceNames,
} from "./traffic-flow-ifindex.js"
import { refreshServerIfaces } from "./traffic-flow-ifaces.js"
import {
isDashDisplayIface,
isJunkFactIface,
isOverlayTunnelIface,
isWanFactIface,
overlayDupLabel,
wanIfaceLabel,
} from "./traffic-flow-facts-filter.js"
import { getServerCatalog, loadFlowTopology, type FlowTopology } from "./traffic-flow-topology.js"
import { OTHER_SERVICE, isNamedInternetService } from "./traffic-flow-brands.js"
/** Так же, как на карте сети: DNS/SSH/BGP/туннели и пустые метки — не отдельные сервисы, а «Прочее». */
export function normalizeFactService(label: string): string {
const s = String(label ?? "").trim()
return isNamedInternetService(s, "") ? s : OTHER_SERVICE
}
const TOP_N = 200
const HOUR_WINDOW_MS = 48 * 3600_000
const PIVOT_ROW_CAP = 50
const PIVOT_COL_CAP = 15
const PIVOT_OTHER_ID = "__other__"
export interface ParsedPeriod {
fromIso: string
toIso: string
fromDay: string
toDayExclusive: string
grain: "hour" | "day"
windowSec: number
}
function pad2(n: number): string {
return String(n).padStart(2, "0")
}
function toUtcDay(d: Date): string {
return `${d.getUTCFullYear()}-${pad2(d.getUTCMonth() + 1)}-${pad2(d.getUTCDate())}`
}
function addUtcDays(day: string, n: number): string {
const d = new Date(`${day}T00:00:00Z`)
d.setUTCDate(d.getUTCDate() + n)
return toUtcDay(d)
}
/** Parse from/to. Date-only `to` is inclusive (end of that UTC day). */
export function parseStatisticsPeriod(fromRaw: string, toRaw: string, nowMs: number = Date.now()): ParsedPeriod | null {
const from = Date.parse(fromRaw.includes("T") ? fromRaw : `${fromRaw}T00:00:00Z`)
const toHasTime = toRaw.includes("T")
const to = Date.parse(toHasTime ? toRaw : `${toRaw}T00:00:00Z`)
if (!Number.isFinite(from) || !Number.isFinite(to)) return null
const fromDate = new Date(from)
let toDate = new Date(to)
let toDayExclusive: string
if (toHasTime) {
toDayExclusive = toUtcDay(toDate)
if (toDate.getUTCHours() !== 0 || toDate.getUTCMinutes() !== 0 || toDate.getUTCSeconds() !== 0) {
toDayExclusive = addUtcDays(toDayExclusive, 1)
}
} else {
toDayExclusive = addUtcDays(toUtcDay(toDate), 1)
toDate = new Date(`${toDayExclusive}T00:00:00Z`)
}
// Конец периода в будущем (например, «сегодня»): окно длится только до сейчас,
// иначе avgBps размывается ещё не наступившими часами суток.
if (toDate.getTime() > nowMs) toDate = new Date(nowMs)
if (toDate.getTime() <= from) return null
const windowSec = Math.max(1, Math.round((toDate.getTime() - from) / 1000))
const grain: "hour" | "day" = toDate.getTime() - from <= HOUR_WINDOW_MS ? "hour" : "day"
return {
fromIso: fromDate.toISOString(),
toIso: toDate.toISOString(),
fromDay: toUtcDay(fromDate),
toDayExclusive,
grain,
windowSec,
}
}
type FactScope = "unique" | "wan" | "overlay"
interface FilterCtx {
fromIso: string
toIso: string
fromDay: string
toDayExclusive: string
serverId?: number
iface?: string
country?: string
service?: string
asn?: number
planes: "unique" | "all"
userIfaces: Array<{ serverId: number; iface: string }> | null
unboundOnly: boolean
boundIfaces: Array<{ serverId: number; iface: string }>
overlayIfaces: Array<{ serverId: number; iface: string }>
wanIfaces: Array<{ serverId: number; iface: string }>
excludeServerIds: number[]
topo: FlowTopology | null
}
function ifaceFilterAliases(iface: string, serverId?: number): string[] {
return factIfaceAliases(iface.trim(), serverId)
}
function looksLikeIfIndex(iface: string): boolean {
const raw = iface.trim()
return /^\d+$/.test(raw) || /^#\d+$/.test(raw)
}
async function warmIfaceCache(ids: Iterable<number>): Promise<void> {
const uniq = [...new Set(ids)].filter((id) => Number.isFinite(id) && id > 0)
if (!uniq.length) return
await Promise.all(uniq.map((id) => refreshServerIfaces(id)))
}
async function warmBindingIfaceCache(): Promise<void> {
const rows = await db.select({ serverId: userInterfaceBindings.serverId }).from(userInterfaceBindings)
await warmIfaceCache(rows.map((r) => r.serverId))
}
function canonicalIfaceDimId(id: string): string {
const colon = id.indexOf(":")
if (colon < 0) return id
const sid = Number(id.slice(0, colon))
if (!Number.isFinite(sid)) return id
return `${sid}:${displayFactIface(sid, id.slice(colon + 1))}`
}
function pushIfaceTuples(
parts: string[],
params: unknown[],
alias: string,
tuples: Array<{ serverId: number; iface: string }>,
op: "IN" | "NOT IN",
): void {
if (!tuples.length) {
if (op === "IN") parts.push("FALSE")
return
}
const sql = tuples.map(() => "(?, ?)").join(", ")
parts.push(`(${alias}.server_id, ${alias}.iface) ${op} (${sql})`)
for (const t of tuples) {
params.push(t.serverId, t.iface)
}
}
function factWhere(
alias: string,
grain: "hour" | "day",
ctx: FilterCtx,
scope: FactScope = "unique",
): { sql: string; params: unknown[] } {
const params: unknown[] = []
const parts: string[] = []
if (grain === "hour") {
params.push(ctx.fromIso, ctx.toIso)
parts.push(`${alias}.bucket_at >= ? AND ${alias}.bucket_at < ?`)
} else {
params.push(ctx.fromDay, ctx.toDayExclusive)
parts.push(`${alias}.day >= ? AND ${alias}.day < ?`)
}
if (ctx.serverId != null) {
parts.push(`${alias}.server_id = ?`)
params.push(ctx.serverId)
}
if (ctx.country) {
parts.push(`${alias}.country = ?`)
params.push(ctx.country.toUpperCase())
}
if (ctx.service) {
parts.push(`${alias}.service = ?`)
params.push(ctx.service)
}
if (ctx.asn != null) {
parts.push(`${alias}.asn = ?`)
params.push(ctx.asn)
}
parts.push(`${alias}.iface NOT IN ('0', '—', '__unknown__', 'wg-flow', '')`)
if (scope === "wan") {
pushIfaceTuples(parts, params, alias, ctx.wanIfaces, "IN")
return { sql: parts.join(" AND "), params }
}
if (scope === "overlay") {
pushIfaceTuples(parts, params, alias, ctx.overlayIfaces, "IN")
return { sql: parts.join(" AND "), params }
}
if (ctx.iface) {
const aliases = ifaceFilterAliases(ctx.iface, ctx.serverId)
if (aliases.length <= 1) {
parts.push(`${alias}.iface = ?`)
params.push(aliases[0] ?? ctx.iface)
} else {
parts.push(`${alias}.iface IN (${aliases.map(() => "?").join(", ")})`)
params.push(...aliases)
}
return { sql: parts.join(" AND "), params }
}
if (ctx.userIfaces) {
pushIfaceTuples(parts, params, alias, ctx.userIfaces, "IN")
return { sql: parts.join(" AND "), params }
}
if (ctx.unboundOnly) {
parts.push("FALSE")
return { sql: parts.join(" AND "), params }
}
pushIfaceTuples(parts, params, alias, ctx.boundIfaces, "IN")
if (ctx.excludeServerIds.length) {
parts.push(`${alias}.server_id NOT IN (${ctx.excludeServerIds.map(() => "?").join(", ")})`)
params.push(...ctx.excludeServerIds)
}
return { sql: parts.join(" AND "), params }
}
function emptyDto(period: ParsedPeriod): StatisticsDto {
return {
from: period.fromIso,
to: period.toIso,
grain: period.grain,
kpis: {
bytes: 0,
packets: 0,
avgBps: 0,
users: 0,
servers: 0,
ifaces: 0,
topCountry: "—",
topService: "—",
},
series: [],
users: [],
servers: [],
interfaces: [],
countries: [],
services: [],
asns: [],
}
}
function toBreakdown(
rows: Array<{ id: string; label: string; bytes: number; packets: number }>,
totalBytes: number,
windowSec: number,
): StatisticsBreakdownRow[] {
const denom = totalBytes || 1
return rows
.sort((a, b) => b.bytes - a.bytes)
.slice(0, TOP_N)
.map((r) => ({
id: r.id,
label: r.label,
bytes: r.bytes,
packets: r.packets,
bps: (r.bytes * 8) / windowSec,
percent: (r.bytes / denom) * 100,
}))
}
interface UserBindTuple {
userId: string
serverId: number
iface: string
}
async function loadBindUserTuples(): Promise<UserBindTuple[]> {
const binds = await db.select().from(userInterfaceBindings)
const seen = new Set<string>()
const out: UserBindTuple[] = []
for (const b of binds) {
for (const iface of factIfaceAliases(b.interfaceName, b.serverId)) {
const k = `${b.userId}\0${b.serverId}\0${iface}`
if (seen.has(k)) continue
seen.add(k)
out.push({ userId: b.userId, serverId: b.serverId, iface })
}
}
return out
}
function uniqueBoundIfaces(tuples: UserBindTuple[]): Array<{ serverId: number; iface: string }> {
const seen = new Set<string>()
const out: Array<{ serverId: number; iface: string }> = []
for (const t of tuples) {
const k = `${t.serverId}\0${t.iface}`
if (seen.has(k)) continue
seen.add(k)
out.push({ serverId: t.serverId, iface: t.iface })
}
return out
}
async function resolveUserIfaces(userId?: string): Promise<Array<{ serverId: number; iface: string }> | null> {
if (!userId || userId === STATISTICS_UNBOUND_USER_ID) return null
const binds = await db.select().from(userInterfaceBindings).where(eq(userInterfaceBindings.userId, userId))
return expandBindingIfaces(binds.map((b) => ({ serverId: b.serverId, iface: b.interfaceName })))
}
function userBindJoinSql(tuples: UserBindTuple[]): { sql: string; params: unknown[] } {
const values = tuples.map(() => "(?::text, ?::int, ?::text)").join(", ")
const params = tuples.flatMap((t) => [t.userId, t.serverId, t.iface])
return {
sql: `JOIN (VALUES ${values}) AS b(user_id, server_id, iface) ON b.server_id = f.server_id AND b.iface = f.iface`,
params,
}
}
function expandIfaceTuples(
items: Array<{ serverId: number; iface: string }>,
): Array<{ serverId: number; iface: string }> {
const seen = new Set<string>()
const out: Array<{ serverId: number; iface: string }> = []
for (const t of items) {
for (const iface of factIfaceAliases(t.iface, t.serverId)) {
const k = `${t.serverId}\0${iface}`
if (seen.has(k)) continue
seen.add(k)
out.push({ serverId: t.serverId, iface })
}
}
return out
}
async function loadPayloadScope(serverId?: number): Promise<{
overlayIfaces: Array<{ serverId: number; iface: string }>
wanIfaces: Array<{ serverId: number; iface: string }>
excludeServerIds: number[]
topo: FlowTopology
}> {
const topo = await loadFlowTopology()
const catalog = await getServerCatalog()
await warmIfaceCache(catalog.list.map((s) => s.id))
const overlayRaw: Array<{ serverId: number; iface: string }> = []
const wanRaw: Array<{ serverId: number; iface: string }> = []
for (const s of catalog.list) {
if (serverId != null && s.id !== serverId) continue
const wanSet = topo.wanIfaces.get(s.id)
const wanNames = wanSet && wanSet.size > 0
? [...wanSet]
: s.type === "home-router" ? [] : ["ether1"]
for (const name of wanNames) wanRaw.push({ serverId: s.id, iface: name })
const names = new Set(listCachedIfaceNames(s.id))
for (const name of topo.tunnelIfaces?.get(s.id) ?? []) names.add(name)
for (const name of names) {
if (isOverlayTunnelIface(topo, s.id, name)) overlayRaw.push({ serverId: s.id, iface: name })
}
}
return {
overlayIfaces: expandIfaceTuples(overlayRaw),
wanIfaces: expandIfaceTuples(wanRaw),
excludeServerIds: serverId != null
? []
: catalog.list.filter((s) => s.type === "exit-node").map((s) => s.id),
topo,
}
}
async function buildFilterCtx(query: StatisticsQuery, period: ParsedPeriod): Promise<FilterCtx | null> {
const bindTuples = await loadBindUserTuples()
const boundIfaces = uniqueBoundIfaces(bindTuples)
const unboundOnly = query.userId === STATISTICS_UNBOUND_USER_ID
const userIfaces = unboundOnly ? null : await resolveUserIfaces(query.userId)
if (userIfaces && userIfaces.length === 0) return null
if (unboundOnly) return null
const scope = await loadPayloadScope(query.serverId)
return {
...period,
serverId: query.serverId,
iface: query.iface,
country: query.country,
service: query.service,
asn: query.asn,
planes: query.planes ?? "unique",
userIfaces,
unboundOnly,
boundIfaces,
overlayIfaces: scope.overlayIfaces,
wanIfaces: scope.wanIfaces,
excludeServerIds: scope.excludeServerIds,
topo: scope.topo,
}
}
export async function getStatistics(query: StatisticsQuery): Promise<StatisticsDto> {
const period = parseStatisticsPeriod(query.from, query.to)
if (!period) return emptyDto({
fromIso: query.from,
toIso: query.to,
fromDay: query.from.slice(0, 10),
toDayExclusive: query.to.slice(0, 10),
grain: "day",
windowSec: 1,
})
await warmBindingIfaceCache()
if (query.serverId) await warmIfaceCache([query.serverId])
const bindTuples = await loadBindUserTuples()
const ctx = await buildFilterCtx(query, period)
if (!ctx) return emptyDto(period)
const table = period.grain === "hour" ? "flow_hour_facts" : "flow_daily_facts"
const timeCol = period.grain === "hour" ? "bucket_at" : "day"
const where = factWhere("f", period.grain, ctx)
const totals = await dbAll<{ bytes: number; packets: number; servers: number }>(`
SELECT
COALESCE(SUM(f.bytes), 0) AS bytes,
COALESCE(SUM(f.packets), 0) AS packets,
COUNT(DISTINCT f.server_id)::int AS servers
FROM ${table} f
WHERE ${where.sql}
`, where.params)
const bytes = Number(totals[0]?.bytes) || 0
const packets = Number(totals[0]?.packets) || 0
const serverCount = Number(totals[0]?.servers) || 0
const seriesRows = await dbAll<{ t: string; bytes: number }>(`
SELECT ${timeCol}::text AS t, SUM(f.bytes) AS bytes
FROM ${table} f
WHERE ${where.sql}
GROUP BY ${timeCol}
ORDER BY ${timeCol}
`, where.params)
const countryRows = await dbAll<{ id: string; bytes: number; packets: number }>(`
SELECT f.country AS id, SUM(f.bytes) AS bytes, SUM(f.packets) AS packets
FROM ${table} f
WHERE ${where.sql}
GROUP BY f.country
`, where.params)
const serviceRows = await dbAll<{ id: string; bytes: number; packets: number }>(`
SELECT f.service AS id, SUM(f.bytes) AS bytes, SUM(f.packets) AS packets
FROM ${table} f
WHERE ${where.sql}
GROUP BY f.service
`, where.params)
const asnRows = await dbAll<{ id: number; bytes: number; packets: number }>(`
SELECT f.asn AS id, SUM(f.bytes) AS bytes, SUM(f.packets) AS packets
FROM ${table} f
WHERE ${where.sql}
GROUP BY f.asn
`, where.params)
const serverRows = await dbAll<{ id: number; bytes: number; packets: number }>(`
SELECT f.server_id AS id, SUM(f.bytes) AS bytes, SUM(f.packets) AS packets
FROM ${table} f
WHERE ${where.sql}
GROUP BY f.server_id
`, where.params)
const ifaceRowsRaw = await dbAll<{ serverId: number; iface: string; bytes: number; packets: number }>(`
SELECT f.server_id AS "serverId", f.iface AS iface, SUM(f.bytes) AS bytes, SUM(f.packets) AS packets
FROM ${table} f
WHERE ${where.sql}
GROUP BY f.server_id, f.iface
`, where.params)
await warmIfaceCache(ifaceRowsRaw.filter((r) => looksLikeIfIndex(r.iface)).map((r) => r.serverId))
const ifaceRows = collapseServerIfaceRows(ifaceRowsRaw).filter((r) => {
if (isJunkFactIface(r.iface) || isDashDisplayIface(r.iface)) return false
if (ctx.iface) return true
if (ctx.topo && isOverlayTunnelIface(ctx.topo, r.serverId, r.iface)) return false
if (ctx.topo && isWanFactIface(ctx.topo, r.serverId, r.iface)) return false
return true
})
const ifaceCount = ifaceRows.length
let dupeIfaceRows: Array<{ serverId: number; iface: string; bytes: number; packets: number; kind: "wan" | "overlay" }> = []
if (ctx.planes === "all" && !ctx.iface) {
const wanWhere = factWhere("f", period.grain, ctx, "wan")
const overlayWhere = factWhere("f", period.grain, ctx, "overlay")
const [wanRaw, overlayRaw] = await Promise.all([
dbAll<{ serverId: number; iface: string; bytes: number; packets: number }>(`
SELECT f.server_id AS "serverId", f.iface AS iface, SUM(f.bytes) AS bytes, SUM(f.packets) AS packets
FROM ${table} f
WHERE ${wanWhere.sql}
GROUP BY f.server_id, f.iface
`, wanWhere.params),
dbAll<{ serverId: number; iface: string; bytes: number; packets: number }>(`
SELECT f.server_id AS "serverId", f.iface AS iface, SUM(f.bytes) AS bytes, SUM(f.packets) AS packets
FROM ${table} f
WHERE ${overlayWhere.sql}
GROUP BY f.server_id, f.iface
`, overlayWhere.params),
])
await warmIfaceCache([
...wanRaw.filter((r) => looksLikeIfIndex(r.iface)).map((r) => r.serverId),
...overlayRaw.filter((r) => looksLikeIfIndex(r.iface)).map((r) => r.serverId),
])
const seen = new Set(ifaceRows.map((r) => `${r.serverId}:${r.iface}`))
for (const r of collapseServerIfaceRows(wanRaw)) {
if (isJunkFactIface(r.iface) || isDashDisplayIface(r.iface)) continue
const key = `${r.serverId}:${r.iface}`
if (seen.has(key)) continue
seen.add(key)
dupeIfaceRows.push({ ...r, kind: "wan" })
}
for (const r of collapseServerIfaceRows(overlayRaw)) {
if (isJunkFactIface(r.iface) || isDashDisplayIface(r.iface)) continue
const key = `${r.serverId}:${r.iface}`
if (seen.has(key)) continue
seen.add(key)
dupeIfaceRows.push({ ...r, kind: "overlay" })
}
}
let userRows: Array<{ id: string; bytes: number; packets: number }> = []
if (bindTuples.length && !ctx.unboundOnly) {
const join = userBindJoinSql(bindTuples)
userRows = await dbAll<{ id: string; bytes: number; packets: number }>(`
SELECT b.user_id AS id, SUM(f.bytes) AS bytes, SUM(f.packets) AS packets
FROM ${table} f
${join.sql}
WHERE ${where.sql}
GROUP BY b.user_id
`, [...join.params, ...where.params])
}
const serverNames = new Map<number, string>()
const allServers = await db.select({ id: servers.id, name: servers.name, host: servers.host }).from(servers)
for (const s of allServers) serverNames.set(s.id, s.name || s.host)
const userNames = new Map<string, string>()
const allUsers = await db.select({ id: appUsers.id, name: appUsers.name, login: appUsers.login }).from(appUsers)
for (const u of allUsers) userNames.set(u.id, u.name || u.login)
const asnHolders = new Map<number, string>()
const asnMeta = await db.select({ asn: flowAsnMeta.asn, holder: flowAsnMeta.holder }).from(flowAsnMeta)
for (const a of asnMeta) asnHolders.set(a.asn, a.holder)
const countries = toBreakdown(
countryRows.map((r) => ({
id: r.id,
label: r.id === "XX" ? "Неизвестно" : r.id,
bytes: Number(r.bytes) || 0,
packets: Number(r.packets) || 0,
})),
bytes,
period.windowSec,
)
const services = toBreakdown(
Object.entries(serviceRows.reduce<Record<string, { bytes: number; packets: number }>>((acc, r) => {
const key = normalizeFactService(r.id)
const prev = acc[key] ?? { bytes: 0, packets: 0 }
prev.bytes += Number(r.bytes) || 0
prev.packets += Number(r.packets) || 0
acc[key] = prev
return acc
}, {})).map(([label, v]) => ({ id: label, label, bytes: v.bytes, packets: v.packets })),
bytes,
period.windowSec,
)
const asns = toBreakdown(
asnRows.map((r) => {
const id = Number(r.id) || 0
const holder = asnHolders.get(id)
return {
id: String(id),
label: id === 0 ? "other" : holder ? `AS${id} · ${holder}` : `AS${id}`,
bytes: Number(r.bytes) || 0,
packets: Number(r.packets) || 0,
}
}),
bytes,
period.windowSec,
)
const serverBreakdown = toBreakdown(
serverRows.map((r) => ({
id: String(r.id),
label: serverNames.get(r.id) || String(r.id),
bytes: Number(r.bytes) || 0,
packets: Number(r.packets) || 0,
})),
bytes,
period.windowSec,
)
const uniqueInterfaces = toBreakdown(
ifaceRows.map((r) => {
const serverName = serverNames.get(r.serverId) || String(r.serverId)
const wan = ctx.topo ? isWanFactIface(ctx.topo, r.serverId, r.iface) : false
return {
id: `${r.serverId}:${r.iface}`,
label: wan ? wanIfaceLabel(serverName, r.iface) : `${serverName} · ${r.iface}`,
bytes: Number(r.bytes) || 0,
packets: Number(r.packets) || 0,
}
}),
bytes,
period.windowSec,
)
const dupeInterfaces: StatisticsBreakdownRow[] = dupeIfaceRows.map((r) => {
const serverName = serverNames.get(r.serverId) || String(r.serverId)
const rowBytes = Number(r.bytes) || 0
const rowPackets = Number(r.packets) || 0
return {
id: `${r.serverId}:${r.iface}`,
label: r.kind === "wan" ? wanIfaceLabel(serverName, r.iface) : overlayDupLabel(serverName, r.iface),
bytes: rowBytes,
packets: rowPackets,
bps: (rowBytes * 8) / period.windowSec,
percent: 0,
}
})
const interfaces = [...uniqueInterfaces, ...dupeInterfaces]
const matchedUsers = toBreakdown(
userRows.map((r) => ({
id: r.id,
label: userNames.get(r.id) || r.id,
bytes: Number(r.bytes) || 0,
packets: Number(r.packets) || 0,
})),
bytes,
period.windowSec,
)
const users = [...matchedUsers]
return {
from: period.fromIso,
to: period.toIso,
grain: period.grain,
kpis: {
bytes,
packets,
avgBps: (bytes * 8) / period.windowSec,
users: matchedUsers.length,
servers: serverCount,
ifaces: ifaceCount,
topCountry: countries[0]?.label || "—",
topService: services[0]?.label || "—",
},
series: seriesRows.map((r) => ({ t: r.t, bytes: Number(r.bytes) || 0 })),
users,
servers: serverBreakdown,
interfaces,
countries,
services,
asns,
}
}
function dimSql(dim: StatisticsPivotDim, factAlias: string, bindAlias: string): string {
if (dim === "country") return `${factAlias}.country`
if (dim === "service") return `${factAlias}.service`
if (dim === "asn") return `${factAlias}.asn::text`
if (dim === "server") return `${factAlias}.server_id::text`
if (dim === "iface") return `(${factAlias}.server_id::text || ':' || ${factAlias}.iface)`
return `${bindAlias}.user_id`
}
function emptyPivot(query: StatisticsPivotQuery): StatisticsPivotDto {
return {
rowDim: query.row,
colDim: query.col,
metric: query.metric,
columns: [],
rows: [],
otherBytes: 0,
}
}
export function pivotDimsConflict(row: StatisticsPivotDim, col: StatisticsPivotDim): boolean {
return row === col
}
export async function getStatisticsPivot(query: StatisticsPivotQuery): Promise<StatisticsPivotDto> {
if (pivotDimsConflict(query.row, query.col)) return emptyPivot(query)
const period = parseStatisticsPeriod(query.from, query.to)
if (!period) return emptyPivot(query)
await warmBindingIfaceCache()
if (query.serverId) await warmIfaceCache([query.serverId])
const bindTuples = await loadBindUserTuples()
const ctx = await buildFilterCtx(query, period)
if (!ctx) return emptyPivot(query)
const needsUser = query.row === "user" || query.col === "user"
if (needsUser && bindTuples.length === 0) return emptyPivot(query)
const table = period.grain === "hour" ? "flow_hour_facts" : "flow_daily_facts"
const where = factWhere("f", period.grain, ctx)
const rowExpr = dimSql(query.row, "f", "b")
const colExpr = dimSql(query.col, "f", "b")
const join = needsUser ? userBindJoinSql(bindTuples) : { sql: "", params: [] as unknown[] }
const raw = await dbAll<{ row_id: string; col_id: string; bytes: number; packets: number }>(`
SELECT ${rowExpr} AS row_id, ${colExpr} AS col_id,
SUM(f.bytes) AS bytes, SUM(f.packets) AS packets
FROM ${table} f
${join.sql}
WHERE ${where.sql}
GROUP BY 1, 2
`, [...join.params, ...where.params])
if (query.row === "service" || query.col === "service") {
for (const r of raw) {
if (query.row === "service") r.row_id = normalizeFactService(String(r.row_id ?? ""))
if (query.col === "service") r.col_id = normalizeFactService(String(r.col_id ?? ""))
}
}
if (query.row === "iface" || query.col === "iface") {
const ifaceServerIds: number[] = []
for (const r of raw) {
for (const dim of [query.row, query.col] as const) {
if (dim !== "iface") continue
const id = dim === query.row ? String(r.row_id ?? "") : String(r.col_id ?? "")
const colon = id.indexOf(":")
if (colon < 0) continue
const sid = Number(id.slice(0, colon))
if (looksLikeIfIndex(id.slice(colon + 1)) && Number.isFinite(sid)) ifaceServerIds.push(sid)
}
}
await warmIfaceCache(ifaceServerIds)
for (const r of raw) {
if (query.row === "iface") r.row_id = canonicalIfaceDimId(String(r.row_id ?? ""))
if (query.col === "iface") r.col_id = canonicalIfaceDimId(String(r.col_id ?? ""))
}
}
const metric = query.metric
type Acc = { bytes: number; packets: number }
const cell = new Map<string, Map<string, Acc>>()
const colTotals = new Map<string, number>()
for (const r of raw) {
const rid = String(r.row_id ?? "")
const cid = String(r.col_id ?? "")
const acc: Acc = { bytes: Number(r.bytes) || 0, packets: Number(r.packets) || 0 }
const val = metric === "packets" ? acc.packets : acc.bytes
let rowMap = cell.get(rid)
if (!rowMap) {
rowMap = new Map()
cell.set(rid, rowMap)
}
const prev = rowMap.get(cid)
if (prev) {
prev.bytes += acc.bytes
prev.packets += acc.packets
} else {
rowMap.set(cid, acc)
}
colTotals.set(cid, (colTotals.get(cid) ?? 0) + val)
}
const topCols = [...colTotals.entries()]
.sort((a, b) => b[1] - a[1])
.slice(0, PIVOT_COL_CAP)
.map(([id]) => id)
const topColSet = new Set(topCols)
const folded = new Map<string, Map<string, number>>()
const foldedColTotals = new Map<string, number>()
let otherBytes = 0
for (const [rid, cols] of cell) {
const rowMap = new Map<string, number>()
for (const [cid, acc] of cols) {
const val = metric === "packets" ? acc.packets : acc.bytes
const dest = topColSet.has(cid) ? cid : PIVOT_OTHER_ID
if (dest === PIVOT_OTHER_ID) otherBytes += val
rowMap.set(dest, (rowMap.get(dest) ?? 0) + val)
foldedColTotals.set(dest, (foldedColTotals.get(dest) ?? 0) + val)
}
folded.set(rid, rowMap)
}
const rowTotals = new Map<string, number>()
for (const [rid, cols] of folded) {
let t = 0
for (const v of cols.values()) t += v
rowTotals.set(rid, t)
}
const topRows = [...rowTotals.entries()]
.sort((a, b) => b[1] - a[1])
.slice(0, PIVOT_ROW_CAP)
.map(([id]) => id)
const topRowSet = new Set(topRows)
const finalRows = new Map<string, Map<string, number>>()
const finalRowTotals = new Map<string, number>()
for (const [rid, cols] of folded) {
const dest = topRowSet.has(rid) ? rid : PIVOT_OTHER_ID
if (dest === PIVOT_OTHER_ID) {
for (const [cid, v] of cols) {
if (cid !== PIVOT_OTHER_ID) otherBytes += v
}
}
let rowMap = finalRows.get(dest)
if (!rowMap) {
rowMap = new Map()
finalRows.set(dest, rowMap)
}
for (const [cid, v] of cols) {
rowMap.set(cid, (rowMap.get(cid) ?? 0) + v)
}
}
for (const [rid, cols] of finalRows) {
let t = 0
for (const v of cols.values()) t += v
finalRowTotals.set(rid, t)
}
const colIds = [...topCols]
if (foldedColTotals.has(PIVOT_OTHER_ID)) colIds.push(PIVOT_OTHER_ID)
const rowIds = [...topRows]
if (finalRows.has(PIVOT_OTHER_ID) && !topRowSet.has(PIVOT_OTHER_ID)) rowIds.push(PIVOT_OTHER_ID)
const labels = await loadPivotLabels(query.row, query.col, rowIds, colIds)
return {
rowDim: query.row,
colDim: query.col,
metric,
columns: colIds.map((id) => ({
id,
label: labels.col.get(id) ?? (id === PIVOT_OTHER_ID ? "Прочие" : id),
total: foldedColTotals.get(id) ?? 0,
})),
rows: rowIds.map((id) => {
const cols = finalRows.get(id) ?? new Map()
const cells: Record<string, number> = {}
for (const cid of colIds) cells[cid] = cols.get(cid) ?? 0
return {
id,
label: labels.row.get(id) ?? (id === PIVOT_OTHER_ID ? "Прочие" : id),
total: finalRowTotals.get(id) ?? 0,
cells,
}
}),
otherBytes,
}
}
async function loadPivotLabels(
rowDim: StatisticsPivotDim,
colDim: StatisticsPivotDim,
rowIds: string[],
colIds: string[],
): Promise<{ row: Map<string, string>; col: Map<string, string> }> {
const topo = await loadFlowTopology()
const serverNames = new Map<string, string>()
const allServers = await db.select({ id: servers.id, name: servers.name, host: servers.host }).from(servers)
for (const s of allServers) serverNames.set(String(s.id), s.name || s.host)
const userNames = new Map<string, string>()
const allUsers = await db.select({ id: appUsers.id, name: appUsers.name, login: appUsers.login }).from(appUsers)
for (const u of allUsers) userNames.set(u.id, u.name || u.login)
const asnHolders = new Map<string, string>()
const asnMeta = await db.select({ asn: flowAsnMeta.asn, holder: flowAsnMeta.holder }).from(flowAsnMeta)
for (const a of asnMeta) asnHolders.set(String(a.asn), a.holder)
function label(dim: StatisticsPivotDim, id: string): string {
if (id === PIVOT_OTHER_ID) return "Прочие"
if (dim === "country") return id === "XX" ? "Неизвестно" : id
if (dim === "server") return serverNames.get(id) || id
if (dim === "user") return userNames.get(id) || id
if (dim === "asn") {
if (id === "0") return "other"
const holder = asnHolders.get(id)
return holder ? `AS${id} · ${holder}` : `AS${id}`
}
if (dim === "iface") {
const colon = id.indexOf(":")
if (colon < 0) return id
const sid = id.slice(0, colon)
const iface = id.slice(colon + 1)
const sidNum = Number(sid)
const name = Number.isFinite(sidNum) ? displayFactIface(sidNum, iface) : iface
const serverName = serverNames.get(sid) || sid
if (Number.isFinite(sidNum) && isWanFactIface(topo, sidNum, name)) {
return wanIfaceLabel(serverName, name)
}
if (Number.isFinite(sidNum) && isOverlayTunnelIface(topo, sidNum, name)) {
return overlayDupLabel(serverName, name)
}
return `${serverName} · ${name}`
}
return id
}
const row = new Map<string, string>()
const col = new Map<string, string>()
for (const id of rowIds) row.set(id, label(rowDim, id))
for (const id of colIds) col.set(id, label(colDim, id))
return { row, col }
}
+6 -13
View File
@@ -1,6 +1,7 @@
import { and, asc, desc, eq, gte, lt } from "drizzle-orm"
import { and, asc, desc, eq, gte } from "drizzle-orm"
import { db } from "../db/index.js"
import { servers, trafficSamples, trafficSettings } from "../db/schema.js"
import { decodeTrafficSampleFlags, encodeTrafficFlags } from "../db/traffic-flags.js"
import type { TrafficRunSnapshot } from "../types/scheduler-run-snapshot.js"
import { SCHEDULER_RUN_SNAPSHOT_VERSION } from "../types/scheduler-run-snapshot.js"
import { MikrotikClient } from "./mikrotik.js"
@@ -79,12 +80,6 @@ async function getSettingsRow() {
return (await db.select().from(trafficSettings).where(eq(trafficSettings.id, 1)).limit(1))[0]
}
async function cleanupOldSamples(retentionDays: number) {
const cutoff = new Date(Date.now() - retentionDays * 24 * 60 * 60 * 1000).toISOString()
await db.delete(trafficSamples)
.where(lt(trafficSamples.sampledAt, cutoff))
}
async function readPreviousWave(serverId: number): Promise<Map<string, { rxBytes: number; txBytes: number; sampledAt: string }>> {
const last = (await db
.select({ sampledAt: trafficSamples.sampledAt })
@@ -164,8 +159,7 @@ export async function collectTrafficOnce(): Promise<TrafficRunSnapshot> {
txBytes,
rxBps,
txBps,
running,
disabled,
flags: encodeTrafficFlags(running, disabled),
}
})
try {
@@ -194,8 +188,7 @@ export async function collectTrafficOnce(): Promise<TrafficRunSnapshot> {
txBytes,
rxBps,
txBps,
running,
disabled,
flags: encodeTrafficFlags(running, disabled),
})
}
} catch {
@@ -224,7 +217,6 @@ export async function collectTrafficOnce(): Promise<TrafficRunSnapshot> {
}
}
await cleanupOldSamples(Math.max(1, settings.retentionDays))
await db.update(trafficSettings).set({
lastCollectedAt: now,
lastDurationMs: Date.now() - startedAt,
@@ -286,11 +278,12 @@ export async function updateTrafficSettings(patch: {
}
export async function readServerSamplesInRange(serverId: number, sinceIso: string) {
return await db.select()
const rows = await db.select()
.from(trafficSamples)
.where(and(
eq(trafficSamples.serverId, serverId),
gte(trafficSamples.sampledAt, sinceIso),
))
.orderBy(asc(trafficSamples.sampledAt))
return rows.map((r) => ({ ...r, ...decodeTrafficSampleFlags(r.flags) }))
}
@@ -16,6 +16,29 @@ import {
seedRipeCacheForTests,
} from "./traffic-flow-ripe.js"
{
const liveErr = await formatLiveSseFromBuilder(() => {
throw new Error("SQLITE_BUSY")
})
assert.equal(liveErr.event, "error")
assert.equal((liveErr.data as { error: string }).error, "SQLITE_BUSY")
const liveOk = await formatLiveSseFromBuilder(() => ({ ok: true }))
assert.equal(liveOk.event, "sample")
const liveAsync = await formatLiveSseFromBuilder(async () => ({
uniqueSrc: 3,
destinations: [{ id: "8.8.8.8", label: "8.8.8.8", bytes: 1, packets: 1, bps: 1, percent: 100 }],
}))
assert.equal(liveAsync.event, "sample")
assert.notEqual(JSON.stringify(liveAsync.data), "{}")
assert.equal((liveAsync.data as { uniqueSrc: number }).uniqueSrc, 3)
assert.ok(Array.isArray((liveAsync.data as { destinations: unknown[] }).destinations))
const liveReject = await formatLiveSseFromBuilder(async () => {
throw new Error("pg down")
})
assert.equal(liveReject.event, "error")
assert.equal((liveReject.data as { error: string }).error, "pg down")
}
if (!(await withPgOrSkip())) {
console.log("traffic-flow-analytics.test.ts: skip")
process.exit(0)
@@ -248,13 +271,6 @@ try {
assert.equal(degraded.degraded, true)
assert.equal(degraded.conversationsList.length, 0)
assert.ok((degraded.bytes ?? 0) >= 12_000)
const liveErr = formatLiveSseFromBuilder(() => {
throw new Error("SQLITE_BUSY")
})
assert.equal(liveErr.event, "error")
assert.equal((liveErr.data as { error: string }).error, "SQLITE_BUSY")
const liveOk = formatLiveSseFromBuilder(() => ({ ok: true }))
assert.equal(liveOk.event, "sample")
const exporters = await listFlowExporters(5)
const clients = await listFlowClients(5)
assert.ok(Array.isArray(exporters.exporters))
@@ -372,8 +388,8 @@ try {
assert.equal(def.excludeOverlayApplied, true)
assert.equal(def.excludeMeshApplied, true)
assert.ok(!def.conversationsList.some((r) => r.proto === 47))
assert.equal(def.conversationsList[0]?.service, "Google")
assert.equal(def.conversationsList[0]?.category, "Веб")
assert.equal(def.conversationsList[0]?.service, "YouTube")
assert.equal(def.conversationsList[0]?.category, "Видео / стриминг")
assert.equal(def.conversationsList[0]?.clientName, "Alice")
assert.equal(def.conversationsList[0]?.enName, "NSK-SERVHOST-RTK")
assert.equal(def.conversationsList[0]?.plane, "payload")
@@ -400,14 +416,44 @@ try {
resetIfaceCacheForTests()
resetRipeCacheForTests()
disableRipeEnqueueForTests()
seedRipeCacheForTests({
prefix: "74.125.0.0/16",
asn: 15169,
country: "US",
lat: null,
lng: null,
holder: "GOOGLE",
ok: true,
fetchedAt: Date.now(),
})
seedRipeCacheForTests({
prefix: "104.18.0.0/16",
asn: 13335,
country: "US",
lat: null,
lng: null,
holder: "CLOUDFLARENET",
ok: true,
fetchedAt: Date.now(),
})
seedRipeCacheForTests({
prefix: "146.75.0.0/16",
asn: 54113,
country: "US",
lat: null,
lng: null,
holder: "FASTLY",
ok: true,
fetchedAt: Date.now(),
})
rememberServerIfaces(7, [{ ".id": "*2", name: "ether1" }])
ingestParsedFlowsForServerForTests(7, [
{
src: "173.194.151.65",
dst: "10.200.100.53",
proto: 6,
src: "74.125.104.196/32",
dst: "10.200.100.53/32",
proto: 17,
srcPort: 443,
dstPort: 57182,
dstPort: 62598,
bytes: 12_000,
packets: 10,
inIface: "2",
@@ -424,15 +470,40 @@ try {
inIface: "2",
outIface: "2",
},
{
src: "146.75.118.132/32",
dst: "10.200.100.53/32",
proto: 6,
srcPort: 80,
dstPort: 35026,
bytes: 4_000,
packets: 5,
inIface: "2",
outIface: "2",
},
])
try {
const rev = await buildFlowAnalytics({ minutes: 5, serverId: 7 })
const google = rev.conversationsList.find((r) => r.src === "173.194.151.65")
const google = rev.conversationsList.find((r) => r.src === "74.125.104.196")
const cf = rev.conversationsList.find((r) => r.src === "104.18.35.51")
assert.equal(google?.service, "Google")
assert.equal(google?.category, "Веб")
const fastly = rev.conversationsList.find((r) => r.src === "146.75.118.132")
assert.equal(google?.service, "YouTube")
assert.equal(google?.category, "Видео / стриминг")
assert.equal(google?.internetPeer, "74.125.104.196")
assert.equal(google?.internetPeerPort, 443)
assert.equal(google?.clientIp, "10.200.100.53")
assert.equal(google?.direction, "to_client")
assert.equal(google?.dstAsn, 15169)
assert.equal(google?.dstCountry, "US")
assert.ok(!String(google?.src).includes("/"), "DTO src без /32")
assert.equal(cf?.service, "Cloudflare")
assert.equal(cf?.category, "CDN")
assert.equal(fastly?.service, "Fastly")
assert.equal(fastly?.dstAsn, 54113)
assert.equal(fastly?.dstCountry, "US")
assert.ok(rev.asns?.some((r) => r.id === "54113"))
assert.ok(rev.countries?.some((r) => r.id === "US"))
assert.ok(!rev.services?.every((s) => s.label === "Прочее"), "сервисы не схлопнуты в Прочее")
} finally {
resetFlowRingsForTests()
resetIfaceCacheForTests()
+36 -16
View File
@@ -26,11 +26,10 @@ import { resolveIfaceName } from "./traffic-flow-ifaces.js"
import { getTrafficFlowSettingsRow, listHostPeers } from "./traffic-flow-settings.js"
import { applicationName, flowRowMatchesFilter } from "./traffic-flow-apps.js"
import { dedupFlowRowsMaxBytes, flowTupleKey } from "./traffic-flow-dedup.js"
import { enqueueRipeMisses, lookupRipeCached } from "./traffic-flow-ripe.js"
import { classifyFlowDst, refreshFlowCatalogInBackground } from "./traffic-flow-classify.js"
import { isIsoCountry } from "./traffic-flow-brands.js"
import { enqueueRipeMisses } from "./traffic-flow-ripe.js"
import { classifyFlowPlane, flowBps, shouldKeepPlane } from "./traffic-flow-planes.js"
import { pickInternetPeer } from "./traffic-flow-ip.js"
import { resolveInternetDest } from "./traffic-flow-dest.js"
import { refreshFlowCatalogInBackground } from "./traffic-flow-classify.js"
import {
enGreIfaceNames,
getServerCatalog,
@@ -252,11 +251,25 @@ async function buildFlowAnalyticsUncached(q: FlowAnalyticsQuery): Promise<FlowAn
totalPackets += r.packets
srcs.add(r.src)
dsts.add(r.dst)
const peer = pickInternetPeer(r.src, r.dst, r.srcPort, r.dstPort)
peers.add(peer)
const app = applicationName(r.proto, r.dstPort, r.srcPort)
const ripe = lookupRipeCached(peer)
const classified = classifyFlowDst(peer, r.proto, r.dstPort, r.srcPort, ripe)
const destMeta = resolveInternetDest({
src: r.src,
dst: r.dst,
proto: r.proto,
srcPort: r.srcPort,
dstPort: r.dstPort,
serverId: r.serverId,
inIface: resolved.name,
topo,
natSrc: r.natSrc,
natDst: r.natDst,
natSrcPort: r.natSrcPort,
natDstPort: r.natDstPort,
})
const ep = destMeta.endpoints
if (destMeta.dest) peers.add(destMeta.dest)
const app = applicationName(r.proto, ep.peerPort || r.dstPort, ep.otherPort || r.srcPort)
const ripe = destMeta.ripe
const classified = destMeta.classified
bump(applications, app, r.bytes, r.packets)
bump(protocols, protoName(r.proto), r.bytes, r.packets)
bump(sources, r.src, r.bytes, r.packets)
@@ -268,7 +281,7 @@ async function buildFlowAnalyticsUncached(q: FlowAnalyticsQuery): Promise<FlowAn
const asnLabel = ripe.holder ? `AS${ripe.asn} ${ripe.holder}` : `AS${ripe.asn}`
bump(asns, asnId, r.bytes, r.packets, asnLabel)
}
const dstCountry = ripe?.ok && isIsoCountry(ripe.country) ? ripe.country : ""
const dstCountry = destMeta.country && destMeta.country !== "unknown" ? destMeta.country : ""
if (dstCountry) {
bump(countries, dstCountry, r.bytes, r.packets)
}
@@ -300,8 +313,8 @@ async function buildFlowAnalyticsUncached(q: FlowAnalyticsQuery): Promise<FlowAn
conv.set(ckey, {
serverId: String(r.serverId),
serverName: nameById.get(r.serverId) ?? String(r.serverId),
src: r.src,
dst: r.dst,
src: ep.packetSrc || r.src,
dst: ep.packetDst || r.dst,
proto: r.proto,
protoName: protoName(r.proto),
srcPort: r.srcPort,
@@ -320,6 +333,10 @@ async function buildFlowAnalyticsUncached(q: FlowAnalyticsQuery): Promise<FlowAn
dstAsn: ripe?.asn || undefined,
clientId: client?.userId,
clientName: client?.name,
clientIp: ep.clientIp || undefined,
internetPeer: ep.internetPeer || undefined,
internetPeerPort: ep.peerPort || undefined,
direction: ep.direction,
enId: en ? String(en.id) : undefined,
enName: en?.name,
plane,
@@ -599,9 +616,12 @@ export async function listFlowClients(minutes: number): Promise<{ clients: { id:
return { clients }
}
export function formatLiveSseFromBuilder(build: () => unknown): { event: "sample" | "error"; data: unknown } {
export async function formatLiveSseFromBuilder(
build: () => unknown | Promise<unknown>,
): Promise<{ event: "sample" | "error"; data: unknown }> {
try {
return { event: "sample", data: build() }
const data = await build()
return { event: "sample", data }
} catch (err) {
const message = err instanceof Error ? err.message : String(err)
return { event: "error", data: { error: message } }
@@ -613,10 +633,10 @@ export function isFlowAnalyticsDegraded(): boolean {
return health.pendingSize >= LIVE_DEGRADED_PENDING
}
export function safeBuildLiveFlowSample(q: Omit<FlowAnalyticsQuery, "minutes" | "skipHeavy">): {
export async function safeBuildLiveFlowSample(q: Omit<FlowAnalyticsQuery, "minutes" | "skipHeavy">): Promise<{
event: "sample" | "error"
data: unknown
} {
}> {
return formatLiveSseFromBuilder(async () => {
const skipHeavy = isFlowAnalyticsDegraded()
return await buildFlowAnalytics({ ...q, minutes: LIVE_ANALYTICS_MINUTES, skipHeavy })
@@ -1,11 +1,16 @@
import assert from "node:assert/strict"
import {
brandByAsn,
brandByHolder,
countryFromHolder,
isSteamGamePort,
lookupBrand,
OTHER_SERVICE,
isNamedInternetService,
mapServiceNodeId,
mapCountryNodeId,
mapCountryServiceNodeId,
resolveFlowBrand,
resolveRipeCountry,
} from "./traffic-flow-brands.js"
@@ -29,6 +34,8 @@ assert.equal(brandByAsn(401115)?.service, "ChatGPT")
assert.equal(lookupBrand("1.1.1.1", 13335)?.service, "Cloudflare")
assert.equal(lookupBrand("104.18.35.51", 0)?.service, "Cloudflare")
assert.equal(lookupBrand("173.194.151.65", 0)?.service, "Google")
assert.equal(lookupBrand("64.233.161.1", 0)?.service, "Google")
assert.equal(lookupBrand("142.250.1.10", 0)?.service, "Google")
assert.equal(lookupBrand("8.8.8.8", 0)?.service, "Google")
assert.equal(lookupBrand("203.0.113.9", 64500), null)
assert.equal(OTHER_SERVICE, "Прочее")
@@ -38,5 +45,77 @@ assert.equal(isNamedInternetService("GRE", "Туннель"), false)
assert.equal(isNamedInternetService("DNS", "DNS"), false)
assert.equal(mapServiceNodeId("AWS"), "svc:aws")
assert.equal(mapServiceNodeId("Cloudflare"), "svc:cloudflare")
assert.equal(mapServiceNodeId("Прочее"), "svc:other")
assert.equal(mapCountryNodeId("US"), "cc:us")
assert.equal(mapCountryNodeId("nl"), "cc:nl")
assert.equal(mapCountryNodeId(""), "cc:other")
assert.equal(mapCountryNodeId("Прочее"), "cc:other")
assert.equal(mapCountryNodeId("EU"), "cc:other")
assert.equal(mapCountryServiceNodeId("cc:us", "svc:google"), "cc:us|svc:google")
assert.equal(mapCountryServiceNodeId("cc:other", "svc:other"), "cc:other|svc:other")
assert.equal(brandByAsn(714)?.service, "Apple")
assert.equal(brandByAsn(714)?.category, "CDN")
assert.equal(brandByAsn(36459)?.service, "GitHub")
assert.equal(brandByAsn(395701)?.service, "Epic")
assert.equal(brandByAsn(6507)?.service, "Riot")
assert.equal(brandByAsn(33353)?.service, "PlayStation")
assert.equal(brandByAsn(14061)?.service, "DigitalOcean")
assert.equal(brandByAsn(24940)?.service, "Hetzner")
assert.equal(brandByAsn(8403)?.service, "Spotify")
assert.equal(brandByAsn(13414)?.service, "X")
assert.equal(brandByAsn(47541)?.service, "VK")
assert.equal(brandByAsn(47764)?.service, "VK")
assert.equal(brandByAsn(30103)?.service, "Zoom")
assert.equal(brandByAsn(19281)?.service, "Quad9")
assert.equal(brandByAsn(9059)?.service, "AWS")
assert.equal(brandByAsn(396982)?.service, "Google")
assert.equal(brandByAsn(400645)?.service, "ChatGPT")
assert.equal(brandByHolder("VALVE-CORPORATION")?.service, "Steam")
assert.equal(brandByHolder("OpenAI, LLC")?.service, "ChatGPT")
assert.equal(brandByHolder("YouTube LLC")?.service, "YouTube")
assert.equal(brandByHolder("AMAZON-AES - Amazon.com, Inc."), null)
assert.equal(isSteamGamePort(17, 27015, 50000), true)
assert.equal(isSteamGamePort(6, 443, 50000), false)
assert.equal(resolveFlowBrand("104.18.35.51", 32590, "VALVE-CORPORATION", 6, 443, 1)?.service, "Cloudflare")
assert.equal(resolveFlowBrand("203.0.113.9", 32590, "", 17, 27015, 50000)?.service, "Steam")
assert.equal(resolveFlowBrand("8.8.8.8", 15169, "GOOGLE", 6, 443, 51234)?.service, "Google")
assert.equal(resolveFlowBrand("173.194.160.163", 15169, "GOOGLE", 6, 443, 51234)?.service, "YouTube")
assert.equal(resolveFlowBrand("64.233.161.1", 0, "", 17, 443, 50000)?.service, "YouTube")
assert.equal(resolveFlowBrand("64.233.161.1", 0, "", 6, 80, 50000)?.service, "Google")
assert.equal(resolveFlowBrand("2001:4860:4860::8888", 15169, "GOOGLE", 17, 53, 53000)?.service, "Google")
assert.equal(resolveFlowBrand("2001:4860:4860::8888", 15169, "GOOGLE", 17, 443, 50000)?.service, "YouTube")
assert.equal(brandByAsn(32934)?.service, "Meta")
assert.equal(lookupBrand("157.240.12.52", 0)?.service, "Meta")
assert.equal(lookupBrand("57.144.22.192", 0)?.service, "Meta")
assert.equal(brandByHolder("Instagram LLC")?.service, "Instagram")
assert.equal(mapServiceNodeId("Instagram"), "svc:instagram")
assert.equal(isNamedInternetService("Instagram", "Видео / стриминг"), true)
assert.equal(
resolveFlowBrand("157.240.12.52", 32934, "FACEBOOK", 6, 443, 51234)?.service,
"Instagram",
"HTTPS на Meta front → Instagram, как YouTube на Google",
)
assert.equal(
resolveFlowBrand("57.144.22.192", 0, "", 17, 443, 50000)?.service,
"Instagram",
"cdninstagram CIDR :443 без ASN → Instagram",
)
assert.equal(
resolveFlowBrand("157.240.12.52", 32934, "FACEBOOK", 6, 80, 50000)?.service,
"Meta",
":80 на Meta остаётся Meta",
)
assert.equal(
resolveFlowBrand("157.240.1.1", 54115, "WHATSAPP", 6, 443, 1)?.service,
"Meta",
"AS54115 WhatsApp не становится Instagram",
)
assert.equal(resolveRipeCountry("", 9059, ""), "IE")
assert.equal(resolveRipeCountry("", 24940, ""), "DE")
console.log("traffic-flow-brands.test.ts: ok")
+279 -49
View File
@@ -7,59 +7,163 @@ export interface BrandHit {
category: string
}
const CDN = { category: "CDN" } as const
const WEB = { category: "Веб" } as const
const VIDEO = { category: "Видео / стриминг" } as const
const GAMES = { category: "Игры" } as const
const VOICE = { category: "Голос" } as const
const AI = { category: "ИИ" } as const
const DNS = { category: "DNS" } as const
const CLOUDFLARE: BrandHit = { service: "Cloudflare", ...CDN }
const FASTLY: BrandHit = { service: "Fastly", ...CDN }
const AKAMAI: BrandHit = { service: "Akamai", ...CDN }
const AWS: BrandHit = { service: "AWS", ...CDN }
const MICROSOFT: BrandHit = { service: "Microsoft", ...CDN }
const YANDEX: BrandHit = { service: "Yandex", ...CDN }
const APPLE: BrandHit = { service: "Apple", ...CDN }
const DIGITALOCEAN: BrandHit = { service: "DigitalOcean", ...CDN }
const HETZNER: BrandHit = { service: "Hetzner", ...CDN }
const OVH: BrandHit = { service: "OVH", ...CDN }
const ORACLE: BrandHit = { service: "Oracle", ...CDN }
const LINODE: BrandHit = { service: "Linode", ...CDN }
const VULTR: BrandHit = { service: "Vultr", ...CDN }
const SCALEWAY: BrandHit = { service: "Scaleway", ...CDN }
const IBM_CLOUD: BrandHit = { service: "IBM Cloud", ...CDN }
const ALIBABA: BrandHit = { service: "Alibaba", ...CDN }
const TENCENT: BrandHit = { service: "Tencent", ...CDN }
const GCORE: BrandHit = { service: "G-Core", ...CDN }
const CDN77: BrandHit = { service: "CDN77", ...CDN }
const SELECTEL: BrandHit = { service: "Selectel", ...CDN }
const TIMEWEB: BrandHit = { service: "Timeweb", ...CDN }
const BEGET: BrandHit = { service: "Beget", ...CDN }
const DDOS_GUARD: BrandHit = { service: "DDoS-Guard", ...CDN }
const META: BrandHit = { service: "Meta", ...CDN }
const INSTAGRAM: BrandHit = { service: "Instagram", ...VIDEO }
const GOOGLE: BrandHit = { service: "Google", ...WEB }
const GITHUB: BrandHit = { service: "GitHub", ...WEB }
const GITLAB: BrandHit = { service: "GitLab", ...WEB }
const X: BrandHit = { service: "X", ...WEB }
const LINKEDIN: BrandHit = { service: "LinkedIn", ...WEB }
const VK: BrandHit = { service: "VK", ...WEB }
const REDDIT: BrandHit = { service: "Reddit", ...WEB }
const DROPBOX: BrandHit = { service: "Dropbox", ...WEB }
const SNAP: BrandHit = { service: "Snap", ...WEB }
const WIKIPEDIA: BrandHit = { service: "Wikipedia", ...WEB }
const PAYPAL: BrandHit = { service: "PayPal", ...WEB }
const SALESFORCE: BrandHit = { service: "Salesforce", ...WEB }
const YOUTUBE: BrandHit = { service: "YouTube", ...VIDEO }
const NETFLIX: BrandHit = { service: "Netflix", ...VIDEO }
const TWITCH: BrandHit = { service: "Twitch", ...VIDEO }
const TIKTOK: BrandHit = { service: "TikTok", ...VIDEO }
const SPOTIFY: BrandHit = { service: "Spotify", ...VIDEO }
const STEAM: BrandHit = { service: "Steam", ...GAMES }
const BLIZZARD: BrandHit = { service: "Blizzard", ...GAMES }
const EPIC: BrandHit = { service: "Epic", ...GAMES }
const RIOT: BrandHit = { service: "Riot", ...GAMES }
const PLAYSTATION: BrandHit = { service: "PlayStation", ...GAMES }
const ROBLOX: BrandHit = { service: "Roblox", ...GAMES }
const UBISOFT: BrandHit = { service: "Ubisoft", ...GAMES }
const DISCORD: BrandHit = { service: "Discord", ...VOICE }
const TELEGRAM: BrandHit = { service: "Telegram", ...VOICE }
const ZOOM: BrandHit = { service: "Zoom", ...VOICE }
const CHATGPT: BrandHit = { service: "ChatGPT", ...AI }
const QUAD9: BrandHit = { service: "Quad9", ...DNS }
const OPENDNS: BrandHit = { service: "OpenDNS", ...DNS }
function brandEntries(hit: BrandHit, asns: number[]): Array<[number, BrandHit]> {
return asns.map((asn) => [asn, hit])
}
function hqEntries(cc: string, asns: number[]): Array<[number, string]> {
return asns.map((asn) => [asn, cc])
}
const ASN_BRANDS = new Map<number, BrandHit>([
[13335, { service: "Cloudflare", category: "CDN" }],
[209242, { service: "Cloudflare", category: "CDN" }],
[54113, { service: "Fastly", category: "CDN" }],
[20940, { service: "Akamai", category: "CDN" }],
[16509, { service: "AWS", category: "CDN" }],
[14618, { service: "AWS", category: "CDN" }],
[8075, { service: "Microsoft", category: "CDN" }],
[13238, { service: "Yandex", category: "CDN" }],
[32590, { service: "Steam", category: "Игры" }],
[57976, { service: "Blizzard", category: "Игры" }],
[2906, { service: "Netflix", category: "Видео / стриминг" }],
[40027, { service: "Netflix", category: "Видео / стриминг" }],
[15169, { service: "Google", category: "Веб" }],
[36040, { service: "YouTube", category: "Видео / стриминг" }],
[46489, { service: "Twitch", category: "Видео / стриминг" }],
[401115, { service: "ChatGPT", category: "ИИ" }],
[49544, { service: "Discord", category: "Голос" }],
[62041, { service: "Telegram", category: "Голос" }],
[59930, { service: "Telegram", category: "Голос" }],
[211157, { service: "Telegram", category: "Голос" }],
[32934, { service: "Meta", category: "CDN" }],
[396986, { service: "TikTok", category: "Видео / стриминг" }],
...brandEntries(CLOUDFLARE, [13335, 209242]),
...brandEntries(FASTLY, [54113]),
...brandEntries(AKAMAI, [20940, 16625, 32787, 35994, 16702, 24319]),
...brandEntries(AWS, [16509, 14618, 8987, 7224, 9059]),
...brandEntries(MICROSOFT, [8075, 8068, 8069, 8070]),
...brandEntries(YANDEX, [13238]),
...brandEntries(APPLE, [714, 6185]),
...brandEntries(DIGITALOCEAN, [14061]),
...brandEntries(HETZNER, [24940, 213230]),
...brandEntries(OVH, [16276]),
...brandEntries(ORACLE, [31898]),
...brandEntries(LINODE, [63949]),
...brandEntries(VULTR, [20473]),
...brandEntries(SCALEWAY, [12876]),
...brandEntries(IBM_CLOUD, [36351]),
...brandEntries(ALIBABA, [45102]),
...brandEntries(TENCENT, [132203]),
...brandEntries(GCORE, [199524]),
...brandEntries(CDN77, [60068]),
...brandEntries(SELECTEL, [50340, 49505]),
...brandEntries(TIMEWEB, [9123]),
...brandEntries(BEGET, [198610]),
...brandEntries(DDOS_GUARD, [57724]),
...brandEntries(META, [32934, 63293, 54115]),
...brandEntries(GOOGLE, [15169, 396982]),
...brandEntries(GITHUB, [36459]),
...brandEntries(GITLAB, [54876]),
...brandEntries(X, [13414]),
...brandEntries(LINKEDIN, [14413, 40793]),
...brandEntries(VK, [47541, 47764]),
...brandEntries(REDDIT, [394706]),
...brandEntries(DROPBOX, [19679]),
...brandEntries(SNAP, [19750]),
...brandEntries(WIKIPEDIA, [14907]),
...brandEntries(PAYPAL, [17012, 26101]),
...brandEntries(SALESFORCE, [14340]),
...brandEntries(YOUTUBE, [36040, 43515]),
...brandEntries(NETFLIX, [2906, 40027]),
...brandEntries(TWITCH, [46489]),
...brandEntries(TIKTOK, [396986, 138699]),
...brandEntries(SPOTIFY, [8403, 34081]),
...brandEntries(STEAM, [32590]),
...brandEntries(BLIZZARD, [57976]),
...brandEntries(EPIC, [395701]),
...brandEntries(RIOT, [6507, 62830]),
...brandEntries(PLAYSTATION, [33353]),
...brandEntries(ROBLOX, [22697]),
...brandEntries(UBISOFT, [197922]),
...brandEntries(DISCORD, [49544, 394141]),
...brandEntries(TELEGRAM, [62041, 59930, 211157]),
...brandEntries(ZOOM, [30103]),
...brandEntries(CHATGPT, [401115, 400645]),
...brandEntries(QUAD9, [19281]),
...brandEntries(OPENDNS, [36692]),
])
const ASN_HQ_COUNTRY = new Map<number, string>([
[13335, "US"],
[209242, "US"],
[54113, "US"],
[20940, "US"],
[16509, "US"],
[14618, "US"],
[8075, "US"],
[15169, "US"],
[32590, "US"],
[57976, "US"],
[2906, "US"],
[40027, "US"],
[36040, "US"],
[46489, "US"],
[401115, "US"],
[49544, "US"],
[32934, "US"],
[13238, "RU"],
[62041, "NL"],
[59930, "NL"],
[211157, "NL"],
...hqEntries("US", [
13335, 209242, 54113, 20940, 16625, 32787, 35994, 16702, 24319,
16509, 14618, 8987, 7224, 8075, 8068, 8069, 8070, 15169, 396982,
32590, 57976, 2906, 40027, 36040, 43515, 46489, 401115, 400645,
49544, 394141, 32934, 63293, 54115, 714, 6185, 36459, 54876, 14061,
31898, 63949, 20473, 36351, 13414, 14413, 40793, 394706, 19679, 19750,
14907, 17012, 26101, 14340, 30103, 36692, 395701, 6507, 62830, 33353, 22697,
]),
...hqEntries("IE", [9059]),
...hqEntries("SG", [138699]),
...hqEntries("DE", [24940, 213230]),
...hqEntries("FR", [16276, 12876, 197922]),
...hqEntries("CN", [45102, 132203]),
...hqEntries("LU", [199524]),
...hqEntries("CZ", [60068]),
...hqEntries("RU", [13238, 50340, 49505, 9123, 198610, 57724, 47541, 47764]),
...hqEntries("SE", [8403, 34081]),
...hqEntries("NL", [62041, 59930, 211157]),
...hqEntries("CH", [19281]),
])
const GOOGLE: BrandHit = { service: "Google", category: "Веб" }
const CLOUDFLARE: BrandHit = { service: "Cloudflare", category: "CDN" }
const YOUTUBE: BrandHit = { service: "YouTube", category: "Видео / стриминг" }
const CIDR_BRANDS: Array<{ cidr: string; prefixLen: number; hit: BrandHit }> = [
{ cidr: "104.16.0.0/13", prefixLen: 13, hit: CLOUDFLARE },
{ cidr: "104.24.0.0/14", prefixLen: 14, hit: CLOUDFLARE },
@@ -71,12 +175,67 @@ const CIDR_BRANDS: Array<{ cidr: string; prefixLen: number; hit: BrandHit }> = [
{ cidr: "172.217.0.0/16", prefixLen: 16, hit: GOOGLE },
{ cidr: "74.125.0.0/16", prefixLen: 16, hit: GOOGLE },
{ cidr: "142.250.0.0/15", prefixLen: 15, hit: GOOGLE },
{ cidr: "64.233.0.0/16", prefixLen: 16, hit: GOOGLE },
{ cidr: "66.102.0.0/16", prefixLen: 16, hit: GOOGLE },
{ cidr: "66.249.64.0/19", prefixLen: 19, hit: GOOGLE },
{ cidr: "72.14.192.0/18", prefixLen: 18, hit: GOOGLE },
{ cidr: "108.177.0.0/16", prefixLen: 16, hit: GOOGLE },
{ cidr: "209.85.128.0/17", prefixLen: 17, hit: GOOGLE },
{ cidr: "216.58.192.0/19", prefixLen: 19, hit: GOOGLE },
{ cidr: "216.239.32.0/19", prefixLen: 19, hit: GOOGLE },
{ cidr: "208.65.152.0/22", prefixLen: 22, hit: YOUTUBE },
{ cidr: "208.117.224.0/19", prefixLen: 19, hit: YOUTUBE },
{ cidr: "31.13.64.0/18", prefixLen: 18, hit: META },
{ cidr: "57.141.0.0/16", prefixLen: 16, hit: META },
{ cidr: "57.142.0.0/15", prefixLen: 15, hit: META },
{ cidr: "57.144.0.0/14", prefixLen: 14, hit: META },
{ cidr: "57.148.0.0/15", prefixLen: 15, hit: META },
{ cidr: "66.220.144.0/20", prefixLen: 20, hit: META },
{ cidr: "69.63.176.0/20", prefixLen: 20, hit: META },
{ cidr: "69.171.224.0/19", prefixLen: 19, hit: META },
{ cidr: "74.119.76.0/22", prefixLen: 22, hit: META },
{ cidr: "129.134.0.0/16", prefixLen: 16, hit: META },
{ cidr: "157.240.0.0/16", prefixLen: 16, hit: META },
{ cidr: "173.252.64.0/18", prefixLen: 18, hit: META },
{ cidr: "179.60.192.0/22", prefixLen: 22, hit: META },
{ cidr: "185.60.216.0/22", prefixLen: 22, hit: META },
{ cidr: "199.201.64.0/22", prefixLen: 22, hit: META },
{ cidr: "204.15.20.0/22", prefixLen: 22, hit: META },
].sort((a, b) => b.prefixLen - a.prefixLen)
const HOLDER_BRANDS: Array<{ re: RegExp; hit: BrandHit }> = [
{ re: /youtube/i, hit: YOUTUBE },
{ re: /instagram/i, hit: INSTAGRAM },
{ re: /valve|\bsteam\b/i, hit: STEAM },
{ re: /blizzard|battle.?net/i, hit: BLIZZARD },
{ re: /openai/i, hit: CHATGPT },
{ re: /riot games/i, hit: RIOT },
{ re: /epic games/i, hit: EPIC },
{ re: /\bapple\b/i, hit: APPLE },
{ re: /github/i, hit: GITHUB },
{ re: /spotify/i, hit: SPOTIFY },
{ re: /twitter|\bx corp\b/i, hit: X },
{ re: /dropbox/i, hit: DROPBOX },
{ re: /akamai/i, hit: AKAMAI },
]
const NON_ISO = new Set(["EU", "AP", "ZZ", "XX", "A1", "A2", "O1"])
const STEAM_ASN = 32590
const GOOGLE_FRONT_ASN = new Set([15169, 396982])
/** AS32934 / AS63293 — Meta front (Facebook + Instagram CDN). AS54115 — WhatsApp, не Instagram. */
const META_FRONT_ASN = new Set([32934, 63293])
const WHATSAPP_ASN = 54115
function isGooglePublicDns(ip: string): boolean {
return ipInCidrV4(ip, "8.8.8.0/24") || ipInCidrV4(ip, "8.8.4.0/24")
}
function isHttpsOrQuic(proto: number, dstPort: number, srcPort: number): boolean {
if (proto !== 6 && proto !== 17) return false
return dstPort === 443 || srcPort === 443
}
export function isIsoCountry(code: string): boolean {
const c = String(code ?? "").trim().toUpperCase()
return /^[A-Z]{2}$/.test(c) && !NON_ISO.has(c)
@@ -114,10 +273,68 @@ export function brandByCidr(ip: string): BrandHit | null {
return null
}
export function brandByHolder(holder: string): BrandHit | null {
const h = String(holder ?? "").trim()
if (!h) return null
for (const row of HOLDER_BRANDS) {
if (row.re.test(h)) return row.hit
}
return null
}
/** Игровые порты Steam — только вместе с AS32590, никогда :80/:443. */
export function isSteamGamePort(proto: number, dstPort: number, srcPort: number): boolean {
if (proto !== 6 && proto !== 17) return false
const port = dstPort || srcPort
if (!port || port === 80 || port === 443) return false
if (port === 4380 || port === 3478) return true
return port >= 27000 && port <= 27100
}
export function lookupBrand(ip: string, asn: number): BrandHit | null {
return brandByCidr(ip) || brandByAsn(asn)
}
function isGoogleFront(asn: number, cidrBrand: BrandHit | null, asnBrand: BrandHit | null): boolean {
return GOOGLE_FRONT_ASN.has(asn) || cidrBrand?.service === "Google" || asnBrand?.service === "Google"
}
function isInstagramFront(asn: number, cidrBrand: BrandHit | null, asnBrand: BrandHit | null): boolean {
if (asn === WHATSAPP_ASN) return false
return META_FRONT_ASN.has(asn) || cidrBrand?.service === "Meta" || asnBrand?.service === "Meta"
}
/**
* Cloudflare CIDR бьёт holder (витрина на CF не становится Steam).
* Holder (YouTube / Instagram и др.) бьёт остальные CIDR/ASN.
* HTTPS/QUIC на Google front YouTube (кроме 8.8.8.8); на Meta front Instagram (кроме WhatsApp ASN).
* Порты Steam только AS32590 и не выше Cloudflare CIDR.
*/
export function resolveFlowBrand(
ip: string,
asn: number,
holder: string,
proto = 0,
dstPort = 0,
srcPort = 0,
): BrandHit | null {
const cidrBrand = brandByCidr(ip)
if (cidrBrand?.service === "Cloudflare") return cidrBrand
const holderBrand = brandByHolder(holder)
if (holderBrand) return holderBrand
const asnBrand = brandByAsn(asn)
if (!isGooglePublicDns(ip) && isHttpsOrQuic(proto, dstPort, srcPort) && isGoogleFront(asn, cidrBrand, asnBrand)) {
return YOUTUBE
}
if (isHttpsOrQuic(proto, dstPort, srcPort) && isInstagramFront(asn, cidrBrand, asnBrand)) {
return INSTAGRAM
}
const fromLookup = cidrBrand || asnBrand
if (fromLookup) return fromLookup
if (asn === STEAM_ASN && isSteamGamePort(proto, dstPort, srcPort)) return STEAM
return null
}
const SKIP_MAP_SERVICES = new Set([
OTHER_SERVICE,
"GRE",
@@ -139,10 +356,23 @@ export function isNamedInternetService(service: string, category: string): boole
}
export function mapServiceNodeId(label: string): string {
const slug = label
.trim()
const raw = label.trim()
if (raw === OTHER_SERVICE) return "svc:other"
const slug = raw
.toLowerCase()
.replace(/[^a-z0-9]+/g, "-")
.replace(/^-+|-+$/g, "")
return `svc:${slug || "unknown"}`
}
/** `US` → `cc:us`; неизвестная / пустая → `cc:other`. */
export function mapCountryNodeId(code: string): string {
const iso = normalizeIsoCountry(code)
if (!iso) return "cc:other"
return `cc:${iso.toLowerCase()}`
}
/** id сервиса внутри страны: `cc:us|svc:google` — Google в US и NL не смешиваются в одном payload. */
export function mapCountryServiceNodeId(countryId: string, serviceId: string): string {
return `${countryId}|${serviceId}`
}
@@ -33,10 +33,10 @@ const google = classifyFlowDst("173.194.160.163", 6, 443, 1, {
ok: true,
fetchedAt: Date.now(),
})
assert.equal(google.service, "Google")
assert.equal(google.category, "Веб")
assert.equal(google.service, "YouTube")
assert.equal(google.category, "Видео / стриминг")
const googleCidr = classifyFlowDst("173.194.151.65", 6, 57182, 443, null)
const googleCidr = classifyFlowDst("173.194.151.65", 6, 80, 50000, null)
assert.equal(googleCidr.service, "Google")
assert.equal(googleCidr.category, "Веб")
@@ -53,9 +53,152 @@ const youtube = classifyFlowDst("173.194.160.163", 6, 443, 1, {
assert.equal(youtube.service, "YouTube")
assert.equal(youtube.category, "Видео / стриминг")
const valve = classifyFlowDst("203.0.113.40", 17, 27015, 50000, {
prefix: "203.0.113.0/24",
asn: 64501,
country: "US",
lat: null,
lng: null,
holder: "VALVE-CORPORATION",
ok: true,
fetchedAt: Date.now(),
})
assert.equal(valve.service, "Steam")
assert.equal(valve.category, "Игры")
const openaiHolder = classifyFlowDst("203.0.113.41", 6, 443, 1, {
prefix: "203.0.113.0/24",
asn: 64502,
country: "US",
lat: null,
lng: null,
holder: "OPENAI, US",
ok: true,
fetchedAt: Date.now(),
})
assert.equal(openaiHolder.service, "ChatGPT")
assert.equal(openaiHolder.category, "ИИ")
const cfNotSteam = classifyFlowDst("104.18.35.51", 6, 443, 1, {
prefix: "104.18.0.0/16",
asn: 32590,
country: "US",
lat: null,
lng: null,
holder: "VALVE-CORPORATION",
ok: true,
fetchedAt: Date.now(),
})
assert.equal(cfNotSteam.service, "Cloudflare")
assert.notEqual(cfNotSteam.service, "Steam")
const awsIeu = classifyFlowDst("203.0.113.42", 6, 443, 1, {
prefix: "203.0.113.0/24",
asn: 9059,
country: "IE",
lat: null,
lng: null,
holder: "AMAZON-02",
ok: true,
fetchedAt: Date.now(),
})
assert.equal(awsIeu.service, "AWS")
assert.equal(awsIeu.category, "CDN")
const googleCloud = classifyFlowDst("203.0.113.43", 6, 443, 1, {
prefix: "203.0.113.0/24",
asn: 396982,
country: "US",
lat: null,
lng: null,
holder: "GOOGLE-CLOUD",
ok: true,
fetchedAt: Date.now(),
})
assert.equal(googleCloud.service, "YouTube")
assert.equal(googleCloud.category, "Видео / стриминг")
const gre = classifyFlowDst("198.51.100.1", 47, 0, 0, null)
assert.equal(gre.service, "GRE")
assert.equal(gre.category, "Туннель")
const greIgnore = classifyFlowDst("8.8.8.8", 47, 0, 0, {
prefix: "8.8.8.0/24",
asn: 15169,
country: "US",
lat: null,
lng: null,
holder: "GOOGLE",
ok: true,
fetchedAt: Date.now(),
}, { ignoreTunnelProto: true })
assert.equal(greIgnore.service, "Google")
const dnsGoogle = classifyFlowDst("8.8.8.8", 17, 53, 53000, {
prefix: "8.8.8.0/24",
asn: 15169,
country: "US",
lat: null,
lng: null,
holder: "GOOGLE",
ok: true,
fetchedAt: Date.now(),
})
assert.equal(dnsGoogle.service, "Google")
assert.notEqual(dnsGoogle.service, "Прочее")
const ipv6Yt = classifyFlowDst("2001:4860:4860::8888", 17, 443, 50000, {
prefix: "2001:4860:4860::8888/128",
asn: 15169,
country: "US",
lat: null,
lng: null,
holder: "GOOGLE",
ok: true,
fetchedAt: Date.now(),
})
assert.equal(ipv6Yt.service, "YouTube")
const instagram = classifyFlowDst("157.240.12.52", 6, 443, 62598, {
prefix: "157.240.0.0/16",
asn: 32934,
country: "US",
lat: null,
lng: null,
holder: "FACEBOOK",
ok: true,
fetchedAt: Date.now(),
})
assert.equal(instagram.service, "Instagram")
assert.equal(instagram.category, "Видео / стриминг")
assert.notEqual(instagram.service, "Meta")
const metaHttp = classifyFlowDst("157.240.12.52", 6, 80, 50000, {
prefix: "157.240.0.0/16",
asn: 32934,
country: "US",
lat: null,
lng: null,
holder: "FACEBOOK",
ok: true,
fetchedAt: Date.now(),
})
assert.equal(metaHttp.service, "Meta")
assert.equal(metaHttp.category, "CDN")
const igHolder = classifyFlowDst("203.0.113.80", 6, 443, 1, {
prefix: "203.0.113.0/24",
asn: 64503,
country: "US",
lat: null,
lng: null,
holder: "Instagram LLC",
ok: true,
fetchedAt: Date.now(),
})
assert.equal(igHolder.service, "Instagram")
const igCidr = classifyFlowDst("57.144.22.192", 17, 443, 50000, null)
assert.equal(igCidr.service, "Instagram")
assert.equal(igCidr.category, "Видео / стриминг")
const esp = classifyFlowDst("198.51.100.1", 50, 0, 0, null)
assert.equal(esp.category, "Туннель")
assert.equal(applicationName(17, 443, 50000), "QUIC")
+12 -11
View File
@@ -1,4 +1,4 @@
import { lookupBrand, OTHER_SERVICE } from "./traffic-flow-brands.js"
import { OTHER_SERVICE, resolveFlowBrand } from "./traffic-flow-brands.js"
import { db } from "../db/index.js"
import { evobgpSettings } from "../db/schema.js"
import { ipInCidrV4, parseCidrV4 } from "./traffic-flow-ip.js"
@@ -47,12 +47,13 @@ export function seedFlowCatalogForTests(input: {
export function categoryFromPurpose(purpose: string, proto: number, dstPort: number, srcPort: number): string {
const p = purpose.toLowerCase()
if (/gaming|steam|epic|riot/.test(p)) return "Игры"
if (/streaming|youtube|netflix|twitch|video/.test(p)) return "Видео / стриминг"
if (/cdn|cloudflare|akamai|fastly/.test(p)) return "CDN"
if (/gaming|steam|epic|riot|playstation|roblox|ubisoft/.test(p)) return "Игры"
if (/streaming|youtube|netflix|twitch|video|spotify|instagram/.test(p)) return "Видео / стриминг"
if (/cdn|cloudflare|akamai|fastly|hetzner|ovh|apple/.test(p)) return "CDN"
if (/voip|discord|zoom/.test(p)) return "Голос"
if (/openai|chatgpt|\bai\b/.test(p)) return "ИИ"
if (/веб|web|google/.test(p)) return "Веб"
if (/quad9|opendns/.test(p)) return "DNS"
if (/веб|web|google|github|paypal|vk|linkedin/.test(p)) return "Веб"
const app = applicationName(proto, dstPort, srcPort)
if (app === "DNS" || app === "SSH" || app === "BGP") return app
if (app === "GRE" || app === "ESP" || app === "WireGuard") return "Туннель"
@@ -72,17 +73,17 @@ export function classifyFlowDst(
dstPort: number,
srcPort: number,
ripe: FlowIpMeta | null,
opts?: { ignoreTunnelProto?: boolean },
): FlowClassification {
if (proto === 47) return { service: "GRE", category: "Туннель" }
if (proto === 50) return { service: "ESP", category: "Туннель" }
if (!opts?.ignoreTunnelProto) {
if (proto === 47) return { service: "GRE", category: "Туннель" }
if (proto === 50) return { service: "ESP", category: "Туннель" }
}
const app = applicationName(proto, dstPort, srcPort)
if (app === "WireGuard") return { service: "WireGuard", category: "Туннель" }
const hit = matchCidr(dst)
const holder = ripe?.holder ?? ""
const youtubeHolder = /youtube/i.test(holder)
const brand = youtubeHolder
? { service: "YouTube", category: "Видео / стриминг" }
: lookupBrand(dst, ripe?.asn ?? 0)
const brand = resolveFlowBrand(dst, ripe?.asn ?? 0, holder, proto, dstPort, srcPort)
const asnName = ripe?.asn ? asnPurpose.get(ripe.asn) : undefined
const service = (hit?.purpose || brand?.service || asnName || OTHER_SERVICE).trim() || OTHER_SERVICE
const category = hit

Some files were not shown because too many files have changed in this diff Show More