Files
EvoFirewall/docs/policy-budget-cache.md
Denozordec 9fa6269174
quality / changes (push) Successful in 7s
quality / commitlint (push) Skipped
quality / openapi (push) Successful in 30s
quality / web (push) Successful in 1m3s
quality / api (push) Successful in 1m15s
quality / linux-agent-smoke (push) Failing after 27s
quality / docker-check (push) Failing after 2m28s
CD / quality (push) Failing after 5m52s
CD / publish (push) Skipped
feat: harden firewall policies telemetry and operations
Implement reliable agent apply, telemetry v2, analytics retention, guarded rollouts, enrollment, labels and webhooks. Improve ReUI frontend data safety and bounded list loading, and add Docker smoke, recovery and resource checks. Document local verification and pending native acceptance.
2026-10-26 13:10:58 +07:00

20 lines
5.0 KiB
Markdown

# Policy publication bounds and conditional polling
The compiler and agent contract uses shared `POLICY_BUDGET`: 50000 effective network memberships, 200000 raw memberships, 1000 serialized Port ACL rules, and an 8 MiB JSON response. The error code is `POLICY_BUDGET_EXCEEDED`. Importing a 100000-entry source list is still permitted; assigning an oversized effective policy is refused without truncation. The last successfully delivered desired hash/revision and the last applied acknowledgement are preserved on refusal.
Effective memberships are canonical unique CIDR strings in each serialized `deny_cidrs`, `allow_cidrs`, and individual `port_rules[].src_cidrs` array, before optional kernel CIDR collapse. Adjacent or overlapping but distinct networks count separately. A network repeated in distinct source sets counts for each set. `protocol:both` produces two serialized rules with the same logical ID and different protocols, and its source memberships count twice. Exact deny/allow duplicate conflicts are removed before the effective L3 count, preserving deny-wins semantics. Raw membership counting precedes deduplication and includes repeated source references and transport expansion. Source prefetches have a combined 200000-entry bound; compilation stops before publishing an over-budget candidate.
`src_kind:all` now means both `0.0.0.0/0` and `::/0`. An explicit IPv4 CIDR `0.0.0.0/0` remains IPv4-only. Agents without the required IPv6/ACL support must refuse the entire candidate and retain their last-good rules; the UI explains missing capabilities as unsupported until upgraded. There is no hidden IPv6 ND/RA exception.
`GET /v1/agent/policy` normally returns HTTP422 with the standard bounded error envelope for a compilation-budget refusal. A RouterOS fetch can opt into `?transport=routeros`: only this known refusal returns HTTP200 `{error:{code:"POLICY_BUDGET_EXCEEDED",message}}`, with no policy hash/generation, `Cache-Control:no-store`, and no ETag. `policyBudgetErrorEnvelopeSchema` is the shared explicit parser contract. Authentication and unexpected failures retain their actual status. Unknown transport values return400. Clients must inspect the error envelope before normalizing or importing a policy.
Each database owns a cache through a WeakMap. LRU limits are128 agents and32 MiB of conservative charged retained structures. Identical content is interned by content hash across agent entries; each entry retains its own generation, chain, summary, sync metadata and expiry. Eviction releases unused content references. Charge includes UTF16/string allowance, object/array headers and slots, map/entry overhead and metadata; repeated references within a representation are conservatively charged repeatedly. This is a structural memory budget rather than an allocator measurement. Temporary compilation/serialization work and the database are outside that cache budget. No serialized JSON response is permanently retained.
The cache key includes agent generation, default action and sync interval setting. List, hostname, set, rule, Port ACL and assignment mutations must publish their generation bumps. Expiry is processed transactionally before every cache lookup, and the earliest pending override expiry also bounds an entry's lifetime. Tests exercise actual list publication for a Port ACL-only dependency and hostname refresh, plus settings, revision and TTL invalidation.
JSON polling returns a strong versioned ETag over the exact complete response JSON, including generation, sync interval and script SHA. RouterOS script polling hashes the actual rendered script. `If-None-Match` can return304 with an empty body; authenticated contact and delivery metadata are still recorded. Returning to an earlier content hash after TTL expiry still produces a new ETag/HTTP200 because the generation changed. Clients without a conditional header continue receiving200. A client enabling304 must atomically retain a validated policy body and its matching ETag and reuse that body on304; merely retaining a hash is insufficient. Agent conditional-cache integration remains a separate script checkpoint.
V2 apply-report acknowledgements echo the pending `report_id` and `sequence` and include `sequence_floor`, the persisted highest accepted sequence. A restored agent receiving a matching authenticated stale-sequence ACK may advance its next sequence beyond that floor while marking lost coverage and rotating counter epochs. It must never rewrite a queued report. A floor equal to `Number.MAX_SAFE_INTEGER` is exhausted and cannot be incremented safely.
Checkpoint validation: the focused61-test suite includes ten compiler/cache tests, nine explain tests and telemetry/analytics/routes/integrity regressions; another27 existing policy/stats/list tests pass (88 distinct tests total). Shared/DB builds and API TypeScript check pass. The script owner validates parser limits, durable ACK recovery and subsequent304 bundle support separately; production kernel/RouterOS runtime acceptance and maximum-load profiling remain pending.