# Operational checkpoint

The deployment remains one Fastify/SPA/SQLite container. On 2026-10-26 Docker CLI is absent; `wsl.exe` exists but reports that WSL is not installed. Linux kernel execution and container resource measurements are pending CI. No throughput or memory improvement is claimed from request-injection tests.

## Authentication, permissions and limits

The operational plugin assigns route security metadata at registration: public, authenticated agent, protected user or the explicit `GET/HEAD /api/v1/ui/summary` section-filtered exception. A new `/api/*` route without a permission mapping or a new `/v1/*` agent route without an authentication mapping fails startup. The catalog coverage regression includes raw statistics, CSV permissions and agent explain. UI summary still filters each section by the caller's permissions. Diagnostics requires `fw:settings:admin`.

Successful agent requests are limited to 120/minute **per authenticated agent ID**, user requests to 600/minute per verified JWT subject, enrollment to 20/minute per source IP and remaining public API requests to 120/minute per IP. Limits are checked after authentication; users and agents behind one NAT do not share their successful-request bucket. Authentication failures are counted separately per IP: after 60 failures/minute, invalid requests return 429. A valid agent behind that same NAT remains usable. Verification still performs the indexed token lookup or JWT check before rejecting an invalid request, so this is not a volumetric network-DOS defense. Both maps cap identity buckets at 10,000 and reclaim expired windows before admitting more identities. Rejection includes `Retry-After: 60`.

Proxy headers are ignored by default. Configure only the actual reverse proxy IP/CIDRs, for example `TRUST_PROXY_CIDRS=127.0.0.1/32,10.20.0.10/32`; arbitrary truthy strings/hostnames are rejected. Audit uses Fastify's derived `request.ip`, never an unchecked first `X-Forwarded-For` value. Incorrectly trusting a whole client network permits spoofed IPs, so keep the list narrow.

Unknown `/api`, `/api/*`, `/v1` and `/v1/*` requests return JSON 404 even when SPA history fallback is enabled. Browser routes continue to serve the SPA.

## Durable audit delivery

Migration `014_audit_outbox.sql` creates a raw SQLite delivery table. Local audit append and outbox enqueue use one SQLite transaction. A single worker sends at most 25 events per batch every five seconds; it allows one active batch, uses an eight-second HTTP timeout, refuses redirects and records only a sanitized status/error. Success deletes the acknowledged batch. Failure retains event IDs and payload bytes and schedules exponential retry from five seconds up to one hour, without a retry-count deletion policy. Restart resumes persisted rows. Shutdown aborts the active HTTP request before DB closure and retains unacknowledged events.

Queue capacity is 10,000 events, with at most 16KiB per event and 8KiB bounded details. Settings audits contain only changed key names; values, tokens and passwords are excluded. Defensive key redaction also applies to other mutation details. An enabled full queue rejects protected writes with 503 before the handler. If disk failure or a race with an already-running asynchronous mutation prevents enqueue, local audit preservation is attempted, an error is logged and `enqueue_failures` increments; such an event needs operator recovery from the local audit log. Previously queued events are retained. This explicit failure mode does not promise delivery after a full disk.

Every enqueue validates the shared `ingestAuditEventSchema`: summaries are capped at 500 characters and invalid placeholder emails such as `dev@local` become null. Stored legacy events are normalized and validated again before delivery. Unrepairable JSON/contract/event-ID mismatches are quarantined individually, so a malformed row cannot poison a healthy batch. Quarantine retains at most 100 delivery rows and exposes only counts and a sanitized recovery instruction in diagnostics. Oldest excess delivery rows are removed while their original local audit records remain; operator repair uses that local log. Known legacy overlong summaries/invalid emails are repaired in place and retain the original event ID.

`GET /api/v1/diagnostics` reports DB/WAL byte sizes, available/total filesystem capacity, outbox volume/age/retry/capacity/enqueue failures, limiter counts, uptime and RSS/heap. It never returns queued payloads, secrets or complete filesystem paths. Unavailable metrics are null. This bounds the audit delivery queue and request buckets; Pino/stdout buffering is unchanged. Compose log rotation bounds stored container logs, and a stalled stdout consumer remains an operational limitation.

## Verification and Linux rehearsal

The in-process NAT regression sends 800 requests from 100 authenticated agents behind one source IP: three v2 reports with exact retries, policy reads and concurrent summary reads. It requires every response to be 200 and verifies retries do not double-count totals. The standalone Docker baseline runner separates a 1000-request cold/burst sweep from a warmed, paced 400-request steady profile. The steady phase adds a fourth unique report per agent with exact retry; final checks require four raw rows, 300/600 per-agent deltas and 30000/60000 fleet totals. It then imports 100k prefixes and requires a bounded summary response. The constrained Docker harness enforces the optional 500ms steady ingestion/timeseries p95 target; unconstrained Windows runs report observations without claiming target acceptance. Every HTTP error including 429 fails. It remains pending execution against a built Linux image.

## F2 webhooks implementation checkpoint

Migration `016_webhooks.sql` implements a separate outbox. Scope: settings administrators manage endpoints; consumers receive versioned event envelopes for agent offline/recovered, apply failure/recovery and source refresh failure/recovery. Payloads contain event ID, event type, occurrence time, agent/list ID and sanitized status codes, never tokens, source URLs, settings values or raw apply errors. Endpoint secrets are write-only, rotated explicitly and omitted from API responses, audit details, diagnostics and logs.

Use a separate durable outbox and single bounded worker polled every 30 seconds, preserving local controller writes when webhook delivery fails. Limits: 20 endpoints, 1000 pending rows globally, 16KiB encoded body, one active request, five deliveries per batch, eight-second deadline and 64KiB maximum response consumption. Persist the exact signed payload bytes and event ID for at-least-once retries; consumers deduplicate by ID. Failures retry with exponential backoff and jitter up to one hour, at most 12 attempts/24 hours before a bounded dead-letter state. Expose only counts, oldest age and sanitized error codes. Capacity exhaustion increments a dropped-notification counter and emits a local audit warning; it must not block agent telemetry or policy application. Retain at most 100 dead letters; no unbounded retry history.

Persist a five-minute cooldown per endpoint/event type/resource, with transition detection so repeated identical failures are suppressed and recovery is emitted once. Commit transition state and enqueue atomically to avoid duplicate storms after restart. A failed delivery does not restart cooldown. Endpoint deletion cancels queued deliveries; secret rotation defines whether old queued messages retain their original signing key (proposed: discard pending messages and audit the count, never retain retired secrets).

Sign `timestamp + '.' + raw_body` with HMAC-SHA256; send version, timestamp, event ID and signature in dedicated headers. Verify timestamps within five minutes and use constant-time comparison at receivers. Retry uses the stable event ID/body with a fresh timestamp/signature. Secrets require the existing `EVOFW_SECRET_KEY` (at least 32 characters), with AES-256-GCM encryption and endpoint-ID authenticated context; there is no plaintext fallback. Missing/weak key disables delivery and makes create/update return clear 503. Back up both the DB and current key securely before rotation. Existing ciphertext must be decrypted using the old key and re-encrypted with the new key during offline maintenance, or explicitly replace each endpoint signing secret under the new key; changing only the environment key prevents existing secrets from decrypting. Keep the old key with its matching backups for recovery.

POST transport must support DNS pinning specifically: HTTPS only, no userinfo/fragments, default port 443, no redirects, validate every resolved address against public ranges, select one validated address and pin the socket lookup while keeping TLS SNI/certificate verification for the hostname. Revalidate on every retry. The existing source GET transport must not be reused blindly: its configured internal-origin exception must never enable webhooks. No cross-origin forwarding of authorization, proxy inheritance or arbitrary request headers. Tests must cover DNS rebinding, mixed public/private answers, IPv4-mapped IPv6, redirects, timeouts, restart replay, cooldown and capacity pressure before enabling delivery.

Native RouterOS validation remains a separate release gate on an explicitly configured disposable router/runner: install the shipped script, apply IPv4 and IPv6 policies, verify counters, retry reports, drift repair and preservation of foreign rules. No RouterOS endpoint or credentials are configured here, so neither a static RSC check nor Linux CI counts as that gate passing. The existing `linux-agent-smoke` workflow is scheduled on API/Docker changes and runs the shipped agent in isolated namespaces; its result still requires an actual CI run.

```sh
# From apps/api
node node_modules/vitest/vitest.mjs run src/services/operational.test.ts src/services/audit.test.ts
# From repository root, on a disposable Linux runner with namespace privileges
sudo bash scripts/ci/linux-netns-smoke.sh
sh scripts/bench/runtime.sh IMAGE runtime-artifacts/candidate
```

The Linux smoke always creates network, mount and PID namespaces before any firewall mutation; it verifies the namespace IDs differ from the parent, makes mounts private and uses temporary `/etc`, state, log and `/run` mounts. A nested client namespace and veth pair exercise real packet denial. It runs the shipped agent against a local fixture, checks drift repair, invalid-candidate preservation and an unrelated nft table. It never applies policies to the host firewall. CI installs prerequisites and runs this script; unavailable namespace capabilities fail the job rather than reporting a skipped check as success. CI now executes separate isolated nft and ipset namespaces with both IPv4 and IPv6 veth HTTP traffic: allow-before-policy, deny-after-policy, drift repair, invalid-candidate preservation and foreign-table preservation. Actual execution is pending CI; Docker-network and RouterOS matrices remain separate. Local validation covers shell/Python syntax only; installation of WSL or host reboot is not part of this work.

Validation: 12 focused webhook tests passed (7 durable-service, 3 mocked transport, 2 route/RBAC tests). They cover encryption, missing-key 503, redacted DTOs, restart/retry byte identity, cooldown recovery, endpoint/outbox/dead-letter bounds, mixed/private DNS, socket pinning, redirect refusal and shutdown cancellation. No real external webhook was sent. Native Docker/Linux/RouterOS validation remains pending CI or an explicitly configured runner.

Webhook cancellation review: endpoint delivery changes increment a persistent generation and cancel queued rows. Each batch candidate is re-read immediately before sending and must still be pending, enabled and match that generation. A request already in progress may finish; canceled unsent bytes cannot be moved to a changed URL/secret. Two paused-transport regressions cover URL/secret change and disable while the first POST is in flight.